If you're willing to share, I'd love to hear about your experience with the Spotify API. What endpoints do you use? How often do you call them? How do you handle rate limiting? Has Spotify reached out regarding storing playlist data (being in violation of their terms of service)? Any other lessons or advice?
Additionally, I wonder if there's any opportunity for collaboration here. Perhaps you can use the archive to fill out your dataset? Maybe it can act as a cache for your system's scraping needs? Perhaps our Web UIs could link to each other or be combined somehow (https://spotifyplaylistarchive.com/)?
> Currently we keep track of 18378 Spotify playlists
How did you find all of these playlists? Querying Spotify for "Spotify"-owned playlists gives me less than 5000 results. I'd love to add more playlists to my archive but I don't have good discovery mechanisms, especially for non-Spotify-owned playlists.
I maintain an archive of Spotify playlists, scraped daily via GitHub Actions: https://github.com/mackorone/spotify-playlist-archive.
If you're willing to share, I'd love to hear about your experience with the Spotify API. What endpoints do you use? How often do you call them? How do you handle rate limiting? Has Spotify reached out regarding storing playlist data (being in violation of their terms of service)? Any other lessons or advice?
Additionally, I wonder if there's any opportunity for collaboration here. Perhaps you can use the archive to fill out your dataset? Maybe it can act as a cache for your system's scraping needs? Perhaps our Web UIs could link to each other or be combined somehow (https://spotifyplaylistarchive.com/)?
> Currently we keep track of 18378 Spotify playlists
How did you find all of these playlists? Querying Spotify for "Spotify"-owned playlists gives me less than 5000 results. I'd love to add more playlists to my archive but I don't have good discovery mechanisms, especially for non-Spotify-owned playlists.