Skip to content

Instantly share code, notes, and snippets.

@TheMuellenator

TheMuellenator/main.py

Forked from angelabauer/main.py
Last active May 6, 2024 15:46
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 17 You must be signed in to fork a gist
  • Save TheMuellenator/c84616c21f0f9ce68c12c357d3e1c794 to your computer and use it in GitHub Desktop.
Save TheMuellenator/c84616c21f0f9ce68c12c357d3e1c794 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
main.py
sp = spotipy.Spotify(
auth_manager=SpotifyOAuth(
scope="playlist-modify-private",
redirect_uri="http://example.com",
client_id=YOUR UNIQUE CLIENT ID,
client_secret= YOUR UNIQUE CLIENT SECRET,
show_dialog=True,
cache_path="token.txt"
)
)
user_id = sp.current_user()["id"]
date = input("Which year do you want to travel to? Type the date in this format YYYY-MM-DD: ")
song_uris = ["The list of", "song URIs", "you got by", "searching Spotify"]
playlist = sp.user_playlist_create(user=user_id, name=f"{date} Billboard 100", public=False)
# print(playlist)
sp.playlist_add_items(playlist_id=playlist["id"], items=song_uris)
@russbrownccie
Copy link

russbrownccie commented Oct 4, 2022 via email

@kakaa2993
Copy link

kakaa2993 commented Oct 5, 2022 •
edited

from bs4 import BeautifulSoup
import requests

from spotipy.oauth2 import SpotifyOAuth
import spotipy

SPOTIPY_REDIRECT_URI='http://localhost:8888/callback/'

BILLBOARD_HOT_TOP_100_URL = "https://www.billboard.com/charts/hot-100/"
Client_ID ='your id'
Client_Secret ='your client sec'

---------Scraping the names of the top 100-------------

date = input("Which year do you want to travel to? type the date in this format 'YYYY-MM-DD':\n")
response = requests.get(url=BILLBOARD_HOT_TOP_100_URL+date)
html_page = response.text
soup = BeautifulSoup(html_page, "html.parser")
music_list = soup.select(selector='li #title-of-a-story')
music_titles = [_.getText().strip() for _ in music_list]
print(music_titles)

------Creating spotify object and search for the songs on spotify ------

Spotify = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id,
client_secret,
redirect_uri="http://localhost:8888/callback/",
scope="playlist-modify-private",
show_dialog=True,
cache_path=".cache"))
user_id = Spotify.current_user()["id"]
musics_dict = {}
for album in music_titles:
try:
result = Spotify.search(q=f"track:{album} year:{date.split('-')[0]}", type="track", limit=1, offset=0)
song_dic = result["tracks"]
song_items = song_dic["items"]
song = song_items[0]["external_urls"]["spotify"]
musics_dict[album] = song
except:
pass
print(musics_dict)

-------------- Create the playlist and add the songs ----------

playlist = Spotify.user_playlist_create(user_id, f"{date} Billboard 100", public=False, collaborative=False, description='')
for _ in musics_dict.keys():
song_id = musics_dict[_].split("/")[-1]
add_song = Spotify.playlist_add_items(
playlist_id=playlist["id"],
items=[f'spotify:track:{song_id}'],
position=None
)

@zeta-bradley81
Copy link

zeta-bradley81 commented Oct 11, 2022 via email

@cosersakura
Copy link

I was struggling with creating the playlist with the exact songs because the code was only searching by title name, the problem this created was that a few times a song would be added but of another artist (who also had a song with the same name), instead of the actual Billboard 100 song being added. Therefore I changed the search by title name AND artist. This is my code:

from bs4 import BeautifulSoup
import requests
import spotipy
from spotipy.oauth2 import SpotifyOAuth


# Scraping Billboard 100
date = "1994-12-24"


response = requests.get("https://www.billboard.com/charts/hot-100/" + date)
soup = BeautifulSoup(response.text, 'html.parser')


# List of songs
song_list_raw = soup.select(selector="div ul li ul li h3")
song_list = []
for s in range(99):
    song_list.append(song_list_raw[s].getText().replace('\n', ''))


# List of artists
span_class = "c-label a-no-trucate a-font-primary-s lrv-u-font-size-14@mobile-max u-line-height-normal@mobile-max u-letter-spacing-0021 lrv-u-display-block a-truncate-ellipsis-2line u-max-width-330 u-max-width-230@tablet-only"
artist_list_raw = soup.select(selector="div ul li ul li span", class_=span_class)
artist_list = []
for a in range(0, 700, 7):
    '''
    I know this is not great code, kind of creative haha. The output I got from web-scraping (soup.select above), the artists were every 7 lines apart . This was my solution (to iterate every 7 lines), and works. If someone is reading this, maybe you have a better solution? 
    '''
    artist_list.append(artist_list_raw[a].getText().replace('\n', ''))



# Zip songs and artists
songs_artists = list(zip(song_list, artist_list))


# Spotify Authentication
sp = spotipy.Spotify(
    auth_manager=SpotifyOAuth(
        scope="playlist-modify-private",
        redirect_uri="http://example.com",
        client_id='your_id',
        client_secret='your_secret',
        show_dialog=True,
        cache_path="token.txt"
    )
)
user_id = sp.current_user()["id"]
print(user_id)


# Searching Spotify for songs by TITLE AND ARTIST!
song_uris = []
for n in range(99):
    result = sp.search(q=f"track:{songs_artists[n][0]} artist:{songs_artists[n][1]}", type="track")  #
    print(result)
    try:
        uri = result["tracks"]["items"][0]["uri"]
        song_uris.append(uri)
    except IndexError:
        print(f"{songs_artists[n][0]} doesn't exist in Spotify. Skipped.")


# Creating a new private playlist in Spotify
playlist = sp.user_playlist_create(user=user_id, name=f"{date} Billboard 100", public=False)
print(playlist)

# Adding songs found into the new playlist
sp.playlist_add_items(playlist_id=playlist["id"], items=song_uris)


print(songs_artists)

Thank you so much! I struggled so long time and did not solve this case still. The document of Spotipy is really confusing.

@d1patel69
Copy link

d1patel69 commented Nov 12, 2022 •
edited

Wasted a lot of time trying to make the API for Spotify work for search (i.e.: https://developer.spotify.com/documentation/web-api/reference/#/operations/search) ...eventually gave up and saw the solution to realize I had to use sp.search...That was useful. Suffice to say the Spotify Web API documentation isn't great (IMHO)...

Anyway, it works now - so happy. And I am going to extend tomorrow to see if I can create Playlists of #'s by Year of choice :) - should be 'easy' now ;)

BTW, one enhancement. When I was using the search by year, it wasn't always finding the songs on Spotify even though they were there, so I ended up enhancing the code a little to include a hierarchy of searches - first by year(range), then by artist, and finally with no restriction on year or artist. Seems to find most songs this way albeit sometimes I end up with a cover. I am sure there is a better way to do this but I am going to move on to the next challenge now.

Code snippets are included below if anyone is interested (have also written the code for all songs that might have been top-5 for a year - not included below). Fun exercise, took way too long but learnt a few things!

-----------Need the artist name to use for searching--------------

Following gives me the list of the artists - matches by index the song list...

results = soup.select("div li ul li span")
artists_long_list = [item.text.strip() for item in results]
artists = []

Extract each artist, incrementing by 7 through the long list because items 2-6 are additional info

related to the entry

for i in range(0, len(artists_long_list), 7):
artists.append(artists_long_list[i])

-----------Snapshot of hierarchy code--------------

Search for each song in the list, using year as a limiter, get the URI into a list so this can

be used to add songs to the playlist. If year doesn't work, then try artist and if that doesn't

then try no restriction.

songs_uri = []
not_found = []
for i in range(0, len(songs)):
year_range = f"{int(year)-1}-{int(year)+1}"

# Following method yields 27 songs not found on my test  date
#
# Following code is a hierarchy of searches, first with year, then with artist then no
# restriction to find as many songs as possible from the list
search_response = sp.search(q=f"track:{songs[i]} year:{year_range} artist:{artists[i]}",
                            limit=1, type="track")

try:
    uri = search_response["tracks"]["items"][0]["uri"]
    songs_uri.append(uri)
except IndexError:
    # First method didn't work, so exception handle to try second method
    print(f"{songs[i]} track by {artists[i]} not found on Spotify, trying a different method")

    # Try artists only instead of year
    # Following method yields 18 songs not found for same test date 
    search_response = sp.search(q=f"track:{songs[i]} artist:{artists[i]}", limit=1, type="track")
    try:
        uri = search_response["tracks"]["items"][0]["uri"]
        songs_uri.append(uri)
    except IndexError:
        # Second method also didn't work, so trying final method
        print(f"{songs[i]} track by {artists[i]} still not found on Spotify, trying final method")

        # Following method yields 0 songs not found for same test date but some songs
        # might be a cover (e.g. I noticed Adele on one of them)
        search_response = sp.search(q=f"track:{songs[i]}", limit=1, type="track")
        try:
            uri = search_response["tracks"]["items"][0]["uri"]
            songs_uri.append(uri)
        except IndexError:
            print(f"{songs[i]} track by {artists[i]} still not found on Spotify, skipping")
            not_found.append(f"{songs[i]} by {artists[i]}")

@hamidshafiei
Copy link

import spotipy
from spotipy.oauth2 import SpotifyClientCredentials
from bs4 import BeautifulSoup
import requests
b = spotipy.client.Spotify(auth_manager=spotipy.oauth2.SpotifyOAuth(client_id="",
client_secret="",
scope="playlist-modify-private",
redirect_uri="http://example.com",
cache_path="token.txt"))

url="https://www.billboard.com/charts/hot-100/"
travel_time = input("Which year do you want to travel to? Type the date in this format YYYY-MM-DD ")
response = requests.get(url+travel_time)
web_html = response.text
soup = BeautifulSoup(web_html, "html.parser")
titles = soup.select("li ul li h3")
a=[title.getText().strip() for title in titles]
year = travel_time[0:4]
x1 = []
for item in a:
h = b.search(f"track: {item} year: {year}", type='track', limit=1)
try:
r = h["tracks"]["items"][0]["uri"]
x1.append(r)
except IndexError:
print(f"{item} doesn't exist in Spotify.")
print(x1)
user_id=b.current_user()["id"]
print(user_id)
playlist_id = b.user_playlist_create(user=user_id,name=f"{travel_time}Billboard 100",public=False)
b.playlist_add_items(playlist_id=playlist_id["id"], items=x1)

@rillyv
Copy link

rillyv commented Nov 30, 2022

This took me all day ^^" but managed to get a 95-100% success from 80's music until today. Most "not found" songs are caused by track/artist differences between what spotify expects and what billboard offers. Some others are simply unavailable in your country, or are rated below covers, remixes and so on.

Here goes my approach. I hope it helps someone struggling with this project:


import requests

import datetime

from spotipy.oauth2 import SpotifyOAuth
from spotipy.client import Spotify
from spotipy.cache_handler import CacheFileHandler


def process_artist(s: str) -> str:
    """ Returns a more 'friendly' string for Spotify search engine. """
    out = s.replace("'", "")

    open_par = out.find("(")
    close_par = out.find(")")
    if 0 <= open_par < close_par:
        out = out.split("(")[0] + out.split(")")[1]

    out = out.split("Feat")[0]

    return out


def process_track(s: str) -> str:
    """ Returns a more 'friendly' string for Spotify search engine. """
    out = s.replace("'", "")

    open_par = out.find("(")
    close_par = out.find(")")
    if 0 <= open_par < close_par:
        out = out.split("(")[0] + out.split(")")[1]

    words = out.split(" ")
    out = ""
    for w in words:
        if w.find('*') < 0:
            out = out + w + " "

    return out


def get_song_uris(client: Spotify, tracks: [str], track_artists: [str]) -> [str]:
    """ Returns song uris from tracks and artists names. """
    uris = []
    for track, artist in zip(tracks, track_artists):
        processed_track = process_track(track)
        processed_artist = process_artist(artist)

        query = f"{processed_track} {processed_artist}"
        results = client.search(q=query, type="track", market="US", limit=1)
        try:
            uris.append(results["tracks"]["items"][0]["uri"])
        except IndexError:
            print(f"'{track} --- {artist}' not found.")
            print(f"{processed_track} --- {processed_artist}.")

    return uris


def get_playlist(client: Spotify, user: int, name: str) -> int:
    """ Returns playlist's id by name. Returns 0 if it already existed. """
    playlists = spotify.user_playlists(user_id)["items"]

    for p in playlists:
        if p["name"] == name:
            return 0

    playlist = client.user_playlist_create(user=user, name=name, public=False)

    return playlist["id"]


def get_spotify_client() -> Spotify:
    """ Returns a spotipy client based on inside-function data. """
    spotify_client_id = "CLIENT_ID"
    spotify_client_secret = "CLIENT_SECRET"
    spotify_redirect_uri = "REDIRECT_URI"
    spotify_scope = "playlist-modify-private playlist-read-private"  # Allows us to read and modify private playlists
    spotify_cache_handler = CacheFileHandler("./token.txt")

    return Spotify(
        auth_manager=SpotifyOAuth(
            client_id=spotify_client_id,
            client_secret=spotify_client_secret,
            redirect_uri=spotify_redirect_uri,
            scope=spotify_scope,
            cache_handler=spotify_cache_handler
        )
    )


def get_billboard_songs_artists(date_str: str) -> ([str], [str]):
    """ Gets top 100 billboard title songs and artists from parameter date.
        Returns ([song_list], [artist_list]). """
    billboard_url = "https://www.billboard.com/charts/hot-100/"
    response = requests.get(billboard_url + date_str)

    soup = BeautifulSoup(response.text, "html.parser")
    # There are a lot of trash '\n' and '\t' in raw content
    raw_titles = [t.getText() for t in soup.select("li > h3.c-title")]
    raw_artists = [artist.getText() for artist in soup.select("li > span.a-no-trucate")]

    # Get rid of those \n and \t
    processed_titles = [t.replace("\n", "").replace("\t", "") for t in raw_titles]
    processed_artists = [a.replace("\n", "").replace("\t", "") for a in raw_artists]

    return processed_titles, processed_artists


def check_date(date_str: str) -> bool:
    """Checks if date format is correct and if date <= today"""
    wrong_format_msg = "Wrong date format. Must be -> YYYY-MM-DD"
    # Date to int
    try:
        ymd = [int(i) for i in date_str.split("-")]
    except ValueError:
        print(wrong_format_msg)
        return False
    # Check that we got three values
    if len(ymd) != 3:
        print(wrong_format_msg)
        return False
    # Check date format is valid
    try:
        date_obj = datetime.date(year=ymd[0], month=ymd[1], day=ymd[2])
    except ValueError as e:
        print(e)
        return False

    today = datetime.date.today()
    # A future date is not allowed
    if date_obj > today:
        print("Input date can't be higher than current date")
        return False
    # Everything is OK
    return True


if __name__ == "__main__":
    date = input("Enter date YYYY-MM-DD: ")
    if check_date(date):
        songs, artists = get_billboard_songs_artists(date)

        spotify = get_spotify_client()
        user_id = spotify.me()["id"]
        playlist_name = "Top 100. " + date

        playlist_id = get_playlist(spotify, user_id, playlist_name)

        if playlist_id != 0:  # Playlist didn't exist
            song_uris = get_song_uris(spotify, songs, artists)
            spotify.playlist_add_items(playlist_id, song_uris)
        else:
            print("That playlist already existed.")

AMAZING, thanks. Learned a lot from your code.

@Tnatiel
Copy link

Tnatiel commented Dec 12, 2022

Hello everyone

I anybody get this bug:

spotipy-create-playlist-error

Try change the public argument to False:
spotipy-create-playlist-solution

@512jay
Copy link

512jay commented Dec 14, 2022 •
edited

For anyone stuck I hope this helps, here is my working code:

from bs4 import BeautifulSoup
import spotipy
from spotipy.oauth2 import SpotifyOAuth
import os

date = input("Which year do you want to travel to? Type the date in this format YYYY-MM-DD: \n")
url = f"https://www.billboard.com/charts/hot-100/{date}/"
response = requests.get(url)
web_page = response.text
soup = BeautifulSoup(web_page, "html.parser")
song_soup = soup.find_all("h3", "a-no-trucate")
song_names = [s.getText(strip=True) for s in song_soup]
spotify = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id=os.environ.get("client_id"),
                                                    client_secret=os.environ.get("client_secret"),
                                                    redirect_uri="http://example.com",
                                                    scope="playlist-modify-private"))

user = spotify.current_user()['id']
name = f"{date} Billboard 100"
print(f"Generating a playlist for user: {user} called {name}")

playlist_id = spotify.user_playlist_create(user, name, public=False, collaborative=False,
                                           description=f'The top 100 songs on {date}')['id']


def get_track_id(song):
    track_id = spotify.search(song, limit=1)['tracks']['items'][0]['uri']
    return track_id


track_ids = [get_track_id(song) for song in song_names]
spotify.playlist_add_items(playlist_id, track_ids)
print("Complete")

@ebetanc
Copy link

ebetanc commented Dec 15, 2022

#--------------------------------Scraping Billboard --------------------------------------------# user_date = input("What year would you like to visit? Type the date in this format YYYY-MM-DD: ")

Create the request to the billboard website

response = requests.get(url=f"{billboard_url}/{user_date}/") billboard_response = response.text

Make soup using the info you got from the website.

soup = BeautifulSoup(billboard_response,"html.parser")

Tap into the section you want. Here, it will be the titles of songs.

song_titles=[song.getText().strip() for song in soup.select(selector="li #title-of-a-story")]

print (song_titles) # Prints the list of songs scraped.

#-------------------Spotify Authentication and token.txt creation ----------------------#

Authenticate yourself on spotify

spotify_auth = spotipy.oauth2.SpotifyOAuth(client_id=Client_ID, client_secret=Client_Secret, redirect_uri=redirect_URI, scope="playlist-modify-private", show_dialog=True, cache_path="token.txt" ) spotify_auth.get_access_token(as_dict=False) s = spotipy.Spotify(oauth_manager=spotify_auth) user_id = s.current_user()["id"] print (user_id) prints the User_ID

#--------------------Searching Spotify for the songs----------------------------#

Searching spotify for songs by their titles and getting their uris

Set a new empty list for the uris

song_uris = [] year = user_date.split("-")[0] for song in song_titles: result = s.search(q=f"track:{song} year:{year}", type="track") # print(result) #Prints the result try: # Handling exception where the song cannot be found. It is skipped in this case. uri = result["tracks"]["items"][0]["uri"] song_uris.append(uri) except IndexError: print(f"{song} doesn't exist in Spotify. Skipped.")

print (song_uris) # You can verify song by visiting https://open.spotify.com/track/URI_ID_GENERATED

Example :https://open.spotify.com/track/2y4lAQpi5VTNLu2ldeTdUH

#-----------------Creating new private playlist in Spotify--------------------------# playlist = s.user_playlist_create(user_id,name=f"{user_date} Billboard 100",public=False, description="Musical Time Machine") #print(playlist)

#---------------- Adding the songs to the playlist----------------------------# s.playlist_add_items(playlist_id=playlist['id'], items=song_uris)

Is this one still working for you? It doesn't for me

@LiamBatiste
Copy link

Created a slightly more optimized version of this program, with respect to the querying of tracks.

I noticed that using the title/track name and the year occasionally resulted in covers of the song being added to the playlist, as opposed to the desired artist. So I scraped the billboard website for the artist and included this in the query when searching for a more specific search.

This did result in slightly reduced success when searching for tracks but did give me the correct artist (the artist that got the track into the billboards).

Here is my version:

import requests
from bs4 import BeautifulSoup
import spotipy
from spotipy.oauth2 import SpotifyOAuth
import pprint
import time
import os

pp = pprint.PrettyPrinter(indent=0)

client_ID = os.environ["client_ID"]
client_secret = os.environ["client_secret"]
scope = "playlist-modify-public"

sp = spotipy.Spotify(
    auth_manager=SpotifyOAuth(
        client_id=client_ID,
        client_secret=client_secret,
        redirect_uri="http://mysite.com/callback/",
        scope=scope,
        show_dialog=True,
        cache_path="token.txt"))

user_id = sp.current_user()["id"]

date = input("What year would you like to create a spotify playlist (please use the YYYY-MM-DD format): ")

while len(date) != 10:
    print("please use correct date format YYYY-MM-DD")
    date = input("What year would you like to create a spotify playlist (please use the YYYY-MM-DD format): ")

year = date.split("-")[0]
month = date.split("-")[1]
day = date.split("-")[2]

print("Creating spotify playlist...")
time.sleep(1)

response = requests.get(f"https://www.billboard.com/charts/hot-100/{date}/")
website = response.text

soup = BeautifulSoup(website, "html.parser")

titles = soup.find_all(name="h3", id="title-of-a-story", class_="a-no-trucate")
artists = soup.find_all(name="span", class_="a-font-primary-s")

t100_list = [title.getText().strip("\n\t") for title in titles]

t100_artist = [artist.getText().strip("\n\t") for artist in artists]

for artist in t100_artist:
    if artist == "RIAA Certification:":
        t100_artist.remove(artist)

song_uris = []

artist_index_count = 0
for tracks in t100_list:
    results = sp.search(q=f"track:{tracks} artist: {t100_artist[artist_index_count]}", type="track")
    artist_index_count += 1
    try:
        song_uri = results["tracks"]["items"][0]["uri"]
        print(song_uri)
        song_uris.append(song_uri)
    except IndexError:
        print("Track number " + str(t100_list.index(tracks) + 1) + " cannot be found")

playlists = sp.user_playlist_create(user=f"{user_id}", name=f"{year} Billboard top 100 ({day}/{month})", public=True, description=f"Top Tracks from the year: {year}")

sp.playlist_add_items(playlist_id=playlists["id"], items=song_uris)

print(f"\n...Your playlist has been created, we managed to find a total of {len(song_uris)}/100 of the top 100 songs! ")

@krishnaraddi
Copy link

Try this for the project:

import requests import spotipy from bs4 import BeautifulSoup from spotipy.oauth2 import SpotifyOAuth billboard_url = "https://www.billboard.com/charts/hot-100/" Client_ID = "Your Client ID" Client_Secret= "Your Client Secret" redirect_URI="http://example.com"

#--------------------------------Scraping Billboard --------------------------------------------# user_date = input("What year would you like to visit? Type the date in this format YYYY-MM-DD: ")

Create the request to the billboard website

response = requests.get(url=f"{billboard_url}/{user_date}/") billboard_response = response.text

Make soup using the info you got from the website.

soup = BeautifulSoup(billboard_response,"html.parser")

Tap into the section you want. Here, it will be the titles of songs.

song_titles=[song.getText().strip() for song in soup.select(selector="li #title-of-a-story")]

print (song_titles) # Prints the list of songs scraped.

#-------------------Spotify Authentication and token.txt creation ----------------------#

Authenticate yourself on spotify

spotify_auth = spotipy.oauth2.SpotifyOAuth(client_id=Client_ID, client_secret=Client_Secret, redirect_uri=redirect_URI, scope="playlist-modify-private", show_dialog=True, cache_path="token.txt" ) spotify_auth.get_access_token(as_dict=False) s = spotipy.Spotify(oauth_manager=spotify_auth) user_id = s.current_user()["id"] print (user_id) prints the User_ID

#--------------------Searching Spotify for the songs----------------------------#

Searching spotify for songs by their titles and getting their uris

Set a new empty list for the uris

song_uris = [] year = user_date.split("-")[0] for song in song_titles: result = s.search(q=f"track:{song} year:{year}", type="track") # print(result) #Prints the result try: # Handling exception where the song cannot be found. It is skipped in this case. uri = result["tracks"]["items"][0]["uri"] song_uris.append(uri) except IndexError: print(f"{song} doesn't exist in Spotify. Skipped.")

print (song_uris) # You can verify song by visiting https://open.spotify.com/track/URI_ID_GENERATED

Example :https://open.spotify.com/track/2y4lAQpi5VTNLu2ldeTdUH

#-----------------Creating new private playlist in Spotify--------------------------# playlist = s.user_playlist_create(user_id,name=f"{user_date} Billboard 100",public=False, description="Musical Time Machine") #print(playlist)

#---------------- Adding the songs to the playlist----------------------------# s.playlist_add_items(playlist_id=playlist['id'], items=song_uris)

END RESULT IN SPOTIFY ACCOUNT: image

@krishnaraddi
Copy link

it helped me !!!! .. clear TODO steps in one page

@lucasps100
Copy link

2023-03-15 (1)

Found a helpful fix for error handling. If you query without the "track:", "artist:" or "year:" keywords, it does a basic search and pulls the most relevant. I also removed the parentheses from any song title. Haven't gotten any errors so far.

@cipa5
Copy link

cipa5 commented Mar 22, 2023

I'm having an error and I can't add tracks to my playlist.
raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 400, code:-1 - https://api.spotify.com/v1/playlists/6WW7HbTGZgkrOcoBL9TNDR/tracks:
Invalid track uri: spotify🧑‍🎨1l7ZsJRRS8wlW3WfJfPfNS, reason: None

I had the same error and the solution was to change the argument public to False because the SCOPE on Spotify0Auth i used "playlist-modify-private", i know you fixed the problem but dont see de solution, just add thid

True!! I changed the Public = False, it worked after getting lots of errors, seems that the scope and this argument has to be the same i.e. Private

Thank you, this helped a lot!

@ezequielo198
Copy link

I was not able to complete the project by myself, maybe I am too stupid, I don't know but here I found a video explaing the process to create and add songs to the playlist and it worked just fine, some of the methods used in the video are deprecated but pycharm will tell you which ones to use instead.

https://www.youtube.com/watch?v=jSOrEmKUd_c&t=2s&ab_channel=KalyanCodes

@Bowler-Dan
Copy link

This project had nothing to do with being a good Python developer or a good web site user. This was all about trying to figure out some very badly written documenation on how to interact with Spotify. The basic Python code for working with lists and dictionaries was fine. So was the basic code for scraping a website once you knew what keys to look for. But trying to figure out the Spotify parts of this assignment were beyond my abilities. I needed help from a lot of the people on this page to finally get my code to work.

Sharing my working code below:

from bs4 import BeautifulSoup # allows us to scrape web pages and find listed items
import requests # lets us retrieve the contents of a web page
import spotipy # this tool helps us to log in and authenticate with Spotify
from spotipy.oauth2 import SpotifyOAuth
import json # lets us format some outputs for easier reading and troubleshooting

--------------------- BILLBOARD ACCESS ----------------------------------------#

date = input("Find songs from the past few decades! Type the date in this format YYYY-MM-DD: ")

this will let us pick the specific date we want to scrape

URL = f"https://www.billboard.com/charts/hot-100/{date}" # now we have the full URL for the billboard site

response = requests.get(URL) # get the HTML from billboard for the chosen date
webpage = response.text # this holds the HTML of the found page

soup = BeautifulSoup(webpage, "html.parser") # build the "object" that we are scraping

song_name_list = soup.find_all(name="h3", id="title-of-a-story", class_="a-no-trucate")

note: there were many, many classes all listed together. Just need to pick 1 that was unique to the titles

song_names = [song.getText().strip() for song in song_name_list]

The song names have lots of leading tabs, spaces and newlines, etc. So strip them off. Left with the titles only.

print(song_names)

now do the same for the artists, not really needed but good practice with scraping

artist_list = soup.find_all(name="span", class_="a-no-trucate")
artists = [artist.getText().strip() for artist in artist_list]

print(artists)

--------------------- SPOTIFY ACCESS ----------------------------------------#

You can store this in environment variables for better security

SPOTIPY_CLIENT_ID = "e31e8565affc445bb9af136de33300de"
SPOTIPY_CLIENT_SECRET = "ab32e0a155f14a7399e2f1972b5cbdf0"
SPOTIPY_REDIRECT_URI = "http://localhost:8888/callback"

Make sure you put the above URI in your Spotify developer application redirect URI, they need to match

This next line is the one that uses the credentials to get you access to Spotify. You will know it works

when you try to retrieve the userid which validates that you got into Spotify properly.

sp = spotipy.Spotify(auth_manager=SpotifyOAuth(scope="user-library-read playlist-modify-private",
redirect_uri=SPOTIPY_REDIRECT_URI,
client_id=SPOTIPY_CLIENT_ID,
client_secret=SPOTIPY_CLIENT_SECRET,
show_dialog=False,
cache_path="token.txt"))

The above line, when executed will capture the authentication token and save it to your working directory

If the above authentication worked, you should now be able to retrieve your userid with the following query

results = sp.current_user() # this is the full dictionary. Now you can pick out your user_id

formatted_results = json.dumps(results, indent=4) # this creates a formatted view of the JSON
with open("results.json", mode="w") as file:
file.write(formatted_results) # useful if you want to see what the function returned, and in a nice format

user_id = results["id"] # query the dictionary for the "id"

print(user_id)

--------------------- Playlist URI Build ----------------------------------------#

build the URI list using the song names. You also need the year, so "split" the date on the - and keep [0]

year = date.split("-")[0]

playlist_uris = [] # want to capture all the individual uri's for the playlist in a single list
for song in song_names: # recall this from billboard, it has all the song titles
result = sp.search(q=f"track:{song} year:{year}", type="track") # this searches Spotify for the specific songs

try:
    uri = result["tracks"]["items"][0]["uri"]  # this builds the uri from the search results
    playlist_uris.append(uri)  # see if we can add the uri to our list.  Will fail if nothing was found
except IndexError:
    print(f"{song} doesn't exist in Spotify. Skipped.")

print(playlist_uris)

--------------------- Spotify Playlist Build ----------------------------------------#

playlist = sp.user_playlist_create(user=user_id, name=f"{date} Billboard 100", public=False)
print(playlist)

sp.playlist_add_items(playlist_id=playlist["id"], items=playlist_uris)

print(f"\nThe playlist has been created. A total of {len(playlist_uris)}/100 of the top 100 songs were found! ")

@Leon-dor
Copy link

Hello everyone

I anybody get this bug:

spotipy-create-playlist-error

Try change the public argument to False: spotipy-create-playlist-solution

Thank you , mate! I had the same problem. And just changed public to False and everything works!

@xdpiqbx
Copy link

xdpiqbx commented Jul 26, 2023

And here is my work. Try in OOP style

@mansibansal23
Copy link

I try this code. token.txt be made and my console said it's successful process. but my spotify dashboard isn't changed. how can i solve this problem???

you need to login to spotify.com and not the developer dashboard. you will see your playlist reflected there. Hope this helps.

@Tejuzz
Copy link

Tejuzz commented Aug 29, 2023

PLEASE I NEED HELP WHY THIS CODE DOESN'T WORK FOR ME?

import requests from bs4 import BeautifulSoup import spotipy from spotipy.oauth2 import SpotifyOAuth

CLIENT_ID = "9067a23713c34ea68d7276265816c720" CLIENT_SECRET = "334e6b70064546b092ad6b355ec171db"

url = "https://www.billboard.com/charts/hot-100/"

date = input("Which year do you want to travel to? Type the date in this Format YY-MM-DD.")

response = requests.get(f"{url}{date}") web = response.text

soup = BeautifulSoup(web, "html.parser")

musics = soup.find_all("h3", "a-no-trucate") music_titles = [] for music in musics: title = music.get_text().strip() music_titles.append(title)

sp = spotipy.Spotify( auth_manager=SpotifyOAuth( scope="playlist-modify-private", redirect_uri="http://example.com", client_id=CLIENT_ID, client_secret=CLIENT_SECRET, show_dialog=True, cache_path="token.txt" ) )

user_id = sp.current_user()["id"]

song_uris = [] year = date.split("-")[0] for song in music_titles: result = sp.search(q=f"track:{song} year:{year}", type="track") # print(result) # print("space\n\n") try: uri = result["tracks"]["items"][0] song_uris.append(uri) except IndexError: print(f"{song} doesn't exist in Spotify. Skipped.")

Creating a new private playlist in Spotify

playlist = sp.user_playlist_create(user=user_id, name=f"{date} Billboard 100", public=False) print(playlist)

Adding songs found into the new playlist

sp.playlist_add_items(playlist_id=playlist['id'], items=song_uris)

print(playlist['id'])

bro the for loop should be like this:
for song in songs_name[:100]:
song = song.get_text()
result = sp.search(q=f"track:{song} year:{year}", type="track")
print(result)
try:

@navalega0109
Copy link

If anyone needs code for this project then check below code:

import requests
from bs4 import BeautifulSoup
from dotenv import dotenv_values
import spotipy
from spotipy.oauth2 import SpotifyOAuth
from flask import Flask, request, url_for, session, redirect
import time

# Part 2: Spotify Authentication and New Playlist Creation based on Date.
app = Flask(__name__)

secrets = dotenv_values(".env")
app.config['SESSION_COOKIE_NAME'] = secrets['SESSION_COOKIE_NAME']
app.secret_key = secrets['SPOTIFY_APP_SECRET_KEY']
TOKEN_INFO = 'token_info'
SPOTIFY_CLIENT_ID = secrets["SPOTIFY_CLIENT_ID"]
SPOTIFY_CLIENT_SECRET = secrets["SPOTIFY_CLIENT_SECRET"]
BILLBOARD_URL = secrets['BILLBOARD_URL']
date = None

if date is None:
    date = input("Which year do you want to travel to? Type date in this format YYYY-MM-DD : \n\t")

time_travel_url = f"{BILLBOARD_URL}/{date}"

res = requests.get(url=time_travel_url)
res.raise_for_status()
time_travel = res.text

soup = BeautifulSoup(time_travel, "html.parser")

# data_results = soup.find_all(name="h3", id="title-of-a-story")
data_results = soup.select(selector="li ul li h3")
# print(data_results)

song_names = [(song.getText()).strip("\n\t") for song in data_results]
with open(f""
          f"songs_{date}.txt", mode="w") as file:
    for song in song_names:
        file.write(f"{song} \n")

# all the top 100 songs name on given date
# print(song_names)
# data_songs = soup.find_all(name="h3", class_="o-chart-results-list__item")
# print(data_songs)
# songs = [song.getText() for song in data_songs]
# print(songs)


@app.route('/')
def login():
    auth_url = create_spotify_oauth().get_authorize_url()
    return redirect(auth_url)


@app.route('/redirect')
def redirect_page():
    session.clear()
    # get the authCode which needs to be exchanged to get in return auth token
    code = request.args.get('code')
    token_info = create_spotify_oauth().get_access_token(code=code)
    session[TOKEN_INFO] = token_info
    return redirect(url_for('save_to_date', _external=True))


@app.route('/saveToDate')
def save_to_date():
    try:
        token_info = get_token()
    except:
        print('User not logged in.')
        return redirect('/')

    # Use this return to check if our OAuth is successful.
    # return 'OAuth Successful'
    sp = spotipy.Spotify(auth=token_info['access_token'])
    user_id = sp.current_user()['id']
    current_playlists = sp.current_user_playlists()['items']

    song_uris = []
    # Song search to add in playlist
    for song in song_names:
        result = sp.search(q=f"track:{song}", type="track", market="US")
        # print track: find the
        # print(result['tracks']['items'][0])
        try:
            uri = result["tracks"]["items"][0]["uri"]
            song_uris.append(uri)
        except IndexError:
            print(f"{song} doesn't exist in Spotify. Skipped.")
    # print(song_uris)

    # Check if the playlist already exist for given date:
    billboard_100_playlist_id = None
    for playlist in current_playlists:
        if playlist['name'] == f'{date} Billboard 100':
            billboard_100_playlist_id = playlist['id']

    if not billboard_100_playlist_id:
        new_playlist = sp.user_playlist_create(user_id, f'{date} Billboard 100', True)
        billboard_100_playlist_id = new_playlist['id']
        # return f"Billboard top 100 playlist id for date {date} is not found."

    # Now use this playlist to add all the song uris:
    sp.playlist_add_items(billboard_100_playlist_id, song_uris, None)

    print(f"All the songs from timeline {date} added to the playlist '{date} Billboard 100' in spotify.")
    return f"All the songs from timeline {date} added to the playlist '{date} Billboard 100' in spotify."


def get_token():
    token_info = session.get(TOKEN_INFO, None)
    if not token_info:
        redirect(url_for('login', _external=False))

    now = int(time.time())

    is_expired = token_info['expires_at'] - now < 60
    if is_expired:
        spotify_oauth = create_spotify_oauth()
        token_info = spotify_oauth.refresh_access_token(token_info['refresh_token'])

    #  return token info if not expired & if expired then refreshed before sending the given token info.
    return token_info


def create_spotify_oauth():
    return SpotifyOAuth(
        client_id=SPOTIFY_CLIENT_ID,
        client_secret=SPOTIFY_CLIENT_SECRET,
        redirect_uri=url_for('redirect_page', _external=True),
        scope='user-library-read playlist-modify-public playlist-modify-private'
    )


app.run(debug=False)

@AboodALhassan
Copy link

Is anyone else finding this project challenging? I'm having trouble grasping the concepts involved. Any help or clarification would be greatly appreciated

@huzaifasaeed123
Copy link

#Here Is The Complete Project In Simplest Way
from bs4 import BeautifulSoup
import requests
import spotipy
from spotipy.oauth2 import SpotifyOAuth

#date=input("Please Enter date in YYYY-MM- Format")

response=requests.get("https://www.billboard.com/charts/hot-100/2023-08-05/")

soup=BeautifulSoup(response.text,"html.parser")
all_titles= soup.select(selector=".o-chart-results-list__item #title-of-a-story")

client_id="05d82555b91a4325b5e1e5563e4b0faa"
client_secret="5c3d0bc4a04d47b38935ee977742c379"

sp = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id=client_id,
client_secret=client_secret,
redirect_uri='https://example.com',
scope='playlist-modify-private'))

user_profile = sp.current_user()

Extract user ID from the profile

user_id = user_profile['id']
print(user_id)

playlist_name = 'Top 100 Songs'
playlist_description = 'Your Playlist Description'
playlist = sp.user_playlist_create(user=user_id, name=playlist_name, public=False, description=playlist_description)

for titles in all_titles:
print(titles.getText().strip())
# Search for the song
song_title = titles.getText().strip()
search_results = sp.search(q=song_title, type='track')

# Extract the track URI
track_uri = search_results['tracks']['items'][0]['uri']  # Assuming the first search result is the desired song
playlist_id = playlist['id']
sp.playlist_add_items(playlist_id=playlist_id, items=[track_uri])

@Moogeee
Copy link

Moogeee commented Feb 24, 2024

Here is my code, this one is definitely challenging because the documentation is not explained very well (at least for me) so it requires trials and lots of errors. Worked it all out in the end. Hope this helps

from bs4 import BeautifulSoup
import requests
import spotipy
from spotipy.oauth2 import SpotifyOAuth

time_input = input("Enter the time you want to jump to in this format YYYY-MM-DD: ")

URL = "https://www.billboard.com/charts/hot-100/"+time_input

TEMP1 = "c-title a-no-trucate a-font-primary-bold-s u-letter-spacing-0021 lrv-u-font-size-18@tablet lrv-u-font-size-16 u-line-height-125 u-line-height-normal@mobile-max a-truncate-ellipsis u-max-width-330 u-max-width-230@tablet-only"
TEMP2 = "c-title a-no-trucate a-font-primary-bold-s u-letter-spacing-0021 u-font-size-23@tablet lrv-u-font-size-16 u-line-height-125 u-line-height-normal@mobile-max a-truncate-ellipsis u-max-width-245 u-max-width-230@tablet-only u-letter-spacing-0028@tablet"
TEMP3 = "c-label a-no-trucate a-font-primary-s lrv-u-font-size-14@mobile-max u-line-height-normal@mobile-max u-letter-spacing-0021 lrv-u-display-block a-truncate-ellipsis-2line u-max-width-330 u-max-width-230@tablet-only"
TEMP4 = "c-label a-no-trucate a-font-primary-s lrv-u-font-size-14@mobile-max u-line-height-normal@mobile-max u-letter-spacing-0021 lrv-u-display-block a-truncate-ellipsis-2line u-max-width-330 u-max-width-230@tablet-only u-font-size-20@tablet"
# These are classes scrapped from the billboard website

CLIENT_ID = (your client_id here)
CLIENT_SECRET = (your client_secret here)
REDIRECT_URI = "http://example.com"
SPOTIFY_URL = "https://api.spotify.com/v1"
USERNAME = (your username here)

# ------------- SCRAPE BILLBOARD AND GET 2 LISTS OF SONGS AND CORRESPONDING ARTISTS -------------
response = requests.get(URL)
data = response.text
soup = BeautifulSoup(data, "html.parser")
songs = [item.getText().strip() for item in soup.find_all(name="h3", id="title-of-a-story", class_=TEMP1)]
first_song = soup.find(name="h3", id="title-of-a-story", class_=TEMP2).getText().strip()
songs.insert(0, first_song)
artists = [item.getText().strip() for item in soup.find_all(name="span", class_=TEMP3)]
first_artist = soup.find(name="span", class_=TEMP4).getText().strip()
artists.insert(0, first_artist)

# ------------- ACCESS SPOTIFY AND MAKE AUTH -------------
scope = "playlist-modify-private"
sp = spotipy.Spotify(
    auth_manager=SpotifyOAuth(
        client_id=CLIENT_ID,
        client_secret=CLIENT_SECRET,
        redirect_uri=REDIRECT_URI,
        cache_path="token.txt",
        scope=scope,
        show_dialog=True,
        username=USERNAME,
    )
)
user_id = sp.current_user()["id"]

# ------------- CREATE PLAYLIST -------------
playlist = sp.user_playlist_create(
    user=user_id,
    name="Top 100 Billboard songs",
    public=False,
    collaborative=False,
    description=f"Top 100 Billboard songs on {time_input}",
)
playlist_id = playlist["id"]

# ------------- SEARCH FOR SONGS -------------


class NoSongFound(Exception):
    pass


uris = []
for index in range(0, len(songs)):
    try:
        search = sp.search(
            q=f"track:{songs[index]} artist:{artists[index]}",
            limit=1,
            offset=0,
            market=None,
            type="track",
        )
        if search["tracks"]["total"] == 0:
            raise NoSongFound
    except NoSongFound:
        pass
    else:
        uri = str(search["tracks"]["items"][0]["id"])
        uris.append(uri)

# ------------- ADD SONGS -------------
sp.playlist_add_items(playlist_id=playlist_id, items=uris)

print(sp.playlist_items(
    playlist_id=playlist_id,
    additional_types="track",
))

@bluebanana18
Copy link

bluebanana18 commented Feb 28, 2024 •
edited

Hey guys, I keep getting an insufficient client scope error. What am I still missing in the code?

Also, when I run Angela's code it creates the playlist but can't add any songs. I am a bit lost there lol.

@drakewilcox
Copy link

Hey guys, I keep getting an insufficient client scope error. What am I still missing in the code?

Also, when I run Angela's code it creates the playlist but can't add any songs. I am a bit lost there lol.

@bluebanana18 I was having the exact same issue.

The solution for me was updating the scope in the Spotify Authentication section to include both private and public playlist-modify scopes. (update to line 3 of Angela's code)

Yes, this seems counter intuitive since on line 15 of the example, the created playlist is set as private and has the playlist-modify-private scope set, so you think it would work. But looks like the Spotify API wants both scopes set.

Also you may need to delete your token.txt file before running in order for the token to reset properly.

Example:

sp = spotipy.Spotify(
  auth_manager=SpotifyOAuth(
    scope="playlist-modify-private playlist-modify-public",
    redirect_uri="http://example.com",
    client_id=CLIENT_ID,
    client_secret=CLIENT_SECRET,
    show_dialog=True,
    cache_path="token.txt"
  )
)

Hope this helps

@bluebanana18
Copy link

@drakewilcox
Mind blown. It was really that simple, huh. Thanks a lot!

@grizzleswens
Copy link

Here is my code, it is working perfectly, I had to have a little bit of hand holding from chat gpt

import requests
from bs4 import BeautifulSoup
import spotipy
from spotipy.oauth2 import SpotifyOAuth

Set your Spotify app credentials

You will need to make a spotify web app to get this data, go to spotify dev tools

SPOTIPY_CLIENT_ID = 'YOUR CLIENT ID'
SPOTIPY_CLIENT_SECRET = 'YOUR CLIENT SECRET'
SPOTIPY_REDIRECT_URI = 'YOUR REDIRECT URI'
SCOPE = 'playlist-modify-public user-read-private'

date = input("What date would you like to travel back in time to? YYYY-MM-DD")
endpoint = f"https://www.billboard.com/charts/hot-100/{date}/"

Scrape web for top 100 songs

response = requests.get(endpoint)
html_data = response.text

soup = BeautifulSoup(html_data, "html.parser")

songs = soup.select("li ul li h3")
songs_titles = [song.getText().strip() for song in songs]

track_uris = []

Authenticate with Spotify

sp = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id=SPOTIPY_CLIENT_ID,
client_secret=SPOTIPY_CLIENT_SECRET,
redirect_uri=SPOTIPY_REDIRECT_URI,
scope=SCOPE))

Find track URIs

for song in songs_titles:
try:
results = sp.search(q=f"track:{song}", type="track")
track_uri = results['tracks']['items'][0]['uri']
track_uris.append(track_uri)
except IndexError:
continue

# Authenticate with Spotify

sp = spotipy.Spotify(auth_manager=SpotifyOAuth(client_id=SPOTIPY_CLIENT_ID,
client_secret=SPOTIPY_CLIENT_SECRET,
redirect_uri=SPOTIPY_REDIRECT_URI,
scope=SCOPE))

Get current user's profile data

user_id = sp.current_user()['id']

Create a new playlist for the current user

playlist_name = f"Billboard top songs on {date}"
playlist_description = "Created with Python"
playlist = sp.user_playlist_create(user=user_id, name=playlist_name, description=playlist_description)

Get the playlist ID

playlist_id = playlist['id']

Add tracks to the playlist

sp.playlist_add_items(playlist_id=playlist_id, items=track_uris)

print(f"Playlist created and tracks added. Playlist ID: {playlist_id}")

@rickycc
Copy link

rickycc commented Apr 18, 2024 •
edited 

# Use this return to check if our OAuth is successful.
# return 'OAuth Successful'
sp = spotipy.Spotify(auth=token_info['access_token'])
user_id = sp.current_user()['id']
current_playlists = sp.current_user_playlists()['items']

song_uris = []
# Song search to add in playlist
for song in song_names:
    result = sp.search(q=f"track:{song}", type="track", market="US")
    # print track: find the
    # print(result['tracks']['items'][0])
    try:
        uri = result["tracks"]["items"][0]["uri"]
        song_uris.append(uri)
    except IndexError:
        print(f"{song} doesn't exist in Spotify. Skipped.")
# print(song_uris)

# Check if the playlist already exist for given date:
billboard_100_playlist_id = None
for playlist in current_playlists:
    if playlist['name'] == f'{date} Billboard 100':
        billboard_100_playlist_id = playlist['id']

if not billboard_100_playlist_id:
    new_playlist = sp.user_playlist_create(user_id, f'{date} Billboard 100', True)
    billboard_100_playlist_id = new_playlist['id']
    # return f"Billboard top 100 playlist id for date {date} is not found."

Awesome work. However, I am having trouble carrying out the playlist check. I cannot retrieve any information running the line below immediately after the authentication steps. I can print out user_id = sp.current_user()["id"] without any problem.
Can you shed some light on what should i do?

current_playlists = sp.current_user_playlists()['items']

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment