Skip to content

Instantly share code, notes, and snippets.

@dimitryzub
Last active July 4, 2022 06:06
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dimitryzub/fbd76be90eadc1660477265e02c600c7 to your computer and use it in GitHub Desktop.
Save dimitryzub/fbd76be90eadc1660477265e02c600c7 to your computer and use it in GitHub Desktop.
Web Scraping YouTube Popular Videos Results Data with Python and SerpApi
# video tutorial: https://www.youtube.com/watch?v=PTLpbBiz_sc
from serpapi import GoogleSearch
from urllib.parse import (parse_qsl, urlsplit)
import json
params = {
"api_key": "your api key", # your serpapi api key
"engine": "youtube", # serpapi parsing engine
"search_query": "blender foundation", # search query
"gl": "US", # country of the search
"hl": "en", # language of the search
# "sp": "EgJ4AQ%253D%253D" # youtube filter
}
search = GoogleSearch(params)
video_results = []
videos_amount = []
while True:
results = search.get_dict()
if "playlist_results" in results:
pass # extract only playlist results or whatever you want from the output
for result in results.get("video_results", []):
print(result["title"])
video_results.append({
"title": result.get("title"),
"link": result.get("link"),
"published_date": result.get("published_date"),
"views": result.get("views"),
"length": result.get("length"),
"channel": result.get("channel")
})
videos_amount.append(result["title"])
if "next" in results.get("serpapi_pagination", {}):
search.params_dict.update(dict(parse_qsl(urlsplit(results.get("serpapi_pagination", {}).get("next")).query)))
else:
break
print(len(videos_amount))
@dimitryzub
Copy link
Author

dimitryzub commented Jul 4, 2022

1. Apply desired YouTube filter on YouTube itself:

2. Copy sp= query parameter value from URL:

CAM%253D = filter by views.

3. Pass this value to SerpApi:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment