Skip to content

Instantly share code, notes, and snippets.

@OldMetalmind

OldMetalmind/grab_json

Created February 13, 2020 16:21
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save OldMetalmind/ac7767a4f526508d273c91b338090370 to your computer and use it in GitHub Desktop.
Save OldMetalmind/ac7767a4f526508d273c91b338090370 to your computer and use it in GitHub Desktop.
Download ZIP
Grab JSON from source
Raw
grab_json
# Core
import time
import json
# Third-party
import requests
from bs4 import BeautifulSoup
SOURCE = requests.get("<ENDPOINT>")
soup = BeautifulSoup(SOURCE.text, "html.parser")
all_scripts = soup.find_all('script')
script_with_data = all_scripts[22]
raw_data = script_with_data.get_text()[2638:22084]
data = json.loads(raw_data)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment