Skip to content

Instantly share code, notes, and snippets.

@farbod-s
Created January 22, 2017 23:56
Show Gist options
  • Save farbod-s/a3ae9e0ecc1b14f953004b24cd88b1c5 to your computer and use it in GitHub Desktop.
Save farbod-s/a3ae9e0ecc1b14f953004b24cd88b1c5 to your computer and use it in GitHub Desktop.
#!/bin/python
import requests
from bs4 import BeautifulSoup
cookies = dict(CHESSCOM_REMEMBERME='YOUR_COOKIE')
s = requests.session()
for i in range(1, 7000):
r = s.get('https://www.chess.com/lessons/view/' + str(i), cookies=cookies)
soup = BeautifulSoup(r.text, 'html.parser')
result = soup.body.main.find('div', {'id':'content'}).div['ng-init']
start = result.find('{')
end = result.rfind('}') + 1
with open('lesson_' + str(i) + '.txt', 'a') as myfile:
myfile.write(str(result[start:end]))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment