Skip to content

Instantly share code, notes, and snippets.

@kognate
Last active January 29, 2017 03:34
Show Gist options
  • Save kognate/b8d3782bf26e7393adc1d976ffff278b to your computer and use it in GitHub Desktop.
Save kognate/b8d3782bf26e7393adc1d976ffff278b to your computer and use it in GitHub Desktop.
from lxml.cssselect import CSSSelector
import lxml.html
import requests
import re
id_finder = re.compile('\/id([0-9]+)')
def extract_id(linkstr):
mid = id_finder.search(linkstr)
if mid:
return mid.group(1)
else:
return "not found"
def main(params):
r = requests.get('http://www.apple.com/itunes/charts/free-apps/')
tree = lxml.html.fromstring(r.text)
sel = CSSSelector('div.section-content ul li h3 a')
results = sel(tree)
return {'result' : [{"id": extract_id(x.get('href')), "title": x.text}
for x in
results]}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment