Skip to content

Instantly share code, notes, and snippets.

@hltbra

hltbra/our-queueing-solution-blog-post-ex1.py

Created May 16, 2019 11:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hltbra/ef27fb4d0c392241ea6b48c522164518 to your computer and use it in GitHub Desktop.
Save hltbra/ef27fb4d0c392241ea6b48c522164518 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
our-queueing-solution-blog-post-ex1.py
@starting_task
def scrape_listing():
for link in get_links(LISTING_URL):
schedule('scrape_link', link, seconds=0)
@subtask
def scrape_link(url):
resp = requests.get(url)
info = get_link_info(resp.content)
save('link', info)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment