Skip to content

Instantly share code, notes, and snippets.

@d0uji
Created March 7, 2018 14:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save d0uji/60d974269f65f7c2f03dd0808beb61c7 to your computer and use it in GitHub Desktop.
Save d0uji/60d974269f65f7c2f03dd0808beb61c7 to your computer and use it in GitHub Desktop.
from multiprocessing.pool import ThreadPool
from six.moves import urllib
def fetch_url(url):
html = urllib.request.urlopen(url)
if 'yadi.sk' in html.read().decode('utf-8'):
return url
urls = ["https://goodgame.ru/user/{0!s}/".format(i) for i in range(1, 9999999)]
results = ThreadPool(100).imap_unordered(fetch_url, urls)
for url in results:
if url is not None:
print(url)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment