Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save fanurs/abd53315c7a2180d9043806f5de92211 to your computer and use it in GitHub Desktop.
Save fanurs/abd53315c7a2180d9043806f5de92211 to your computer and use it in GitHub Desktop.
Demo: Download images from URLs using `ThreadPoolExecutor`
import concurrent.futures
import urllib
import numpy as np
links = {
'IU': 'https://upload.wikimedia.org/wikipedia/commons/thumb/3/33/IU_at_%22Midnight_Runners%22_VIP_premiere%2C_7_August_2017_03.jpg/800px-IU_at_%22Midnight_Runners%22_VIP_premiere%2C_7_August_2017_03.jpg',
'Stephen Curry': 'https://www.si.com/.image/ar_233:100%2Cc_fill%2Ccs_srgb%2Cg_faces:center%2Cq_auto:good%2Cw_1920/MTg4MDIzNjkxMDA5ODYxNDE4/stephen-curry-smiles-iso.webp',
'Syed Saddiq': 'https://media2.malaymail.com/uploads/articles/2018/2018-08/20180802YM21.jpg',
'Terence Tao': 'https://s3.amazonaws.com/cms.ipressroom.com/173/files/20147/53e288debd26f511d800600e_Terence_Tao/Terence_Tao_c4560e18-0bf8-4bab-8daa-7e74d3acf907-prv.jpg',
}
def download(url):
with urllib.request.urlopen(url) as req:
return np.asarray(bytearray(req.read()), dtype=np.int8))
with concurrent.futures.ThreadPoolExecutor() as executor:
results = {
name: executor.submit(download, url)
for name, url in links.items()
}
results = {name: fut.result() for name, fut in results.items()}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment