Skip to content

Instantly share code, notes, and snippets.

@Addono
Last active December 16, 2019 21:37
Show Gist options
  • Save Addono/17b23f8746d1293ecc5c903cabb53947 to your computer and use it in GitHub Desktop.
Save Addono/17b23f8746d1293ecc5c903cabb53947 to your computer and use it in GitHub Desktop.
File downloader Python, both sequential including status updating with tqdm or
from tqdm import tqdm
import urllib.request
urls = ["https://www.photobookselysee.ch/iiif/77-071_OOR_Een-Staat-In-Wording___%s/full/full/0/default.png" % i for i in range(1, 70)]
print("Downloading:")
print("\n".join(urls))
for index, url in tqdm(enumerate(urls), unit="pages", total=len(urls)):
file_name = "OOR_Een-Staat-In-Wording_Page_%s.png" % index
urllib.request.urlretrieve(url, file_name)
import urllib.request
import multiprocessing
urls = ["https://www.photobookselysee.ch/iiif/77-071_OOR_Een-Staat-In-Wording___%s/full/full/0/default.png" % i for i in range(1, 70)]
file_names = ["OOR_Een-Staat-In-Wording_Page_%s.png" % index for index, _ in enumerate(urls)]
print("Downloading:")
print("\n".join(urls))
with multiprocessing.Pool(processes=multiprocessing.cpu_count()) as pool:
pool.starmap(urllib.request.urlretrieve, zip(urls, file_names))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment