Skip to content

Instantly share code, notes, and snippets.

@chbrandt
Forked from wy193777/download.py
Last active September 9, 2019 11:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save chbrandt/0be76f471c3e947a8e8e1f9332fe1189 to your computer and use it in GitHub Desktop.
Save chbrandt/0be76f471c3e947a8e8e1f9332fe1189 to your computer and use it in GitHub Desktop.
Download file through HTTP using requests.py and tqdm
import os
import requests
from tqdm import tqdm
def download_from_url(url, dst):
"""
@param: download file URL
@param: output file name
"""
file_size = int(requests.head(url).headers["Content-Length"])
if os.path.exists(dst):
first_byte = os.path.getsize(dst)
else:
first_byte = 0
if first_byte >= file_size:
return file_size
header = {"Range": "bytes=%s-%s" % (first_byte, file_size)}
pbar = tqdm(
total=file_size, initial=first_byte,
unit='B', unit_scale=True, desc=url.split('/')[-1])
req = requests.get(url, headers=header, stream=True)
with(open(dst, 'ab')) as f:
for chunk in req.iter_content(chunk_size=1024):
if chunk:
f.write(chunk)
pbar.update(1024)
pbar.close()
return file_size
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment