Skip to content

Instantly share code, notes, and snippets.

@Nilpo
Forked from triple-j/download.py
Created October 26, 2019 03:44
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save Nilpo/ad303807af5f44a19fd0ca72862e3fa2 to your computer and use it in GitHub Desktop.
Save Nilpo/ad303807af5f44a19fd0ca72862e3fa2 to your computer and use it in GitHub Desktop.
Download file through HTTP using requests.py and tqdm
import os.path
from urllib.request import urlopen
import requests
from tqdm import tqdm
def download_from_url(url, dst):
"""
@param: url to download file
@param: dst place to put the file
@author: wy193777
"""
file_size = int(urlopen(url).info().get('Content-Length', -1))
if os.path.exists(dst):
first_byte = os.path.getsize(dst)
else:
first_byte = 0
if first_byte >= file_size:
return file_size
header = {"Range": "bytes=%s-%s" % (first_byte, file_size)}
pbar = tqdm(
total=file_size,
initial=first_byte,
unit='B',
unit_scale=True,
desc=url.split('/')[-1],
ncols=80,
bar_format="{l_bar}{bar}| {n_fmt}/{total_fmt} [{remaining}, {rate_fmt}{postfix}]")
req = requests.get(url, headers=header, stream=True)
with(open(dst, 'ab')) as f:
for chunk in req.iter_content(chunk_size=1024):
if chunk:
f.write(chunk)
pbar.update(1024)
pbar.close()
return file_size
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment