Skip to content

Instantly share code, notes, and snippets.

Last active September 10, 2020 03:47
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Sample program to stream large data from a REST or HTTP endpoint using Python. For more details refer to
import requests
session = requests.Session()
authentication = {"USER":"", "PASSWORD":""}
payload = {"query":"some query"}
local_file = "data.json"
# This is a dummy URL. You can replace this with the actual URL
URL = ""
# This is a POST request
with, stream=True, data=payload, auth=(authentication["USER"], authentication["PASSWORD"]), verify=False) as r:
with open(local_file, 'wb') as f:
for chunk in r.iter_content(chunk_size=128):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment