Skip to content

Instantly share code, notes, and snippets.

@imankulov
Created January 6, 2023 21:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save imankulov/27e0a5f7fc1bb0435d286f10ac9c79ec to your computer and use it in GitHub Desktop.
Save imankulov/27e0a5f7fc1bb0435d286f10ac9c79ec to your computer and use it in GitHub Desktop.
A Python request.Session() instance with reasonable defaults for retries.
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
USER_AGENT = "My project (https://example.com)"
def get_default_session() -> requests.Session:
"""Return a request.Session() instance with reasonable defaults for retries.
Instead of using requests.get() or requests.post(), use this function as follows:
session = get_default_session()
response = session.get(url)
"""
# A helpful guideline for configuring requests properly:
# https://findwork.dev/blog/advanced-usage-python-requests-timeouts-retries-hooks/
session = requests.Session()
session.headers["User-Agent"] = USER_AGENT
retries = Retry(
total=4,
backoff_factor=1.0,
status_forcelist=[429, 500, 502, 503, 504],
)
session.mount("http://", HTTPAdapter(max_retries=retries))
session.mount("https://", HTTPAdapter(max_retries=retries))
return session
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment