Skip to content

Instantly share code, notes, and snippets.

@HebeHH
Created August 19, 2020 15:22
Show Gist options
  • Save HebeHH/14382d210c89781c582af6a5a2dadb71 to your computer and use it in GitHub Desktop.
Save HebeHH/14382d210c89781c582af6a5a2dadb71 to your computer and use it in GitHub Desktop.
Extracting Headers from cURL request as a dictionary for the python Requests library
# The function:
get_headers = lambda curl: {k:v for k, v in
[[i.strip().strip(r"'-H \$'") for i in
[x.split(': ')[0], ': '.join(x.split(': ')[1:])]]
for x in curl.split('\n')
if '-H' in x]}
# You get a curl request in this kind of format from eg: Chrome Developer Tools
# Important: all arguments must be on a new line and it must be a raw string (the r before the triple quotes)
example_curl = r"""curl 'https://urllib3.readthedocs.io/en/latest/_static/alabaster.css' \
-H 'authority: urllib3.readthedocs.io' \
-H 'dnt: 1' \
-H 'accept: text/css,*/*;q=0.1' \
-H 'sec-fetch-site: same-origin' \
-H 'referer: https://urllib3.readthedocs.io/en/latest/reference/index.html' \
--compressed"""
# Using the function:
get_headers(example_curl)
# Output from this request:
"""
{'authority': 'sg.element14.com',
'upgrade-insecure-requests': '1',
'dnt': '1',
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36',
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
'sec-fetch-site': 'same-origin',
'sec-fetch-mode': 'navigate',
'sec-fetch-user': '?1',
'sec-fetch-dest': 'document',
'referer': 'https://sg.element14.com/pro-signal/abm-715-rc/electret-microphone-omni-leads/dp/2066501?st=electret%20microphone',
'accept-language': 'en-US,en;q=0.9'}
"""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment