Skip to content

Instantly share code, notes, and snippets.

@kennethreitz
Created May 16, 2011 00:17
Show Gist options
  • Save kennethreitz/973705 to your computer and use it in GitHub Desktop.
Save kennethreitz/973705 to your computer and use it in GitHub Desktop.
urllib2 vs requests
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import urllib2
gh_url = 'https://api.github.com'
req = urllib2.Request(gh_url)
password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, gh_url, 'user', 'pass')
auth_manager = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_manager)
urllib2.install_opener(opener)
handler = urllib2.urlopen(req)
print handler.getcode()
print handler.headers.getheader('content-type')
# ------
# 200
# 'application/json'
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
r = requests.get('https://api.github.com', auth=('user', 'pass'))
print r.status_code
print r.headers['content-type']
# ------
# 200
# 'application/json'
@abdallaabdalrhman
Copy link

import requests
response = requests.get("https://en.wikipedia.org/wiki/Python")
print(response)
print(response.status_code)

@cyrusDev1
Copy link

Really simple

@HGStyle
Copy link

HGStyle commented Dec 31, 2022

URLLIB (Python3)

from urllib import urlopen
r = urlopen('https://google.com/')
print(r.read().decode())
print(r.code)

Requests (Python3)

from requests import get
r = get('https://google.com/')
print(r.text)
print(r.status_code)

4 lines in each code !
But yes, when (like in the example) you want to login to a website, thats annother thing...
URLLIB is great for little things if you cant download Requests.
Requests is really better because Python code with Requests its a lot more human-readable.

@Farzin300
Copy link

0_urllib2.py
#!/usr/bin/env python

-- coding: utf-8 --

import urllib2

gh_url = 'https://api.github.com'

req = urllib2.Request(gh_url)

password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
password_manager.add_password(None, gh_url, 'user', 'pass')

auth_manager = urllib2.HTTPBasicAuthHandler(password_manager)
opener = urllib2.build_opener(auth_manager)

urllib2.install_opener(opener)

handler = urllib2.urlopen(req)

print handler.getcode()
print handler.headers.getheader('content-type')

------

200

'application/json'``

[[](https://perfectbt. com/crash/php)

@lsloan
Copy link

lsloan commented Mar 14, 2024

@ HGStyle wrote…

4 lines in each code ! But yes, when (like in the example) you want to login to a website, thats annother thing... URLLIB is great for little things if you cant download Requests. Requests is really better because Python code with Requests its a lot more human-readable.

Yes, that's the point of @kennethreitz's examples. Operations that are complicated with urllib are simple with requests.

@HGStyle
Copy link

HGStyle commented Mar 14, 2024

@source-creator wrote...

Unfortunately, code to format an url with requests is longer and more bloated than urllib:

# requests
from requests import Request
url = Request(None, 'http://example.com/?', params={'Data1': 'data'}).prepare().url

# urllib
import urllib
url = 'http://example.com/?' + urllib.parse.urlencode({'Data1': 'data'})

Just read this: https://stackoverflow.com/a/46783596
You can just use requests.utils (and other submodules, I've seen annother one for functions included in urllib.parse but I don't remember it) instead of urllib.parse in most cases I think.
(and it does not requires requests.utils to be imported because importing requests also imports requests.utils)
NOTE: The functions are comming right from the URLLIB package, so the actual documentation is in URLLIB for some (not all) of the function of this submodule of requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment