Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Requests and Beautiful Soup example, following the form of
from BeautifulSoup import BeautifulSoup
import requests
import urlparse
URL = ''
s = requests.Session()
def fetch(url, data=None):
if data is None:
return s.get(url).content
return, data=data).content
soup = BeautifulSoup(fetch(URL))
form = soup.find('form')
fields = form.findAll('input')
formdata = dict( (field.get('name'), field.get('value')) for field in fields)
formdata['username'] = u'username'
formdata['password'] = u'password'
print formdata
posturl = urlparse.urljoin(URL, form['action'])
print posturl
r =, data=formdata)
print r.text
print s.get(URL).text
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment