Skip to content

Instantly share code, notes, and snippets.

@Ricky-Wilson
Created March 14, 2014 08:19
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Ricky-Wilson/9543932 to your computer and use it in GitHub Desktop.
Save Ricky-Wilson/9543932 to your computer and use it in GitHub Desktop.
Scrape word definitions from dictionary.com with python
#!/usr/bin/python
from bs4 import BeautifulSoup as bs
import re
from requests import get
class dictionary:
def remove_non_ascii(self,text):
return re.sub(r'[^\x00-\x7F]+','', text)
def get_soup(self,url):
raw = self.remove_non_ascii(get(url).content)
soup = bs(raw)
return soup.select("#MainTxt")[0].select('.ds-single')[0].text.strip()
def lookup(self,word):
base_url = "http://www.thefreedictionary.com/"
query_url = base_url + word
return self.get_soup(query_url)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment