Skip to content

Instantly share code, notes, and snippets.

@moubaba

moubaba/create_vocab.py Secret

Created January 11, 2019 15:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save moubaba/312ecf51b9dd5eb4f5db51da256c4ba7 to your computer and use it in GitHub Desktop.
Save moubaba/312ecf51b9dd5eb4f5db51da256c4ba7 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
create_vocab.py
text_stems_sid,text_lems_sid = process_data(" ".join(homer_data.split(".")[10:13]))
vocab = list(set(text_stems_sid))
print(" ".join(homer_data.split(".")[10:13]))
Raw
get_book.py
homer_response = requests.get("https://www.gutenberg.org/files/52927/52927-0.txt")
homer_data = homer_response.text
homer_data = homer_data.split("***")[2]
Raw
onehot.py
stems,lems = process_data(homer_data.split(".")[12])
print(homer_data.split(".")[12])
onehot_encoded = list()
for word in stems:
letter = [0 for _ in range(len(set(text_stems_sid)))]
print(word,vocab.index(word))
letter[vocab.index(word)] = 1
onehot_encoded.append(letter)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment