Skip to content

Instantly share code, notes, and snippets.

@aravindpai

aravindpai/10_2.py

Created Mar 12, 2020
Embed
What would you like to do?
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
#Tokenize the sentences
tokenizer = Tokenizer()
#preparing vocabulary
tokenizer.fit_on_texts(list(x_tr))
#converting text into integer sequences
x_tr_seq = tokenizer.texts_to_sequences(x_tr)
x_val_seq = tokenizer.texts_to_sequences(x_val)
#padding to prepare sequences of same length
x_tr_seq = pad_sequences(x_tr_seq, maxlen=100)
x_val_seq = pad_sequences(x_val_seq, maxlen=100)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.