Skip to content

Instantly share code, notes, and snippets.

@amankharwal
Created November 1, 2020 06:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save amankharwal/80b82076a546b58053a29dbbe3b4e3ec to your computer and use it in GitHub Desktop.
Save amankharwal/80b82076a546b58053a29dbbe3b4e3ec to your computer and use it in GitHub Desktop.
vocab_size = 1000
embedding_dim = 16
max_len = 20
oov_token = "<OOV>"
tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_token)
tokenizer.fit_on_texts(training_sentences)
word_index = tokenizer.word_index
sequences = tokenizer.texts_to_sequences(training_sentences)
padded_sequences = pad_sequences(sequences, truncating='post', maxlen=max_len)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment