Skip to content

Instantly share code, notes, and snippets.

@arshjat

arshjat/1.7.py

Created January 1, 2019 10:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save arshjat/2f6e6ea26cb3bf1c0581e8a164d9b5cb to your computer and use it in GitHub Desktop.
Save arshjat/2f6e6ea26cb3bf1c0581e8a164d9b5cb to your computer and use it in GitHub Desktop.
Download ZIP
Raw
1.7.py
tk = Tokenizer(lower = True, filters='')
tk.fit_on_texts(full_text)
train_tokenized = tk.texts_to_sequences(train['tweet'])
test_tokenized = tk.texts_to_sequences(test['tweet'])
max_len = 50
X_train = pad_sequences(train_tokenized, maxlen = max_len)
X_test = pad_sequences(test_tokenized, maxlen = max_len)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment