Skip to content

Instantly share code, notes, and snippets.

@mohdsanadzakirizvi

mohdsanadzakirizvi/lm_val.py

Created August 7, 2019 02:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mohdsanadzakirizvi/1197d5561a5b7b8effda9f4fd39aab60 to your computer and use it in GitHub Desktop.
Save mohdsanadzakirizvi/1197d5561a5b7b8effda9f4fd39aab60 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
lm_val.py
from sklearn.model_selection import train_test_split
# vocabulary size
vocab = len(mapping)
sequences = np.array(sequences)
# create X and y
X, y = sequences[:,:-1], sequences[:,-1]
# one hot encode y
y = to_categorical(y, num_classes=vocab)
# create train and validation sets
X_tr, X_val, y_tr, y_val = train_test_split(X, y, test_size=0.1, random_state=42)
print('Train shape:', X_tr.shape, 'Val shape:', X_val.shape)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment