Skip to content

Instantly share code, notes, and snippets.

@zredlined

zredlined/create_training_set.py

Last active January 29, 2020 22:00
Show Gist options
  • Save zredlined/a6d39721e7054835d8f24a6ce3f6a3c8 to your computer and use it in GitHub Desktop.
Save zredlined/a6d39721e7054835d8f24a6ce3f6a3c8 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
create_training_set.py
def split_input_target(chunk):
input_text = chunk[:-1]
target_text = chunk[1:]
return input_text, target_text
BATCH_SIZE = 64
BUFFER_SIZE = 10000
# Create TensorFlow training examples and targets
char_dataset = tf.data.Dataset.from_tensor_slices(text_as_int)
sequences = char_dataset.batch(seq_length+1, drop_remainder=True)
dataset = sequences.map(split_input_target)
dataset = dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE, drop_remainder=True)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment