Skip to content

Instantly share code, notes, and snippets.

@mohdsanadzakirizvi

mohdsanadzakirizvi/lm_seq.py

Last active August 7, 2019 02:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mohdsanadzakirizvi/33488ad0b88fcff255494c79ebce3ee5 to your computer and use it in GitHub Desktop.
Save mohdsanadzakirizvi/33488ad0b88fcff255494c79ebce3ee5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
lm_seq.py
def create_seq(text):
length = 30
sequences = list()
for i in range(length, len(text)):
# select sequence of tokens
seq = text[i-length:i+1]
# store
sequences.append(seq)
print('Total Sequences: %d' % len(sequences))
return sequences
# create sequences
sequences = create_seq(data_new)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment