Skip to content

Instantly share code, notes, and snippets.



Created May 22, 2020
What would you like to do?
#applying BPE to OOV
oov ='lowest'
#tokenize OOV into characters
oov = " ".join(list(oov))
#append </w>
oov = oov + ' </w>'
#create a dictionary
oov = { oov : 1}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment