antondevv/punct.py

Last active November 3, 2021 10:35

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/antondevv/c03fd7da0f15657faf766003033a8b76.js"></script>
Save antondevv/c03fd7da0f15657faf766003033a8b76 to your computer and use it in GitHub Desktop.

Download ZIP

remove punct

Raw

	remove_punctuation = re.compile('[%s]' % re.escape(string.punctuation))
	tokens = [remove_punctuation.sub('', w) for w in tokenized]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment