antondevv/split_text.py

Last active November 3, 2021 23:17

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/antondevv/393adef279fe4b3aa6d1f367db3ac0d7.js"></script>
Save antondevv/393adef279fe4b3aa6d1f367db3ac0d7 to your computer and use it in GitHub Desktop.

Download ZIP

Split text

Raw

split_text.py

	text = 'What a beautiful day to be outside, incredibly beautiful day!'
	text = text.lower()
	tokenized = text.split()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment