Skip to content

Instantly share code, notes, and snippets.

Avatar

Jacob Perkins japerk

View GitHub Profile
View gist:3d7e998d72460ada20c1
Verifying that +japerk is my openname (Bitcoin username). https://onename.io/japerk
@japerk
japerk / nltk_tokenize_tag_chunk.rst
Created Feb 25, 2012
NLTK Tokenization, Tagging, Chunking, Treebank
View nltk_tokenize_tag_chunk.rst

Sentence Tokenization

>>> from nltk import tokenize
>>> para = "Hello. My name is Jacob. Today you'll be learning NLTK."
>>> sents = tokenize.sent_tokenize(para)
>>> sents
['Hello.', 'My name is Jacob.', "Today you'll be learning NLTK."]