Created

Embed URL

HTTPS clone URL

SSH clone URL

You can clone with HTTPS or SSH.

Download Gist

Basic example of using NLTK for name entity extraction.

View example1.py
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
import nltk
 
with open('sample.txt', 'r') as f:
sample = f.read()
sentences = nltk.sent_tokenize(sample)
tokenized_sentences = [nltk.word_tokenize(sentence) for sentence in sentences]
tagged_sentences = [nltk.pos_tag(sentence) for sentence in tokenized_sentences]
chunked_sentences = nltk.batch_ne_chunk(tagged_sentences, binary=True)
 
def extract_entity_names(t):
entity_names = []
if hasattr(t, 'node') and t.node:
if t.node == 'NE':
entity_names.append(' '.join([child[0] for child in t]))
else:
for child in t:
entity_names.extend(extract_entity_names(child))
return entity_names
 
entity_names = []
for tree in chunked_sentences:
# Print results per sentence
# print extract_entity_names(tree)
entity_names.extend(extract_entity_names(tree))
 
# Print all entity names
#print entity_names
 
# Print unique entity names
print set(entity_names)

Im sorry but your code isnt working ---- nltk.batch_ne_chunk : 'module' object has no attribute 'batch_ne_chunk'
Please suggest what to do

hi Rsingh, the NLTK 3.0 docs say :smile: chunk.batch_ne_chunk() → chunk.ne_chunk_sents()
i replaced that and script works again ...

hugo

also seems there is more changes in NLTK 3.0

also change this 'node' to 'label()' :

if hasattr(t, 'label') and t.label:
    if t.label() == 'NE':
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.