NLTK Named Entity recognition to a Python list

by

nltk.ne_chunk returns a nested nltk.tree.Tree object so you would have to traverse the Tree object to get to the NEs.

Take a look at Named Entity Recognition with Regular Expression: NLTK

>>> from nltk import ne_chunk, pos_tag, word_tokenize
>>> from nltk.tree import Tree
>>> 
>>> def get_continuous_chunks(text):
...     chunked = ne_chunk(pos_tag(word_tokenize(text)))
...     continuous_chunk = []
...     current_chunk = []
...     for i in chunked:
...             if type(i) == Tree:
...                     current_chunk.append(" ".join([token for token, pos in i.leaves()]))
...             if current_chunk:
...                     named_entity = " ".join(current_chunk)
...                     if named_entity not in continuous_chunk:
...                             continuous_chunk.append(named_entity)
...                             current_chunk = []
...             else:
...                     continue
...     return continuous_chunk
... 
>>> my_sent = "WASHINGTON -- In the wake of a string of abuses by New York police officers in the 1990s, Loretta E. Lynch, the top federal prosecutor in Brooklyn, spoke forcefully about the pain of a broken trust that African-Americans felt and said the responsibility for repairing generations of miscommunication and mistrust fell to law enforcement."
>>> get_continuous_chunks(my_sent)
['WASHINGTON', 'New York', 'Loretta E. Lynch', 'Brooklyn']


>>> my_sent = "How's the weather in New York and Brooklyn"
>>> get_continuous_chunks(my_sent)
['New York', 'Brooklyn']

More Related Contents:

  1. How to use Stanford Parser in NLTK using Python
  2. Creating a new corpus with NLTK
  3. Python NLTK pos_tag not returning the correct part-of-speech tag
  4. Ordinal numbers replacement
  5. How to config nltk data directory from code?
  6. Classification using movie review corpus in NLTK/Python
  7. Convert words between verb/noun/adjective forms
  8. nltk NaiveBayesClassifier training for sentiment analysis
  9. Computing N Grams using Python
  10. Using NLTK and WordNet; how do I convert simple tense verb into its present, past or past participle form?
  11. How do I tokenize a string sentence in NLTK?
  12. How to get rid of punctuation using NLTK tokenizer?
  13. English grammar for parsing in NLTK
  14. NLTK and language detection
  15. Fast/Optimize N-gram implementations in python
  16. Extract Word from Synset using Wordnet in NLTK 3.0
  17. How do I do dependency parsing in NLTK?
  18. Extract list of Persons and Organizations using Stanford NER Tagger in NLTK
  19. training data format for NLTK punkt
  20. Difference between Python’s collections.Counter and nltk.probability.FreqDist
  21. Saving nltk drawn parse tree to image file
  22. Creating a custom categorized corpus in NLTK and Python
  23. tag generation from a text content
  24. NLTK WordNet Lemmatizer: Shouldn’t it lemmatize all inflections of a word?
  25. What is NLTK POS tagger asking me to download?
  26. Implementing Bag-of-Words Naive-Bayes classifier in NLTK
  27. Fast n-gram calculation
  28. Using the Python NLTK (2.0b5) on the Google App Engine
  29. How to use malt parser in python nltk
  30. How to tweak the NLTK sentence tokenizer

Leave a Comment