wordnet lemmatization and pos tagging in python

by

First of all, you can use nltk.pos_tag() directly without training it.
The function will load a pretrained tagger from a file. You can see the file name
with nltk.tag._POS_TAGGER:

nltk.tag._POS_TAGGER
>>> 'taggers/maxent_treebank_pos_tagger/english.pickle'

As it was trained with the Treebank corpus, it also uses the Treebank tag set.

The following function would map the treebank tags to WordNet part of speech names: 

from nltk.corpus import wordnet

def get_wordnet_pos(treebank_tag):

    if treebank_tag.startswith('J'):
        return wordnet.ADJ
    elif treebank_tag.startswith('V'):
        return wordnet.VERB
    elif treebank_tag.startswith('N'):
        return wordnet.NOUN
    elif treebank_tag.startswith('R'):
        return wordnet.ADV
    else:
        return ''

You can then use the return value with the lemmatizer:

from nltk.stem.wordnet import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()
lemmatizer.lemmatize('going', wordnet.VERB)
>>> 'go'

Check the return value before passing it to the Lemmatizer because an empty string would give a KeyError.

More Related Contents:

  1. How to check if a word is an English word with Python?
  2. Convert words between verb/noun/adjective forms
  3. All synonyms for word in python? [duplicate]
  4. Using NLTK and WordNet; how do I convert simple tense verb into its present, past or past participle form?
  5. How to get synonyms from nltk WordNet Python
  6. Extract Word from Synset using Wordnet in NLTK 3.0
  7. How to get all the hyponyms of a word/synset in python nltk and wordnet?
  8. Refering to a directory in a Flask app doesn’t work unless the path is absolute
  9. Ordinal numbers replacement
  10. pip issue installing almost any library
  11. Opening A large JSON file
  12. NLTK Named Entity recognition to a Python list
  13. Computing N Grams using Python
  14. NLTK Tagging spanish words using a corpus
  15. How to get rid of punctuation using NLTK tokenizer?
  16. English grammar for parsing in NLTK
  17. NLTK and language detection
  18. How to apply NLTK word_tokenize library on a Pandas dataframe for Twitter data?
  19. How do I do dependency parsing in NLTK?
  20. Extract list of Persons and Organizations using Stanford NER Tagger in NLTK
  21. SSL error downloading NLTK data
  22. Difference between Python’s collections.Counter and nltk.probability.FreqDist
  23. Spell Checker for Python
  24. Tokenize a paragraph into sentence and then into words in NLTK
  25. Saving nltk drawn parse tree to image file
  26. Creating a custom categorized corpus in NLTK and Python
  27. Save Naive Bayes Trained Classifier in NLTK
  28. NLTK WordNet Lemmatizer: Shouldn’t it lemmatize all inflections of a word?
  29. Topic distribution: How do we see which document belong to which topic after doing LDA in python
  30. How to tweak the NLTK sentence tokenizer

Leave a Comment