Implementing Bag-of-Words Naive-Bayes classifier in NLTK

by

scikit-learn has an implementation of multinomial naive Bayes, which is the right variant of naive Bayes in this situation. A support vector machine (SVM) would probably work better, though.

As Ken pointed out in the comments, NLTK has a nice wrapper for scikit-learn classifiers. Modified from the docs, here’s a somewhat complicated one that does TF-IDF weighting, chooses the 1000 best features based on a chi2 statistic, and then passes that into a multinomial naive Bayes classifier. (I bet this is somewhat clumsy, as I’m not super familiar with either NLTK or scikit-learn.)

import numpy as np
from nltk.probability import FreqDist
from nltk.classify import SklearnClassifier
from sklearn.feature_extraction.text import TfidfTransformer
from sklearn.feature_selection import SelectKBest, chi2
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

pipeline = Pipeline([('tfidf', TfidfTransformer()),
                     ('chi2', SelectKBest(chi2, k=1000)),
                     ('nb', MultinomialNB())])
classif = SklearnClassifier(pipeline)

from nltk.corpus import movie_reviews
pos = [FreqDist(movie_reviews.words(i)) for i in movie_reviews.fileids('pos')]
neg = [FreqDist(movie_reviews.words(i)) for i in movie_reviews.fileids('neg')]
add_label = lambda lst, lab: [(x, lab) for x in lst]
classif.train(add_label(pos[:100], 'pos') + add_label(neg[:100], 'neg'))

l_pos = np.array(classif.classify_many(pos[100:]))
l_neg = np.array(classif.classify_many(neg[100:]))
print "Confusion matrix:\n%d\t%d\n%d\t%d" % (
          (l_pos == 'pos').sum(), (l_pos == 'neg').sum(),
          (l_neg == 'pos').sum(), (l_neg == 'neg').sum())

This printed for me:

Confusion matrix:
524     376
202     698

Not perfect, but decent, considering it’s not a super easy problem and it’s only trained on 100/100.

More Related Contents:

  1. Python NLTK pos_tag not returning the correct part-of-speech tag
  2. tag generation from a text content
  3. Save Naive Bayes Trained Classifier in NLTK
  4. How to use Stanford Parser in NLTK using Python
  5. Creating a new corpus with NLTK
  6. Stopword removal with NLTK
  7. Ordinal numbers replacement
  8. How to config nltk data directory from code?
  9. Python: tf-idf-cosine: to find document similarity
  10. nltk NaiveBayesClassifier training for sentiment analysis
  11. NLTK Named Entity recognition to a Python list
  12. Computing N Grams using Python
  13. Using NLTK and WordNet; how do I convert simple tense verb into its present, past or past participle form?
  14. How do I tokenize a string sentence in NLTK?
  15. How to get rid of punctuation using NLTK tokenizer?
  16. English grammar for parsing in NLTK
  17. NLTK and language detection
  18. Fast/Optimize N-gram implementations in python
  19. Extract Word from Synset using Wordnet in NLTK 3.0
  20. How do I do dependency parsing in NLTK?
  21. Difference between Python’s collections.Counter and nltk.probability.FreqDist
  22. Saving nltk drawn parse tree to image file
  23. Creating a custom categorized corpus in NLTK and Python
  24. How to compute precision, recall, accuracy and f1-score for the multiclass case with scikit learn?
  25. NLTK WordNet Lemmatizer: Shouldn’t it lemmatize all inflections of a word?
  26. What is NLTK POS tagger asking me to download?
  27. Fast n-gram calculation
  28. How to use malt parser in python nltk
  29. How to get most informative features for scikit-learn classifier for different class?
  30. How to tweak the NLTK sentence tokenizer

Leave a Comment