classification

What is the difference between a sigmoid followed by the cross entropy and sigmoid_cross_entropy_with_logits in TensorFlow?

You’re confusing the cross-entropy for binary and multi-class problems. Multi-class cross-entropy The formula that you use is correct and it directly corresponds to tf.nn.softmax_cross_entropy_with_logits: -tf.reduce_sum(p * tf.log(q), axis=1) p and q are expected to be probability distributions over N classes. In particular, N can be 2, as in the following example: p = tf.placeholder(tf.float32, shape=[None, … Read more

Use scikit-learn to classify into multiple categories

What you want is called multi-label classification. Scikits-learn can do that. See here: http://scikit-learn.org/dev/modules/multiclass.html. I’m not sure what’s going wrong in your example, my version of sklearn apparently doesn’t have WordNGramAnalyzer. Perhaps it’s a question of using more training examples or trying a different classifier? Though note that the multi-label classifier expects the target to … Read more

Predict classes or class probabilities?

In principle & in theory, hard & soft classification (i.e. returning classes & probabilities respectively) are different approaches, each one with its own merits & downsides. Consider for example the following, from the paper Hard or Soft Classification? Large-margin Unified Machines: Margin-based classifiers have been popular in both machine learning and statistics for classification problems. … Read more

FailedPreconditionError: Attempting to use uninitialized in Tensorflow

The FailedPreconditionError arises because the program is attempting to read a variable (named “Variable_1″) before it has been initialized. In TensorFlow, all variables must be explicitly initialized, by running their “initializer” operations. For convenience, you can run all of the variable initializers in the current session by executing the following statement before your training loop: … Read more

How to get most informative features for scikit-learn classifiers?

The classifiers themselves do not record feature names, they just see numeric arrays. However, if you extracted your features using a Vectorizer/CountVectorizer/TfidfVectorizer/DictVectorizer, and you are using a linear model (e.g. LinearSVC or Naive Bayes) then you can apply the same trick that the document classification example uses. Example (untested, may contain a bug or two): … Read more

Save classifier to disk in scikit-learn

Classifiers are just objects that can be pickled and dumped like any other. To continue your example: import cPickle # save the classifier with open(‘my_dumped_classifier.pkl’, ‘wb’) as fid: cPickle.dump(gnb, fid) # load it again with open(‘my_dumped_classifier.pkl’, ‘rb’) as fid: gnb_loaded = cPickle.load(fid) Edit: if you are using a sklearn Pipeline in which you have custom … Read more

Loss & accuracy – Are these reasonable learning curves?

A little understanding of the actual meanings (and mechanics) of both loss and accuracy will be of much help here (refer also to this answer of mine, although I will reuse some parts)… For the sake of simplicity, I will limit the discussion to the case of binary classification, but the idea is generally applicable; … Read more