Tackling Class Imbalance: scaling contribution to loss and sgd

by

Why don’t you use the InfogainLoss layer to compensate for the imbalance in your training set?

The Infogain loss is defined using a weight matrix H (in your case 2-by-2) The meaning of its entries are

[cost of predicting 1 when gt is 0,    cost of predicting 0 when gt is 0
 cost of predicting 1 when gt is 1,    cost of predicting 0 when gt is 1]

So, you can set the entries of H to reflect the difference between errors in predicting 0 or 1.

You can find how to define matrix H for caffe in this thread.

Regarding sample weights, you may find this post interesting: it shows how to modify the SoftmaxWithLoss layer to take into account sample weights.

Recently, a modification to cross-entropy loss was proposed by Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár Focal Loss for Dense Object Detection, (ICCV 2017).
The idea behind focal-loss is to assign different weight for each example based on the relative difficulty of predicting this example (rather based on class size etc.). From the brief time I got to experiment with this loss, it feels superior to "InfogainLoss" with class-size weights.

More Related Contents:

  1. Does Caffe need data to be shuffled?
  2. Common causes of nans during training of neural networks
  3. What is a `”Python”` layer in caffe?
  4. How to interpret caffe log with debug_info?
  5. Caffe | solver.prototxt values setting strategy
  6. What is `weight_decay` meta parameter in Caffe?
  7. Caffe: What can I do if only a small batch fits into memory?
  8. How to create caffe.deploy from train.prototxt
  9. How to reduce a fully-connected (`”InnerProduct”`) layer using truncated SVD
  10. What is `lr_policy` in Caffe?
  11. Why binary_crossentropy and categorical_crossentropy give different performances for the same problem?
  12. Test labels for regression caffe, float not allowed?
  13. Many to one and many to many LSTM examples in Keras
  14. How to initialize weights in PyTorch?
  15. InfogainLoss layer
  16. How do I initialize weights in PyTorch?
  17. What is the difference between loss function and metric in Keras? [duplicate]
  18. How do I load a caffe model and convert to a numpy array?
  19. Why must a nonlinear activation function be used in a backpropagation neural network? [closed]
  20. How to feed caffe multi label data in HDF5 format?
  21. LSTM module for Caffe
  22. caffe data layer example step by step
  23. Is deep learning bad at fitting simple non linear functions outside training scope (extrapolating)?
  24. multi-layer perceptron (MLP) architecture: criteria for choosing number of hidden layers and size of the hidden layer? [closed]
  25. How to interpret loss and accuracy for a machine learning model [closed]
  26. scale the loss value according to “badness” in caffe
  27. keras: how to save the training history attribute of the history object
  28. What is the difference between Keras model.evaluate() and model.predict()?
  29. caffe: model definition: write same layer with different phase using caffe.NetSpec()
  30. Choosing number of Steps per Epoch

Leave a Comment