Does Caffe need data to be shuffled?

by

Should you shuffle the samples? Think about the learning process if you don’t shuffle; caffe sees only 0 samples – what do you expect the algorithm to deduce? simply predict 0 all the time and everything is cool. If you have plenty of 0 before you hit the first 1 caffe will be very confident in predicting always 0. It will be very difficult to move the model from this point.
On the other hand, if it constantly sees a mix of 0 and 1 it learns from the beginning meaningful features for separating the examples.
Bottom line: it is very advantageous to shuffle the training samples, especially when using SGD-based approaches.

AFAIK, caffe does not randomly sample batch_size samples, but rather goes sequentially over the input DB batch_size after batch_size samples.

TL;DR
shuffle.

More Related Contents:

  1. Tackling Class Imbalance: scaling contribution to loss and sgd
  2. Common causes of nans during training of neural networks
  3. What is a `”Python”` layer in caffe?
  4. How to interpret caffe log with debug_info?
  5. Caffe | solver.prototxt values setting strategy
  6. What is `weight_decay` meta parameter in Caffe?
  7. Caffe: What can I do if only a small batch fits into memory?
  8. How to create caffe.deploy from train.prototxt
  9. How to reduce a fully-connected (`”InnerProduct”`) layer using truncated SVD
  10. What is `lr_policy` in Caffe?
  11. Why binary_crossentropy and categorical_crossentropy give different performances for the same problem?
  12. Test labels for regression caffe, float not allowed?
  13. Many to one and many to many LSTM examples in Keras
  14. How to initialize weights in PyTorch?
  15. InfogainLoss layer
  16. How do I initialize weights in PyTorch?
  17. What is the difference between loss function and metric in Keras? [duplicate]
  18. How do I load a caffe model and convert to a numpy array?
  19. Why must a nonlinear activation function be used in a backpropagation neural network? [closed]
  20. How to feed caffe multi label data in HDF5 format?
  21. LSTM module for Caffe
  22. caffe data layer example step by step
  23. Is deep learning bad at fitting simple non linear functions outside training scope (extrapolating)?
  24. multi-layer perceptron (MLP) architecture: criteria for choosing number of hidden layers and size of the hidden layer? [closed]
  25. How to interpret loss and accuracy for a machine learning model [closed]
  26. scale the loss value according to “badness” in caffe
  27. keras: how to save the training history attribute of the history object
  28. What is the difference between Keras model.evaluate() and model.predict()?
  29. caffe: model definition: write same layer with different phase using caffe.NetSpec()
  30. Choosing number of Steps per Epoch

Leave a Comment