memory issues when transforming np.array using to_categorical

by

You don’t need to use to_categorical since I guess you are doing multi-label classification. To avoid any confusion once and for all(!), let me explain this.

If you are doing binary classification, meaning each sample may belong to only one
of two classes e.g. cat vs dog or happy vs sad or positive review vs negative review, then:

  • The labels should be like [0 1 0 0 1 ... 0] with shape of (n_samples,) i.e. each sample has a one (e.g. cat) or zero (e.g. dog) label.
  • The activation function used for the last layer is usually sigmoid (or any other function that outputs a value in range [0,1]).
  • The loss function usually used is binary_crossentropy.

If you are doing multi-class classification, meaning each sample may belong to only one of many classes e.g. cat vs dog vs lion or happy vs neutral vs sad or positive review vs neutral review vs negative review, then:

  • The labels should be either one-hot encoded, i.e. [1, 0, 0] corresponds to cat, [0, 1, 0] corresponds to dog and [0, 0, 1] corresponds to lion, which in this case the labels have a shape of (n_samples, n_classes); Or they can be integers (i.e. sparse labels), i.e. 1 for cat, 2 for dog and 3 for lion, which in this case the labels have a shape of (n_samples,). The to_categorical function is used to convert sparse labels to one-hot encoded labels, of course if you wish to do so.
  • The activation function used is usually softmax.
  • The loss function used depends on the format of labels: if they are one-hot encoded, categorical_crossentropy is used and if they are sparse then sparse_categorical_crossentropy is used.

If you are doing multi-label classification, meaning each sample may belong to zero, one or more than one classes e.g. an image may contain both cat and dog, then:

  • The labels should be like [[1 0 0 1 ... 0], ..., [0 0 1 0 ... 1]] with shape of (n_samples, n_classes). For example, a label [1 1] means that the corresponding sample belong to both classes (e.g. cat and dog).
  • The activation function used is sigmoid since presumably each class is independent of another class.
  • The loss function used is binary_crossentropy.

More Related Contents:

  1. How can I use a weight matrix created by a trained neural network to make predictions in another file?
  2. Multiclass Classification
  3. Convert array of indices to 1-hot encoded numpy array
  4. Keras Dense layer’s input is not flattened
  5. Keras not training on entire dataset
  6. How to get reproducible results in keras
  7. How to implement the Softmax function in Python
  8. Deep-Learning Nan loss reasons
  9. How to apply gradient clipping in TensorFlow?
  10. gradient descent using python and numpy
  11. How does Keras calculate the accuracy?
  12. Can Keras with Tensorflow backend be forced to use CPU or GPU at will?
  13. Proper way to feed time-series data to stateful LSTM?
  14. TypeError: __array__() takes 1 positional argument but 2 were given (Image classification Keras)
  15. Keras custom loss function: Accessing current input pattern
  16. Custom transformer for sklearn Pipeline that alters both X and y
  17. How can I use a pre-trained neural network with grayscale images?
  18. How to know if underfitting or overfitting is occuring?
  19. How to save final model using keras?
  20. How to fix ‘Object arrays cannot be loaded when allow_pickle=False’ for imdb.load_data() function?
  21. Unknown initializer: GlorotUniform when loading Keras model
  22. How to use fit_generator with multiple inputs
  23. How to create a keras layer with a custom gradient in TF2.0?
  24. keras: how to save the training history attribute of the history object
  25. ModuleNotFoundError: No module named ‘numpy.testing.nosetester’
  26. How to fix “ResourceExhaustedError: OOM when allocating tensor”
  27. Can I send callbacks to a KerasClassifier?
  28. Numpy Broadcast to perform euclidean distance vectorized
  29. How is the Keras Conv1D input specified? I seem to be lacking a dimension
  30. ValueError: x and y must be the same size

Leave a Comment