Tensorflow One Hot Encoder?

by

As of TensorFlow 0.8, there is now a native one-hot op, tf.one_hot that can convert a set of sparse labels to a dense one-hot representation. This is in addition to tf.nn.sparse_softmax_cross_entropy_with_logits, which can in some cases let you compute the cross entropy directly on the sparse labels instead of converting them to one-hot.

Previous answer, in case you want to do it the old way:
@Salvador’s answer is correct – there (used to be) no native op to do it. Instead of doing it in numpy, though, you can do it natively in tensorflow using the sparse-to-dense operators:

num_labels = 10

# label_batch is a tensor of numeric labels to process
# 0 <= label < num_labels

sparse_labels = tf.reshape(label_batch, [-1, 1])
derived_size = tf.shape(label_batch)[0]
indices = tf.reshape(tf.range(0, derived_size, 1), [-1, 1])
concated = tf.concat(1, [indices, sparse_labels])
outshape = tf.pack([derived_size, num_labels])
labels = tf.sparse_to_dense(concated, outshape, 1.0, 0.0)

The output, labels, is a one-hot matrix of batch_size x num_labels.

Note also that as of 2016-02-12 (which I assume will eventually be part of a 0.7 release), TensorFlow also has the tf.nn.sparse_softmax_cross_entropy_with_logits op, which in some cases can let you do training without needing to convert to a one-hot encoding.

Edited to add: At the end, you may need to explicitly set the shape of labels. The shape inference doesn’t recognize the size of the num_labels component. If you don’t need a dynamic batch size with derived_size, this can be simplified.

Edited 2016-02-12 to change the assignment of outshape per comment below.

More Related Contents:

  1. Tensor is not an element of this graph
  2. How can I use a weight matrix created by a trained neural network to make predictions in another file?
  3. Make a custom loss function in keras
  4. How to make a custom activation function with only Python in Tensorflow?
  5. Keras not training on entire dataset
  6. What is a `”Python”` layer in caffe?
  7. Deep-Learning Nan loss reasons
  8. How to apply gradient clipping in TensorFlow?
  9. How to assign a value to a TensorFlow variable?
  10. How does Keras calculate the accuracy?
  11. Tensorflow: How to replace or modify gradient?
  12. How do I initialize weights in PyTorch?
  13. Can Keras with Tensorflow backend be forced to use CPU or GPU at will?
  14. Tensorflow: How to write op with gradient in python?
  15. TensorFlow operator overloading
  16. Negative dimension size caused by subtracting 3 from 1 for ‘conv2d_2/convolution’
  17. How to get other metrics in Tensorflow 2.0 (not only accuracy)?
  18. How can I use a pre-trained neural network with grayscale images?
  19. How to understand the term `tensor` in TensorFlow?
  20. Removing then Inserting a New Middle Layer in a Keras Model
  21. How do I set TensorFlow RNN state when state_is_tuple=True?
  22. How to know if underfitting or overfitting is occuring?
  23. Loading a trained Keras model and continue training
  24. Unbalanced data and weighted cross entropy
  25. Save and load model optimizer state
  26. How to handle non-determinism when training on a GPU?
  27. Neural network XOR gate not learning
  28. How to implement the ReLU function in Numpy
  29. How to fix “ResourceExhaustedError: OOM when allocating tensor”
  30. How is the Keras Conv1D input specified? I seem to be lacking a dimension

Leave a Comment