Tensorflow NaN bug?

by

Actually, it turned out to be something stupid. I’m posting this in case anyone else would run into a similar error.

cross_entropy = -tf.reduce_sum(y_*tf.log(y_conv))

is actually a horrible way of computing the cross-entropy. In some samples, certain classes could be excluded with certainty after a while, resulting in y_conv=0 for that sample. That’s normally not a problem since you’re not interested in those, but in the way cross_entropy is written there, it yields 0*log(0) for that particular sample/class. Hence the NaN.

Replacing it with 

cross_entropy = -tf.reduce_sum(y_*tf.log(tf.clip_by_value(y_conv,1e-10,1.0)))

solved all my problems.

More Related Contents:

  1. Make a custom loss function in keras
  2. Why is NaN not equal to NaN? [duplicate]
  3. Issue installing Tensorflow — not a CUDA/CuDNN issue
  4. What’s the difference of name scope and a variable scope in tensorflow?
  5. How do you test to see if a double is equal to NaN?
  6. How could I use batch normalization in TensorFlow?
  7. FailedPreconditionError: Attempting to use uninitialized in Tensorflow
  8. NotImplementedError: Layers with arguments in `__init__` must override `get_config`
  9. Tensorflow error : DLL load failed: The specified procedure could not be found
  10. How to set adaptive learning rate for GradientDescentOptimizer?
  11. Tensorflow Different ways to Export and Run graph in C++
  12. Tensorflow: How to replace a node in a calculation graph?
  13. How can I use a pre-trained neural network with grayscale images?
  14. Tensorflow r1.0 : could not a find a version that satisfies the requirement tensorflow
  15. TensorFlow ValueError: Cannot feed value of shape (64, 64, 3) for Tensor u’Placeholder:0′, which has shape ‘(?, 64, 64, 3)’
  16. TypeError: ‘Tensor’ object does not support item assignment in TensorFlow
  17. How do I set TensorFlow RNN state when state_is_tuple=True?
  18. Tensorflow (.pb) format to Keras (.h5)
  19. Save and load model optimizer state
  20. Keras misinterprets training data shape
  21. Why can I not import Tensorflow.contrib I get an error of No module named ‘tensorflow.python.saved
  22. Problems in Android repository Flutter TensorFlow-lite by Bintray 502
  23. Sum numbers returns NaN
  24. Remove NaN/NULL columns in a Pandas dataframe?
  25. How do I get the weights of a layer in Keras?
  26. Installing TensorFlow on Windows (Python 3.6.x)
  27. PHP: How to encode infinity or NaN numbers to JSON?
  28. How to fix “AttributeError: module ‘tensorflow’ has no attribute ‘get_default_graph'”?
  29. What is right batch normalization function in Tensorflow?
  30. Checkpointing keras model: TypeError: can’t pickle _thread.lock objects

Leave a Comment