How Can I Define Only the Gradient for a Tensorflow Subgraph?

by

Here’s a trick from Sergey Ioffe:

Suppose you want group of ops that behave as f(x) in forward mode, but as g(x) in the backward mode. You implement it as

t = g(x)
y = t + tf.stop_gradient(f(x) - t)

So in your case your g(x) could be an identity op, with a custom gradient using gradient_override_map

More Related Contents:

  1. How to compile Tensorflow with SSE4.2 and AVX instructions?
  2. On Windows, running “import tensorflow” generates No module named “_pywrap_tensorflow” error
  3. Meaning of buffer_size in Dataset.map , Dataset.prefetch and Dataset.shuffle
  4. What is the meaning of the word logits in TensorFlow? [duplicate]
  5. How to set specific gpu in tensorflow?
  6. What is the meaning of the “None” in model.summary of KERAS?
  7. Module ‘tensorflow’ has no attribute ‘contrib’
  8. Understanding TensorBoard (weight) histograms
  9. How do I split Tensorflow datasets?
  10. Multivariate LSTM with missing values
  11. TensorFlow: numpy.repeat() alternative
  12. Tensorflow: ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory
  13. Keras Loss Function with Additional Dynamic Parameter
  14. Create keras callback to save model predictions and targets for each batch during training
  15. Working with multiple graphs in TensorFlow
  16. How to extract data/labels back from TensorFlow dataset
  17. Obtaining output of an Intermediate layer in TensorFlow/Keras
  18. Making predictions with a TensorFlow model
  19. tf.data with multiple inputs / outputs in Keras
  20. Update only part of the word embedding matrix in Tensorflow
  21. Could not install packages due to an EnvironmentError: [WinError 5] Access is denied:
  22. Keras split train test set when using ImageDataGenerator
  23. CUDA_ERROR_OUT_OF_MEMORY in tensorflow
  24. Implementing contrastive loss and triplet loss in Tensorflow
  25. Confusion about keras Model: __call__ vs. call vs. predict methods
  26. What does batch, repeat, and shuffle do with TensorFlow Dataset?
  27. Checkpointing keras model: TypeError: can’t pickle _thread.lock objects
  28. How to replace (or insert) intermediate layer in Keras model?
  29. Choosing number of Steps per Epoch
  30. Parallelism isn’t reducing the time in dataset map

Leave a Comment