Holding variables constant during optimizer

by

tf.stop_gradient(tensor) might be what you are looking for. The tensor will be treated as constant for gradient computation purposes. You can create two losses with different parts treated as constants.

The other option (and often better) would be to create 2 optimizers but explicitly optimize only subsets of variables, e.g.

train_a = tf.train.GradientDescentOptimizer(0.1).minimize(loss_a, var_list=[A])
train_b = tf.train.GradientDescentOptimizer(0.1).minimize(loss_b, var_list=[B])

and you can iterate between them on the updates.

More Related Contents:

  1. How to compile Tensorflow with SSE4.2 and AVX instructions?
  2. On Windows, running “import tensorflow” generates No module named “_pywrap_tensorflow” error
  3. Keras : How should I prepare input data for RNN?
  4. Meaning of buffer_size in Dataset.map , Dataset.prefetch and Dataset.shuffle
  5. How does TensorFlow name tensors?
  6. What is the meaning of the word logits in TensorFlow? [duplicate]
  7. How to set specific gpu in tensorflow?
  8. Can I use TensorBoard with Google Colab?
  9. Understanding TensorBoard (weight) histograms
  10. How do I split Tensorflow datasets?
  11. After building TensorFlow from source, seeing libcudart.so and libcudnn errors
  12. Multivariate LSTM with missing values
  13. Specify connections in NN (in keras)
  14. What are the differences between all these cross-entropy losses in Keras and TensorFlow?
  15. Tensorflow: ImportError: libcusolver.so.8.0: cannot open shared object file: No such file or directory
  16. Tensorflow crashes with CUBLAS_STATUS_ALLOC_FAILED
  17. Keras Loss Function with Additional Dynamic Parameter
  18. Create keras callback to save model predictions and targets for each batch during training
  19. Working with multiple graphs in TensorFlow
  20. How to extract data/labels back from TensorFlow dataset
  21. Making predictions with a TensorFlow model
  22. tf.data with multiple inputs / outputs in Keras
  23. Update only part of the word embedding matrix in Tensorflow
  24. Could not install packages due to an EnvironmentError: [WinError 5] Access is denied:
  25. Convert between NHWC and NCHW in TensorFlow
  26. How to properly use tf.metrics.accuracy?
  27. How can I convert a trained Tensorflow model to Keras?
  28. ImportError: libcudnn when running a TensorFlow program
  29. Google Storage (gs) wrapper file input/out for Cloud ML?
  30. parallelising tf.data.Dataset.from_generator

Leave a Comment