“freeze” some variables/scopes in tensorflow: stop_gradient vs passing variables to minimize

by

The easiest way to achieve this, as you mention in your question, is to create two optimizer operations using separate calls to opt.minimize(cost, ...). By default, the optimizer will use all of the variables in tf.trainable_variables(). If you want to filter the variables to a particular scope, you can use the optional scope argument to tf.get_collection() as follows:

optimizer = tf.train.AdagradOptimzer(0.01)

first_train_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,
                                     "scope/prefix/for/first/vars")
first_train_op = optimizer.minimize(cost, var_list=first_train_vars)

second_train_vars = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES,
                                      "scope/prefix/for/second/vars")                     
second_train_op = optimizer.minimize(cost, var_list=second_train_vars)

More Related Contents:

  1. What are logits? What is the difference between softmax and softmax_cross_entropy_with_logits?
  2. TensorFlow Only running on 1/32 of the Training data provided [duplicate]
  3. How to get current available GPUs in tensorflow?
  4. How to export Keras .h5 to tensorflow .pb?
  5. Keras + Tensorflow and Multiprocessing in Python
  6. Is it still necessary to install CUDA before using the conda tensorflow-gpu package?
  7. Meaning of inter_op_parallelism_threads and intra_op_parallelism_threads
  8. How do I use TensorFlow GPU?
  9. Is Tensorflow compatible with a Windows workflow?
  10. Tensorflow, best way to save state in RNNs?
  11. Remove nodes from graph or reset entire default graph
  12. How does Keras 1d convolution layer work with word embeddings – text classification problem? (Filters, kernel size, and all hyperparameter)
  13. Streaming large training and test files into Tensorflow’s DNNClassifier
  14. What do I need K.clear_session() and del model for (Keras with Tensorflow-gpu)?
  15. Tensorflow: Using Adam optimizer
  16. Why can’t I get reproducible results in Keras even though I set the random seeds?
  17. How can I visualize the weights(variables) in cnn in Tensorflow?
  18. overcome Graphdef cannot be larger than 2GB in tensorflow
  19. Training a simple model in Tensorflow GPU slower than CPU
  20. How to solve “AttributeError: module ‘google.protobuf.descriptor’ has no attribute ‘_internal_create_key”?
  21. Tensorflow Precision / Recall / F1 score and Confusion matrix
  22. ‘Tensor’ object has no attribute ‘lower’
  23. Unexpected keyword argument ‘ragged’ in Keras
  24. Tensorflow and Multiprocessing: Passing Sessions
  25. Sparse Tensor (matrix) from a dense Tensor Tensorflow
  26. how to convert numpy to tfrecords and then generate batches?
  27. How to fix ipykernel_launcher.py: error: unrecognized arguments in jupyter?
  28. TensorFlow, “‘module’ object has no attribute ‘placeholder'”
  29. How to prefetch data using a custom python function in tensorflow
  30. Tensorflow image reading & display

Leave a Comment