Meaning of inter_op_parallelism_threads and intra_op_parallelism_threads

by

The inter_op_parallelism_threads and intra_op_parallelism_threads options are documented in the source of the tf.ConfigProto protocol buffer. These options configure two thread pools used by TensorFlow to parallelize execution, as the comments describe:

// The execution of an individual op (for some op types) can be
// parallelized on a pool of intra_op_parallelism_threads.
// 0 means the system picks an appropriate number.
int32 intra_op_parallelism_threads = 2;

// Nodes that perform blocking operations are enqueued on a pool of
// inter_op_parallelism_threads available in each process.
//
// 0 means the system picks an appropriate number.
//
// Note that the first Session created in the process sets the
// number of threads for all future sessions unless use_per_session_threads is
// true or session_inter_op_thread_pool is configured.
int32 inter_op_parallelism_threads = 5;

There are several possible forms of parallelism when running a TensorFlow graph, and these options provide some control multi-core CPU parallelism:

  • If you have an operation that can be parallelized internally, such as matrix multiplication (tf.matmul()) or a reduction (e.g. tf.reduce_sum()), TensorFlow will execute it by scheduling tasks in a thread pool with intra_op_parallelism_threads threads. This configuration option, therefore, controls the maximum parallel speedup for a single operation. Note that if you run multiple operations in parallel, these operations will share this thread pool.

  • If you have many operations that are independent in your TensorFlow graph— because there is no directed path between them in the dataflow graph— TensorFlow will attempt to run them concurrently, using a thread pool with inter_op_parallelism_threads threads. If those operations have a multithreaded implementation, they will (in most cases) share the same thread pool for intra-op parallelism.

Finally, both configuration options take a default value of 0, which means “the system picks an appropriate number.” Currently, this means that each thread pool will have one thread per CPU core in your machine.

More Related Contents:

  1. Tensorflow and Multiprocessing: Passing Sessions
  2. How do I parallelize a simple Python loop?
  3. Make a custom loss function in keras
  4. Could not load dynamic library ‘cudart64_101.dll’ on tensorflow CPU-only installation
  5. How could I use batch normalization in TensorFlow?
  6. FailedPreconditionError: Attempting to use uninitialized in Tensorflow
  7. NotImplementedError: Layers with arguments in `__init__` must override `get_config`
  8. Tensorflow error : DLL load failed: The specified procedure could not be found
  9. can’t import tensorflow in python, windows 10 64 bit
  10. How to set adaptive learning rate for GradientDescentOptimizer?
  11. Tensorflow Different ways to Export and Run graph in C++
  12. Tensorflow: How to replace a node in a calculation graph?
  13. This model has not yet been built error on model.summary()
  14. How can I use a pre-trained neural network with grayscale images?
  15. How to understand the term `tensor` in TensorFlow?
  16. Tensorflow r1.0 : could not a find a version that satisfies the requirement tensorflow
  17. TensorFlow ValueError: Cannot feed value of shape (64, 64, 3) for Tensor u’Placeholder:0′, which has shape ‘(?, 64, 64, 3)’
  18. TypeError: ‘Tensor’ object does not support item assignment in TensorFlow
  19. How do I set TensorFlow RNN state when state_is_tuple=True?
  20. In Tensorflow, get the names of all the Tensors in a graph
  21. Custom loss function in Keras
  22. Save and load model optimizer state
  23. Keras misinterprets training data shape
  24. In TensorFlow, what is the difference between Session.run() and Tensor.eval()?
  25. Why can I not import Tensorflow.contrib I get an error of No module named ‘tensorflow.python.saved
  26. How do I get the weights of a layer in Keras?
  27. Installing TensorFlow on Windows (Python 3.6.x)
  28. cannot import name ‘_registerMatType’ from ‘cv2.cv2’
  29. How to do Xavier initialization on TensorFlow
  30. aiohttp: rate limiting parallel requests

Leave a Comment