looking for source code of from gen_nn_ops in tensorflow

by

You can’t find this source because the source is automatically generated by bazel. If you build from source, you’ll see this file inside bazel-genfiles. It’s also present in your local distribution which you can find using inspect module. The file contains automatically generated Python wrappers to underlying C++ implementations, so it basically consists of a bunch of 1-line functions. A shortcut to find underlying C++ implementation of such generated Python op is to convert snake case to camel case, ie conv2d_backprop_input -> Conv2dBackpropInput

# figure out where gen_nn_ops is
print(tf.nn.conv2d_transpose.__globals__['gen_nn_ops'])

from tensorflow.python.ops import gen_nn_ops
import inspect
inspect.getsourcefile('gen_nn_ops.conv2d_backprop_input')
'/Users/yaroslav/anaconda/lib/python3.5/site-packages/tensorflow/python/ops/gen_nn_ops.py'

If you cared to find out how this file really came about, you could follow the trail of bazel dependencies in BUILD files. It to find Bazel target that generated it from tensorflow source tree:

fullname=$(bazel query tensorflow/python/ops/gen_nn_ops.py)
bazel query "attr('srcs', $fullname, ${fullname//:*/}:*)"

//tensorflow/python:nn_ops_gen

So now going to BUILD file inside tensorflow/python you see that this is a target of type tf_gen_op_wrapper_private_py which is defined here and calls gen_op_wrapper_py from tensorflow/tensorflow.bzl which looks like this

def tf_gen_op_wrapper_py(name, out=None, hidden=None, visibility=None, deps=[],
....
      native.cc_binary(
      name = tool_name,

This native.cc_binary construct is a way to have Bazel target that represents execution of an arbitrary command. In this case it calls tool_name with some arguments. With a couple more steps you can find that “tool” here is compiled from framework/python_op_gen_main.cc

The reason for this complication is that TensorFlow was designed to be language agnostic. So in ideal world you would have each op described in ops.pbtxt, and then each op would have one implementation per hardware type using REGISTER_KERNEL_BUILDER, so all implementations would be done in C++/CUDA/Assembly and become automatically available to all language front-ends. There would be an equivalent translator op like “python_op_gen_main” for every language and all client library code would be automatically generated. However, because Python is so dominant, there was pressure to add features on the Python side. So now there are two kinds of ops — pure TensorFlow ops seen in files like gen_nn_ops.py, and Python-only ops in files like nn_ops.py which typically wrap ops automatically generated files gen_nn_ops.py but add extra features/syntax sugar. Also, originally all names were camel-case, but it was decided that public facing release should be PEP compliant with more common Python syntax, so this is a reason for camel-case/snake-case mismatch between C++/Python interfaces of same op

More Related Contents:

  1. Keras, How to get the output of each layer?
  2. How to make a custom activation function with only Python in Tensorflow?
  3. Using a pre-trained word embedding (word2vec or Glove) in TensorFlow
  4. What is the difference between ‘SAME’ and ‘VALID’ padding in tf.nn.max_pool of tensorflow?
  5. How to apply gradient clipping in TensorFlow?
  6. How to assign a value to a TensorFlow variable?
  7. Keras replacing input layer
  8. Keras custom loss function: Accessing current input pattern
  9. How can I use a pre-trained neural network with grayscale images?
  10. How to understand the term `tensor` in TensorFlow?
  11. Dimension of shape in conv1D
  12. TensorFlow ValueError: Cannot feed value of shape (64, 64, 3) for Tensor u’Placeholder:0′, which has shape ‘(?, 64, 64, 3)’
  13. Removing then Inserting a New Middle Layer in a Keras Model
  14. Unbalanced data and weighted cross entropy
  15. How to set layer-wise learning rate in Tensorflow?
  16. Custom loss function in Keras
  17. How do I get the weights of a layer in Keras?
  18. Multiple outputs in Keras
  19. How to handle non-determinism when training on a GPU?
  20. ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: [8, 28, 28]
  21. Keras LSTM input dimension setting
  22. How to implement Grad-CAM on a trained network
  23. ValueError: Tensor must be from the same graph as Tensor with Bidirectinal RNN in Tensorflow
  24. What is right batch normalization function in Tensorflow?
  25. How to fix “ResourceExhaustedError: OOM when allocating tensor”
  26. How do I plot a Keras/Tensorflow subclassing API model?
  27. Strange behaviour of the loss function in keras model, with pretrained convolutional base
  28. Multi-input Multi-output Model with Keras Functional API
  29. Sparse Tensor (matrix) from a dense Tensor Tensorflow
  30. how to convert numpy to tfrecords and then generate batches?

Leave a Comment