What is the difference between ‘SAME’ and ‘VALID’ padding in tf.nn.max_pool of tensorflow?

by

If you like ascii art:

  • "VALID" = without padding:

       inputs:         1  2  3  4  5  6  7  8  9  10 11 (12 13)
                  |________________|                dropped
                                 |_________________|

  • "SAME" = with zero padding:

                   pad|                                      |pad
   inputs:      0 |1  2  3  4  5  6  7  8  9  10 11 12 13|0  0
               |________________|
                              |_________________|
                                             |________________|

In this example:

  • Input width = 13
  • Filter width = 6
  • Stride = 5

Notes:

  • "VALID" only ever drops the right-most columns (or bottom-most rows).
  • "SAME" tries to pad evenly left and right, but if the amount of columns to be added is odd, it will add the extra column to the right, as is the case in this example (the same logic applies vertically: there may be an extra row of zeros at the bottom).

Edit:

About the name:

  • With "SAME" padding, if you use a stride of 1, the layer’s outputs will have the same spatial dimensions as its inputs.
  • With "VALID" padding, there’s no “made-up” padding inputs. The layer only uses valid input data.

More Related Contents:

  1. Keras, How to get the output of each layer?
  2. How to make a custom activation function with only Python in Tensorflow?
  3. Using a pre-trained word embedding (word2vec or Glove) in TensorFlow
  4. How to apply gradient clipping in TensorFlow?
  5. How to assign a value to a TensorFlow variable?
  6. Keras replacing input layer
  7. Keras custom loss function: Accessing current input pattern
  8. How can I use a pre-trained neural network with grayscale images?
  9. How to understand the term `tensor` in TensorFlow?
  10. Dimension of shape in conv1D
  11. TensorFlow ValueError: Cannot feed value of shape (64, 64, 3) for Tensor u’Placeholder:0′, which has shape ‘(?, 64, 64, 3)’
  12. Removing then Inserting a New Middle Layer in a Keras Model
  13. Unbalanced data and weighted cross entropy
  14. How to set layer-wise learning rate in Tensorflow?
  15. Custom loss function in Keras
  16. How do I get the weights of a layer in Keras?
  17. Multiple outputs in Keras
  18. How to handle non-determinism when training on a GPU?
  19. ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: [8, 28, 28]
  20. Keras LSTM input dimension setting
  21. How to implement Grad-CAM on a trained network
  22. looking for source code of from gen_nn_ops in tensorflow
  23. ValueError: Tensor must be from the same graph as Tensor with Bidirectinal RNN in Tensorflow
  24. What is right batch normalization function in Tensorflow?
  25. How to fix “ResourceExhaustedError: OOM when allocating tensor”
  26. How do I plot a Keras/Tensorflow subclassing API model?
  27. Strange behaviour of the loss function in keras model, with pretrained convolutional base
  28. Multi-input Multi-output Model with Keras Functional API
  29. TensorFlow: Remember LSTM state for next batch (stateful LSTM)
  30. Tensorflow set CUDA_VISIBLE_DEVICES within jupyter

Leave a Comment