What’s the difference between “hidden” and “output” in PyTorch LSTM?

by

I made a diagram. The names follow the PyTorch docs, although I renamed num_layers to w.

output comprises all the hidden states in the last layer (“last” depth-wise, not time-wise). (h_n, c_n) comprises the hidden states after the last timestep, t = n, so you could potentially feed them into another LSTM.

LSTM diagram

The batch dimension is not included.

More Related Contents:

  1. Why do we “pack” the sequences in PyTorch?
  2. Best way to save a trained model in PyTorch? [closed]
  3. Keras : How should I prepare input data for RNN?
  4. How do I save a trained model in PyTorch?
  5. Pytorch – RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed
  6. LSTM module for Caffe
  7. Understanding Keras LSTMs
  8. Keras input explanation: input_shape, units, batch_size, dim, etc
  9. Many to one and many to many LSTM examples in Keras
  10. How to initialize weights in PyTorch?
  11. How do I initialize weights in PyTorch?
  12. Why do we need to call zero_grad() in PyTorch?
  13. How does Pytorch’s “Fold” and “Unfold” work?
  14. What does model.eval() do in pytorch?
  15. TensorFlow: Remember LSTM state for next batch (stateful LSTM)
  16. How to understand the term `tensor` in TensorFlow?
  17. Calculate the output size in convolution layer [closed]
  18. How to speed up Gensim Word2vec model load time?
  19. Error when checking model input: expected lstm_1_input to have 3 dimensions, but got array with shape (339732, 29)
  20. What’s the difference between torch.stack() and torch.cat() functions?
  21. How do I create a variable-length input LSTM in Keras?
  22. Keras LSTM input dimension setting
  23. Error when checking model input: expected convolution2d_input_1 to have 4 dimensions, but got array with shape (32, 32, 3)
  24. Is .data still useful in pytorch?
  25. ValueError: Input 0 is incompatible with layer lstm_13: expected ndim=3, found ndim=4
  26. ValueError: Tensor must be from the same graph as Tensor with Bidirectinal RNN in Tensorflow
  27. RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
  28. How do I split a custom dataset into training and test datasets?
  29. RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! when resuming training
  30. PyTorch Binary Classification – same network structure, ‘simpler’ data, but worse performance?

Leave a Comment