In Keras, what exactly am I configuring when I create a stateful `LSTM` layer with N `units`?

by

You can check this question for further information, although it is based on Keras-1.x API.

Basically, the unit means the dimension of the inner cells in LSTM. Because in LSTM, the dimension of inner cell (C_t and C_{t-1} in the graph), output mask (o_t in the graph) and hidden/output state (h_t in the graph) should have the SAME dimension, therefore you output’s dimension should be unit-length as well.

And LSTM in Keras only define exactly one LSTM block, whose cells is of unit-length. If you set return_sequence=True, it will return something with shape: (batch_size, timespan, unit). If false, then it just return the last output in shape (batch_size, unit).

As for the input, you should provide input for every timestamp. Basically, the shape is like (batch_size, timespan, input_dim), where input_dim can be different from the unit. If you just want to provide input at the first step, you can simply pad your data with zeros at other time steps.

More Related Contents:

  1. Multivariate LSTM with missing values
  2. Keras : How should I prepare input data for RNN?
  3. Choosing number of Steps per Epoch
  4. TensorFlow and person recognition in video stream
  5. Tensorflow – ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type float)
  6. Keras + Tensorflow and Multiprocessing in Python
  7. How to Properly Combine TensorFlow’s Dataset API and Keras?
  8. What is the meaning of the word logits in TensorFlow? [duplicate]
  9. Proper way to feed time-series data to stateful LSTM?
  10. What is the meaning of the “None” in model.summary of KERAS?
  11. Specify connections in NN (in keras)
  12. What are the differences between all these cross-entropy losses in Keras and TensorFlow?
  13. expected ndim=3, found ndim=2
  14. Passing output of a CNN to BILSTM
  15. Does model.compile() initialize all the weights and biases in Keras (tensorflow backend)?
  16. Keras Loss Function with Additional Dynamic Parameter
  17. Using Gensim Fasttext model with LSTM nn in keras
  18. Create keras callback to save model predictions and targets for each batch during training
  19. Loading a trained Keras model and continue training
  20. How do I check if keras is using gpu version of tensorflow?
  21. ValueError: Shapes (None, 1) and (None, 2) are incompatible
  22. Multiple outputs in Keras
  23. Keras LSTM input dimension setting
  24. Keras split train test set when using ImageDataGenerator
  25. Confusion about keras Model: __call__ vs. call vs. predict methods
  26. How can I convert a trained Tensorflow model to Keras?
  27. How to display custom images in TensorBoard using Keras?
  28. Using pre-trained word2vec with LSTM for word generation
  29. Checkpointing keras model: TypeError: can’t pickle _thread.lock objects
  30. How to replace (or insert) intermediate layer in Keras model?

Leave a Comment