Pytorch – RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed

by

The problem is from my training loop: it doesn’t detach or repackage the hidden state in between batches? If so, then loss.backward() is trying to back-propagate all the way through to the start of time, which works for the first batch but not for the second because the graph for the first batch has been discarded.

there are two possible solutions.

1) detach/repackage the hidden state in between batches. There are (at
least) three ways to do this (and I chose this solution):

 hidden.detach_()
 hidden = hidden.detach()

2) replace loss.backward() with loss.backward(retain_graph=True) but know that each successive batch will take more time than the previous one because it will have to back-propagate all the way through to the start of the first batch.

Example

More Related Contents:

  1. Best way to save a trained model in PyTorch? [closed]
  2. How do I save a trained model in PyTorch?
  3. How to initialize weights in PyTorch?
  4. How do I initialize weights in PyTorch?
  5. Why do we need to call zero_grad() in PyTorch?
  6. How does Pytorch’s “Fold” and “Unfold” work?
  7. What does model.eval() do in pytorch?
  8. PyTorch NotImplementedError in forward
  9. What’s the difference between torch.stack() and torch.cat() functions?
  10. ValueError: Tensor must be from the same graph as Tensor with Bidirectinal RNN in Tensorflow
  11. RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same
  12. How do I split a custom dataset into training and test datasets?
  13. RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu! when resuming training
  14. PyTorch Binary Classification – same network structure, ‘simpler’ data, but worse performance?
  15. Understanding Keras LSTMs
  16. How to get the location of all text present in an image using OpenCV?
  17. Keras Sequential model input layer
  18. Heroku: slug size too large after installing Pytorch
  19. Does Any one got “AttributeError: ‘str’ object has no attribute ‘decode’ ” , while Loading a Keras Saved Model
  20. Removing then Inserting a New Middle Layer in a Keras Model
  21. Unbalanced data and weighted cross entropy
  22. What is the role of TimeDistributed layer in Keras?
  23. ModuleNotFoundError: No module named ‘tools.nnwrap’
  24. Keras LSTM input dimension setting
  25. keras: how to save the training history attribute of the history object
  26. Is .data still useful in pytorch?
  27. What is right batch normalization function in Tensorflow?
  28. Data Augmentation in PyTorch
  29. how to implement custom metric in keras?
  30. Install PyTorch from requirements.txt

Leave a Comment