TL;DR Your input data is not normalized. use x_data = (x_data – x_data.mean()) / x_data.std() increase the learning rate optimizer = torch.optim.Adam(model.parameters(), lr=0.01) You’ll get convergence in only 1000 iterations. More details The key difference between the two examples you have is that the data x in the first example is centered around (0, 0) … Read more
Your NetActor does not directly store any nn.Parameter. Moreover, all other layers it eventually uses in forward are stored as a simple list in self.nn_layers. If you want self.actor_nn.parameters() to know that the items stored in the list self.nn_layers may contain trainable parameters, you should work with containers. Specifically, making self.nn_layers to be a nn.ModuleList … Read more
Add –find-links in requirements.txt before torch –find-links https://download.pytorch.org/whl/torch_stable.html torch==1.2.0+cpu Source: https://github.com/pytorch/pytorch/issues/29745#issuecomment-553588171
There might be an issue with the device parameters are on: If you need to move a model to GPU via .cuda() , please do so before constructing optimizers for it. Parameters of a model after .cuda() will be different objects with those before the call. In general, you should make sure that optimized parameters … Read more
Starting in PyTorch 0.4.1 you can use random_split: train_size = int(0.8 * len(full_dataset)) test_size = len(full_dataset) – train_size train_dataset, test_dataset = torch.utils.data.random_split(full_dataset, [train_size, test_size])
I assume you are asking whether these data augmentation transforms (e.g. RandomHorizontalFlip) actually increase the size of the dataset as well, or are they applied on each item in the dataset one by one and not adding to the size of the dataset. Running the following simple code snippet we could observe that the latter … Read more
The error occurs because you ran out of memory on your GPU. One way to solve it is to reduce the batch size until your code runs without this error.
You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU. inputs, labels = data # this is what you had inputs, labels = inputs.cuda(), labels.cuda() # add this line Or like this, to stay consistent with … Read more
By “combine them” I assume you mean to concatenate the two inputs. Assuming you concat along the second dimension: import torch from torch import nn class TwoInputsNet(nn.Module): def __init__(self): super(TwoInputsNet, self).__init__() self.conv = nn.Conv2d( … ) # set up your layer here self.fc1 = nn.Linear( … ) # set up first FC layer self.fc2 = … Read more
Here is how you do this: In your Module’s forward return final output and layers’ output for which you want to apply L1 regularization loss variable will be sum of cross entropy loss of output w.r.t. targets and L1 penalties. Here’s an example code import torch from torch.autograd import Variable from torch.nn import functional as … Read more