TL;DR Your input data is not normalized. use x_data = (x_data – x_data.mean()) / x_data.std() increase the learning rate optimizer = torch.optim.Adam(model.parameters(), lr=0.01) You’ll get convergence in only 1000 iterations. More details The key difference between the two examples you have is that the data x in the first example is centered around (0, 0) … Read more
Based on what you said it sounds like you need a larger batch_size, and of course there are implications with that which could impact the steps_per_epoch and number of epochs. To solve for jumping-around A larger batch size will give you a better gradient and will help to prevent jumping around You may also want … Read more
Here I’m answering to OP’s topic question rather than his exact problem. I’m doing this as the question shows up in the top when I google the topic problem. You can implement a custom metric in two ways. As mentioned in Keras docu. import keras.backend as K def mean_pred(y_true, y_pred): return K.mean(y_pred) model.compile(optimizer=”sgd”, loss=”binary_crossentropy”, metrics=[‘accuracy’, … Read more
There might be an issue with the device parameters are on: If you need to move a model to GPU via .cuda() , please do so before constructing optimizers for it. Parameters of a model after .cuda() will be different objects with those before the call. In general, you should make sure that optimized parameters … Read more
Yes, it is possible and you should also use a doubly block circulant matrix (which is a special case of Toeplitz matrix). I will give you an example with a small size of kernel and the input, but it is possible to construct Toeplitz matrix for any kernel. So you have a 2d input x … Read more
Starting in PyTorch 0.4.1 you can use random_split: train_size = int(0.8 * len(full_dataset)) test_size = len(full_dataset) – train_size train_dataset, test_dataset = torch.utils.data.random_split(full_dataset, [train_size, test_size])
You get this error because your model is on the GPU, but your data is on the CPU. So, you need to send your input tensors to the GPU. inputs, labels = data # this is what you had inputs, labels = inputs.cuda(), labels.cuda() # add this line Or like this, to stay consistent with … Read more
We can do that easily in tf. keras using its awesome Functional API. Here we will walk you through how to build multi-out with a different type (classification and regression) using Functional API. According to your last diagram, you need one input model and three outputs of different types. To demonstrate, we will use MNIST … Read more
I assume you mean how to define phase when writing a prototxt using caffe.NetSpec? from caffe import layers as L, params as P, to_proto import caffe ns = caffe.NetSpec() ns.data = L.Data(name=”data”, data_param={‘source’:’/path/to/lmdb’,’batch_size’:32}, include={‘phase’:caffe.TEST}) If you want to have BOTH train and test layers in the same prototxt, what I usually do is making one … Read more
Looks like I found the solution. As I have suggested the problem is with BatchNormalization layers. They make tree things subtract mean and normalize by std collect statistics on mean and std using running average train two additional parameters (two per node). When one sets trainable to False, these two parameters freeze and layer also … Read more