Yes, it is possible and you should also use a doubly block circulant matrix (which is a special case of Toeplitz matrix). I will give you an example with a small size of kernel and the input, but it is possible to construct Toeplitz matrix for any kernel. So you have a 2d input x … Read more
I assume you mean how to define phase when writing a prototxt using caffe.NetSpec? from caffe import layers as L, params as P, to_proto import caffe ns = caffe.NetSpec() ns.data = L.Data(name=”data”, data_param={‘source’:’/path/to/lmdb’,’batch_size’:32}, include={‘phase’:caffe.TEST}) If you want to have BOTH train and test layers in the same prototxt, what I usually do is making one … Read more
You are correct in observing that the scaling you are doing for the log(prob) is exactly what “InfogainLoss” layer is doing (You can read more about it here and here). As for the derivative (back-prop): the loss computed by this layer is L = – sum_j infogain_mat[label * dim + j] * log( prob(j) ) … Read more
Loop over the shape of every variable in tf.trainable_variables(). total_parameters = 0 for variable in tf.trainable_variables(): # shape is an array of tf.Dimension shape = variable.get_shape() print(shape) print(len(shape)) variable_parameters = 1 for dim in shape: print(dim) variable_parameters *= dim.value print(variable_parameters) total_parameters += variable_parameters print(total_parameters) Update: I wrote an article to clarify the dynamic/static shapes in … Read more
Explanation For neural networks, we usually use loss to assess how well the network has learned to classify the input image (or other tasks). The loss term is usually a scalar value. In order to update the parameters of the network, we need to calculate the gradient of loss w.r.t to the parameters, which is … Read more
I’ll try to answer the original question of why model.add_loss() is being used instead of specifying a custom loss function to model.compile(loss=…). All loss functions in Keras always take two parameters y_true and y_pred. Have a look at the definition of the various standard loss functions available in Keras, they all have these two parameters. … Read more
You can use a “Python” layer: a layer implemented in python to feed data into your net. (See an example for adding a type: “Python” layer here). import sys, os sys.path.insert(0, os.environ[‘CAFFE_ROOT’]+’/python’) import caffe class myInputLayer(caffe.Layer): def setup(self,bottom,top): # read parameters from `self.param_str` … def reshape(self,bottom,top): # no “bottom”s for input layer if len(bottom)>0: raise … Read more
I know Jeff Donahue worked on LSTM models using Caffe. He also gave a nice tutorial during CVPR 2015. He has a pull-request with RNN and LSTM. Update: there is a new PR by Jeff Donahue including RNN and LSTM. This PR was merged on June 2016 to master.
Let’s first look at how the number of learnable parameters is calculated for each individual type of layer you have, and then calculate the number of parameters in your example. Input layer: All the input layer does is read the input image, so there are no parameters you could learn here. Convolutional layers: Consider a … Read more
Ok I think this is about the simplest way to explain it all. Your example is 1 image, size 2×2, with 1 channel. You have 1 filter, with size 1×1, and 1 channel (size is height x width x channels x number of filters). For this simple case the resulting 2×2, 1 channel image (size … Read more