keras - w3toppers.com

How to add attention layer to a Bi-LSTM

This can be a possible custom solution with a custom layer that computes attention on the positional/temporal dimension from tensorflow.keras.layers import Layer from tensorflow.keras import backend as K class Attention(Layer): def __init__(self, return_sequences=True): self.return_sequences = return_sequences super(Attention,self).__init__() def build(self, input_shape): self.W=self.add_weight(name=”att_weight”, shape=(input_shape[-1],1), initializer=”normal”) self.b=self.add_weight(name=”att_bias”, shape=(input_shape[1],1), initializer=”zeros”) super(Attention,self).build(input_shape) def call(self, x): e = K.tanh(K.dot(x,self.W)+self.b) a = … Read more

How to export Keras .h5 to tensorflow .pb?

Keras does not include by itself any means to export a TensorFlow graph as a protocol buffers file, but you can do it using regular TensorFlow utilities. Here is a blog post explaining how to do it using the utility script freeze_graph.py included in TensorFlow, which is the “typical” way it is done. However, I … Read more

Keras input explanation: input_shape, units, batch_size, dim, etc

Units: The amount of “neurons”, or “cells”, or whatever the layer has inside it. It’s a property of each layer, and yes, it’s related to the output shape (as we will see later). In your picture, except for the input layer, which is conceptually different from other layers, you have: Hidden layer 1: 4 units … Read more

Keras not training on entire dataset

The number 1875 shown during fitting the model is not the training samples; it is the number of batches. model.fit includes an optional argument batch_size, which, according to the documentation: If unspecified, batch_size will default to 32. So, what happens here is – you fit with the default batch size of 32 (since you have … Read more

Make a custom loss function in keras

There are two steps in implementing a parameterized custom loss function in Keras. First, writing a method for the coefficient/metric. Second, writing a wrapper function to format things the way Keras needs them to be. It’s actually quite a bit cleaner to use the Keras backend instead of tensorflow directly for simple custom loss functions … Read more

TensorFlow Only running on 1/32 of the Training data provided [duplicate]

This is a common misconception, there have been updates to Keras and it now shows batches, not samples, in the progress bar. And this is perfectly consistent because you say 1/32 of the data provided, and 32 is the default batch size in keras.

Loss & accuracy – Are these reasonable learning curves?

A little understanding of the actual meanings (and mechanics) of both loss and accuracy will be of much help here (refer also to this answer of mine, although I will reuse some parts)… For the sake of simplicity, I will limit the discussion to the case of binary classification, but the idea is generally applicable; … Read more

Keras Dense layer’s input is not flattened

Currently, contrary to what has been stated in documentation, the Dense layer is applied on the last axis of input tensor: Contrary to the documentation, we don’t actually flatten it. It’s applied on the last axis independently. In other words, if a Dense layer with m units is applied on an input tensor of shape … Read more

Why binary_crossentropy and categorical_crossentropy give different performances for the same problem?

The reason for this apparent performance discrepancy between categorical & binary cross entropy is what user xtof54 has already reported in his answer below, i.e.: the accuracy computed with the Keras method evaluate is just plain wrong when using binary_crossentropy with more than 2 labels I would like to elaborate more on this, demonstrate the … Read more

Keras, How to get the output of each layer?

You can easily get the outputs of any layer by using: model.layers[index].output For all layers use this: from keras import backend as K inp = model.input # input placeholder outputs = [layer.output for layer in model.layers] # all layer outputs functors = [K.function([inp, K.learning_phase()], [out]) for out in outputs] # evaluation functions # Testing test … Read more