Using Gensim Fasttext model with LSTM nn in keras

by

here the procedure to incorporate the fasttext model inside an LSTM Keras network

# define dummy data and precproces them

docs = ['Well done',
        'Good work',
        'Great effort',
        'nice work',
        'Excellent',
        'Weak',
        'Poor effort',
        'not good',
        'poor work',
        'Could have done better']

docs = [d.lower().split() for d in docs]

# train fasttext from gensim api

ft = FastText(size=10, window=2, min_count=1, seed=33)
ft.build_vocab(docs)
ft.train(docs, total_examples=ft.corpus_count, epochs=10)

# prepare text for keras neural network

max_len = 8

tokenizer = tf.keras.preprocessing.text.Tokenizer(lower=True)
tokenizer.fit_on_texts(docs)

sequence_docs = tokenizer.texts_to_sequences(docs)
sequence_docs = tf.keras.preprocessing.sequence.pad_sequences(sequence_docs, maxlen=max_len)

# extract fasttext learned embedding and put them in a numpy array

embedding_matrix_ft = np.random.random((len(tokenizer.word_index) + 1, ft.vector_size))

pas = 0
for word,i in tokenizer.word_index.items():
    
    try:
        embedding_matrix_ft[i] = ft.wv[word]
    except:
        pas+=1

# define a keras model and load the pretrained fasttext weights matrix

inp = Input(shape=(max_len,))
emb = Embedding(len(tokenizer.word_index) + 1, ft.vector_size, 
                weights=[embedding_matrix_ft], trainable=False)(inp)
x = LSTM(32)(emb)
out = Dense(1)(x)

model = Model(inp, out)

model.predict(sequence_docs)

how to deal unseen text

unseen_docs = ['asdcs work','good nxsqa zajxa']
unseen_docs = [d.lower().split() for d in unseen_docs]

sequence_unseen_docs = tokenizer.texts_to_sequences(unseen_docs)
sequence_unseen_docs = tf.keras.preprocessing.sequence.pad_sequences(sequence_unseen_docs, maxlen=max_len)

model.predict(sequence_unseen_docs)

More Related Contents:

  1. TensorFlow and person recognition in video stream
  2. How to add attention layer to a Bi-LSTM
  3. Keras : How should I prepare input data for RNN?
  4. How to Properly Combine TensorFlow’s Dataset API and Keras?
  5. What is the meaning of the “None” in model.summary of KERAS?
  6. How does Keras 1d convolution layer work with word embeddings – text classification problem? (Filters, kernel size, and all hyperparameter)
  7. Multivariate LSTM with missing values
  8. Specify connections in NN (in keras)
  9. What are the differences between all these cross-entropy losses in Keras and TensorFlow?
  10. Does model.compile() initialize all the weights and biases in Keras (tensorflow backend)?
  11. Keras Loss Function with Additional Dynamic Parameter
  12. Create keras callback to save model predictions and targets for each batch during training
  13. Obtaining output of an Intermediate layer in TensorFlow/Keras
  14. tf.data with multiple inputs / outputs in Keras
  15. ValueError: Shapes (None, 1) and (None, 2) are incompatible
  16. Update only part of the word embedding matrix in Tensorflow
  17. Keras split train test set when using ImageDataGenerator
  18. In Keras, what exactly am I configuring when I create a stateful `LSTM` layer with N `units`?
  19. Confusion about keras Model: __call__ vs. call vs. predict methods
  20. How can I convert a trained Tensorflow model to Keras?
  21. How to display custom images in TensorBoard using Keras?
  22. What is the difference between keras and tf.keras?
  23. Checkpointing keras model: TypeError: can’t pickle _thread.lock objects
  24. How to replace (or insert) intermediate layer in Keras model?
  25. Choosing number of Steps per Epoch
  26. Make a custom loss function in keras
  27. Keras replacing input layer
  28. expected ndim=3, found ndim=2
  29. Custom loss function in Keras
  30. Strange behaviour of the loss function in keras model, with pretrained convolutional base

Leave a Comment