How to use Bert for long text classification?

by

You have basically three options:

  1. You can cut the longer texts off and only use the first 512 Tokens. The original BERT implementation (and probably the others as well) truncates longer sequences automatically. For most cases, this option is sufficient.
  2. You can split your text in multiple subtexts, classify each of them and combine the results back together ( choose the class which was predicted for most of the subtexts for example). This option is obviously more expensive.
  3. You can even feed the output token for each subtext (as in option 2) to another network (but you won’t be able to fine-tune) as described in this discussion.

I would suggest to try option 1, and only if this is not good enough to consider the other options.

More Related Contents:

  1. What are some simple NLP projects that a CS undergrad can try implementing? [closed]
  2. Detecting syllables in a word
  3. Stemmers vs Lemmatizers
  4. How do you implement a “Did you mean”? [duplicate]
  5. How do I do word Stemming or Lemmatization?
  6. How do I open multiple instances of Visual Studio Code?
  7. Anyone know of some good Word Sense Disambiguation software? [closed]
  8. What is the difference between lemmatization vs stemming?
  9. OpenAI GPT-3 API: How do I make sure answers are from a customized (fine-tuning) dataset?
  10. Machine understand natural language – nlp [closed]
  11. Speeker Identification (Machine Learning)
  12. How to use Stanford Parser in NLTK using Python
  13. Calculate cosine similarity given 2 sentence strings
  14. How does the Google “Did you mean?” Algorithm work? [closed]
  15. How to add attention layer to a Bi-LSTM
  16. How can I plot a confusion matrix? [duplicate]
  17. php sentence boundaries detection [duplicate]
  18. Computing N Grams using Python
  19. How can I split a text into sentences using the Stanford parser?
  20. Python – RegEx for splitting text into sentences (sentence-tokenizing) [duplicate]
  21. ArrayList as key in HashMap
  22. ROC for multiclass classification
  23. English grammar for parsing in NLTK
  24. Extract Word from Synset using Wordnet in NLTK 3.0
  25. How do I do dependency parsing in NLTK?
  26. Fuzzy String Comparison
  27. Natural Language date and time parser for java [closed]
  28. Pandas dataframe groupby text value that occurs in two columns
  29. Fast n-gram calculation
  30. Using the Python NLTK (2.0b5) on the Google App Engine

Leave a Comment