n-grams in python, four, five, six grams?

by

Great native python based answers given by other users. But here’s the nltk approach (just in case, the OP gets penalized for reinventing what’s already existing in the nltk library).

There is an ngram module that people seldom use in nltk. It’s not because it’s hard to read ngrams, but training a model base on ngrams where n > 3 will result in much data sparsity.

from nltk import ngrams

sentence="this is a foo bar sentences and i want to ngramize it"

n = 6
sixgrams = ngrams(sentence.split(), n)

for grams in sixgrams:
  print grams

More Related Contents:

  1. NLTK-based text processing with pandas
  2. Computing N Grams using Python
  3. Fast/Optimize N-gram implementations in python
  4. Fast n-gram calculation
  5. Convert bytes to a string
  6. How do I lowercase a string in Python?
  7. Random string generation with upper case letters and digits
  8. How to split the definition of a long string over multiple lines?
  9. Find and replace string values in list
  10. pandas dataframe str.contains() AND operation
  11. How to insert a character after every 2 characters in a string
  12. Python convert tuple to string
  13. Longest common substring from more than two strings
  14. How can I escape latex code received through user input?
  15. re.sub erroring with “Expected string or bytes-like object”
  16. Count consecutive characters
  17. Confused about __str__ on list in Python [duplicate]
  18. Find all upper, lower and mixed case combinations of a string
  19. How does my input not equal the answer?
  20. Having both single and double quotation in a Python string
  21. shuffle string in python
  22. removing newlines from messy strings in pandas dataframe cells?
  23. Convert a columns of string to list in pandas
  24. pandas: replace string with another string
  25. What’s the difference between str.isdigit(), isnumeric() and isdecimal() in Python?
  26. How does python startswith work?
  27. How can I match an exact word in a string?
  28. Why is ‘\x’ invalid in Python?
  29. Join a list of strings in python and wrap each string in quotation marks
  30. pandas – convert string into list of strings [duplicate]

Leave a Comment