How to remove English words from a file containing Dari words?

by

You could install and use the nltk library. This provides you with a list of English words and a means to split each line into words:

from nltk.tokenize import word_tokenize
from nltk.corpus import words

english = words.words()

with open('Dari.pos') as f_input, open('DariNER.txt', 'w') as f_output:
    for line in f_input:
        f_output.write(' '.join(word for word in word_tokenize(line) if word.lower() not in english) + '\n')

After installing nltk, you should run:

import nltk
nltk.download()

and use it to download words

More Related Contents:

  1. How to add a label to all words in a file? [closed]
  2. Change sign of elements with an odd sum of indices
  3. Function Arguments – Clarification
  4. How can I read inputs as numbers?
  5. Why is a list comprehension so much faster than appending to a list?
  6. How to print a string at a fixed width?
  7. How to use pip with Python 3.x alongside Python 2.x
  8. Why is parenthesis in print voluntary in Python 2.7?
  9. How can I make a for-loop pyramid more concise in Python? [duplicate]
  10. Backporting Python 3 open(encoding=”utf-8″) to Python 2
  11. pip or pip3 to install packages for Python 3?
  12. Accessing attributes on literals work on all types, but not `int`; why? [duplicate]
  13. operator.itemgetter or lambda
  14. Accessing attributes on literals work on all types, but not `int`; why? [duplicate]
  15. How to print a list with integers without the brackets, commas and no quotes? [duplicate]
  16. How do I run python 2 and 3 in windows 7? [duplicate]
  17. Python3 correct way to import relative or absolute?
  18. Is Python’s order of evaluation of function arguments and operands deterministic (+ where is it documented)?
  19. Pip freeze vs. pip list
  20. PyOpenGL glutInit NullFunctionError
  21. How to apply a function to each sublist of a list in python?
  22. Python 3 urllib produces TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str
  23. How to read the last line of a file in Python? [duplicate]
  24. Multiple keys per value
  25. Type error Unhashable type:set
  26. Loading text file containing both float and string using numpy.loadtxt
  27. What is absolute import in python?
  28. Run process with realtime output to a Tkinter GUI
  29. What does the slice() function do in Python?
  30. Skip elements on a condition based in a list comprehension in python

Leave a Comment