iterating over file object in Python does not work, but readlines() does but is inefficient

by

the syntax for line in fin can only be used once. After you do that, you’ve exhausted the file and you can’t read it again unless you “reset the file pointer” by fin.seek(0). Conversely, fin.readlines() will give you a list which you can iterate over and over again.

I think a simple refactor with Counter (python2.7+) could save you this headache:

from collections import Counter
with open('file') as fin:
    result = Counter()
    for line in fin:
        result += Counter(set(line.strip().lower()))

which will count the number of words in your file (1 word per line) that contain a particular character (which is what your original code does I believe … Please correct me if I’m wrong)

You could also do this easily with a defaultdict (python2.5+):

from collections import defaultdict
with open('file') as fin:
    result = defaultdict(int)
    for line in fin:
        chars = set(line.strip().lower())
        for c in chars:
            result[c] += 1

And finally, kicking it old-school — I don’t even know when setdefault was introduced…:

fin = open('file')
result = dict()
for line in fin:
    chars = set(line.strip().lower())
    for c in chars:
        result[c] = result.setdefault(c,0) + 1

fin.close()

More Related Contents:

  1. Remove some duplicates from list in python
  2. How can I count the number of times each word appears in a txt file?
  3. “is” operator behaves unexpectedly with integers
  4. How to avoid having class data shared among instances?
  5. Make a dictionary with duplicate keys in Python
  6. What exactly does “import *” import?
  7. Python: Ignore ‘Incorrect padding’ error when base64 decoding
  8. How can you print a variable name in python? [duplicate]
  9. Reshape wide to long in pandas
  10. Pass input/variables to command/script over SSH using Python Paramiko
  11. Caesar Cipher Function in Python
  12. binning data in python with scipy/numpy
  13. How do I count the occurrence of a certain item in an ndarray?
  14. Calculating Pearson correlation and significance in Python
  15. Passing command line arguments to argv in jupyter/ipython notebook
  16. Floor division with negative number
  17. Python + Django page redirect
  18. deciding among subprocess, multiprocessing, and thread in Python?
  19. How to do sed like text replace with python?
  20. Apply function to pandas DataFrame that can return multiple rows
  21. Scikit Learn OneHotEncoder fit and transform Error: ValueError: X has different shape than during fitting
  22. How to load a different file than index.html in FastAPI root path?
  23. Pandas Dataframe: split column into multiple columns, right-align inconsistent cell entries
  24. Pandas in AWS lambda gives numpy error
  25. Easiest way to rename a model using Django/South?
  26. How to display progress of scipy.optimize function?
  27. argparse subcommands with nested namespaces
  28. Python functions and their __call__ attribute
  29. Download a folder from S3 using Boto3
  30. how to understand closed and label arguments in pandas resample method?

Leave a Comment