iterating over file object in Python does not work, but readlines() does but is inefficient

Solution 1:

the syntax for line in fin can only be used once. After you do that, you've exhausted the file and you can't read it again unless you "reset the file pointer" by fin.seek(0). Conversely, fin.readlines() will give you a list which you can iterate over and over again.

I think a simple refactor with Counter (python2.7+) could save you this headache:

from collections import Counter
with open('file') as fin:
    result = Counter()
    for line in fin:
        result += Counter(set(line.strip().lower()))

which will count the number of words in your file (1 word per line) that contain a particular character (which is what your original code does I believe ... Please correct me if I'm wrong)

You could also do this easily with a defaultdict (python2.5+):

from collections import defaultdict
with open('file') as fin:
    result = defaultdict(int)
    for line in fin:
        chars = set(line.strip().lower())
        for c in chars:
            result[c] += 1

And finally, kicking it old-school -- I don't even know when setdefault was introduced...:

fin = open('file')
result = dict()
for line in fin:
    chars = set(line.strip().lower())
    for c in chars:
        result[c] = result.setdefault(c,0) + 1

fin.close()

Solution 2:

You have three options:

1. Read in the whole file anyway.
2. Seek back to the beginning of the file before attempting to iterate over it again.
3. Rearchitect your code so that it doesn't need to iterate over the file more than once.