CSVWriter not saving data to file the moment I write it

Python newbie getting a bit frustrated with the csv module. At this rate, it would have been easier if I wrote the file parser myself, but I want to do things the Pythonic way ....

I have written a little python script that should save my data into a CSV file.

Here is a snippet of my code:

import csv

wrtr = csv.writer(open('myfile.csv','wb'),delimiter=',', quotechar='"')

for row in rows:
    wrtr.writerow([row.field1,row.field2,row.field3])

The file myfile.csv is created successfully, yet it is empty - but has a lock on it, as its still being used by the Python process. It seems that the data has been written to the file in memory, but it has not yet been flushed to disk.

Since the Python process is holding a lock on the file, then I assume that I am responsible for releasing the lock. Here are my questions:

1. How do I get python to flush to disk
2. How do I close the file that was opened in the csv.writer() method?

Solution 1:

Use

with open('myfile.csv','wb') as myfile:
    wrtr = csv.writer(myfile, delimiter=',', quotechar='"')
    for row in rows:
        wrtr.writerow([row.field1,row.field2,row.field3])
        myfile.flush() # whenever you want

or

myfile = open('myfile.csv','wb')
wrtr = csv.writer(myfile, delimiter=',', quotechar='"')
for row in rows:
    wrtr.writerow([row.field1,row.field2,row.field3])
    myfile.flush() # whenever you want, and/or
myfile.close() # when you're done.

The nice thing about the first approach is that your file will also be automatically properly closed in case of an Exception.

If you want your file object to be anonymous, then it will only be closed when the program exits. When or whether it is flushed depends on the OS - so it might be never until exit.

Solution 2:

The flush() and close() methods of the file object. Or use with.