Easiest way to persist a data structure to a file in python?
Let's say I have something like this:
d = { "abc" : [1, 2, 3], "qwerty" : [4,5,6] }
What's the easiest way to progammatically get that into a file that I can load from python later?
Can I somehow save it as python source (from within a python script, not manually!), then import it later?
Or should I use JSON or something?
Solution 1:
Use the pickle module.
import pickle
d = { "abc" : [1, 2, 3], "qwerty" : [4,5,6] }
afile = open(r'C:\d.pkl', 'wb')
pickle.dump(d, afile)
afile.close()
#reload object from file
file2 = open(r'C:\d.pkl', 'rb')
new_d = pickle.load(file2)
file2.close()
#print dictionary object loaded from file
print new_d
Solution 2:
Take your pick: Python Standard Library - Data Persistance. Which one is most appropriate can vary by what your specific needs are.
pickle is probably the simplest and most capable as far as "write an arbitrary object to a file and recover it" goes—it can automatically handle custom classes and circular references.
For the best pickling performance (speed and space), use cPickle at HIGHEST_PROTOCOL.
Solution 3:
Try the shelve module which will give you persistent dictionary, for example:
import shelve
d = { "abc" : [1, 2, 3], "qwerty" : [4,5,6] }
shelf = shelve.open('shelf_file')
for key in d:
shelf[key] = d[key]
shelf.close()
....
# reopen the shelf
shelf = shelve.open('shelf_file')
print(shelf) # => {'qwerty': [4, 5, 6], 'abc': [1, 2, 3]}
Solution 4:
JSON has faults, but when it meets your needs, it is also:
- simple to use
- included in the standard library as the
jsonmodule - interface somewhat similar to
pickle, which can handle more complex situations - human-editable text for debugging, sharing, and version control
- valid Python code
- well-established on the web (if your program touches any of that domain)
Solution 5:
You also might want to take a look at Zope's Object Database the more complex you get:-) Probably overkill for what you have, but it scales well and is not too hard to use.