How to append data to a json file?

json might not be the best choice for on-disk formats; The trouble it has with appending data is a good example of why this might be. Specifically, json objects have a syntax that means the whole object must be read and parsed in order to understand any part of it.

Fortunately, there are lots of other options. A particularly simple one is CSV; which is supported well by python's standard library. The biggest downside is that it only works well for text; it requires additional action on the part of the programmer to convert the values to numbers or other formats, if needed.

Another option which does not have this limitation is to use a sqlite database, which also has built-in support in python. This would probably be a bigger departure from the code you already have, but it more naturally supports the 'modify a little bit' model you are apparently trying to build.

You probably want to use a JSON list instead of a dictionary as the toplevel element.

So, initialize the file with an empty list:

with open(DATA_FILENAME, mode='w', encoding='utf-8') as f:
    json.dump([], f)

Then, you can append new entries to this list:

with open(DATA_FILENAME, mode='w', encoding='utf-8') as feedsjson:
    entry = {'name': args.name, 'url': args.url}
    feeds.append(entry)
    json.dump(feeds, feedsjson)

Note that this will be slow to execute because you will rewrite the full contents of the file every time you call add. If you are calling it in a loop, consider adding all the feeds to a list in advance, then writing the list out in one go.

Append entry to the file contents if file exists, otherwise append the entry to an empty list and write in in the file:

a = []
if not os.path.isfile(fname):
    a.append(entry)
    with open(fname, mode='w') as f:
        f.write(json.dumps(a, indent=2))
else:
    with open(fname) as feedsjson:
        feeds = json.load(feedsjson)

    feeds.append(entry)
    with open(fname, mode='w') as f:
        f.write(json.dumps(feeds, indent=2))

Using a instead of w should let you update the file instead of creating a new one/overwriting everything in the existing file.

See this answer for a difference in the modes.

One possible solution is do the concatenation manually, here is some useful code:

import json
def append_to_json(_dict,path): 
    with open(path, 'ab+') as f:
        f.seek(0,2)                                #Go to the end of file    
        if f.tell() == 0 :                         #Check if file is empty
            f.write(json.dumps([_dict]).encode())  #If empty, write an array
        else :
            f.seek(-1,2)           
            f.truncate()                           #Remove the last character, open the array
            f.write(' , '.encode())                #Write the separator
            f.write(json.dumps(_dict).encode())    #Dump the dictionary
            f.write(']'.encode())                  #Close the array

You should be careful when editing the file outside the script not add any spacing at the end.

How to append data to a json file?

Related

Recent Posts