How to efficiently fix JSON file converted from pandas dataframe

Solution 1:

The issue is that you are adding an index at two places.

Once while writing your file to csv. This adds the "Unnamed: 0" fields in the final JSON files. You can use index = False in the to_csv method while writing CSV to disk or specify the index_col parameter while reading the saved CSV in read_csv.

Secondly you are adding an index while writing the df to json with orient="index". This adds the outermost indices such as "0", "1" in the final JSON file. You should use orient="records" if you intend to save the json in a similar format to it was loaded in.

To understand how the orient parameter works, refer to pandas.DataFrame.to_json