Convert pandas DataFrame to a nested dict
Solution 1:
I don't understand why there isn't a B2
in your dict. I'm also not sure what you want to happen in the case of repeated column values (every one except the last, I mean.) Assuming the first is an oversight, we could use recursion:
def recur_dictify(frame):
if len(frame.columns) == 1:
if frame.values.size == 1: return frame.values[0][0]
return frame.values.squeeze()
grouped = frame.groupby(frame.columns[0])
d = {k: recur_dictify(g.ix[:,1:]) for k,g in grouped}
return d
which produces
>>> df
name v1 v2 v3
0 A A1 A11 1
1 A A2 A12 2
2 B B1 B12 3
3 C C1 C11 4
4 B B2 B21 5
5 A A2 A21 6
>>> pprint.pprint(recur_dictify(df))
{'A': {'A1': {'A11': 1}, 'A2': {'A12': 2, 'A21': 6}},
'B': {'B1': {'B12': 3}, 'B2': {'B21': 5}},
'C': {'C1': {'C11': 4}}}
It might be simpler to use a non-pandas approach, though:
def retro_dictify(frame):
d = {}
for row in frame.values:
here = d
for elem in row[:-2]:
if elem not in here:
here[elem] = {}
here = here[elem]
here[row[-2]] = row[-1]
return d
Solution 2:
You can reconstruct your dictionary as easy as follows
>>> result = {}
>>> for lst in df.values:
... leaf = result
... for path in lst[:-2]:
... leaf = leaf.setdefault(path, {})
... leaf.setdefault(lst[-2], list()).append(lst[-1])
...
>>> result
{'A': {'A1': {'A11': [1]}, 'A2': {'A21': [6], 'A12': [2]}}, 'C': {'C1': {'C11': [4]}}, 'B': {'B1': {'B12': [3]}, 'B2': {'B21': [5]}}}
If you're sure your leafs won't overlap, replace last line
... leaf.setdefault(lst[-2], list()).append(lst[-1])
with
... leaf[lst[-2]] = lst[-1]
to get output you desired:
>>> result
{'A': {'A1': {'A11': 1}, 'A2': {'A21': 6, 'A12': 2}}, 'C': {'C1': {'C11': 4}}, 'B': {'B1': {'B12': 3}, 'B2': {'B21': 5}}}
Sample data used for tests:
import pandas as pd
data = {'name': ['A','A','B','C','B','A'],
'v1': ['A1','A2','B1','C1','B2','A2'],
'v2': ['A11','A12','B12','C11','B21','A21'],
'v3': [1,2,3,4,5,6]}
df = pd.DataFrame.from_dict(data)
Solution 3:
see here as their are some options that you can pass to get the output in several different forms.
In [5]: df
Out[5]:
name v1 v2 v3
0 A A1 A11 1
1 A A2 A12 2
2 B B1 B12 3
3 C C1 C11 4
4 B B2 B21 5
5 A A2 A21 6
In [6]: df.to_dict()
Out[6]:
{'name': {0: 'A', 1: 'A', 2: 'B', 3: 'C', 4: 'B', 5: 'A'},
'v1': {0: 'A1', 1: 'A2', 2: 'B1', 3: 'C1', 4: 'B2', 5: 'A2'},
'v2': {0: 'A11', 1: 'A12', 2: 'B12', 3: 'C11', 4: 'B21', 5: 'A21'},
'v3': {0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6}}
Here is a way to create a json format, then literally eval it to create an actual dict
In [11]: import ast
In [15]: ast.literal_eval(df.to_json(orient='values'))
Out[15]:
[['A', 'A1', 'A11', 1],
['A', 'A2', 'A12', 2],
['B', 'B1', 'B12', 3],
['C', 'C1', 'C11', 4],
['B', 'B2', 'B21', 5],
['A', 'A2', 'A21', 6]]
Solution 4:
data.groupby(by='name', sort=False).apply(lambda x: x.to_dict(orient='records'))
Should help and is the simplest way.