List of dictionaries: remove duplicates while keeping part of the dictionary
I have a list of dictionaries and I would like to remove duplicates while keeping some informations.
For contexts, these are blockchain transactions. The informations are:
-
transaction hash,
-
ID of the NFT,
-
value of the transaction (it can be buying, selling or minting price).
In the original list below every action is separated. However, I realized that some have the same hash so they are just one transaction with multiple actions (in this case minting 2 NFTs in one transaction). So y goal is to have a cleaned list with only the individual transactions (identified by their hash) and the multiple IDs separated by a comma if there are multiple actions in one transaction, and the total price.
original_list = [
{
'hash': '12345',
'ID': '355',
'price': 12
},
{
'hash': '12345',
'ID': '356',
'price': 12
},
{
'hash': '635',
'ID': '355',
'price': 30
},
{
'hash': '637',
'ID': '356',
'price': 35
}
]
Here is the end result I want:
clean_list = [
{
'hash': '12345',
'ID': '355, 356',
'price': 12
},
{
'hash': '635',
'ID': '355',
'price': 30
},
{
'hash': '637',
'ID': '356',
'price': 35
}
]
How can I do this?
You can just loop through your list and create a new dictionary where you merge the data:
def exist_in_list[hash,lst]: #function that checks if the hash is already in the list and returns the index if it does
for i in range(len(lst)):
if lst[i]['hash']==hash:
return i
return -1
new_list = []
for dict in clean_list:
index = exist_in_list(dict['hash'],new_list):
if index<0:
new_list.append(dict)
else:
new_list[index]['ID'] += f", {dict['ID']}"
There may be some small syntax errors but the logic is there.
You loop through the first list, check if the dictionary is in the new one, if it isn't then you add it and if it is, you add the new ID in it.