json.loads allows duplicate keys in a dictionary, overwriting the first value

>>> raw_post_data = request.raw_post_data
>>> print raw_post_data
{"group":{"groupId":"2", "groupName":"GroupName"}, "members":{"1":{"firstName":"fName","lastName":"LName","address":"address"},"1": {"firstName":"f_Name","lastName":"L_Name","address":"_address"}}}
>>> create_request = json.loads(raw_post_data)
>>> print create_request
{u'group': {u'groupName': u'GroupName', u'groupId': u'2'}, u'members': {u'1': {u'lastName': u'L_Name', u'firstName': u'f_Name', u'address': u'_address'}}}

As you can see members with key '1' is overwritten when I use json.dumps()

Is there any way to catch it as exception in python, saying found duplicate keys in request from client ?

The rfc 4627 for application/json media type recommends unique keys but it doesn't forbid them explicitly:

The names within an object SHOULD be unique.

From rfc 2119:

SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.

import json

def dict_raise_on_duplicates(ordered_pairs):
    """Reject duplicate keys."""
    d = {}
    for k, v in ordered_pairs:
        if k in d:
           raise ValueError("duplicate key: %r" % (k,))
        else:
           d[k] = v
    return d

json.loads(raw_post_data, object_pairs_hook=dict_raise_on_duplicates)
# -> ValueError: duplicate key: u'1'

This is a linter-fixed and type-annotated version of the answer by jfs. Issues highlighted by various linters were addressed. It is also modernized for Python 3.6+ to use f-strings.

import json
from typing import Any, Dict, Hashable, List, Tuple

def raise_on_duplicate_keys(ordered_pairs: List[Tuple[Hashable, Any]]) -> Dict:
    """Raise ValueError if a duplicate key exists in provided ordered list of pairs, otherwise return a dict."""
    dict_out = {}
    for key, val in ordered_pairs:
        if key in dict_out:
            raise ValueError(f'Duplicate key: {key}')
        else:
            dict_out[key] = val
    return dict_out

json.loads('{"x": 1, "x": 2}', object_pairs_hook=raise_on_duplicate_keys)

ordered_pairs above is a list of tuples, with each tuple having a key and a value. Refer to the docs for object_pairs_hook.

Alternatively if you want to catch all the duplicate keys (per level) you can use a collections.Counter

from collections import Counter

class KeyWatcher(dict):

    def __init__(self, *args):
        duplicates = [d for d,i in Counter([pair[0] for pair in args[0]]).items() if i > 0]
        if duplicates:
            raise KeyError("Can't add duplicate keys {} to a json message".format(duplicates))
        self.update(*args[0])

json.loads(raw_post_data, object_pairs_hook=KeyWatcher)

json.loads allows duplicate keys in a dictionary, overwriting the first value

Related

Recent Posts