How to know bytes size of python object like arrays and dictionaries? - The simple way

Solution 1:

There's:

>>> import sys
>>> sys.getsizeof([1,2, 3])
96
>>> a = []
>>> sys.getsizeof(a)
72
>>> a = [1]
>>> sys.getsizeof(a)
80

But I wouldn't say it's that reliable, as Python has overhead for each object, and there are objects that contain nothing but references to other objects, so it's not quite the same as in C and other languages.

Have a read of the docs on sys.getsizeof and go from there I guess.

Solution 2:

None of the answers here are truly generic.

The following solution will work with any type of object recursively, without the need for an expensive recursive implementation:

import gc
import sys

def get_obj_size(obj):
    marked = {id(obj)}
    obj_q = [obj]
    sz = 0

    while obj_q:
        sz += sum(map(sys.getsizeof, obj_q))

        # Lookup all the object referred to by the object in obj_q.
        # See: https://docs.python.org/3.7/library/gc.html#gc.get_referents
        all_refr = ((id(o), o) for o in gc.get_referents(*obj_q))

        # Filter object that are already marked.
        # Using dict notation will prevent repeated objects.
        new_refr = {o_id: o for o_id, o in all_refr if o_id not in marked and not isinstance(o, type)}

        # The new obj_q will be the ones that were not marked,
        # and we will update marked with their ids so we will
        # not traverse them again.
        obj_q = new_refr.values()
        marked.update(new_refr.keys())

    return sz

For example:

>>> import numpy as np
>>> x = np.random.rand(1024).astype(np.float64)
>>> y = np.random.rand(1024).astype(np.float64)
>>> a = {'x': x, 'y': y}
>>> get_obj_size(a)
16816

See my repository for more information, or simply install my package (objsize):

$ pip install objsize

Then:

>>> from objsize import get_deep_size
>>> get_deep_size(a)
16816

Solution 3:

a bit late to the party but an easy way to get size of dict is to pickle it first.

Using sys.getsizeof on python object (including dictionary) may not be exact since it does not count referenced objects.

The way to handle it is to serialize it into a string and use sys.getsizeof on the string. Result will be much closer to what you want.

import cPickle

mydict = {'key1':'some long string, 'key2':[some, list], 'key3': whatever other data}

doing sys.getsizeof(mydict) is not exact so, pickle it first

mydict_as_string = cPickle.dumps(mydict)

now we can know how much space it takes by

print sys.getsizeof(mydict_as_string)