Convert Django Model object to dict with all of the fields intact
How does one convert a django Model object to a dict with all of its fields? All ideally includes foreign keys and fields with editable=False.
Let me elaborate. Let's say I have a django model like the following:
from django.db import models
class OtherModel(models.Model): pass
class SomeModel(models.Model):
normal_value = models.IntegerField()
readonly_value = models.IntegerField(editable=False)
auto_now_add = models.DateTimeField(auto_now_add=True)
foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")
In the terminal, I have done the following:
other_model = OtherModel()
other_model.save()
instance = SomeModel()
instance.normal_value = 1
instance.readonly_value = 2
instance.foreign_key = other_model
instance.save()
instance.many_to_many.add(other_model)
instance.save()
I want to convert this to the following dictionary:
{'auto_now_add': datetime.datetime(2015, 3, 16, 21, 34, 14, 926738, tzinfo=<UTC>),
'foreign_key': 1,
'id': 1,
'many_to_many': [1],
'normal_value': 1,
'readonly_value': 2}
Questions with unsatisfactory answers:
Django: Converting an entire set of a Model's objects into a single dictionary
How can I turn Django Model objects into a dictionary and still have their foreign keys?
Solution 1:
There are many ways to convert an instance to a dictionary, with varying degrees of corner case handling and closeness to the desired result.
1. instance.__dict__
instance.__dict__
which returns
{'_foreign_key_cache': <OtherModel: OtherModel object>,
'_state': <django.db.models.base.ModelState at 0x7ff0993f6908>,
'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
'foreign_key_id': 2,
'id': 1,
'normal_value': 1,
'readonly_value': 2}
This is by far the simplest, but is missing many_to_many
, foreign_key
is misnamed, and it has two unwanted extra things in it.
2. model_to_dict
from django.forms.models import model_to_dict
model_to_dict(instance)
which returns
{'foreign_key': 2,
'id': 1,
'many_to_many': [<OtherModel: OtherModel object>],
'normal_value': 1}
This is the only one with many_to_many
, but is missing the uneditable fields.
3. model_to_dict(..., fields=...)
from django.forms.models import model_to_dict
model_to_dict(instance, fields=[field.name for field in instance._meta.fields])
which returns
{'foreign_key': 2, 'id': 1, 'normal_value': 1}
This is strictly worse than the standard model_to_dict
invocation.
4. query_set.values()
SomeModel.objects.filter(id=instance.id).values()[0]
which returns
{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
'foreign_key_id': 2,
'id': 1,
'normal_value': 1,
'readonly_value': 2}
This is the same output as instance.__dict__
but without the extra fields.
foreign_key_id
is still wrong and many_to_many
is still missing.
5. Custom Function
The code for django's model_to_dict
had most of the answer. It explicitly removed non-editable fields, so removing that check and getting the ids of foreign keys for many to many fields results in the following code which behaves as desired:
from itertools import chain
def to_dict(instance):
opts = instance._meta
data = {}
for f in chain(opts.concrete_fields, opts.private_fields):
data[f.name] = f.value_from_object(instance)
for f in opts.many_to_many:
data[f.name] = [i.id for i in f.value_from_object(instance)]
return data
While this is the most complicated option, calling to_dict(instance)
gives us exactly the desired result:
{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
'foreign_key': 2,
'id': 1,
'many_to_many': [2],
'normal_value': 1,
'readonly_value': 2}
6. Use Serializers
Django Rest Framework's ModelSerialzer allows you to build a serializer automatically from a model.
from rest_framework import serializers
class SomeModelSerializer(serializers.ModelSerializer):
class Meta:
model = SomeModel
fields = "__all__"
SomeModelSerializer(instance).data
returns
{'auto_now_add': '2018-12-20T21:34:29.494827Z',
'foreign_key': 2,
'id': 1,
'many_to_many': [2],
'normal_value': 1,
'readonly_value': 2}
This is almost as good as the custom function, but auto_now_add is a string instead of a datetime object.
Bonus Round: better model printing
If you want a django model that has a better python command-line display, have your models child-class the following:
from django.db import models
from itertools import chain
class PrintableModel(models.Model):
def __repr__(self):
return str(self.to_dict())
def to_dict(instance):
opts = instance._meta
data = {}
for f in chain(opts.concrete_fields, opts.private_fields):
data[f.name] = f.value_from_object(instance)
for f in opts.many_to_many:
data[f.name] = [i.id for i in f.value_from_object(instance)]
return data
class Meta:
abstract = True
So, for example, if we define our models as such:
class OtherModel(PrintableModel): pass
class SomeModel(PrintableModel):
normal_value = models.IntegerField()
readonly_value = models.IntegerField(editable=False)
auto_now_add = models.DateTimeField(auto_now_add=True)
foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")
Calling SomeModel.objects.first()
now gives output like this:
{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
'foreign_key': 2,
'id': 1,
'many_to_many': [2],
'normal_value': 1,
'readonly_value': 2}
Solution 2:
I found a neat solution to get to result:
Suppose you have an model object o
:
Just call:
type(o).objects.filter(pk=o.pk).values().first()
Solution 3:
just vars(obj)
, it will state the whole values of the object
>>> obj_attrs = vars(obj)
>>> obj_attrs
{'_file_data_cache': <FileData: Data>,
'_state': <django.db.models.base.ModelState at 0x7f5c6733bad0>,
'aggregator_id': 24,
'amount': 5.0,
'biller_id': 23,
'datetime': datetime.datetime(2018, 1, 31, 18, 43, 58, 933277, tzinfo=<UTC>),
'file_data_id': 797719,
}
You can add this also
>>> keys = obj_attrs.keys()
>>> temp = [obj_attrs.pop(key) if key.startswith('_') else None for key in keys]
>>> del temp
>>> obj_attrs
{
'aggregator_id': 24,
'amount': 5.0,
'biller_id': 23,
'datetime': datetime.datetime(2018, 1, 31, 18, 43, 58, 933277, tzinfo=<UTC>),
'file_data_id': 797719,
}
Solution 4:
@Zags solution was gorgeous!
I would add, though, a condition for datefields in order to make it JSON friendly.
Bonus Round
If you want a django model that has a better python command-line display, have your models child class the following:
from django.db import models
from django.db.models.fields.related import ManyToManyField
class PrintableModel(models.Model):
def __repr__(self):
return str(self.to_dict())
def to_dict(self):
opts = self._meta
data = {}
for f in opts.concrete_fields + opts.many_to_many:
if isinstance(f, ManyToManyField):
if self.pk is None:
data[f.name] = []
else:
data[f.name] = list(f.value_from_object(self).values_list('pk', flat=True))
elif isinstance(f, DateTimeField):
if f.value_from_object(self) is not None:
data[f.name] = f.value_from_object(self).timestamp()
else:
data[f.name] = None
else:
data[f.name] = f.value_from_object(self)
return data
class Meta:
abstract = True
So, for example, if we define our models as such:
class OtherModel(PrintableModel): pass
class SomeModel(PrintableModel):
value = models.IntegerField()
value2 = models.IntegerField(editable=False)
created = models.DateTimeField(auto_now_add=True)
reference1 = models.ForeignKey(OtherModel, related_name="ref1")
reference2 = models.ManyToManyField(OtherModel, related_name="ref2")
Calling SomeModel.objects.first()
now gives output like this:
{'created': 1426552454.926738,
'value': 1, 'value2': 2, 'reference1': 1, u'id': 1, 'reference2': [1]}
Solution 5:
Update
The newer aggregated answer posted by @zags is more complete and elegant than my own. Please refer to that answer instead.
Original
If you are willing to define your own to_dict method like @karthiker suggested, then that just boils this problem down to a sets problem.
>>># Returns a set of all keys excluding editable = False keys
>>>dict = model_to_dict(instance)
>>>dict
{u'id': 1L, 'reference1': 1L, 'reference2': [1L], 'value': 1}
>>># Returns a set of editable = False keys, misnamed foreign keys, and normal keys
>>>otherDict = SomeModel.objects.filter(id=instance.id).values()[0]
>>>otherDict
{'created': datetime.datetime(2014, 2, 21, 4, 38, 51, tzinfo=<UTC>),
u'id': 1L,
'reference1_id': 1L,
'value': 1L,
'value2': 2L}
We need to remove the mislabeled foreign keys from otherDict.
To do this, we can use a loop that makes a new dictionary that has every item except those with underscores in them. Or, to save time, we can just add those to the original dict since dictionaries are just sets under the hood.
>>>for item in otherDict.items():
... if "_" not in item[0]:
... dict.update({item[0]:item[1]})
...
>>>
Thus we are left with the following dict:
>>>dict
{'created': datetime.datetime(2014, 2, 21, 4, 38, 51, tzinfo=<UTC>),
u'id': 1L,
'reference1': 1L,
'reference2': [1L],
'value': 1,
'value2': 2L}
And you just return that.
On the downside, you can't use underscores in your editable=false field names. On the upside, this will work for any set of fields where the user-made fields do not contain underscores.
This is not the best way of doing this, but it could work as a temporary solution until a more direct method is found.
For the example below, dict would be formed based on model_to_dict and otherDict would be formed by filter's values method. I would have done this with the models themselves, but I can't get my machine to accept otherModel.
>>> import datetime
>>> dict = {u'id': 1, 'reference1': 1, 'reference2': [1], 'value': 1}
>>> otherDict = {'created': datetime.datetime(2014, 2, 21, 4, 38, 51), u'id': 1, 'reference1_id': 1, 'value': 1, 'value2': 2}
>>> for item in otherDict.items():
... if "_" not in item[0]:
... dict.update({item[0]:item[1]})
...
>>> dict
{'reference1': 1, 'created': datetime.datetime(2014, 2, 21, 4, 38, 51), 'value2': 2, 'value': 1, 'id': 1, 'reference2': [1]}
>>>
That should put you in a rough ballpark of the answer to your question, I hope.