Converting XML to JSON using Python?

I've seen a fair share of ungainly XML->JSON code on the web, and having interacted with Stack's users for a bit, I'm convinced that this crowd can help more than the first few pages of Google results can.

So, we're parsing a weather feed, and we need to populate weather widgets on a multitude of web sites. We're looking now into Python-based solutions.

This public weather.com RSS feed is a good example of what we'd be parsing (our actual weather.com feed contains additional information because of a partnership w/them).

In a nutshell, how should we convert XML to JSON using Python?

Solution 1:

xmltodict (full disclosure: I wrote it) can help you convert your XML to a dict+list+string structure, following this "standard". It is Expat-based, so it's very fast and doesn't need to load the whole XML tree in memory.

Once you have that data structure, you can serialize it to JSON:

import xmltodict, json

o = xmltodict.parse('<e> <a>text</a> <a>text</a> </e>')
json.dumps(o) # '{"e": {"a": ["text", "text"]}}'

Solution 2:

There is no "one-to-one" mapping between XML and JSON, so converting one to the other necessarily requires some understanding of what you want to do with the results.

That being said, Python's standard library has several modules for parsing XML (including DOM, SAX, and ElementTree). As of Python 2.6, support for converting Python data structures to and from JSON is included in the json module.

So the infrastructure is there.

Solution 3:

You can use the xmljson library to convert using different XML JSON conventions.

For example, this XML:

<p id="1">text</p>

translates via the BadgerFish convention into this:

{
  'p': {
    '@id': 1,
    '$': 'text'
  }
}

and via the GData convention into this (attributes are not supported):

{
  'p': {
    '$t': 'text'
  }
}

... and via the Parker convention into this (attributes are not supported):

{
  'p': 'text'
}

It's possible to convert from XML to JSON and from JSON to XML using the same conventions:

>>> import json, xmljson
>>> from lxml.etree import fromstring, tostring
>>> xml = fromstring('<p id="1">text</p>')
>>> json.dumps(xmljson.badgerfish.data(xml))
'{"p": {"@id": 1, "$": "text"}}'
>>> xmljson.parker.etree({'ul': {'li': [1, 2]}})
# Creates [<ul><li>1</li><li>2</li></ul>]

Disclosure: I wrote this library. Hope it helps future searchers.

Solution 4:

If some time you get only response code instead of all data then error like json parse will be there so u need to convert it as text

import xmltodict

data = requests.get(url)
xpars = xmltodict.parse(data.text)
json = json.dumps(xpars)
print json

Solution 5:

To anyone that may still need this. Here's a newer, simple code to do this conversion.

from xml.etree import ElementTree as ET

xml    = ET.parse('FILE_NAME.xml')
parsed = parseXmlToJson(xml)


def parseXmlToJson(xml):
  response = {}

  for child in list(xml):
    if len(list(child)) > 0:
      response[child.tag] = parseXmlToJson(child)
    else:
      response[child.tag] = child.text or ''

    # one-liner equivalent
    # response[child.tag] = parseXmlToJson(child) if len(list(child)) > 0 else child.text or ''

  return response