JSON Parser Using Python

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. More information about JSON can be found at http://www.json.org/. Yahoo's Douglas Crockford developed this format and within a short period, many web based services have adopted JSON as the primary data interchange format. In this log, we will look at one such service.

Background

There are a bunch of JSON parsers out there, such as:

• simplejson
• json-py

An example JSON document appears as follows:

{
  "Image": {
  "Width": 800,
  "Height": 600,
  "Title": "View from 15th Floor",
  "Thumbnail": {
    "Url": "http://www.example.com/image/481989943",
    "Height": 125,
    "Width": "100"
    },
  "IDs": [116, 943, 234, 38793]
  }
}

For a Python developer, a JSON document is nothing but a dictionary with each element being either an element, array or another dictionary. We can use this fact to our advantage, by simply using the eval() function provided by Python to not only parse this document, but to create a data structure in one statement!

The following code snippet shows how to use the eval() function to parse JSON object. This snippet is based on http://developer.yahoo.com/python/python-json.html:

# If needed set the proxy address
import os
os.environ['http_proxy'] = 'http://[proxy_host]:[proxy_port]/'
import urllib
APP_ID = 'YahooDemo' # Change this to your API key
SEARCH_BASE = 'http://api.search.yahoo.com/WebSearchService/V1/webSearch'

class YahooSearchError(Exception):
    pass

def search(query, results=20, start=1, **kwargs):
    kwargs.update({
        'appid': APP_ID,
        'query': query,
        'results': results,
        'start': start,
        'output': 'json'
    })
    url = SEARCH_BASE + '?' + urllib.urlencode(kwargs)
    f = urllib.urlopen(url)
    buff = f.read().replace('\\/', '/')
    f.close()
    result = eval(buff)
    if 'Error' in result:
        # An error occurred; raise an exception
        raise YahooSearchError, result['Error']
    return result['ResultSet']

info = search('json python')
results = info['Result']
for result in results:
    print result['Title'], result['Url']

Caveats

The eval() function is not safe from programmer errors. Putting a try...catch around the function call fixes that problem. But there is another sinister problem with eval(), not unlike SQL. The eval() function will not distinguish between a JSON object and a malicious statement. Consider the following:

import os
.
.
eval("os.remove('something_very_important')")

Care should be taken to use eval when the string to be evaluated is from a trusted source. I consider Yahoo! or Google servers trusted.

Summary

This article briefly touches upon using the eval() function to understand JSON documents. This will be further elaborated when we will look at Google Local client-server interactions later. I hope you find this trick useful...