JSON filter tool

README.md

JSON filter tool

Syntax

json takes input from stdin and pretty-prints to stdout. With no arguments it behaves like python -mjson.tool.

When given arguments, each argument is either a field selector or a filter.

echo '{ "a": { "b": "c"} }' | json a selects the field a and prints {"b": "c"}

echo '{ "a": [{ "b": "c"}, {"b": "d"}] }' | json a b=d b does the following:

• select the field a
• filter by b=d
• select the field b

and finally prints ["d"]

Examples

curl http://v11.cat.api.dev.streamco.com.au/feeds/programs.json | json entries 'classifications.0.consumerAdvice.0=strong language' title,url

plucks the URL and title of all programs the first consumerAdvice equalling 'strong language' (you can use a regex with ~ instead of =). nulls are gracefully skipped, etc.

[
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0044706.json", 
    "title": "High Noon (1952)"
  }, 
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0040746.json", 
    "title": "Rope (1948)"
  }, 
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0107290.json", 
    "title": "Jurassic Park (1993)"
  }, 
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0103064.json", 
    "title": "Terminator 2: Judgment Day (1991)"
  }, 
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0046359.json", 
    "title": "Stalag 17 (1953)"
  }, 
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0993846.json", 
    "title": "The Wolf of Wall Street (2013)"
  }, 
  {
    "url": "http://v11.cat.api.dev.streamco.com.au/programs/tt0167404.json", 
    "title": "The Sixth Sense (1999)"
  }
]

json

#!/usr/bin/env python
import json
import sys
import re
from operator import eq

def getin(val, field):
    path = field.split('.')
    for p in path:
        if not val: return
        if isinstance(val, list) and p.isdigit() and len(v) > int(p):
            val = val[int(p)]
        elif isinstance(val, dict):
            val = val.get(p)
    return val

def matches(v, key, filter, matchfunc):
    if key.isdigit(): key = int(key)
    result = getin(v, key)
    if not result: return False
    if isinstance(result, list):
        return any(matchfunc(filter, r) for r in result)
    else:
        return matchfunc(filter, result)

def pluck(fieldspec, val):
    if ',' in fieldspec:
        fields = fieldspec.split(',')
        return {field: getin(val, field) for field in fields}
    else:
        return getin(val, fieldspec)

val = json.load(sys.stdin)
for arg in sys.argv[1:]:
    if '~' in arg:
        key, filter = arg.split('~')
        val = [v for v in val if matches(v, key, filter, re.search)]
    elif '=' in arg:
        key, filter = arg.split('=')
        val = [v for v in val if matches(v, key, filter, eq)]
    elif isinstance(val, list):
        val = [pluck(arg, v) for v in val]
    else:
        val = pluck(arg, val)

print json.dumps(val, indent=2)