Extract a JSON value from a BASH script

jsonval.sh

#!/bin/bash
function jsonval {
    temp=`echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop`
    echo ${temp##*|}
}

json=`curl -s -X GET http://twitter.com/users/show/$1.json`
prop='profile_image_url'
picurl=`jsonval`

`curl -s -X GET $picurl -o $1.png`

A bash script which demonstrates parsing a JSON string to extract a property value. The script contains a jsonval function which operates on two variables, json and prop. When the script is passed the name of a twitter user it attempts to download the user's profile picture.

Nice works like a charm!

Very nice. Thank you so much.

To return only the field value, just add " | cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g' " to the end of commands string:

function jsonval {
temp=echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop| cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g'
echo ${temp##*|}
}

At a quick glance, looks like some things could be done a little more efficiently -- for example instead of starting up multiple sed processes, just use one and multiple -e arguments to it. And why is awk defined with a -v variable named k that is not used within the awk?

tks guys,

It could far more Easy to do and read with less code as well.

Key for what you want to find

num occurrence of value if it multiple time

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

To print your values give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url 1

To print all values don't give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url

Another Example

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

URL Array to

urlArray=( "curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json")

Run below command with for loop

echo ${urlArray[$x]} | jsonValue profile_image_url 1
echo ${urlArray[$x]} | jsonValue profile_image_url 2

^ Thanks

how to replace the value of a json node in a file?

Could you speak as how this is licensed? An important detail for those wanting to integrate this with existing code :)

Perfect, Thanks!

Thank you!

Thanks for that simpler snippet @itstayyab!
I tweaked this line abit so it matches full quotes. So subtotal will not be matched when $KEY is total. But unfortunately it still suffers if a JSON value has same $KEY value or JSON contains any FS.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p

Awesome, thanks!

Great, thanks!!!

+1 with the kudos!

I think sed grouping could also be used, something like:

echo $json | sed 's/{.*$key":"*\([0-9a-zA-Z]*\)"*,*.*}/\1/'

And you can probably tweak it so the grouping regex is just .* instead of [0-9a-zA-Z]*

How about jq @ http://stedolan.github.io/jq/manual?

hello,

i tried but it only return one instance value, my json file has multiple values under same prop names (play_url)
how can i print all of them?

thanks

I found this Github project incredibly useful for JSON parsing:
https://github.com/trentm/json

Thanks for this, saved my day ;)

I have the same issue as @ikelca mentioned, any solutions by using this script?

none of these solutions work for a json response that includes a URL (i.e: https://some.url")

{"response":{"code":"200","message":"OK"},"item":"https://someurl.com/api/"}

works great, thanks

great, thx.

Hi,

if i have a json file or json string, and then i need to extract all the key from it, then what changes i need to do?

thanks in advance.

gnar gnar

Genius, thanks

Great tips and examples, saved the evening :)... Thanks!

Or just use jq...

jq is that bomb

@itstayyab,
Your function jsonValue() works pretty great, but only in Bash! Cause For is not supported in Shell.
Could you please rewrite it with a While loop?

Small modification to display only one result for our $KEY.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'

But... How to display whole result if it contains "," or ":" character?

@psmanek. I am facing similar issue. my value string contains , and =. Please let me know if you or anyone on this thread were able to resolve this.

If my json contains a list of object, how can I extract 2 properties of each object?
Want to use one depending of the second one for each object.
Thanks!

nice. works very well.

Great it works very well with adding --silent option in curl

it works! tkz, great job!

{ "array": [ "foo", "bar" ], "other": "entry", "page": 1 }

what do I need to do, to get 'foo' and 'bar' ?
currently jsonValue array returns only foo

I would advise to use jq, like this : cat file.json | jq .array would give:
[
"foo",
"bar"
]

Small modification to display only one result for our $KEY.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'

But... How to display whole result if it contains "," or ":" character?

jsonValue()
{
    num=$2
    grep \""$1"\" | sed  "s/\:/\n/"  | grep -v \""$1"\"  | sed "s/\"\,//g"  | sed 's/["]*$//' | sed 's/[,]*$//'| sed 's/^[ \t]*//'  | sed s/\"// | sed -n "${num}"p
}

Functionally same as @itstayyab, but , and : are also handled

Yes, i know, pretty late to the party.

Small modification to display only one result for our $KEY.
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'
But... How to display whole result if it contains "," or ":" character?

jsonValue()
{
    num=$2
    grep \""$1"\" | sed  "s/\:/\n/"  | grep -v \""$1"\"  | sed "s/\"\,//g"  | sed 's/["]*$//' | sed 's/[,]*$//'| sed 's/^[ \t]*//'  | sed s/\"// | sed -n "${num}"p
}

Functionally same as @itstayyab, but , and : are also handled

Yes, i know, pretty late to the party.

Works perfectly. Thanks

Small modification to display only one result for our $KEY.
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'
But... How to display whole result if it contains "," or ":" character?

jsonValue()
{
    num=$2
    grep \""$1"\" | sed  "s/\:/\n/"  | grep -v \""$1"\"  | sed "s/\"\,//g"  | sed 's/["]*$//' | sed 's/[,]*$//'| sed 's/^[ \t]*//'  | sed s/\"// | sed -n "${num}"p
}

Functionally same as @itstayyab, but , and : are also handled
Yes, i know, pretty late to the party.

Works perfectly. Thanks

Just got the notification, glad it helped you. I have made some improvements since the last time.

###################################################
# Method to extract specified field data from json
# Globals: None
# Arguments: 2
#   ${1} - value of field to fetch from json
#   ${2} - Optional, nth number of value from extracted values, by default shows all.
# Input: file | here string | pipe
#   _json_value "Arguments" < file
#   _json_value "Arguments <<< "${varibale}"
#   echo something | _json_value "Arguments"
# Result: print extracted value
###################################################
_json_value() {
    declare LC_ALL=C num="${2:-}"
    grep -o "\"""${1}""\"\:.*" | sed -e "s/.*\"""${1}""\": //" -e 's/[",]*$//' -e 's/["]*$//' -e 's/[,]*$//' -e "s/\"//" -n -e "${num}"p
}

To avoid forking of multiple sed processes

Got my, mentioned after 6 years :) Glad that I contributed to something.

Awesome bit of code. Thank you saved me a ton of time and frustrations. Why there is no string compare like a contains just baffles me. The heavy handed use of grep is so silly. All I wanted was a simple if (something.contains(this)); then this allows me to achieve this by parsing the JSON response and then very easily create the if statement. Thank you! You are a time saver!

Given this JSON in data.json:

{
  "id": "001",
  "color": "blue"
}

And I want to extract value "blue" from JSON property color in Windows git bash (no jq dependency), here's how I do it in a pinch:

grep color data.json | cut -d'"' -f4

Another way to do it:
https://raymii.org/s/snippets/Get_json_value_with_sed.html