Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Extract a JSON value from a BASH script
#!/bin/bash
function jsonval {
temp=`echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop`
echo ${temp##*|}
}
json=`curl -s -X GET http://twitter.com/users/show/$1.json`
prop='profile_image_url'
picurl=`jsonval`
`curl -s -X GET $picurl -o $1.png`
@cjus

This comment has been minimized.

Copy link
Owner Author

@cjus cjus commented Jun 26, 2011

A bash script which demonstrates parsing a JSON string to extract a property value. The script contains a jsonval function which operates on two variables, json and prop. When the script is passed the name of a twitter user it attempts to download the user's profile picture.

@dadicool

This comment has been minimized.

Copy link

@dadicool dadicool commented Dec 2, 2011

Nice works like a charm!

@EthraZa

This comment has been minimized.

Copy link

@EthraZa EthraZa commented Mar 22, 2013

Very nice. Thank you so much.

To return only the field value, just add " | cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g' " to the end of commands string:

function jsonval {
temp=echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop| cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g'
echo ${temp##*|}
}

@xaprb

This comment has been minimized.

Copy link

@xaprb xaprb commented Jan 30, 2014

At a quick glance, looks like some things could be done a little more efficiently -- for example instead of starting up multiple sed processes, just use one and multiple -e arguments to it. And why is awk defined with a -v variable named k that is not used within the awk?

@saulovenancio

This comment has been minimized.

Copy link

@saulovenancio saulovenancio commented Apr 28, 2014

tks guys,

@itstayyab

This comment has been minimized.

Copy link

@itstayyab itstayyab commented Jun 20, 2014

It could far more Easy to do and read with less code as well.

Key for what you want to find

num occurrence of value if it multiple time

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

To print your values give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url 1

To print all values don't give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url

@itstayyab

This comment has been minimized.

Copy link

@itstayyab itstayyab commented Jun 20, 2014

Another Example

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

URL Array to

urlArray=( "curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json")

Run below command with for loop

echo ${urlArray[$x]} | jsonValue profile_image_url 1
echo ${urlArray[$x]} | jsonValue profile_image_url 2

@jorgechen

This comment has been minimized.

Copy link

@jorgechen jorgechen commented Jul 28, 2014

^ Thanks

@nirajkrz

This comment has been minimized.

Copy link

@nirajkrz nirajkrz commented Aug 5, 2014

how to replace the value of a json node in a file?

@nhoad

This comment has been minimized.

Copy link

@nhoad nhoad commented Aug 8, 2014

Could you speak as how this is licensed? An important detail for those wanting to integrate this with existing code :)

@phillpafford

This comment has been minimized.

Copy link

@phillpafford phillpafford commented Aug 23, 2014

Perfect, Thanks!

@anorsich

This comment has been minimized.

Copy link

@anorsich anorsich commented Sep 3, 2014

Thank you!

@fatbattk

This comment has been minimized.

Copy link

@fatbattk fatbattk commented Oct 3, 2014

Thanks for that simpler snippet @itstayyab!
I tweaked this line abit so it matches full quotes. So subtotal will not be matched when $KEY is total. But unfortunately it still suffers if a JSON value has same $KEY value or JSON contains any FS.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
@Jaykah

This comment has been minimized.

Copy link

@Jaykah Jaykah commented Dec 4, 2014

Awesome, thanks!

@vasilemihali

This comment has been minimized.

Copy link

@vasilemihali vasilemihali commented Feb 26, 2015

Great, thanks!!!

@bk138

This comment has been minimized.

Copy link

@bk138 bk138 commented Mar 14, 2015

+1 with the kudos!

@bcopos

This comment has been minimized.

Copy link

@bcopos bcopos commented Jul 31, 2015

I think sed grouping could also be used, something like:

echo $json | sed 's/{.*$key":"*\([0-9a-zA-Z]*\)"*,*.*}/\1/'

And you can probably tweak it so the grouping regex is just .* instead of [0-9a-zA-Z]*

@evandrix

This comment has been minimized.

Copy link

@evandrix evandrix commented Aug 6, 2015

@ilovefood2

This comment has been minimized.

Copy link

@ilovefood2 ilovefood2 commented Aug 26, 2015

hello,

i tried but it only return one instance value, my json file has multiple values under same prop names (play_url)
how can i print all of them?

thanks

@boomshadow

This comment has been minimized.

Copy link

@boomshadow boomshadow commented Aug 31, 2015

I found this Github project incredibly useful for JSON parsing:
https://github.com/trentm/json

@jfig

This comment has been minimized.

Copy link

@jfig jfig commented Jan 12, 2016

Thanks for this, saved my day ;)

@xianlin

This comment has been minimized.

Copy link

@xianlin xianlin commented Jan 23, 2016

I have the same issue as @ikelca mentioned, any solutions by using this script?

@behelit

This comment has been minimized.

Copy link

@behelit behelit commented Apr 28, 2016

none of these solutions work for a json response that includes a URL (i.e: https://some.url")

{"response":{"code":"200","message":"OK"},"item":"https://someurl.com/api/"}

@pmckelvy1

This comment has been minimized.

Copy link

@pmckelvy1 pmckelvy1 commented Apr 28, 2016

works great, thanks

@chenqiangzhishen

This comment has been minimized.

Copy link

@chenqiangzhishen chenqiangzhishen commented Jul 20, 2016

great, thx.

@ggupta01

This comment has been minimized.

Copy link

@ggupta01 ggupta01 commented Apr 28, 2017

Hi,

if i have a json file or json string, and then i need to extract all the key from it, then what changes i need to do?

thanks in advance.

@ORESoftware

This comment has been minimized.

Copy link

@ORESoftware ORESoftware commented Nov 11, 2017

gnar gnar

@otherpirate

This comment has been minimized.

Copy link

@otherpirate otherpirate commented Feb 6, 2018

Genius, thanks

@EricW1970

This comment has been minimized.

Copy link

@EricW1970 EricW1970 commented Apr 10, 2018

Great tips and examples, saved the evening :)... Thanks!

@amedee

This comment has been minimized.

Copy link

@amedee amedee commented Apr 27, 2018

Or just use jq...

@frazras

This comment has been minimized.

Copy link

@frazras frazras commented May 6, 2018

jq is that bomb

@i3laze

This comment has been minimized.

Copy link

@i3laze i3laze commented May 10, 2018

@itstayyab,
Your function jsonValue() works pretty great, but only in Bash! Cause For is not supported in Shell.
Could you please rewrite it with a While loop?

@psmanek

This comment has been minimized.

Copy link

@psmanek psmanek commented Aug 4, 2018

Small modification to display only one result for our $KEY.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'

But... How to display whole result if it contains "," or ":" character?

@joshimanish1986

This comment has been minimized.

Copy link

@joshimanish1986 joshimanish1986 commented Oct 11, 2018

@psmanek. I am facing similar issue. my value string contains , and =. Please let me know if you or anyone on this thread were able to resolve this.

@thisis-mental

This comment has been minimized.

Copy link

@thisis-mental thisis-mental commented Oct 17, 2018

If my json contains a list of object, how can I extract 2 properties of each object?
Want to use one depending of the second one for each object.
Thanks!

@rkewlani

This comment has been minimized.

Copy link

@rkewlani rkewlani commented Nov 26, 2018

nice. works very well.

@wshbair

This comment has been minimized.

Copy link

@wshbair wshbair commented Mar 11, 2019

Great it works very well with adding --silent option in curl

@flinox

This comment has been minimized.

Copy link

@flinox flinox commented Apr 3, 2019

it works! tkz, great job!

@hilmarf

This comment has been minimized.

Copy link

@hilmarf hilmarf commented Sep 2, 2019

{ "array": [ "foo", "bar" ], "other": "entry", "page": 1 }

what do I need to do, to get 'foo' and 'bar' ?
currently jsonValue array returns only foo

@wsieburgh

This comment has been minimized.

Copy link

@wsieburgh wsieburgh commented Dec 3, 2019

I would advise to use jq, like this : cat file.json | jq .array would give:
[
"foo",
"bar"
]

@Akianonymus

This comment has been minimized.

Copy link

@Akianonymus Akianonymus commented Mar 26, 2020

Small modification to display only one result for our $KEY.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'

But... How to display whole result if it contains "," or ":" character?

jsonValue()
{
    num=$2
    grep \""$1"\" | sed  "s/\:/\n/"  | grep -v \""$1"\"  | sed "s/\"\,//g"  | sed 's/["]*$//' | sed 's/[,]*$//'| sed 's/^[ \t]*//'  | sed s/\"// | sed -n "${num}"p
}

Functionally same as @itstayyab, but , and : are also handled

Yes, i know, pretty late to the party.

@impressto

This comment has been minimized.

Copy link

@impressto impressto commented May 21, 2020

Small modification to display only one result for our $KEY.
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'
But... How to display whole result if it contains "," or ":" character?

jsonValue()
{
    num=$2
    grep \""$1"\" | sed  "s/\:/\n/"  | grep -v \""$1"\"  | sed "s/\"\,//g"  | sed 's/["]*$//' | sed 's/[,]*$//'| sed 's/^[ \t]*//'  | sed s/\"// | sed -n "${num}"p
}

Functionally same as @itstayyab, but , and : are also handled

Yes, i know, pretty late to the party.

Works perfectly. Thanks

@Akianonymus

This comment has been minimized.

Copy link

@Akianonymus Akianonymus commented May 22, 2020

Small modification to display only one result for our $KEY.
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -e 's/^[[:space:]]*//'
But... How to display whole result if it contains "," or ":" character?

jsonValue()
{
    num=$2
    grep \""$1"\" | sed  "s/\:/\n/"  | grep -v \""$1"\"  | sed "s/\"\,//g"  | sed 's/["]*$//' | sed 's/[,]*$//'| sed 's/^[ \t]*//'  | sed s/\"// | sed -n "${num}"p
}

Functionally same as @itstayyab, but , and : are also handled
Yes, i know, pretty late to the party.

Works perfectly. Thanks

Just got the notification, glad it helped you. I have made some improvements since the last time.

###################################################
# Method to extract specified field data from json
# Globals: None
# Arguments: 2
#   ${1} - value of field to fetch from json
#   ${2} - Optional, nth number of value from extracted values, by default shows all.
# Input: file | here string | pipe
#   _json_value "Arguments" < file
#   _json_value "Arguments <<< "${varibale}"
#   echo something | _json_value "Arguments"
# Result: print extracted value
###################################################
_json_value() {
    declare LC_ALL=C num="${2:-}"
    grep -o "\"""${1}""\"\:.*" | sed -e "s/.*\"""${1}""\": //" -e 's/[",]*$//' -e 's/["]*$//' -e 's/[,]*$//' -e "s/\"//" -n -e "${num}"p
}

To avoid forking of multiple sed processes

@itstayyab

This comment has been minimized.

Copy link

@itstayyab itstayyab commented May 22, 2020

Got my, mentioned after 6 years :) Glad that I contributed to something.

@grndvl1

This comment has been minimized.

Copy link

@grndvl1 grndvl1 commented Sep 22, 2020

Awesome bit of code. Thank you saved me a ton of time and frustrations. Why there is no string compare like a contains just baffles me. The heavy handed use of grep is so silly. All I wanted was a simple if (something.contains(this)); then this allows me to achieve this by parsing the JSON response and then very easily create the if statement. Thank you! You are a time saver!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.