Create a gist now

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Extract a JSON value from a BASH script
#!/bin/bash
function jsonval {
temp=`echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop`
echo ${temp##*|}
}
json=`curl -s -X GET http://twitter.com/users/show/$1.json`
prop='profile_image_url'
picurl=`jsonval`
`curl -s -X GET $picurl -o $1.png`
@cjus

This comment has been minimized.

Show comment
Hide comment
@cjus

cjus Jun 26, 2011

A bash script which demonstrates parsing a JSON string to extract a property value. The script contains a jsonval function which operates on two variables, json and prop. When the script is passed the name of a twitter user it attempts to download the user's profile picture.

Owner

cjus commented Jun 26, 2011

A bash script which demonstrates parsing a JSON string to extract a property value. The script contains a jsonval function which operates on two variables, json and prop. When the script is passed the name of a twitter user it attempts to download the user's profile picture.

@dadicool

This comment has been minimized.

Show comment
Hide comment
@dadicool

dadicool Dec 2, 2011

Nice works like a charm!

dadicool commented Dec 2, 2011

Nice works like a charm!

@EthraZa

This comment has been minimized.

Show comment
Hide comment
@EthraZa

EthraZa Mar 22, 2013

Very nice. Thank you so much.

To return only the field value, just add " | cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g' " to the end of commands string:

function jsonval {
temp=echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop| cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g'
echo ${temp##*|}
}

EthraZa commented Mar 22, 2013

Very nice. Thank you so much.

To return only the field value, just add " | cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g' " to the end of commands string:

function jsonval {
temp=echo $json | sed 's/\\\\\//\//g' | sed 's/[{}]//g' | awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | sed 's/\"\:\"/\|/g' | sed 's/[\,]/ /g' | sed 's/\"//g' | grep -w $prop| cut -d":" -f2| sed -e 's/^ *//g' -e 's/ *$//g'
echo ${temp##*|}
}

@xaprb

This comment has been minimized.

Show comment
Hide comment
@xaprb

xaprb Jan 30, 2014

At a quick glance, looks like some things could be done a little more efficiently -- for example instead of starting up multiple sed processes, just use one and multiple -e arguments to it. And why is awk defined with a -v variable named k that is not used within the awk?

xaprb commented Jan 30, 2014

At a quick glance, looks like some things could be done a little more efficiently -- for example instead of starting up multiple sed processes, just use one and multiple -e arguments to it. And why is awk defined with a -v variable named k that is not used within the awk?

@saulovenancio

This comment has been minimized.

Show comment
Hide comment

tks guys,

@itstayyab

This comment has been minimized.

Show comment
Hide comment
@itstayyab

itstayyab Jun 20, 2014

It could far more Easy to do and read with less code as well.

Key for what you want to find

num occurrence of value if it multiple time

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

To print your values give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url 1

To print all values don't give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url

It could far more Easy to do and read with less code as well.

Key for what you want to find

num occurrence of value if it multiple time

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

To print your values give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url 1

To print all values don't give second argument

curl -s -X GET http://twitter.com/users/show/$1.json | jsonValue profile_image_url

@itstayyab

This comment has been minimized.

Show comment
Hide comment
@itstayyab

itstayyab Jun 20, 2014

Another Example

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

URL Array to

urlArray=( "curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json")

Run below command with for loop

echo ${urlArray[$x]} | jsonValue profile_image_url 1
echo ${urlArray[$x]} | jsonValue profile_image_url 2

Another Example

function jsonValue() {
KEY=$1
num=$2
awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
}

URL Array to

urlArray=( "curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json"
"curl -s -X GET http://twitter.com/users/show/$1.json")

Run below command with for loop

echo ${urlArray[$x]} | jsonValue profile_image_url 1
echo ${urlArray[$x]} | jsonValue profile_image_url 2

@jorgechen

This comment has been minimized.

Show comment
Hide comment

^ Thanks

@nirajkrz

This comment has been minimized.

Show comment
Hide comment
@nirajkrz

nirajkrz Aug 5, 2014

how to replace the value of a json node in a file?

nirajkrz commented Aug 5, 2014

how to replace the value of a json node in a file?

@nathan-hoad

This comment has been minimized.

Show comment
Hide comment
@nathan-hoad

nathan-hoad Aug 8, 2014

Could you speak as how this is licensed? An important detail for those wanting to integrate this with existing code :)

Could you speak as how this is licensed? An important detail for those wanting to integrate this with existing code :)

@phillpafford

This comment has been minimized.

Show comment
Hide comment
@phillpafford

phillpafford Aug 23, 2014

Perfect, Thanks!

Perfect, Thanks!

@anorsich

This comment has been minimized.

Show comment
Hide comment
@anorsich

anorsich Sep 3, 2014

Thank you!

anorsich commented Sep 3, 2014

Thank you!

@fatbattk

This comment has been minimized.

Show comment
Hide comment
@fatbattk

fatbattk Oct 3, 2014

Thanks for that simpler snippet @itstayyab!
I tweaked this line abit so it matches full quotes. So subtotal will not be matched when $KEY is total. But unfortunately it still suffers if a JSON value has same $KEY value or JSON contains any FS.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p

fatbattk commented Oct 3, 2014

Thanks for that simpler snippet @itstayyab!
I tweaked this line abit so it matches full quotes. So subtotal will not be matched when $KEY is total. But unfortunately it still suffers if a JSON value has same $KEY value or JSON contains any FS.

awk -F"[,:}]" '{for(i=1;i<=NF;i++){if($i~/\042'$KEY'\042/){print $(i+1)}}}' | tr -d '"' | sed -n ${num}p
@Jaykah

This comment has been minimized.

Show comment
Hide comment
@Jaykah

Jaykah Dec 4, 2014

Awesome, thanks!

Jaykah commented Dec 4, 2014

Awesome, thanks!

@vasilemihali

This comment has been minimized.

Show comment
Hide comment
@vasilemihali

vasilemihali Feb 26, 2015

Great, thanks!!!

Great, thanks!!!

@bk138

This comment has been minimized.

Show comment
Hide comment
@bk138

bk138 Mar 14, 2015

+1 with the kudos!

bk138 commented Mar 14, 2015

+1 with the kudos!

@bcopos

This comment has been minimized.

Show comment
Hide comment
@bcopos

bcopos Jul 31, 2015

I think sed grouping could also be used, something like:

echo $json | sed 's/{.*$key":"*\([0-9a-zA-Z]*\)"*,*.*}/\1/'

And you can probably tweak it so the grouping regex is just .* instead of [0-9a-zA-Z]*

bcopos commented Jul 31, 2015

I think sed grouping could also be used, something like:

echo $json | sed 's/{.*$key":"*\([0-9a-zA-Z]*\)"*,*.*}/\1/'

And you can probably tweak it so the grouping regex is just .* instead of [0-9a-zA-Z]*

@evandrix

This comment has been minimized.

Show comment
Hide comment
@ikelca

This comment has been minimized.

Show comment
Hide comment
@ikelca

ikelca Aug 26, 2015

hello,

i tried but it only return one instance value, my json file has multiple values under same prop names (play_url)
how can i print all of them?

thanks

ikelca commented Aug 26, 2015

hello,

i tried but it only return one instance value, my json file has multiple values under same prop names (play_url)
how can i print all of them?

thanks

@boomshadow

This comment has been minimized.

Show comment
Hide comment
@boomshadow

boomshadow Aug 31, 2015

I found this Github project incredibly useful for JSON parsing:
https://github.com/trentm/json

I found this Github project incredibly useful for JSON parsing:
https://github.com/trentm/json

@jfig

This comment has been minimized.

Show comment
Hide comment
@jfig

jfig Jan 12, 2016

Thanks for this, saved my day ;)

jfig commented Jan 12, 2016

Thanks for this, saved my day ;)

@xianlin

This comment has been minimized.

Show comment
Hide comment
@xianlin

xianlin Jan 23, 2016

I have the same issue as @ikelca mentioned, any solutions by using this script?

xianlin commented Jan 23, 2016

I have the same issue as @ikelca mentioned, any solutions by using this script?

@behelit

This comment has been minimized.

Show comment
Hide comment
@behelit

behelit Apr 28, 2016

none of these solutions work for a json response that includes a URL (i.e: https://some.url")

{"response":{"code":"200","message":"OK"},"item":"https://someurl.com/api/"}

behelit commented Apr 28, 2016

none of these solutions work for a json response that includes a URL (i.e: https://some.url")

{"response":{"code":"200","message":"OK"},"item":"https://someurl.com/api/"}

@pmckelvy1

This comment has been minimized.

Show comment
Hide comment
@pmckelvy1

pmckelvy1 Apr 28, 2016

works great, thanks

works great, thanks

@chenqiangzhishen

This comment has been minimized.

Show comment
Hide comment

great, thx.

@ggupta01

This comment has been minimized.

Show comment
Hide comment
@ggupta01

ggupta01 Apr 28, 2017

Hi,

if i have a json file or json string, and then i need to extract all the key from it, then what changes i need to do?

thanks in advance.

Hi,

if i have a json file or json string, and then i need to extract all the key from it, then what changes i need to do?

thanks in advance.

@ORESoftware

This comment has been minimized.

Show comment
Hide comment

gnar gnar

@otherpirate

This comment has been minimized.

Show comment
Hide comment
@otherpirate

otherpirate Feb 6, 2018

Genius, thanks

Genius, thanks

@EricW1970

This comment has been minimized.

Show comment
Hide comment
@EricW1970

EricW1970 Apr 10, 2018

Great tips and examples, saved the evening :)... Thanks!

Great tips and examples, saved the evening :)... Thanks!

@amedee

This comment has been minimized.

Show comment
Hide comment
@amedee

amedee Apr 27, 2018

Or just use jq...

amedee commented Apr 27, 2018

Or just use jq...

@frazras

This comment has been minimized.

Show comment
Hide comment
@frazras

frazras May 6, 2018

jq is that bomb

frazras commented May 6, 2018

jq is that bomb

@i3laze

This comment has been minimized.

Show comment
Hide comment
@i3laze

i3laze May 10, 2018

@itstayyab,
Your function jsonValue() works pretty great, but only in Bash! Cause For is not supported in Shell.
Could you please rewrite it with a While loop?

i3laze commented May 10, 2018

@itstayyab,
Your function jsonValue() works pretty great, but only in Bash! Cause For is not supported in Shell.
Could you please rewrite it with a While loop?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment