Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Read YAML file from Bash script
#!/bin/sh
parse_yaml() {
local prefix=$2
local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
-e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" $1 |
awk -F$fs '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, $2, $3);
}
}'
}
#!/bin/sh
# include parse_yaml function
. parse_yaml.sh
# read yaml file
eval $(parse_yaml zconfig.yml "config_")
# access yaml content
echo $config_development_database
development:
adapter: mysql2
encoding: utf8
database: my_database
username: root
password:
@FlipMartin

This comment has been minimized.

Copy link

commented Jun 25, 2014

Hi,
This is great, but is it intended to work only with indentations made of two spaces? Our yaml files use four spaces indentations and it doesn't work with them.
Thanks.

@cedrictemple

This comment has been minimized.

Copy link

commented Jul 23, 2014

Hi,

This is great. Many thanks!

@stoivo

This comment has been minimized.

Copy link

commented Aug 25, 2014

good, works good to me as well

@sasikumar87

This comment has been minimized.

Copy link

commented Sep 10, 2014

Works good, Thanks.

@ghost

This comment has been minimized.

Copy link

commented Sep 16, 2014

Great job, thanks!

@dbenque

This comment has been minimized.

Copy link

commented Jan 9, 2015

Very useful, thanks.

@droath

This comment has been minimized.

Copy link

commented Jan 17, 2015

Nice, good work! Has anybody got it to work with YAML arrays syntax?

arrays:
  array1:
    - str1
    - str2
    - str3
  array2:
    key1: value1
    key2: value2
    key3: value3
  array3: [ str1, str2, str3 ]
@linuxdan

This comment has been minimized.

Copy link

commented Feb 10, 2015

Pardon me for pointing this out, droath, but arrays and array2 are hashes.
Only array1 and array3 are proper arrays.

@klb-rodriguez

This comment has been minimized.

Copy link

commented Mar 16, 2015

@FlipMartin if you are using 4 spaces the variables will get two underscores as delimiter, e.g. $config_development__database instead of $config_development_database

@renanreismartins

This comment has been minimized.

Copy link

commented Aug 6, 2015

thanks a lot!

@etiennemarais

This comment has been minimized.

Copy link

commented Sep 22, 2015

Really good, thank you

@l5x

This comment has been minimized.

Copy link

commented Oct 2, 2015

Made it to work with yaml arrays.

https://gist.github.com/epiloque/8cf512c6d64641bde388

@inodb

This comment has been minimized.

Copy link

commented Oct 9, 2015

Awesome!

@bajzarpa

This comment has been minimized.

Copy link

commented Oct 13, 2015

If you want to indent with 4 spaces you must replace the indent = length($1)/2; to indent = length($1)/7; at the 6th line

@vitorreis

This comment has been minimized.

Copy link

commented Oct 27, 2015

thank you @bajzarpa I was searching this.

@sameersbn

This comment has been minimized.

Copy link

commented Dec 18, 2015

@pkuczynski

This comment has been minimized.

Copy link
Owner Author

commented Jan 19, 2016

Easier option I recommend is using my gem https://github.com/railsconfig/config
Much more advanced but requires Ruby...

@ankitdixit

This comment has been minimized.

Copy link

commented Feb 17, 2016

Can any body please help me understand this script as i do not understand what most of these lines do

@zilongqiu

This comment has been minimized.

Copy link

commented Feb 28, 2016

Awesome ! thanks !

@xiurobert

This comment has been minimized.

Copy link

commented Mar 18, 2016

Does this work with comments?

@The-Judge

This comment has been minimized.

Copy link

commented Apr 1, 2016

Thanks a lot for sharing!

@Gerst20051

This comment has been minimized.

Copy link

commented Apr 4, 2016

👍 love the version that works with yaml arrays

@Eisum

This comment has been minimized.

Copy link

commented Apr 21, 2016

Dose not work with my .yml

file.yml

683:
    activities:
        copying:
            time: 4800
        invention:
            materials:
            -   quantity: 2
                typeID: 20416
            -   quantity: 2
                typeID: 25887
            products:
            -   probability: 0.3
                quantity: 1
                typeID: 39581
            skills:
            -   level: 1
                typeID: 11442
            -   level: 1
                typeID: 11454
            -   level: 1
                typeID: 21790
            time: 63900
        manufacturing:
            materials:
            -   quantity: 2
                typeID: 38
            -   quantity: 4
                typeID: 39
            -   quantity: 500
                typeID: 37
            -   quantity: 2444
                typeID: 36
            -   quantity: 8000
                typeID: 35
            -   quantity: 22222
                typeID: 34
            products:
            -   quantity: 1
                typeID: 582
            skills:
            -   level: 1
                typeID: 3380
            time: 6000
        research_material:
            time: 2100
        research_time:
            time: 2100
    blueprintTypeID: 683
    maxProductionLimit: 30

output

prefix_683________quantity=1
prefix_683________typeID=3380
prefix_683______time=6000
prefix_683____research_material__time=2100
prefix_683____research_time__time=2100
prefix_683__activities__copying__time=4800
prefix_683__blueprintTypeID=683
prefix_683__maxProductionLimit=30
"''prefix_683__maxProductionLimit="30
@seth-reeser

This comment has been minimized.

Copy link

commented May 26, 2016

Take a look at https://github.com/0k/shyaml for those that need more flexibility.

@rc1021

This comment has been minimized.

Copy link

commented Jun 13, 2016

it's work, thank you

@nareshganesan

This comment has been minimized.

Copy link

commented Jul 21, 2016

thanks @pkuczynski. @klb-rodriguez.

Here are the changes which i did to test.sh, for four spaced yaml file.

#!/bin/sh

# include parse_yaml function
. parse_yaml.sh

# read yaml file
eval $(parse_yaml zconfig.yml "config__")

# access yaml content
echo $config__development__database
@iolloyd

This comment has been minimized.

Copy link

commented Sep 9, 2016

@FlipMartin you could pre-process your file to only have 2 spaces s/____/__/g where _ is a space.

@kkozmic-seek

This comment has been minimized.

Copy link

commented Sep 13, 2016

Works well. Would be nice if it could handle comments, like

foo: bar # NOTE: this is bar
@senthilmkumar

This comment has been minimized.

Copy link

commented Oct 24, 2016

how to read 'development:'

@jquinter

This comment has been minimized.

Copy link

commented Oct 26, 2016

if the yaml file contains carriage returns, the processing will fail. In this case, I made a fix on this line (currently line 4)
local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')

so it became
local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034'|tr -d '\015')

the additional pipe removes the carriage return, if any.

@briceburg

This comment has been minimized.

Copy link

commented Nov 15, 2016

FWIW we're working on a multi-purpose tool that allows XML, JSON, and YAML to be consumed in shell. Here's an example on handling multi-content arrays in shell; jasmith590/COP#17 (comment)

@fengtan

This comment has been minimized.

Copy link

commented Nov 25, 2016

Works well but does not seem to be able to parse keys containing hyphens, for instance:

foo-bar:
  key1: value1
  key2: value2

This changeset seems to fix it:

-   local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
+   local s='[[:space:]]*' w='[a-zA-Z0-9_\-]*' fs=$(echo @|tr @ '\034')
@jeangb

This comment has been minimized.

Copy link

commented Mar 2, 2017

Thanks a lot ! Very usefull.

@EasyAsABC123

This comment has been minimized.

Copy link

commented Mar 10, 2017

@fengtanyou shouldn't need to escape inside a character array [\-] can just be [-] in regex

@lermit4

This comment has been minimized.

Copy link

commented Mar 13, 2017

fix for space on path file Yaml
- -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" $1 |
+ -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" "$1" |

@mikefarah

This comment has been minimized.

Copy link

commented Apr 12, 2017

#selfpromotion: If you're looking for something more sophisticated but still portable (no dependencies required and no installation) take a look at https://github.com/mikefarah/yaml

@CelticParser

This comment has been minimized.

Copy link

commented Jul 27, 2017

@pkuczynski I'm trying to use this in a Makefile and having trouble converting the commands - Returns blank.
Any experience?
Thnx!

Makefile:

BASEDIR     =$(CURDIR)
CONFDIR     =$(BASEDIR)
OUTPUTDIR   =$/../build

DEVDIR      =$(OUTPUTDIR)/dev
PRODDIR     =$(OUTPUTDIR)/prod
STAGEDIR    =$(OUTPUTDIR)/stage

DEVCONF     =$(CONFDIR)/config.dev.yml
PRODCONF    =$(CONFDIR)/config.prod.yml
STAGECONF   =$(CONFDIR)/config.stage.yml
USRCONFIG   =$/../usr/config.yaml

dev:
	# include parse_yaml function
	. $(BASEDIR)/parse_yaml.sh
	# read yaml file
	$(eval $(parse_yaml $(USRCONFIG) 'config_'))
	$(eval DEVURL := $(config_baseurl_dev))
	@echo $(DEVURL)
	sleep 1
	rm -rfv $(DEVDIR)
	sleep 1
	$(BUILDER) --config=$(DEVCONF) -b=$(DEVURL)

yaml:

baseurl:
    dev:    "https://some-dev-url/"
    stage:  "https://a-staging-url/"
    prod:   "https://production-url"

Outputs:

# include parse_yaml function
. /home/ubuntu/workspace/app/parse_yaml.sh
# read yaml file
eval 

sleep 1
rm -rfv ../build/dev
sleep 1
hugo --config=/home/ubuntu/workspace/app/config.dev.yml -b=
@zvikorn

This comment has been minimized.

Copy link

commented Aug 1, 2017

How do I change the parse_yaml function to keep parsing more than one record (with same keys):
key:John
value:Smith

key:Adam
value:Lambert

expected output:
John:Smith
Adam:Lambert

@jasperes

This comment has been minimized.

Copy link

commented Aug 10, 2017

To preserve double quotes, i'm trying it:

cat "$1" | sed 's/\"/\\\"/g' |
sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
        -e "s|^\($s\)\($w\)$s[:-]$s\(.*\)$s\$|\1$fs\2$fs\3|p" |
...

Currently i didn't see any problem... someone know better fix?

@bherto39android

This comment has been minimized.

Copy link

commented Aug 11, 2017

similar @zvikorn 's question above I also tried the script using more than 1 yaml in a single file

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: test-ui-deployment
spec:
  replicas: 1
  revisionHistoryLimit: 0
  template:
    metadata:
      labels:
        app: myapp-test-ui
    spec:
      imagePullSecrets:
      - name: myregistrykey
      containers:
      - name: myapp-test-ui
        image: imageImage:1234
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8443      
---
apiVersion: v1
kind: Service
metadata:
  name: test-ui-service
spec:
  type: NodePort
  ports:
    - port: 8443
      nodePort: 31000
  selector:
    app: myapp-test-ui

doing this

#eval $(parse_yaml mytest.yaml)
echo kind: "$kind"

it shows the last kind which is "Service", is there a way to get the first kind "Deployment"? or does it return an array of those record found?

@jasperes

This comment has been minimized.

Copy link

commented Aug 11, 2017

@bherto39android try something like this:

printf("%s%s%s+=(\"%s\")\n", "'"$prefix"'",vn, $2, $3);

@jasperes

This comment has been minimized.

Copy link

commented Aug 11, 2017

Created a project and updated this script, working with list, object list (parcial), inline comments, etc.

Project: https://github.com/jasperes/bash-yaml

current script:

function parse_yaml() {
    local yaml_file=$1
    local prefix=$2
    local s
    local w
    local fs

    s='[[:space:]]*'
    w='[a-zA-Z0-9_]*'
    fs="$(echo @|tr @ '\034')"

    (
        sed -ne 's/--//g; s/\"/\\\"/g; s/\#.*//g; s/\s*$//g;' \
            -e  "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
            -e "s|^\($s\)\($w\)$s[:-]$s\(.*\)$s\$|\1$fs\2$fs\3|p" |
        awk -F"$fs" '{
            indent = length($1)/2;
            if (length($2) == 0) { conj[indent]="+";} else {conj[indent]="";}
            vname[indent] = $2;
            for (i in vname) {if (i > indent) {delete vname[i]}}
                if (length($3) > 0) {
                    vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
                    printf("%s%s%s%s=(\"%s\")\n", "'"$prefix"'",vn, $2, conj[indent-1],$3);
                }
            }' |
        sed 's/_=/+=/g'
    ) < "$yaml_file"
}

edit: this script are updated on my repository.

@nitinsatish

This comment has been minimized.

Copy link

commented Sep 28, 2017

Works Great! Thanks

@Raneomik

This comment has been minimized.

Copy link

commented Nov 16, 2017

@jasperes It's really great, but it deletes double dashes "--" : problematic for example in storing commands in yml config (such as ls -l --human-readable -> ls -l human-readable) and strings after "#" (included, for example "example#1" -> val: "example") in a property values.
Do you have any particular reason to sed -ne 's/--//g; in your code ?
I'd suggest you to also add a space in s/\#.*//g; (-> s/\ #.*//g;) to have the commenting behaviour, but it'll have a side effect : top keys beginning with "#" could be read (is it too bad ?)

@jasperes

This comment has been minimized.

Copy link

commented Nov 19, 2017

@Raneomik i find a solution:

  • for comments: /#.*"/!s| #.*||g; /^#/s|#.*||g;
  • for dashes: /^--/s|--||g;

Here some tests i did...

yaml file:

more_tests:
  double_dashes: --ok
  inline_comment: something #comment
  hash1: a#hash
  hash2: "a#hash"
  hash3: "a #hash"
  hash4: #hi
  hash5: "#hi"
  comment_with_hash: an#hash #with an comment
  comment_in_string: "a string..." #a comment

Result:

more_tests_double_dashes=("--ok")
more_tests_inline_comment=("something")
more_tests_hash1=("a#hash")
more_tests_hash2=("\"a#hash\"")
more_tests_hash3=("\"a #hash\"")
more_tests_hash5=("\"#hi\"")
more_tests_comment_with_hash=("an#hash")
more_tests_comment_in_string=("\"a string...\"")

I have updated my repository.

@jasperes

This comment has been minimized.

Copy link

commented Nov 19, 2017

I just did see another problem, with single quotes. Fixed: /#.*[\"\']/!s| #.*||g;.

@Askerad

This comment has been minimized.

Copy link

commented Dec 27, 2017

Warning : This does not escape ">" and "<" signs. I had some lines with HTML content and it crashed the eval.
(For future reference if anyone has the same problem: i put a grep inside the $() to grab only the line i needed.
eval $(parse_yaml myfile.yml | grep the_line_i_wanted))

@jasperes

This comment has been minimized.

Copy link

commented Jan 1, 2018

@Askerad tested hello: <ok>ok!</ok> with my alterations, commented above, and worked.

@hgomez-sonarsource

This comment has been minimized.

Copy link

commented Jan 16, 2018

@jasperes list didn't seems to works on, I used test/tesh.sh, complex_test_simple_obj_a_list is empty (created by create_variables). But parse_yaml report element members

@jasperes

This comment has been minimized.

Copy link

commented Jan 24, 2018

@hgomez-sonarsource tests fixed.

@wadewegner

This comment has been minimized.

Copy link

commented Feb 10, 2018

It doesn't seem to work with dashes ...

i-have-dashes: true

... as you can't echo ...

echo $config_i-have-dashes

Could this replace dashes with underscores only for names, ignoring the values? So that ...

echo $config_i_have_dashes

... returns true?

I was able to start this by adding ...

-e "s/-/_/g" \

... after line 5 in parse_yaml.sh but unfortunately it also replaces dashes in the value with underscores too. Any other ideas?

@wadewegner

This comment has been minimized.

Copy link

commented Feb 10, 2018

The following update solves my issue ...

parse_yaml() {
  
  local prefix=$2
  local s='[[:space:]]*'
  local w='[a-zA-Z0-9_]*'
  local fs=$(echo @|tr @ '\034')
  
  sed "h;s/^[^:]*//;x;s/:.*$//;y/-/_/;G;s/\n//" $1 |
  sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
      -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" |
  awk -F$fs '{
    indent = length($1)/2;
    vname[indent] = $2;

    for (i in vname) {if (i > indent) {delete vname[i]}}
    if (length($3) > 0) {
        vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
        printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, $2, $3);
    }
  }'
}

... but I've noticed that arrays like the following doesn't work:

data-plans:
  - ./data/YourCustomObject1__c-plan.json
  - ./data/YourCustomObject2__c-plan.json
@4383

This comment has been minimized.

Copy link

commented Feb 12, 2018

Niet is a tool that help you to extract data from json or yaml file directly in your shell/bash CLI.

$ pip install niet

Consider a yaml file named project.yaml with the following contents:

project:
   meta:
       name: project-sample

You can use niet like this:

$ PROJECT_NAME=$(niet project.yaml project.meta.name)
$ echo ${PROJECT_NAME}
project-sample
@wadewegner

This comment has been minimized.

Copy link

commented Feb 12, 2018

What if it was proj-ect instead of project? That's what I'm trying to solve.

UPDATE: it works!

Here's a test.yaml:

test: "test output"
my-test: "my-test output"
my-array:
  - one
  - two

And the output:

❯ niet test.yml test
test output
❯ niet test.yml my-test
my-test output
❯ niet test.yml my-array
'one' 'two'

Sweet!

@4383

This comment has been minimized.

Copy link

commented Apr 5, 2018

@wadewegner Thanks for reply!

A new stable release of niet (1.0.0) is available and fix some bad behaviors like list handling (remove ticle).

You can install/update this by using:

$ pip install -U niet

Consider your example test.yaml:

$ cat test.yaml
# test.yaml
test: "test output"
my-test: "my-test output"
my-array:
  - one
  - two

New niet behavior with update corresponding to your commands:

$ niet test.yml test
test output
$ niet test.yml my-test
my-test output
$ niet test.yml my-array # remove ticle on items
one two
$ for el in $(niet test.yaml my-array); do echo ${el}; done
one
two

Manage niet errors more properly:

$ for el in $(niet test.yaml my-array); do echo ${el}; done
one
two
$ echo $? # exit code is now available so you can handle errors more properly
0
$ niet test.yaml element.not.found
Element not found: element.not.found
$ echo $? # error during execution
1
$ niet fake.yaml test
Yaml file not found! Abort!
$ echo $? # error during execution
1
$ # You can now deal correctly with errors

Advanced usages

$ RESULTS=$(niet test.yaml element.not.found)
$ if [ "$?" == "1" ]; then echo ${RESULTS}; else echo "it's work!"; fi
Element not found: element.not.found
$ RESULTS=$(niet test.yaml my-array)
$ if [ "$?" == "1" ]; then echo ${RESULTS}; else echo "it's work!"; fi
it's work!
$ # implement your own treatments on available results

Also you can test with samples available in niet source code:

$ git clone https://github.com/gr0und-s3ct0r/niet.git
$ cd niet
$ pipenv shell
$ python setup.py install 
$ niet tests/samples/sample.yaml project.meta.tags
example sample for testing purpose
@jainapatel13

This comment has been minimized.

Copy link

commented Apr 20, 2018

If my .ymal file is in the subdirectory how to parse that? my makefile is in top directory and my parse_yml.sh and .ymal files are in subdirectory.

@v1k0d3n

This comment has been minimized.

Copy link

commented Apr 29, 2018

i've been using @jasperes repo with some really good success. for a bash-only method of reading, parsing yaml into variables it's been working great! thanks for the work. i'll have to try out niet as well and see how that turns out. at least in my case, i'm obsessively trying to limit other language requirements as my intention is to run this on container linux. my whole project will switch to golang soon anyway, but @jasperes solution was great for laying out the general concepts for my team. i really appreciate this thread! thanks, everyone.

@byronmansfield

This comment has been minimized.

Copy link

commented Sep 6, 2018

This is fantastic! Thank you all who have contributed to this. It was exactly what I was looking for. I think it was mentioned above by @wadewegner, it has issues with hyphenated hash keys such as proj-ect:. wadewegner you said that your first pure bash change worked for you, but had array issues. I was unsuccessful with using your approach. And unfortunately I am not able to introduce other tooling like python packages such as niet into my project. Has anyone been able to get this working for hyphenated keys?

@misterboe

This comment has been minimized.

Copy link

commented Sep 20, 2018

How could i "echo" all given variables?

@BGMP

This comment has been minimized.

Copy link

commented Feb 3, 2019

Thank you very much!

@djentlguy

This comment has been minimized.

Copy link

commented Mar 13, 2019

Trying to use Martins modified parser from stckovrflow. Input file:

schemas:
- name: exports
  tables:
  - name: wolverine
    description: sample message here
      may overflow next line
    active_date: 2019-01-07 00:00:00
    columns:
    - name: height
      type: timestamp without time zone
    - name: strength
      type: bigint
      description: sample message here
        may overflow next line
      example: 21352352
    - name: power
      type: bigint
      description: sample message here
        may overflow next line
      example: 10001
  - name: cyclops
    description: sample message here
      may overflow next line
    active_date: 2018-12-15 00:00:00
    columns:
    - name: size
      type: datetime
      description: sample message here
        may overflow next line
      example: 2018-03-03 12:30:00
    - name: power
      type: timestamp without time zone
      description: sample message here
        may overflow next line

Parse function:

function parse_yaml {
   local prefix=$2
   local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
   sed -ne "s|,$s\]$s\$|]|" \
        -e ":1;s|^\($s\)\($w\)$s:$s\[$s\(.*\)$s,$s\(.*\)$s\]|\1\2: [\3]\n\1  - \4|;t1" \
        -e "s|^\($s\)\($w\)$s:$s\[$s\(.*\)$s\]|\1\2:\n\1  - \3|;p" $1 | \
   sed -ne "s|,$s}$s\$|}|" \
        -e ":1;s|^\($s\)-$s{$s\(.*\)$s,$s\($w\)$s:$s\(.*\)$s}|\1- {\2}\n\1  \3: \4|;t1" \
        -e    "s|^\($s\)-$s{$s\(.*\)$s}|\1-\n\1  \2|;p" | \
   sed -ne "s|^\($s\):|\1|" \
        -e "s|^\($s\)-$s[\"']\(.*\)[\"']$s\$|\1$fs$fs\2|p" \
        -e "s|^\($s\)-$s\(.*\)$s\$|\1$fs$fs\2|p" \
        -e "s|^\($s\)\($w\)$s:$s[\"']\(.*\)[\"']$s\$|\1$fs\2$fs\3|p" \
        -e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" | \
   awk -F$fs '{
      indent = length($1)/2;
      vname[indent] = $2;
      for (i in vname) {if (i > indent) {delete vname[i]; idx[i]=0}}
      if(length($2)== 0){  vname[indent]= ++idx[indent] };
      if (length($3) > 0) {
         vn=""; for (i=0; i<indent; i++) { vn=(vn)(vname[i])("_")}
         printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, vname[indent], $3);
      }
   }'
}

Current Output :

1="name: exports"
1_1="name: wolverine"
1_1_description="sample message here"
1_1_active_date="2019-01-07 00:00:00"
1_1_1="name: height"
1_1_1_type="timestamp without time zone"
1_1_2="name: strength"
1_1_2_type="bigint"
1_1_2_description="sample message here"
1_1_2_example="21352352"
1_1_3="name: power"
1_1_3_type="bigint"
1_1_3_description="sample message here"
1_1_3_example="10001"
1_2="name: cyclops"
1_2_description="sample message here"
1_2_active_date="2018-12-15 00:00:00"
1_2_1="name: size"
1_2_1_type="datetime"
1_2_1_description="sample message here"
1_2_1_example="2018-03-03 12:30:00"
1_2_2="name: power"
1_2_2_type="timestamp without time zone"
1_2_2_description="sample message here"

But I need to get the values in nested format csv instead of synthetic numbered markers. The idea is to populate a nested relationship from parent to child elements . Anything in the same level will be written delimited by comma like below:

Wolverine.height,"timestamp without time zone",,
Wolverine.strength,bigint,"sample message here may overflow next line",21352352
Wolverine.power,bigint,"sample message here may overflow next line",10001
Cyclops.size,datetime,"sample message here may overflow next line","2018-03-03 12:30:00"
Cyclops.power,"timestamp without time zone",something,"sample message here may overflow next line",,

How can I reformat it? Can someone help ?

@tec82263

This comment has been minimized.

Copy link

commented May 25, 2019

Hi,
I tried to use this in Jenkins for job config parsing.
I ran successfully with Python 2.6.6 but not with Python 2.7.5
Help is really appreciated.

Ran with Python 2.6.6

  • echo 'Get Job Properties'
    Get Job Properties
  • /usr/bin/python -V
    Python 2.6.6
  • cd /bms/webapps/jenkins/workspace/pro-group/test/yaml_parser
  • . parse_yaml.sh
    ++ parse_yaml access_info.yaml accessconfig_

Ran with Python 2.7.5

  • echo 'Get Job Properties'
    Get Job Properties
  • /usr/bin/python -V
    Python 2.7.5
  • cd /bms/webapps/jenkins/workspace/pro-group/test/yaml_parser
  • . parse_yaml.sh
    /bms/webapps/jenkins/jenkins881879139786658308.sh: line 29: .: parse_yaml.sh: file not found
    Build step 'Execute shell' marked build as failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.