Skip to content

Instantly share code, notes, and snippets.

@drjerry
Created August 26, 2012 17:22
Show Gist options
  • Star 9 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save drjerry/3481798 to your computer and use it in GitHub Desktop.
Save drjerry/3481798 to your computer and use it in GitHub Desktop.
Converts stream of tabular records to stream of JSON records.
{
if (NR == 1) {
split($0, tags);
if (EC == "") EC = "\"";
}
else {
split($0, vals);
jrec = "{";
for (i = 1; i <= NF; ++i) {
if (vals[i] ~ /[^0-9.]/)
jrec = jrec EC tags[i] EC ":" EC vals[i] EC;
else
jrec = jrec EC tags[i] EC ":" vals[i];
if (i < NF)
jrec = jrec ", ";
else
jrec = jrec "},";
}
print jrec;
}
}
@drjerry
Copy link
Author

drjerry commented Aug 26, 2012

The column headers must be the first record of the stream (or file) and are used as the field keys. By default, in the output, all keys are enclosed in quotes, as are non-numeric values. The field separator of the input stream should be specified via the -F argument.

Example: Convert plain CSV into JSON

 $ cat myfile.csv
 text,number,timestamp
 some text,123.4,2012-08-26 12:00:00
 $ awk -F "," -f tab2json.awk myfile.csv 
 { "text":"some text", "number":123.4, "timestamp":"2012-08-26 12:00:00" },

Example 2:

MySQL uses the tab character as its default field separator:

 $ mysql -e "select * from db.mytable;" |awk -F "\\t" -f tab2json.awk

The Enclosing Character

By default double-quotes are used to enclose string values, but this can be changed to any other delimiter via the EC variable. For example, to transform the CSV file using single-quotes instead of double quotes:

 $ awk -F "," --assign EC=\' -f tab2json.awk myfile.csv 
 { 'text':'some text', 'number':123.4, 'timestamp':'2012-08-26 12:00:00' },

@yasirbam
Copy link

great script,
One small bug, is it possible to remove the last comma on the last record?

@minkymorgan
Copy link

the last comma problem is corrected by adding two lines to the script, seen here:
minkymorgan / tab2json.awk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment