CSV to JSON converter using BASH. Original script from http://blog.secaserver.com/2013/12/convert-csv-json-bash/

csv2json.sh

#!/bin/bash

# CSV to JSON converter using BASH
# original script from https://gist.github.com/dsliberty/3de707bc656cf757a0cb
# Usage ./csv2json.sh input.csv > output.json

shopt -s extglob

input="${1:-/dev/stdin}"
SEP=","

[ -z "${input}" ] && printf "No CSV input file specified" && exit 1
[ ! -e "${input}" ] && printf "Unable to locate ${input}" && exit 1

data=$(sed '/^$/d' "${input}")
line_count=$(printf "${data}" | wc -l)

printf "[\n"
row=0
while IFS=$'\n\r' read -r line; do
    if [[ ${row} -eq 0 ]]; then
        IFS="$SEP" read -ra head_items <<< "${line}"
    else
        IFS="$SEP" read -ra line_items <<< "${line}"
        printf "\t{\n"
        col=0
        for item in "${line_items[@]}"; do
            printf  "\t\t\"${head_items[${col}]}\": "
            case "${item}" in
                \"\")
                    printf "null"
                    ;;
                \"*\")
                    printf "${item}"
                    ;;
                *.*.*.*)
                    printf "\"${item}\""
                    ;;
                null|true|false|+([0-9.]))
                    printf "${item}"
                    ;;
                *)
                    printf "\"${item}\""
                    ;;
            esac
            (( col++ ))
            [[ ${col} -lt ${#head_items[@]} ]] && printf ",\n" || printf "\n"
        done
        printf "\t}"
        [[ ${row} -lt ${line_count} ]] && printf ",\n" || printf "\n"
    fi
    (( row++ ))
done <<< "${data}"
printf "]"

Hi lino,

I have been trying to use your csv to json converter (https://gist.github.com/linosteenkamp/98c5e3557f8654e4a3ed405859b46b83) - which is the best I’ve seen on the web!

But I’m having problems.

It isn’t able to convert certain characters (like "%”)

Eg
http://01.shgrasp.vip/V3MZW/setup/%E4%BB%BB%E6%88%91%E8%A1%8C%E6%B5%8F%E8%A7%88%E5%99%A8.exe
or
"http://157.185.172.139/5072507.s21d-5.faiusrd.com/0/abuiabblgaag2_uotauol7b28gi?amp&f=sx_server.exe&v=1451472344&wsiphost=local&wsrid_tag=5ddf4a04_psmgxytsea1ub80_20673-58204\\\\%BB%BB%E6%88%91\\\\\\\\\\\\\\\\\\\\\\\\"",\\\\\\\ %E6%B5%8F\\\\%25"

So if my CSV was:

alert,test,2021-04-12T23:37:33Z,MaliciousUrl,60,MaliciousUrl,malware,MaliciousUrl,"http://01.shgrasp.vip/V3MZW/setup/%E4%BB%BB%E6%88%91%E8%A1%8C%E6%B5%8F%E8%A7%88%E5%99%A8.exe”

It isn’t able to convert it to json…

Can you please help, because your script is very elegant and very fast…it would be great if it could cater for all strange combinations of characters.

Thanks in advance