Often you are handed-out big CSV files and need to sort, filter, update some cells or other tedious work.
You tried sed, jq, and maybe the mighty awk.
But at the end of the day: You’re using the wrong tool for the job.
Here come csvkit’s csvsql, a small Python script to parse your CSV files and create the corresponding database. You now have the best tool, SQL, to answer your data questions:
# Output CREATE TABLE statement
# Here for SQLite:
csvsql -i sqlite data.csv
# Create the DB and insert the data:
csvsql --db sqlite:///data.db --insert data.csv
# And if you just want to run a single query
# no need to create the DB, just run:
csvsql --query "your query here" data.csv | csvlook
# csvlook will prettify the output