Skip to content

Instantly share code, notes, and snippets.

@martijngastkemper
Last active May 26, 2022 07:54
Show Gist options
  • Star 6 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save martijngastkemper/6d62245565a49081db6024f5f432fe63 to your computer and use it in GitHub Desktop.
Save martijngastkemper/6d62245565a49081db6024f5f432fe63 to your computer and use it in GitHub Desktop.
CSVkit example to convert CSV to SQLLite and query the data
# This example requires CSVkit (https://github.com/wireservice/csvkit). A Python toolset with a lot of very cool CSV tools.
# IMPORTANT NOTE: make sure to use a proper csv file. I had a lot of trouble with a csv file created by a service with Dutch
# as locale. Changing it to US solved the problem. Some locales use comma's to seperate point numbers. A semicolon is then
# used.
# Create table example and Load file-a.csv into it
csvsql --db sqlite:///example.db --table example --insert file-a.csv
# Add an extra file to table example
csvsql --db sqlite:///example.db --table example --no-create --insert file-b.csv
# Query the database with direct CSV output.
sql2csv --db sqlite:///example.db --query 'select avg(age) from example'
# Connect directly to the database
sqlite3 example.db
$ .tables # show the available tables
$ .schema example # show the structure of table example
$ select column-b, count(*) as total from example group by column-b having count(*) > 2;
name age
Lis 32
Vera 27
name age
Micheal 29
Nick 25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment