Skip to content

Instantly share code, notes, and snippets.

@tanyuan

tanyuan/csv-statistics.md

Created February 29, 2016 03:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tanyuan/e32b743e8d926a7ea800 to your computer and use it in GitHub Desktop.
Save tanyuan/e32b743e8d926a7ea800 to your computer and use it in GitHub Desktop.
Download ZIP
Useful CSV commands.
Raw
csv-statistics.md

CSV Command Line Statistics

tree list a tree view of current directory recursively, better than ls.

Check line counts for all data:

    wc -l *.csv

Install Python toolkit csvkit for csv* commands.

Show CSV as a easy-to-read table:

    csvlook FILE

Compute mean for each column (no header):

    csvstat -H --mean FILE

Compute mean for multiple files:

    cat *.csv | csvstat -H --mean

Find a certain file:

    find DIR -name 'EdgeVibExp-4-1-*'
    find DIR -name '*-user-0001-*'
    find DIR -name '*-date-2016-2-4-*'

Compute mean for certain files:

    cat `find DIR -name '*-user-0002-*'` | csvstat -H --mean

Copy certain files to a directory

    cp `find DIR -name '*-user-0002-*'` DIR

Sort CSV files by column 2:

    csvsort -c 2 FILE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment