Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save damc-dev/29d3a162c53265cd9e3590fa28f756c7 to your computer and use it in GitHub Desktop.
Save damc-dev/29d3a162c53265cd9e3590fa28f756c7 to your computer and use it in GitHub Desktop.
Bash CSV 'split' Example
# The following will split a CSV (file.csv) into multiple parts of 1 million lines each
# with each part having its own header.
#
# PREFIX denotes the filename to use for the parts. A number will be added to the end.
tail -n +2 file.csv |
split -d -l 1000000 - --filter='sh -c "{ head -n1 file.csv; cat; } > $FILE"' PREFIX
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment