Skip to content

Instantly share code, notes, and snippets.

@rockwood
Last active June 29, 2018 00:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rockwood/a1bd27f5e11aeff794060955bac9ce97 to your computer and use it in GitHub Desktop.
Save rockwood/a1bd27f5e11aeff794060955bac9ce97 to your computer and use it in GitHub Desktop.
Split CSV Files
#!/usr/bin/env bash
input_file=$1
header_file=./output/headers.csv
rows_file=./output/rows.csv
mkdir -p ./output
head -1 $input_file > $header_file
tail -n +2 $input_file > $rows_file
split -l 5000 $rows_file ./output/tmp_
index=1
for chunck_filename in ./output/tmp_*; do
cat $header_file $chunck_filename > ./output/${index}.csv
rm $chunck_filename
let index=${index}+1
done
rm $header_file
rm $rows_file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment