Skip to content

Instantly share code, notes, and snippets.

@phoolish
Forked from steezeburger/splitter.sh
Created June 13, 2019 15:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save phoolish/59965dac1b88d275f520347433d03b99 to your computer and use it in GitHub Desktop.
Save phoolish/59965dac1b88d275f520347433d03b99 to your computer and use it in GitHub Desktop.
Bash script for splitting large CSV files into 100 lines while keeping the header.
#!/bin/bash
FILENAME=file-to-split.csv
HDR=$(head -1 ${FILENAME})
split -l 100 ${FILENAME} xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${HDR} > part-${n}-${FILENAME}.csv
fi
cat ${f} >> part-${n}-${FILENAME}.csv
rm ${f}
((n++))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment