Skip to content

Instantly share code, notes, and snippets.

@senagbe
Forked from steezeburger/splitter.sh
Created February 16, 2022 21:25
Show Gist options
  • Save senagbe/c8fede9f1007c4ce13615f5a26b8d971 to your computer and use it in GitHub Desktop.
Save senagbe/c8fede9f1007c4ce13615f5a26b8d971 to your computer and use it in GitHub Desktop.
Bash script for splitting large CSV files into 100 lines while keeping the header.
#!/bin/bash
FILENAME=file-to-split.csv
HDR=$(head -1 ${FILENAME})
split -l 100 ${FILENAME} xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${HDR} > part-${n}-${FILENAME}.csv
fi
cat ${f} >> part-${n}-${FILENAME}.csv
rm ${f}
((n++))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment