Skip to content

Instantly share code, notes, and snippets.

@iwalucas
Forked from steezeburger/splitter.sh
Created September 19, 2019 02:07
Show Gist options
  • Save iwalucas/be2cd3773e47f99f09702222c7693537 to your computer and use it in GitHub Desktop.
Save iwalucas/be2cd3773e47f99f09702222c7693537 to your computer and use it in GitHub Desktop.
Bash script for splitting large CSV files into 100 lines while keeping the header.
#!/bin/bash
FILENAME=$1
HDR=$(head -1 ${FILENAME})
split -l 100 ${FILENAME} xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${HDR} > part-${n}-${FILENAME}.csv
fi
cat ${f} >> part-${n}-${FILENAME}.csv
rm ${f}
((n++))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment