Bash script for splitting large CSV files into 100 lines while keeping the header.

splitter.sh

#!/bin/bash
FILENAME=file-to-split.csv
HDR=$(head -1 ${FILENAME})
split -l 100 ${FILENAME} xyz
n=1
for f in xyz*
do
    if [[ ${n} -ne 1 ]]; then
        echo ${HDR} > part-${n}-${FILENAME}.csv
    fi
    cat ${f} >> part-${n}-${FILENAME}.csv
    rm ${f}
    ((n++))
done

This writes the header twice in the first file.

Also, there's an error in line 9: missing .csv at the end.

@arobinski Thanks for catching those errors! I've updated the script.

Thanks for posting this.. save me some time.

Thanks.

A couple of improvements can be done though.

1. The first set takes including the header so the data count always stays as n-1 for the first one.
2. adding the extension on lines 9 and 11 makes doubled up when writing the files.