Skip to content

Instantly share code, notes, and snippets.

@charlespwd
Last active November 23, 2020 18:56
Show Gist options
  • Save charlespwd/a4ef9018b12322218a7cec45f77d6214 to your computer and use it in GitHub Desktop.
Save charlespwd/a4ef9018b12322218a7cec45f77d6214 to your computer and use it in GitHub Desktop.
#!/usr/bin/env bash
file=$1
n=$2
if [[ $# -lt 2 ]]; then
cat <<- EOD
Usage: $0 <path> <number_of_files>
EOD
fi
split_csv() {
header=$(head -1 $1)
dir=$(dirname $1)
base=$(basename $1 ".csv")
mkdir -p "$base"
lines=$(wc -l $1 | xargs echo | cut -d ' ' -f1)
chunk=$(node -e "console.log(Math.ceil($lines / $n))")
tail -n +2 $1 | gsplit -l $chunk -d - "$base/$base."
for i in $base/$base*; do
[[ $i = $1 ]] && continue
echo "$header" > $i.csv
cat $i >> $i.csv
rm "$i"
done
}
split_csv $1 $2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment