Skip to content

Instantly share code, notes, and snippets.

@wutangpaul

wutangpaul/split_csv.sh

Created April 13, 2021 14:18
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save wutangpaul/90e6964a25ac0fa6ac9e99ce7ba97dbb to your computer and use it in GitHub Desktop.
Save wutangpaul/90e6964a25ac0fa6ac9e99ce7ba97dbb to your computer and use it in GitHub Desktop.
Download ZIP
Split a large CSV file into smaller files
Raw
split_csv.sh
#!/bin/bash
FILENAME=orders.csv
HDR=$(head -1 $FILENAME)
split -l 5000 $FILENAME xyz
n=1
for f in xyz*
do
if [ $n -gt 1 ]; then
echo $HDR > Part${n}-$FILENAME
fi
cat $f >> Part${n}-$FILENAME
rm $f
((n++))
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment