Skip to content

Instantly share code, notes, and snippets.

Created Jul 20, 2018
What would you like to do?
resumable gigantic csv importer for postgresql. csv column count must match with your table
## usage:
## > filename.csv table database
mkdir $folder
split -l 10000 $1 $folder/part
cd $folder
psql -c "copy $2 FROM 'partaa' DELIMITER ',' CSV HEADER;" $3;
rm partaa
for x in $(ls part*); do
echo "importing: $x";
psql -c "copy $2 FROM '$x' DELIMITER ',';" $3;
rm $x
echo "remaining blocks:";
ls | wc -l
echo 'completed';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment