Skip to content

Instantly share code, notes, and snippets.

Created July 20, 2018 11:54
What would you like to do?
resumable gigantic csv importer for postgresql. csv column count must match with your table
## usage:
## > filename.csv table database
mkdir $folder
split -l 10000 $1 $folder/part
cd $folder
psql -c "copy $2 FROM 'partaa' DELIMITER ',' CSV HEADER;" $3;
rm partaa
for x in $(ls part*); do
echo "importing: $x";
psql -c "copy $2 FROM '$x' DELIMITER ',';" $3;
rm $x
echo "remaining blocks:";
ls | wc -l
echo 'completed';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment