Skip to content

Instantly share code, notes, and snippets.

@PsyChip PsyChip/import.sh
Created Jul 20, 2018

Embed
What would you like to do?
resumable gigantic csv importer for postgresql. csv column count must match with your table
#!/bin/bash
## usage:
## >import.sh filename.csv table database
folder=__csv
mkdir $folder
split -l 10000 $1 $folder/part
cd $folder
psql -c "copy $2 FROM 'partaa' DELIMITER ',' CSV HEADER;" $3;
rm partaa
for x in $(ls part*); do
echo "importing: $x";
psql -c "copy $2 FROM '$x' DELIMITER ',';" $3;
rm $x
echo "remaining blocks:";
ls | wc -l
done
echo 'completed';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.