Skip to content

Instantly share code, notes, and snippets.

@podhy
Last active October 17, 2017 13:19
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save podhy/4bd37e38022fc6ad534a85b8d3e6743e to your computer and use it in GitHub Desktop.
Save podhy/4bd37e38022fc6ad534a85b8d3e6743e to your computer and use it in GitHub Desktop.
Psql parallel processing in Bash
#!/bin/bash
workers=4
STEP=$(psql -U svetluska -c "SELECT ceil(COUNT(*) / $workers) FROM svetluska.domination" -qtA svetluska)
time for ((i=0; i<$workers; i++))
do
echo "UPDATE svetluska.domination SET geometry = t.geometry_wkt FROM (SELECT id, geometry_wkt FROM svetluska.domination ORDER BY id LIMIT $STEP OFFSET $STEP*$i) t WHERE domination.id = t.id";
done | xargs -P $workers -I % psql -U svetluska -c "%" svetluska
#done | xargs -P 3 -I % echo %
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment