Skip to content

Instantly share code, notes, and snippets.

@RhinoDevel
Forked from mdeweerd/update_database.sh
Last active June 9, 2020 18:11
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save RhinoDevel/8a35ebd2a08166f328eca01ab005c6de to your computer and use it in GitHub Desktop.
Save RhinoDevel/8a35ebd2a08166f328eca01ab005c6de to your computer and use it in GitHub Desktop.
Updated to make it work with Nominatim v3.2.0, optionally with Photon v0.3.0 and other changes.
#!/bin/bash -xv
# Modified version of script by mdeweerd (originally by spin0us)
# to work with Nominatim v3.2.0, etc.
#
# Source: https://gist.githubusercontent.com/mdeweerd/9bc5f60f2d6733e907f3/raw/cf307d83adb2b308fc00e612ff58875dcb4972e0/update_database.sh
# Hint:
#
# Use "bashdb ./update_database.sh" and bashdb's "next" command for step-by-step
# execution.
# ******************************************************************************
# REPLACE WITH LIST OF YOUR "COUNTRIES":
#
COUNTRIES="europe/germany/hamburg europe/germany/schleswig-holstein"
# SET TO YOUR NOMINATIM build FOLDER PATH:
#
NOMINATIMBUILD="/home/developer/Documents/Nominatim-3.2.0/build"
# SET TO YOUR update data FOLDER PATH:
#
UPDATEDIR="/home/developer/Documents/map_update/nominatim/update"
UPDATEBASEURL="https://download.geofabrik.de"
UPDATECOUNTRYPOSTFIX="-updates"
# If you do not use Photon, let Nominatim handle (re-)indexing:
#
FOLLOWUP="$NOMINATIMBUILD/utils/update.php --index"
#
# If you use Photon, update Photon and let it handle the index
# (Photon server must be running and must have been started with "-database",
# "-user" and "-password" parameters):
#
#FOLLOWUP="curl http://localhost:2322/nominatim-update"
# ******************************************************************************
UPDATEDONE=0 # 0 = no, 1 = yes.
# For each country, check, if configuration exists (if not, create one)
# and then import the diff:
#
for COUNTRY in $COUNTRIES;
do
DIR="$UPDATEDIR/$COUNTRY"
FILE="$DIR/configuration.txt"
BASEURL="$UPDATEBASEURL/$COUNTRY$UPDATECOUNTRYPOSTFIX"
FILENAME=${COUNTRY//[\/]/_}
if [ ! -f ${FILE} ];
then
mkdir -p ${DIR}
osmosis --rrii workingDirectory=${DIR}/.
echo baseUrl=${BASEURL} > ${FILE}
echo maxInterval = 0 >> ${FILE}
cd ${DIR}
wget ${BASEURL}/state.txt
fi
osmosis --rri workingDirectory=${DIR}/. --wxc ${DIR}/${FILENAME}.osc.gz
# For each diff file, do the import:
#
LIST1=$DIR/*.osc.gz # Maybe not necessary (file empty?)?
LIST2=*.osc.gz
LIST="$LIST1 $LIST2"
for OSC in $LIST;
do
if [ -f ${OSC} ];
then
${NOMINATIMBUILD}/utils/update.php --import-diff ${OSC}
rm ${OSC}
UPDATEDONE=1 # 0 = No, 1 = Yes.
fi
done
done
# Re-index, if (maybe) needed:
#
if ((${UPDATEDONE}));
then
${FOLLOWUP}
fi
@GoNzCiD
Copy link

GoNzCiD commented Feb 19, 2020

Hi, I'm using this script and it was woking in the past, but I have an error when executing /app/src/build/utils/update.php --import-diff xxx.osc.gz

Processing: Node(80k 20.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)DB writer thread failed due to ERROR: result COPY_END for planet_osm_nodes failed: ERROR:  duplicate key value violates unique constraint "planet_osm_nodes_pkey"
DETAIL:  Key (id)=(434418828) already exists.
CONTEXT:  COPY planet_osm_nodes, line 675

I was looking about the error and it seems to be external to the script but I would like to ask if you know anything about it?

Thanks

@RhinoDevel
Copy link
Author

Hi, I'm using this script and it was woking in the past, but I have an error when executing /app/src/build/utils/update.php --import-diff xxx.osc.gz

Processing: Node(80k 20.0k/s) Way(0k 0.00k/s) Relation(0 0.00/s)DB writer thread failed due to ERROR: result COPY_END for planet_osm_nodes failed: ERROR:  duplicate key value violates unique constraint "planet_osm_nodes_pkey"
DETAIL:  Key (id)=(434418828) already exists.
CONTEXT:  COPY planet_osm_nodes, line 675

I was looking about the error and it seems to be external to the script but I would like to ask if you know anything about it?

Thanks

Sorry, did not happen here, yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment