Skip to content

Instantly share code, notes, and snippets.

@husseycoding
Last active April 27, 2019 20:13
Show Gist options
  • Save husseycoding/7ac5cea12017cc3e0ad7 to your computer and use it in GitHub Desktop.
Save husseycoding/7ac5cea12017cc3e0ad7 to your computer and use it in GitHub Desktop.
A bash script for syncing Magento 1 databases
#!/bin/bash
#### - log function - ####
function logoutput {
case $output in
t)
echo "$(date): $1"
;;
b)
echo "$(date): $1"
echo "$(date): $1" >> var/log/syncmagento.log
;;
l)
echo "$(date): $1" >> var/log/syncmagento.log
;;
esac
}
#### - cleanup from previous run - ####
if [[ -f /tmp/syncmagento.sql ]]; then
rm /tmp/syncmagento.sql
fi
found=0
magerun=n98-magerun.phar
working=$(pwd)
output="t"
extra=()
OLDIFS=$IFS
#### - parse script arguments - ####
while getopts "m:a:n:e:lbq" opt; do
case $opt in
m)
### - check we are in a magento root directory - ###
if [[ -d $OPTARG && -n $OPTARG ]]; then
cd "$OPTARG"
if [[ ! -f app/etc/local.xml ]]; then
echo "not magento root directory"
exit
fi
else
echo "not magento root directory"
exit
fi
((found++))
## - can we write to the log file - ##
if [[ ! -d var/log ]]; then
mkdir -p var/log
fi
if [[ ! -w var/log ]]; then
echo "log directory not writeable"
output="n"
fi
if [[ ! -f var/log/syncmagento.log ]]; then
touch var/log/syncmagento.log
fi
if [[ ! -w var/log/syncmagento.log ]]; then
echo "log file not writeable"
output="n"
fi
;;
esac
done
OPTIND=1
while getopts "m:a:n:e:lbq" opt; do
case $opt in
a)
### - check database archive exists - ###
if [[ $OPTARG == /* ]]; then
archive=$OPTARG
else
archive=$working/$OPTARG
fi
if [[ ! -f $archive ]]; then
logoutput "database archive not found"
exit
fi
if [[ ! -r $archive ]]; then
logoutput "database archive not readable"
exit
fi
if [[ ! $archive == *\.sql* ]]; then
logoutput "archive must be an sql file"
exit
fi
logoutput "archive checked ok"
((found++))
;;
n)
### - store n98-magerun.phar run command - ###
magerun=$OPTARG
logoutput "n98-magerun.phar executable name set as $OPTARG"
;;
e)
### - store extra config variables - ###
IFS=$','
read -a extra <<< $OPTARG
IFS=$OLDIFS
;;
### - set output type - ###
l)
if [[ ! $output == n ]]; then
output="$opt"
fi
;;
b)
if [[ ! $output == n ]]; then
output="$opt"
fi
;;
q)
if [[ ! $output == n ]]; then
output="$opt"
fi
;;
esac
done
if [[ $output == n ]]; then
output="t"
fi
### - check magento directory and database archive arguments both exist - ###
if [[ ! $found == 2 ]]; then
logoutput "magento root directory (-m) and database archive (-a) arguments must both exist"
exit;
fi
#### - check magerun is installed - ####
installed=$(command -v "$magerun" 2>&1)
if [[ ! -n $installed ]]; then
logoutput "$magerun is not installed"
exit
fi
#### - get store url config values - ####
logoutput "collecting store url config values"
IFS=$'\n'
configs=()
### - unsecure - ###
configs+=("config:delete --all --quiet web/unsecure/base_url")
for i in $($magerun config:get --magerun-script web/unsecure/base_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/unsecure/base_link_url")
for i in $($magerun config:get --magerun-script web/unsecure/base_link_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/unsecure/base_skin_url")
for i in $($magerun config:get --magerun-script web/unsecure/base_skin_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/unsecure/base_media_url")
for i in $($magerun config:get --magerun-script web/unsecure/base_media_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/unsecure/base_js_url")
for i in $($magerun config:get --magerun-script web/unsecure/base_js_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
### - secure - ###
configs+=("config:delete --all --quiet web/secure/base_url")
for i in $($magerun config:get --magerun-script web/secure/base_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/secure/base_link_url")
for i in $($magerun config:get --magerun-script web/secure/base_link_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/secure/base_skin_url")
for i in $($magerun config:get --magerun-script web/secure/base_skin_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/secure/base_media_url")
for i in $($magerun config:get --magerun-script web/secure/base_media_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/secure/base_js_url")
for i in $($magerun config:get --magerun-script web/secure/base_js_url); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
#### - get secure front/backend config values - ####
logoutput "collecting secure front/backend config values"
configs+=("config:delete --all --quiet web/secure/use_in_frontend")
for i in $($magerun config:get --magerun-script web/secure/use_in_frontend); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet web/secure/use_in_adminhtml")
for i in $($magerun config:get --magerun-script web/secure/use_in_adminhtml); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
#### - get cookie domain config values - ####
logoutput "collecting cookie domain config values"
configs+=("config:delete --all --quiet web/cookie/cookie_domain")
for i in $($magerun config:get --magerun-script web/cookie/cookie_domain); do
if [[ $i == config:set* ]]; then
configs+=($i)
else
configs+=("config:delete --all --quiet web/cookie/cookie_domain")
fi
done
#### - get admin url config values - ####
logoutput "collecting admin url config values"
configs+=("config:delete --all --quiet admin/url/use_custom")
for i in $($magerun config:get --magerun-script admin/url/use_custom); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet admin/url/custom")
for i in $($magerun config:get --magerun-script admin/url/custom); do
if [[ $i == config:set* ]]; then
configs+=($i)
else
configs+=("config:delete --all --quiet admin/url/custom")
fi
done
configs+=("config:delete --all --quiet admin/url/use_custom_path")
for i in $($magerun config:get --magerun-script admin/url/use_custom_path); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
configs+=("config:delete --all --quiet admin/url/custom_path")
for i in $($magerun config:get --magerun-script admin/url/custom_path); do
if [[ $i == config:set* ]]; then
configs+=($i)
fi
done
IFS=$OLDIFS
#### - get extra config variables passed to script - ####
if [ ${#extra[@]} -gt 0 ]; then
logoutput "collecting extra config values"
for i in $extra; do
logoutput $i
IFS=$'\n'
configs+=("config:delete --all --quiet $i")
for j in $($magerun config:get --magerun-script $i); do
if [[ $j == config:set* ]]; then
configs+=($j)
fi
done
IFS=$OLDIFS
done
fi
#### - get database archive and restore - ####
compression=$(file "$archive")
if [[ $compression == *compressed\ data* ]]; then
compression=$(echo "$compression" | grep -oP "[a-z0-9]+ compressed data" | grep -oP "[a-z0-9]+" | head -n 1)
### - handle gzip archive - ###
if [[ $compression == gzip ]]; then
installed=$(command -v "$compression" 2>&1)
if [[ ! -n $installed ]]; then
logoutput "database archive is gzip compressed but gzip is not installed"
exit
fi
if [[ $archive == *\.tar.gz ]]; then
installed=$(command -v tar 2>&1)
if [[ ! -n $installed ]]; then
logoutput "database archive is tarred but tar is not installed"
exit
fi
logoutput "extracting gzip tar archive"
tar -xOzf "$archive" > /tmp/syncmagento.sql
else
logoutput "extracting gzip archive"
gzip -dc "$archive" > /tmp/syncmagento.sql
fi
### - handle bzip2 archive - ###
elif [[ $compression == bzip2 ]]; then
installed=$(command -v "$compression" 2>&1)
if [[ ! -n $installed ]]; then
logoutput "database archive is bzip2 compressed but bzip2 is not installed"
exit
fi
if [[ $archive == *\.tar.bz2 ]]; then
installed=$(command -v tar 2>&1)
if [[ ! -n $installed ]]; then
logoutput "database archive is tarred but tar is not installed"
exit
fi
logoutput "extracting bzip2 tar archive"
tar -xOjf "$archive" > /tmp/syncmagento.sql
else
logoutput "extracting bzip2 archive"
bzip2 -dc "$archive" > /tmp/syncmagento.sql
fi
else
logoutput "database archive compression method not supported, must be gzipped or bzipped"
exit
fi
if [[ -f /tmp/syncmagento.sql ]]; then
first=$(head -n 1 /tmp/syncmagento.sql)
last=$(tail -n 1 /tmp/syncmagento.sql)
if [[ ! $first == *MySQL\ dump* || ! $last == *Dump\ completed* ]]; then
logoutput "archive does not appear to be a valid database dump"
rm /tmp/syncmagento.sql
exit
fi
logoutput "importing database archive"
$magerun db:import --drop --quiet /tmp/syncmagento.sql
rm /tmp/syncmagento.sql
fi
else
### - handle uncompressed archive - ###
first=$(head -n 1 "$archive")
last=$(tail -n 1 "$archive")
if [[ ! $first == *MySQL\ dump* || ! $last == *Dump\ completed* ]]; then
logoutput "archive does not appear to be a valid database dump"
exit
fi
logoutput "importing database archive"
$magerun db:import --drop --quiet "$archive"
fi
#### restore all config values for new database
logoutput "restoring config values"
IFS=$'\n'
for i in "${configs[@]}"; do
if [[ $i == config:set* ]]; then
logoutput $(eval "$magerun" "$i")
else
logoutput "deleting config values for ${i##* }"
$(eval "$magerun" "$i")
fi
done
IFS=$OLDIFS
#### - clear cache - ####
logoutput "clearing cache"
$magerun --quiet cache:clean
$magerun --quiet cache:flush
$magerun --quiet cache:dir:flush
logoutput "script complete"
exit
@husseycoding
Copy link
Author

husseycoding commented Jan 26, 2016

This is a bash script which will take an existing database archive (either a straight .sql file or compressed as .tar.gz, .gz, .tar.bz2 or .bz2) and import the database dump you define and then apply various config values once imported for things like base url.

The script accepts a number of arguments as follows:

a, absolute or relative path to the database archive to be imported.
m, absolute or relative path to the root directory of the Magento install the database is to be restored to.
n, optional argument containing the n98-magerun executable name (as it can be changed to anything from the default of n98-magerun.phar). If omitted the default of n98-magerun.phar will be used.
e, pass a comma separated list of paths to config variables that should be kept from database being replaced.

Logging has 3 arguments, the log file if used can be found at var/log/syncmagento.log inside the install the database is being restored to. If no argument is given output is shown in the terminal window only.

l, (lowercase L) output is sent only to the logfile nothing is printed in the terminal window
q, no output to either the terminal window or the logfile
b, output to both the terminal window and the logfile 

If the logfile cannot be output to (i.e permissions etc) then the script falls back to the default of outputting to the terminal and all logging arguments are ignored.

An example command to run the script might be:

./syncmagento.sh -bm /absolute/path/to/magento/root -a relative/path/to/database/archive/archive.sql.tar.bz2 -e google/analytics/account,paypal/general/business_account

File names for the database archive should contain .sql, so either just somefile.sql, or one of somefile.sql.gz, somefile.sql.tar.gz, somefile.sql.bz2, somefile.sql.tar.bz2. The contents of the sql dump is also checked to make sure the first line contains MySQL dump and the last line contains Dump completed and so appears to be a valid dump - a check is also first run to make sure the archive is actually readable.

The script must be executable for the user running it, also n98-magerun must be installed.

Before importing the new database dump the script will pull all of the secure and unsecure base url values which have been set at all store scopes (including for other paths like js and media), the same for the cookie domain (which can kill the ability to read session cookies and essentially kill the store if wrong), the same for secure front/backend settings and custom admin url configuration. Once the import is complete these config values are set against this new database.

Also as the databases can be pretty big it's worth using n98-magerun to actually create the database backup for use by this script so you can strip out unneeded data like log and session information to reduce the size of the dump.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment