Migrate Redis data with ease using yannh/redis-dump-go. Simple, efficient, and ready-to-use script!

README.md

Redis Migration Script

This script is used to migrate Redis data from one server to another. It supports custom ports, passwords, and the ability to select specific Redis databases to transfer.

Dependencies

Before using the redis-migrate.sh script, ensure that you have the following dependencies installed on your system:

• wget: Used for retrieving files from the web (if your script requires it).
• redis-tools: Provides necessary tools for interacting with Redis, such as redis-cli.

Usage

Before running the script, make sure you have the necessary permissions to execute it. You can set the execution permission using the following command:

chmod +x redis-migrate.sh

To migrate Redis data, run the script with the appropriate options as shown in the example below:

./redis-migrate.sh \
  --src-host "redis-01" \       # Source Redis server host
  --src-port "6379" \           # Source Redis server port
  --src-password "password123" \ # Source Redis server password, if required
  --dest-host "localhost" \     # Destination Redis server host
  --dest-port "6379" \          # Destination Redis server port
  --dest-password "password456" \ # Destination Redis server password, if required
  --dbs "1,0"                  # Comma-separated list of database indexes to migrate

Replace password123 and password456 with the actual passwords for your source and destination Redis instances, respectively. If there is no password, you can simply omit the --src-password and --dest-password options or leave them empty like this --src-password "".

The --dbs option allows you to specify which databases to migrate. In the example above, databases with indexes 1 and 0 will be migrated. If you want to migrate all databases, you can omit this option.

Note

• Make sure that the source and destination Redis servers are accessible from the host where the script is running.
• Verify that there is enough network bandwidth and the destination Redis server has sufficient memory to handle the data being migrated.
• It is recommended to perform the migration during a maintenance window or when the load on the Redis servers is minimal to avoid any performance impact.

Remember to customize the command with the actual data from your environment.

redis-migrate.sh

#!/bin/bash

# Default installation directory for redis-dump-go
INSTALL_DIR="/usr/local/bin"

# Default redis port
DEFAULT_PORT="6379"

# Function to check for required tools
check_requirements() {
  local missing_counter=0
  local dependencies=('wget' 'redis-cli')

  for dep in "${dependencies[@]}"; do
    if ! type "$dep" > /dev/null 2>&1; then
      echo >&2 "Error: $dep is required but it's not installed."
      ((missing_counter++))
    fi
  done

  if (( missing_counter > 0 )); then
    echo "Please install the missing tools before running this script."
    exit 1
  fi
}

check_requirements

# Function to detect the architecture and set the corresponding binary name
get_arch_name() {
    ARCH=$(uname -m)
    case $ARCH in
        x86_64)
            BIN_ARCH="amd64"
            ;;
        i386 | i686)
            BIN_ARCH="386"
            ;;
        aarch64 | arm64)
            BIN_ARCH="arm64"
            ;;
        arm*)
            BIN_ARCH="armv6"
            ;;
        *)
            echo "Architecture not supported"
            exit 1
            ;;
    esac

    # Always Linux as we run this in a Docker container
    BIN_OS="linux"
    BIN_NAME="redis-dump-go-${BIN_OS}-${BIN_ARCH}.tar.gz"
}

# Function to parse named parameters
parse_params() {
    # Default values of variables set from params
    SRC_HOST=''
    SRC_PORT=$DEFAULT_PORT
    SRC_PASSWORD=''
    DEST_HOST=''
    DEST_PORT=$DEFAULT_PORT
    DEST_PASSWORD=''
    DB_NAMES=() # Array to hold database names

    while :; do
        case "${1-}" in
        -h | --help)
            usage ;;
        --src-host) # Source host (required)
            SRC_HOST="${2-}"
            shift
            ;;
        --src-port) # Source port (optional, default is 6379)
            SRC_PORT="${2-}"
            shift
            ;;
        --src-password) # Source password (optional)
            SRC_PASSWORD="${2-}"
            shift
            ;;
        --dest-host) # Destination host (required)
            DEST_HOST="${2-}"
            shift
            ;;
        --dest-port) # Destination port (optional, default is 6379)
            DEST_PORT="${2-}"
            shift
            ;;
        --dest-password) # Destination password (optional)
            DEST_PASSWORD="${2-}"
            shift
            ;;
        --dbs) # Comma-separated list of database names (optional)
            IFS=',' read -r -a DB_NAMES <<< "${2-}"
            shift
            ;;
        -?*)
            echo "Unknown option: $1" >&2
            exit 1
            ;;
        *) # No more options
            break
            ;;
        esac
        shift
    done

    # Check required parameters
    [[ -z "$SRC_HOST" ]] && { echo "Missing required parameter: --src-host"; exit 1; }
    [[ -z "$DEST_HOST" ]] && { echo "Missing required parameter: --dest-host"; exit 1; }
}

# Usage info
usage() {
    cat <<EOF
    Usage: ${0##*/} [OPTION]...

    Options:
      --src-host HOST             Source Redis host
      --src-port PORT             Source Redis port (default: $DEFAULT_PORT)
      --src-password PASSWORD     Source Redis password (if any)
      --dest-host HOST            Destination Redis host
      --dest-port PORT            Destination Redis port (default: $DEFAULT_PORT)
      --dest-password PASSWORD    Destination Redis password (if any)
      --dbs DBS                   Redis databases (a list of db indexes, comma-separated, default: all databases)
EOF
    exit 1
}

# Parse named parameters
parse_params "$@"

# Detect the architecture and set the binary name
get_arch_name


# Download and install redis-dump-go
echo "Downloading redis-dump-go for ${BIN_OS}-${BIN_ARCH}..."
wget -q -O "/tmp/${BIN_NAME}" "https://github.com/yannh/redis-dump-go/releases/latest/download/${BIN_NAME}"
tar -xzf "/tmp/${BIN_NAME}" -C "${INSTALL_DIR}"
rm "/tmp/${BIN_NAME}"
chmod +x "${INSTALL_DIR}/redis-dump-go"

# Build the base redis-dump-go command with optional password
DUMP_COMMAND_BASE="${INSTALL_DIR}/redis-dump-go -host ${SRC_HOST} -port ${SRC_PORT}"
if [[ -n $SRC_PASSWORD ]]; then
    export REDISDUMPGO_AUTH="$SRC_PASSWORD"
fi

# Build the base redis-cli command with optional password
LOAD_COMMAND_BASE="redis-cli -h ${DEST_HOST} -p ${DEST_PORT}"
if [[ -n $DEST_PASSWORD ]]; then
    LOAD_COMMAND_BASE+=" -a ${DEST_PASSWORD}"
fi

# Check if specific databases were provided
if [ ${#DB_NAMES[@]} -eq 0 ]; then
    # No databases specified, dump all databases
    DUMP_COMMAND="${DUMP_COMMAND_BASE}"
    LOAD_COMMAND="${LOAD_COMMAND_BASE}"

    echo "Dumping all databases from ${SRC_HOST} and restoring to ${DEST_HOST}..."
    $DUMP_COMMAND | $LOAD_COMMAND --pipe
else
    # Loop over each database name and perform the dump
    for DB_NAME in "${DB_NAMES[@]}"; do
        DUMP_COMMAND="${DUMP_COMMAND_BASE} -db $DB_NAME"
        LOAD_COMMAND="${LOAD_COMMAND_BASE}"

        echo "Dumping data from ${SRC_HOST} (DB: $DB_NAME) and restoring to ${DEST_HOST}..."
        $DUMP_COMMAND | $LOAD_COMMAND --pipe
    done
fi

if [[ -n $SRC_PASSWORD ]]; then
    unset REDISDUMPGO_AUTH
fi

echo "Data transfer complete."