A simple script to migrate all keys from one Redis to another

redis_migrate.py

#!/usr/bin/env python
import argparse
import redis


def connect_redis(conn_dict):
    conn = redis.StrictRedis(host=conn_dict['host'],
                             port=conn_dict['port'],
                             db=conn_dict['db'])
    return conn


def conn_string_type(string):
    format = '<host>:<port>/<db>'
    try:
        host, portdb = string.split(':')
        port, db = portdb.split('/')
        db = int(db)
    except ValueError:
        raise argparse.ArgumentTypeError('incorrect format, should be: %s' % format)
    return {'host': host,
            'port': port,
            'db': db}


def migrate_redis(source, destination):
    src = connect_redis(source)
    dst = connect_redis(destination)
    for key in src.keys('*'):
        ttl = src.ttl(key)
        # we handle TTL command returning -1 (no expire) or -2 (no key)
        if ttl < 0:
            ttl = 0
        print "Dumping key: %s" % key
        value = src.dump(key)
        if not value:
            print "Skipping none"
            continue

        print "Restoring key: %s" % key
        try:
            dst.restore(key, ttl * 1000, value, replace=True)
        except redis.exceptions.ResponseError:
            print "Failed to restore key: %s" % key
            pass
    return


def run():
    parser = argparse.ArgumentParser()
    parser.add_argument('source', type=conn_string_type)
    parser.add_argument('destination', type=conn_string_type)
    options = parser.parse_args()
    migrate_redis(options.source, options.destination)

if __name__ == '__main__':
    run()

Usage:

virtualenv migrate-redis
source migrate-redis/bin/activate
pip install -U git+git://github.com/andymccurdy/redis-py.git
nano migrate.py # add script here
python migrate.py

works great!

Thanks for your effort, this saved me 👍

A cool feature would be to save the dump locally instead of migrating to another redis instance

is it working for live migration? also between redis single instance to redis cluster. Please give me some advice.
Thanks!

very useful thank you!

It was very helpful for me. Thanks for your contribution~!

Thanks for your effort, this saved me 👍

A cool feature would be to save the dump locally instead of migrating to another redis instance

Just what I was looking for / about to write! As AWS-managed Redis deployments don't let you use MIGRATE >:(

One question though @josegonzalez - why do you set the TTL in the destination multiplied by 1000? i.e. dst.restore(key, ttl * 1000, value, replace=True)

Oh never mind - I see TTL returns the key's ttl value in seconds while RESTORE expects the ttl value in milliseconds. 👏 thorough work!

Thanks for your effort! Worked like a charm!

Can

Thanks for your effort, this saved me 👍

A cool feature would be to save the dump locally instead of migrating to another redis instance

Can you help in editing the script as I am not a devloper and i need to migrate redis data to another redis db

Thanks a lot. You saved my time 👍 👍 👍 👍