Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
A simple script to migrate all keys from one Redis to another
#!/usr/bin/env python
import argparse
import redis
def connect_redis(conn_dict):
conn = redis.StrictRedis(host=conn_dict['host'],
port=conn_dict['port'],
db=conn_dict['db'])
return conn
def conn_string_type(string):
format = '<host>:<port>/<db>'
try:
host, portdb = string.split(':')
port, db = portdb.split('/')
db = int(db)
except ValueError:
raise argparse.ArgumentTypeError('incorrect format, should be: %s' % format)
return {'host': host,
'port': port,
'db': db}
def migrate_redis(source, destination):
src = connect_redis(source)
dst = connect_redis(destination)
for key in src.keys('*'):
ttl = src.ttl(key)
# we handle TTL command returning -1 (no expire) or -2 (no key)
if ttl < 0:
ttl = 0
print "Dumping key: %s" % key
value = src.dump(key)
print "Restoring key: %s" % key
try:
dst.restore(key, ttl * 1000, value, replace=True)
except redis.exceptions.ResponseError:
print "Failed to restore key: %s" % key
pass
return
def run():
parser = argparse.ArgumentParser()
parser.add_argument('source', type=conn_string_type)
parser.add_argument('destination', type=conn_string_type)
options = parser.parse_args()
migrate_redis(options.source, options.destination)
if __name__ == '__main__':
run()
@josegonzalez

This comment has been minimized.

Copy link
Owner Author

@josegonzalez josegonzalez commented Mar 7, 2017

Usage:

virtualenv migrate-redis
source migrate-redis/bin/activate
pip install -U git+git://github.com/andymccurdy/redis-py.git
nano migrate.py # add script here
python migrate.py
@vaibhavkaul

This comment has been minimized.

Copy link

@vaibhavkaul vaibhavkaul commented Mar 7, 2017

works great!

@giefferre

This comment has been minimized.

Copy link

@giefferre giefferre commented May 17, 2018

Thanks for your effort, this saved me 👍

A cool feature would be to save the dump locally instead of migrating to another redis instance

@jsimanjuntak

This comment has been minimized.

Copy link

@jsimanjuntak jsimanjuntak commented Nov 12, 2018

is it working for live migration? also between redis single instance to redis cluster. Please give me some advice.
Thanks!

@feliperazeek

This comment has been minimized.

Copy link

@feliperazeek feliperazeek commented Mar 26, 2019

very useful thank you!

@blackaib

This comment has been minimized.

Copy link

@blackaib blackaib commented May 24, 2019

It was very helpful for me. Thanks for your contribution~!

@RATHOREDPS03

This comment has been minimized.

Copy link

@RATHOREDPS03 RATHOREDPS03 commented Dec 4, 2019

Thanks for your effort, this saved me 👍

A cool feature would be to save the dump locally instead of migrating to another redis instance

@izakp

This comment has been minimized.

Copy link

@izakp izakp commented Jan 22, 2020

Just what I was looking for / about to write! As AWS-managed Redis deployments don't let you use MIGRATE >:(

@izakp

This comment has been minimized.

Copy link

@izakp izakp commented Jan 22, 2020

One question though @josegonzalez - why do you set the TTL in the destination multiplied by 1000? i.e. dst.restore(key, ttl * 1000, value, replace=True)

@izakp

This comment has been minimized.

Copy link

@izakp izakp commented Jan 22, 2020

Oh never mind - I see TTL returns the key's ttl value in seconds while RESTORE expects the ttl value in milliseconds. 👏 thorough work!

@cheryomushkin

This comment has been minimized.

Copy link

@cheryomushkin cheryomushkin commented Dec 13, 2020

Thanks for your effort! Worked like a charm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment