Prepares data for Redis Mass Insertion with the redis-cli --pipe command

redis-mass-insert.md

redis-mass-insertion

Lightweight Python script to prepare Redis commands for mass insertion.

Code

#!/usr/bin/env python
"""
    redis-mass.py
    ~~~~~~~~~~~~~
    Prepares a newline-separated file of Redis commands for mass insertion.
    from https://github.com/Squab/redis-mass-insertion
    :copyright: (c) 2015 by Tim Simmons.
    :license: BSD, see LICENSE for more details.
"""
import sys

def proto(line):
    result = "*%s\r\n$%s\r\n%s\r\n" % (str(len(line)), str(len(line[0])), line[0])
    for arg in line[1:]:
        result += "$%s\r\n%s\r\n" % (str(len(arg)), arg)
    return result

if __name__ == "__main__":
    try:
        filename = sys.argv[1]
        f = open(filename, 'r')
    except IndexError:
        f = sys.stdin.readlines()

    for line in f:
        print proto(line.rstrip().split(' ')),

Usage

redis-mass.py can be used in two ways, either way assumes input with the following format.

SET key value
SET key2 value2
SADD someset value
SADD someset value2
SADD someset value3 value4

You can specify a file to the script as a command-line argument

python redis-mass.py input.txt | redis-cli --pipe

or pipe input from stdin

cat input.txt | python redis-mass.py | redis-cli --pipe