Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Bash script to prepare Redis commands for mass insertion via `redis-cli --pipe`

Redis supports mass insert via redis-cli --pipe, but commands need to be written in redis protocol. For more details, check out the Redis mass insertion page. This script takes care of converting ordinary redis commands to redis protocol.

#!/usr/bin/env bash

while read CMD; do
  # each command begins with *{number arguments in command}\r\n
  XS=($CMD); printf "*${#XS[@]}\r\n"
  # for each argument, we append ${length}\r\n{argument}\r\n
  for X in $CMD; do printf "\$${#X}\r\n$X\r\n"; done
done
Example:
$ for N in $(seq 1 1000); do echo "SADD test $N"; done > data.txt
$ cat data.txt | sh redis-pipe.sh | redis-cli --pipe
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 1000
$ echo "SCARD test" | redis-cli
(integer) 1000
@luyishisi

This comment has been minimized.

Copy link

commented Sep 20, 2016

这个脚本在部分电脑,比如我自己的会报错
Syntax error: "(" unexpected (expecting "done")
解决方式是
cat data.txt | bash crate.sh | redis-cli --pipe
将sh 改成bash 。。

@abtrout

This comment has been minimized.

Copy link
Owner Author

commented Sep 28, 2016

hi @luyishisi. i didn't notice this comment, sorry for the delay.

is it possible your bash is actually dash? a google search suggests that syntax error is due to bash/dash incompatibilities.

@MatthewLQM

This comment has been minimized.

Copy link

commented Jul 19, 2017

hi @abtrout, I have a problem with your code. when my value is json data, this code may not work well.
(By the way, I'm sorry for my bad English.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.