Skip to content

Instantly share code, notes, and snippets.

@wgbartley wgbartley/drain-list.sh
Last active Sep 26, 2017

Embed
What would you like to do?
Drain Redis List to X elements
#!/bin/sh
# Max number of concurrent Redis connections before we back off
CONN_LIMIT=10000
# Check to make sure all arguments were passed
DO_EXIT=0
if [ "$1" == "" ]; then
echo "Please provide a list name (in quotes) as first argument."
DO_EXIT=1
fi
if [ "$2" == "" ]; then
echo "Please provide the max list size as second argument."
DO_EXIT=1
fi
if [ $DO_EXIT -gt 0 ]; then
exit
fi
# Initialize variables
LIST="$1"
MAX_COUNT=$2
COUNTER=0
LEN=$(redis-cli llen $LIST)
# Get memory usage before drain
MEM_BEFORE=$(redis-cli info | grep 'used_memory:' | tr ':' "\n" | grep -o '[0-9]\+')
MEM_BEFORE_HR=$(redis-cli info | grep 'used_memory_human:' | tr ':' "\n" | grep -o '[A-Z0-9\.]\+')
# Drain!
while [ $LEN -ge $MAX_COUNT ]; do
CONNS=$(netstat -tna | grep '6379' | wc -l)
if [ $CONNS -ge $CONN_LIMIT ]; then
echo "Too many connections ($CONNS). Sleeping . . ."
sleep 10
else
TMP=$(redis-cli lpop $LIST)
LEN=$(redis-cli llen $LIST)
COUNTER=$[$COUNTER +1]
echo $LEN
fi
done
# Get memory usage after drain
MEM_AFTER=$(redis-cli info | grep 'used_memory:' | tr ':' "\n" | grep -o '[0-9]\+')
MEM_AFTER_HR=$(redis-cli info | grep 'used_memory_human:' | tr ':' "\n" | grep -o '[A-Z0-9\.]\+')
# Calculate memory freed
MEM_FREED=$((MEM_BEFORE - MEM_AFTER))
MEM_FREED=$(printf "%'d" $MEM_FREED)
# Report stats
echo "Done! Drained $COUNTER elements from $LIST. $MEM_FREED bytes freed ($MEM_BEFORE_HR -> $MEM_AFTER_HR)."
@wgbartley

This comment has been minimized.

Copy link
Owner Author

wgbartley commented Sep 26, 2017

Added connection monitor to prevent running out of TCP ports

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.