Skip to content

Instantly share code, notes, and snippets.

View sennajox's full-sized avatar

Sennajox sennajox

  • Shanghai, China
View GitHub Profile
@juris
juris / redis-cluster-backup.sh
Last active March 21, 2023 14:27
Redis Cluster backup script
#!/bin/sh
readonly cluster_topology=$(redis-cli -h redis-cluster cluster nodes)
readonly slaves=$(echo "${cluster_topology}" | grep slave | cut -d' ' -f2,4 | tr ' ' ',')
readonly backup_dir="/opt/redis-backup"
mkdir -p ${backup_dir}
for slave in ${slaves}; do
master_id=$(echo "${slave}" | cut -d',' -f2)
@marwei
marwei / how_to_reset_kafka_consumer_group_offset.md
Created November 9, 2017 23:39
How to Reset Kafka Consumer Group Offset

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)
@baymaxium
baymaxium / content.md
Created October 18, 2017 09:00
zetcd:让应用解除对ZooKeeper的依赖

原文:Docker

分布式系统通常都依赖一个仲裁系统协同工作,一般这样的系统通过仲裁来保证信息的准确传达,以避免出现脑裂。这类系统通过牺牲通用性换来了充分的设计余地,这种做法显然已经被不断扩散的各种具体实现所例证。这样的系统有很多,例如:chubby,ZooKeeper,etcd和consul等。尽管这些系统的理念和协议不同,但是提供的都是类似的基于key-value的分布式仲裁。作为将etcd打造成分布式系统最受瞩目的基础组件计划的一部分,etcd团队开发了一款全新的代理,zetcd,无需变动就可以让etcd集群消费ZooKeeper的服务请求。

ZooKeeper是第一个开源实现的仲裁软件,这促使它成为众多分布式系统偏好的后端。理论上来说这些系统应该可以跟etcd兼容,但由于历史原因,事实并非如此;etcd集群无法替代ZooKeeper,其数据模型和客户端协议跟ZooKeeper应用不兼容;反之亦然。倘若该系统已经投产,那么几乎没什么动力可以推动它接入新的后端,引入新的复杂度。幸运的是,etcd v3 API正准备通过一个标准代理zetcd实现对ZooKeeper数据模型的模拟支持,zetcd是一个由etcd团队开发的全新开源项目,今天发布了zetcd第一个beta版本,v0.0.1,目标是在生产系统中管理和部署zetcd系统。

zetcd 代理部署在etcd集群前面,服务于一个模拟的ZooKeeper客户端端口,使得ZooKeeper应用可以在

@hydra35
hydra35 / destroy_gw.sh
Created August 24, 2012 03:28
destroy NAT gateway on OSX
#!/bin/bash
# All these steps look excessive but address
# network instability issues created by not doing them
gwdev=`netstat -nr | grep default | awk '{ print $6 }' | head -1`
if [ -z "$gwdev" ]; then
gwdev=en1
fi
@hydra35
hydra35 / create_gw.sh
Created August 24, 2012 03:28
create NAT gateway on OSX
#!/bin/bash
# Get the interface name for the gateway
gwdev=`netstat -nr | grep default | awk '{ print $6 }' | head -1`
# If none are found, set the gateway to en1 (generally Wifi on OS X)
if [ -z "$gwdev" ]; then
gwdev=en1
fi