1.首先安装zookeeper机群: 修改其config目录下的zoo.cfg,主要修改dataDir、server.A=B:C:D参数即可; 然后在dataDir下创建myid文件,内容为当前节点的id;
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/data/soft/zookeeper/data
# the port at which the clients will connect
#clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
# Requires updates to be synced to media of the transaction log before finishing
# processing the update. If this option is set to 'no', ZooKeeper will not require
# updates to be synced to the media.
# WARNING: it's not recommended to run a production ZK cluster with forceSync disabled.
forceSync=yes
skipACL=yes
reconfigEnabled=true
dynamicConfigFile=./zookeeper.properties.dynamic
server.1=10.206.28.67:2888:3888:participant;2181
server.2=10.206.28.68:2888:3888:participant;2181
server.3=10.206.28.69:2888:3888:participant;2181
server.4=10.206.28.70:2888:3888:observer;2181
server.5=10.206.28.71:2888:3888:observer;2181
server.6=10.206.28.72:2888:3888:observer;2181
server.7=10.206.28.73:2888:3888:observer;2181
2.安装kafka机群: 修改config/server.properties文件:
broker.id
log.dirs
num.partitions
zookeeper.connect
#(zookeeper.connect=10.10.82.173:2181,10.10.82.174:2181,10.10.82.175:2181/kafka)
##### Topic #####
auto.create.topics.enable=false
delete.topic.enable=true
修改kafka-server-start.sh将其jvm内存加大,如4G
启动即可:
JMX_PORT=6062 ./bin/kafka-server-start.sh -daemon config/server.properties
JVM参数配置见: http://kafka.apache.org/documentation.html#java
3.安装kafka-manager管理后台: 参见kafka-manager-README
# Protocol
listeners=SASL_PLAINTEXT://:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256
sasl.enabled.mechanisms=SCRAM-SHA-256
# ACL
allow.everyone.if.no.acl.found=false
super.users=User:admin
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
# 用户
# ./bin/kafka-configs.sh --zookeeper 10.206.28.67:2181/k1 --alter --add-config 'SCRAM-SHA-256=[password=admin-secret-sfkafka]' --entity-type users --entity-name admin
# ./bin/kafka-configs.sh --zookeeper 10.206.28.67:2181/k1 --alter --add-config 'SCRAM-SHA-256=[password=kafka-manager-secret]' --entity-type users --entity-name kafka-manager
# ./bin/kafka-configs.sh --zookeeper 10.206.28.67:2181/k1 --describe --entity-type users
# ./bin/kafka-configs.sh --zookeeper 10.206.28.67:2181/k1 --alter --delete-config SCRAM-SHA-256 --entity-type users --entity-name kafka-manager
# ./bin/kafka-configs.sh --zookeeper 10.206.28.67:2181/k1 --alter --add-config 'SCRAM-SHA-256=[password=spring-kafka-producer-secret]' --entity-type users --entity-name spring-kafka-producer
# ./bin/kafka-configs.sh --zookeeper 10.206.28.67:2181/k1 --alter --add-config 'SCRAM-SHA-256=[password=spring-kafka-consumer-secret]' --entity-type users --entity-name spring-kafka-consumer
# 权限
# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=10.206.28.67:2181/k1 --add --cluster --allow-principal User:kafka-manager
# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=10.206.28.67:2181/k1 --list
# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=10.206.28.67:2181/k1 --remove --cluster --allow-principal User:kafka-manager
# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=10.206.28.67:2181/k1 --add --consumer --topic topic1 --topic topic1.DLT --group fooGroup --allow-principal User:spring-kafka-consumer
# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=10.206.28.67:2181/k1 --add --producer --topic topic1 --allow-principal User:spring-kafka-producer
# ./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=10.206.28.67:2181/k1 --add --producer --topic topic1.DLT --allow-principal User:spring-kafka-consumer