Install Apache Kafka - CentOS 8

kafka-centos-8.md

Passos para instalar Apache Kafka no CentOS 8 com DOCKER

Requerimentos de mémoria mínimo: 1.0 Gb (recomendado 2gb - POC) / 4gb-8gb em produção

Modificar os arquivos em /etc/yum.repos.d

Pré update:

sudo sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sudo sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*

Atualizar com: yum update

Pós update:

sudo sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-Linux*
sudo sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-Linux*

Instala dependências:

yum install -y yum-utils device-mapper-persistent-data lvm2

Adicionar repo do docker no yum:

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

Instalar o docker:

yum install -y docker-ce

Habilitar o serviço:

systemctl enable docker

Iniciar o docker:

service docker start

Instalando o docker compose:

curl -L "https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/bin/docker-compose

Adicionando permissão para executar:

chmod +x /usr/bin/docker-compose

Instalando o Zookeeper e o Kafka via docker-compose

Criar um diretório kafka-docker na home do usuário:

mkdir kafka-docker

Criar um arquivo docker-compose.yml nesse diretório e colar o conteúdo abaixo:

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

  kafka:
    image: confluentinc/cp-kafka
    container_name: kafka
    ports:
    # To learn about configuring Kafka for access across networks see
    # https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/
      - "9092:9092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://kafka:29092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
      KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
      

Subir os serviços com:

docker-compose up -d

Thats it.