redis + sentinel + haproxy + keepalived

1 - Instalação REDIS.md

Instalação do Redis no CentOs 7

• Executar todos os comandos abaixo com usuário root

Pacote adicional para enterprise LINUX (EPEL)

yum -y install epel-release

Instalação do pacote do Redis

yum -y install redis

Criação de diretório e permissão para unixsocket

mkdir /var/run/redis
chown redis:root /var/run/redis

Configuração de portas no firewall

firewall-cmd --permanent --add-port=6379/tcp
firewall-cmd --permanent --add-port=26379/tcp
firewall-cmd --reload

Inicializar o rc-local no CentOS 7

chmod +x /etc/rc.d/rc.local
systemctl enable rc-local
systemctl start rc-local

Removendo wanings do Redis

echo 'net.core.somaxconn = 65535' >> /etc/sysctl.conf
echo 'vm.overcommit_memory = 1' >> /etc/sysctl.conf
echo 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' >> /etc/rc.local

Reinicializar o CentOS

reboot

2 - Configurações do Redis e Sentinels.md

Configurações do Redis e Sentinels

Configuração do arquivo /etc/redis.conf

1) Na configuração 'bind 127.0.0.1' alterar para:

bind 127.0.0.1 <IP-DA-MAQUINA>
unixsocket /var/run/redis/redis.sock
unixsocketperm 755

2) Comentar as configurações de 'save':

# save 900 1
# save 300 10
# save 60 10000

3) Configurar a senha de acesso em todos os redis

• Descomentar a linha com a configuração requirepass:

requirepass <i>SENHA</i>

4) Configurar o masterauth em todos os redis com a senha definida no passo 3)

• Descomentar a linha com a configuração masterauth

masterauth <i>SENHA</i>

5) Configurar 'slaveof' somente nos redis SLAVE:

• Descomentar a linha com a configuração 'slaveof' e definir o ip e porta do MASTER:

slaveof <i>IP-DO-REDIS-MASTER</i> <i>PORTA-REDIS-MASTER</i>

Configuração do arquivo /etc/redis-sentinel.conf

bind 0.0.0.0
protected-mode no

dir "/"
port 26379
sentinel monitor mymaster <IP-REDIS-MASTER> <PORTA-REDIS-MASTER> 2
sentinel down-after-milliseconds mymaster 10000
sentinel failover-timeout mymaster 30000
sentinel auth-pass mymaster <SENHA-DO-REDIS>
logfile "/var/log/redis/sentinel.log"

Habilitar e Inicializar Redis e Sentinel

systemctl enable redis
systemctl enable redis-sentinel
systemctl start redis
systemctl start redis-sentinel

3 - Instalar HAProxy e Keepalived.md

Instalar HAProxy e Keepalived

• Executar todos os comandos abaixo com usuário root

Pacote adicional para enterprise LINUX (EPEL)

yum -y install epel-release

Instalar o HAPROXY

yum -y install haproxy

Instalar Keepalived

yum -y install keepalived

Habilitar conexões no haproxy

setsebool -P haproxy_connect_any=1

Configuração de portas no firewall

firewall-cmd --permanent --add-port=6378/tcp
firewall-cmd --reload

Alteração de parametros de ipv4 para haproxy

echo 'net.ipv4.ip_nonlocal_bind = 1' >> /etc/sysctl.conf
echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf

Reinicializar o CentOS

reboot

4 - Configurações do HAProxy e Keepalived.md

Configurações do HAProxy e Keepalived

Configuração do arquivo /etc/haproxy/haproxy.cfg

global
log 127.0.0.1 local0
maxconn 4096
user haproxy
group haproxy
daemon

defaults
mode tcp
timeout connect 3s
timeout server 6s
timeout client 6s
retries 3
option redispatch
option tcplog

frontend RedisFrontend
bind *:6378
default_backend RedisBackend
log 127.0.0.1 local0

backend RedisBackend
option tcp-check
tcp-check connect
tcp-check send AUTH\ <SENHA-REDIS>\r\n
tcp-check send PING\r\n
tcp-check expect string +PONG
tcp-check send info\ replication\r\n
tcp-check expect string role:master
tcp-check send QUIT\r\n
tcp-check expect string +OK
server <NOME-SERVIDOR-REDIS> <IP-REDIS>:<PORTA-REDIS> check inter 2s
<ADICIONAR TODOS OS SERVERS REDIS IGUAL A LINHA ACIMA>

Configuração do arquivo /etc/keepalived/keepalived.conf MASTER

global_defs {
    enable_script_security
}

vrrp_script chk_haproxy {
    script          "/etc/keepalived/script_haproxy.sh"
    user            root
    interval        2
    weight          2
}

vrrp_instance VI_01 {
    state                   MASTER
    interface               <NOME-DA-INTERFACE-DE-REDE>
    virtual_router_id       51
    priority                101
    advert_int              1

    virtual_ipaddress {
        <ENDERECO-IP-VIRTUAL> dev <NOME-DA-INTERFACE-DE-REDE>
    }

    track_script {
        chk_haproxy
    }
}

Configuração do arquivo /etc/keepalived/keepalived.conf BACKUP

global_defs {
    enable_script_security
}

vrrp_script chk_haproxy {
    script          "/etc/keepalived/script_haproxy.sh"
    user            root
    interval        2
    weight          2
}

vrrp_instance VI_01 {
    state                   BACKUP
    interface               <NOME-DA-INTERFACE-DE-REDE>
    virtual_router_id       51
    priority                100
    advert_int              1

    virtual_ipaddress {
        <ENDERECO-IP-VIRTUAL> dev <NOME-DA-INTERFACE-DE-REDE>
    }

    track_script {
        chk_haproxy
    }
}

Criar o arquivo /etc/keepalived/script_haproxy.sh

• Executar os comandos como root

touch /etc/keepalived/script_haproxy.sh
chmod +x /etc/keepalived/script_haproxy.sh

• Adicionar o conteudo abaixo no arquivo:

#!/bin/bash

haproxy=$(pidof haproxy)

if [ -z "$haproxy" ]
then
  $(pkill keepalived)
  exit 1
fi

exit 0

Habilitar e Inicializar HAproxy e Keepalived

systemctl enable haproxy
systemctl enable keepalived
systemctl start haproxy
systemctl start keepalived

5 - Testes.md

Testes

Verificar se o REDIS está rodando corretamente

systemctl status -l redis

• Retorno:

● redis.service - Redis persistent key-value database
   Loaded: loaded (/usr/lib/systemd/system/redis.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/redis.service.d
           └─limit.conf
   Active: active (running) since
  Process: 5884 ExecStop=/usr/libexec/redis-shutdown (code=exited, status=0/SUCCESS)
 Main PID: 5907 (redis-server)
   CGroup: /system.slice/redis.service
           └─5907 /usr/bin/redis-server 127.0.0.1:6379

cat /var/log/redis/redis.log

• Retorno:

5308:C 17 May 17:39:27.991 * supervised by systemd, will signal readiness
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.12 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 5308
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

# Server started, Redis version 3.2.12
* DB loaded from disk: 0.009 seconds
* The server is now ready to accept connections on port 6379
* The server is now ready to accept connections at /var/run/redis/redis.sock
* SLAVE OF xxx.xxx.xxx.xxx:6379 enabled (user request from 'id=2 addr=xxx.xxx.xxx.xxx:44472 fd=7 name=sentinel-91713035-cmd age=9 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=0 qbuf-free=32768 obl=36 oll=0 omem=0 events=r cmd=exec')
# CONFIG REWRITE failed: Permission denied
* Connecting to MASTER xxx.xxx.xxx.xxx:6379
* MASTER <-> SLAVE sync started
* Non blocking connect for SYNC fired the event.
* Master replied to PING, replication can continue...
* Partial resynchronization not possible (no cached master)
* Full resync from master: d67d719d70a9f683f323f2c92486620eefb5a941:2720686
* MASTER <-> SLAVE sync: receiving 5060 bytes from master
* MASTER <-> SLAVE sync: Flushing old data
* MASTER <-> SLAVE sync: Loading DB in memory
* MASTER <-> SLAVE sync: Finished with success

Verificar se o SENTINEL está rodando corretamente

systemctl status -l redis-sentinel

• Retorno:

● redis-sentinel.service - Redis Sentinel
   Loaded: loaded (/usr/lib/systemd/system/redis-sentinel.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/redis-sentinel.service.d
           └─limit.conf
   Active: active (running) since
 Main PID: 5316 (redis-sentinel)
   CGroup: /system.slice/redis-sentinel.service
           └─5316 /usr/bin/redis-sentinel 0.0.0.0:26379 [sentinel]

cat /var/log/redis/sentinel.log

• Retorno:

5316:X 17 May 17:39:27.971 * supervised by systemd, will signal readiness
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.12 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in sentinel mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 26379
 |    `-._   `._    /     _.-'    |     PID: 5316
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

# Sentinel ID is d9713737232859bfe55f59ae25de1daebb2b9fd7

Verificar se o HAProxy está rodando corretamente

systemctl status -l haproxy

• Retorno:

● haproxy.service - HAProxy Load Balancer
   Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor preset: disabled)
   Active: active (running) since Sex 2019-05-17 18:26:28 UTC; 12min ago
 Main PID: 6054 (haproxy-systemd)
   CGroup: /system.slice/haproxy.service
           ├─6054 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
           ├─6055 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
           └─6056 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

systemd[1]: Started HAProxy Load Balancer.
haproxy-systemd-wrapper[6054]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds

Verificar se o Keepalived está rodando corretamente

systemctl status -l keepalioved

• Retorno:

● keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/keepalived.service.d
           └─override.conf
   Active: active (running) since
  Process: 6235 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 6236 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─6236 /usr/sbin/keepalived -D
           ├─6237 /usr/sbin/keepalived -D
           └─6238 /usr/sbin/keepalived -D

Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: VRRP_Instance(VI_01) Sending/queueing gratuitous ARPs on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx
Keepalived_vrrp[6238]: Sending gratuitous ARP on ens160 for xxx.xxx.xxx.xxx

Realizar os testes de desligar os servidores (Redis e Load balancer)