ScyllaIO

001-Scylla.md

Esse DB utiliza alto processamento de IO, por isso algumas horas sobem, outras demoram pra subir e algumas ele só trava

O próprio DB tem seus scripts que ajustam essas configs para o sistema rodando fora do developer-mode, em especifico o script scylla_io_setup que ajusta a propriedade fs.aio-max-nr que normalmente é algo em torno de 65k.

Tem um calculo em cima disso no proprio DockerHub do ScyllaDB

Available AIO on the system - (request AIO per-cpu * ncpus) =
aio_max_nr - aio_nr < (reactor::max_aio + detect_aio_poll + reactor_backend_aio::max_polls) * cpu_cores =
aio_max_nr - aio_nr < (1024 + 2 + 10000) * cpu_cores =
aio_max_nr - aio_nr < 11026 * cpu_cores

where

reactor::max_aio = max_aio_per_queue * max_queues,
max_aio_per_queue = 128,
max_queues = 8.

• Comando para ajustar essa configuração no linux

echo fs.aio-max-nr=1048576 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
echo fs.aio-max-nr=1048576 | sudo tee /etc/sysctl.d/41-aio_max_nr.conf && sudo sysctl --system

• Refs
  • Github
  • Veritas

002-Dockerfile

FROM ubuntu:18.04

ARG DEBIAN_FRONTEND=noninteractive
ARG DUMB_INIT_URL=https://github.com/Yelp/dumb-init/releases/download/v1.2.5/dumb-init_1.2.5_x86_64
ARG DUMB_INIT_PATH=/usr/local/bin/dumb-init

ARG USERNAME=php
ARG USER_UID=1000
ARG USER_GID=$USER_UID

ENV TZ=America/Sao_Paulo
ENV TERM=xterm-256color

SHELL [ "/bin/bash", "-c" ]

ADD ${DUMB_INIT_URL} ${DUMB_INIT_PATH}

RUN set -xeu; \
  groupadd --gid $USER_GID $USERNAME; \
  useradd --uid $USER_UID --gid $USER_GID -m $USERNAME; \
  apt-get update; \
  apt-get install -y sudo; \
  echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME; \
  chmod 0440 /etc/sudoers.d/$USERNAME;

RUN set -xeu; \
  apt-get -qq update; \
  apt-get -qq install -y locales apt-utils nano; \
  sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen; \
  dpkg-reconfigure --frontend=noninteractive locales; \
  update-locale LANG=en_US.UTF-8; \
  apt-get -qq clean; \
  chmod +x ${DUMB_INIT_PATH};

ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8

RUN set -xeu; \
  apt-get -qq update; \
  apt-get -qq install -y tzdata; \
  ln -snf /usr/share/zoneinfo/${TZ} /etc/localtime; \
  echo ${TZ} > /etc/timezone; \
  dpkg-reconfigure --frontend=noninteractive tzdata; \
  apt-get -qq clean;

RUN set -xeu; \
  apt-get -qq update; \
  apt-get -qq install -y nano build-essential curl cmake g++ libpcre3-dev libssl-dev make openssl libgmp-dev git curl ca-certificates software-properties-common wget zip unzip > /dev/null; \
  LC_ALL=C.UTF-8 add-apt-repository -y ppa:ondrej/php; \
  apt-get -qq update; \
  apt-get -qq install php7.1 php7.1-dev php7.1-xml php7.1-mbstring php7.1-curl php7.1-zip php7.1-xdebug; \
  apt-get -qq clean;

RUN set -xeu; \
  curl -fsSL "https://downloads.datastax.com/cpp-driver/ubuntu/18.04/dependencies/libuv/v1.35.0/libuv1_1.35.0-1_amd64.deb" -o /tmp/libuv1_1.35.0-1_amd64.deb; \
  curl -fsSL "https://downloads.datastax.com/cpp-driver/ubuntu/18.04/dependencies/libuv/v1.35.0/libuv1-dev_1.35.0-1_amd64.deb" -o /tmp/libuv1-dev_1.35.0-1_amd64.deb; \
  curl -fsSL "https://downloads.datastax.com/cpp-driver/ubuntu/18.04/cassandra/v2.16.0/cassandra-cpp-driver_2.16.0-1_amd64.deb" -o /tmp/cassandra-cpp-driver_2.16.0-1_amd64.deb; \
  curl -fsSL "https://downloads.datastax.com/cpp-driver/ubuntu/18.04/cassandra/v2.16.0/cassandra-cpp-driver-dev_2.16.0-1_amd64.deb" -o /tmp/cassandra-cpp-driver-dev_2.16.0-1_amd64.deb; \
  dpkg -i /tmp/libuv1_1.35.0-1_amd64.deb; \
  dpkg -i /tmp/libuv1-dev_1.35.0-1_amd64.deb; \
  dpkg -i /tmp/cassandra-cpp-driver_2.16.0-1_amd64.deb; \
  dpkg -i /tmp/cassandra-cpp-driver-dev_2.16.0-1_amd64.deb;

RUN set -xeu; \
  update-alternatives --set php /usr/bin/php7.1; \
  update-alternatives --set phar /usr/bin/phar7.1; \
  update-alternatives --set phar.phar /usr/bin/phar.phar7.1; \
  pecl install cassandra > /dev/null; \
  echo "extension=cassandra.so" | tee -a /etc/php/7.1/cli/conf.d/20-cassandra.ini; \
  curl -fsSL "https://getcomposer.org/installer" | php -- --filename=composer --install-dir=/usr/local/bin;

USER ${USERNAME}

WORKDIR /var/www

ENTRYPOINT [ "/usr/local/bin/dumb-init", "--" ]

CMD [ "bash" ]

003-docker-compose.yml

version: '3.8'

services:
  workspace:
    build:
      context: docker
      dockerfile: Dockerfile
    tty: true
    image: workspace-php:local
    container_name: workspace-php
    hostname: workspace
    ports:
      - 8000:8000
    volumes:
      - ./:/var/www
    depends_on:
      - carepet-scylla1
      - carepet-scylla2
      - carepet-scylla3
    networks:
      - php
  carepet-scylla1:
    image: scylladb/scylla
    container_name: carepet-scylla1
    command: --smp 1
    environment:
      - "SCYLLA_JMX_ADDR=-ja 0.0.0.0"
      - "SCYLLA_JMX_REMOTE=-r"
    expose:
      - "7199"
    networks:
      php:
        ipv4_address: 10.10.5.2
  carepet-scylla2:
    image: scylladb/scylla
    container_name: carepet-scylla2
    command: --smp 1 --seeds=carepet-scylla1
    environment:
      - "SCYLLA_JMX_ADDR=-ja 0.0.0.0"
      - "SCYLLA_JMX_REMOTE=-r"
    expose:
      - "7199"
    networks:
      php:
        ipv4_address: 10.10.5.3
  carepet-scylla3:
    image: scylladb/scylla
    container_name: carepet-scylla3
    command: --smp 1 --seeds=carepet-scylla1
    environment:
      - "SCYLLA_JMX_ADDR=-ja 0.0.0.0"
      - "SCYLLA_JMX_REMOTE=-r"
    expose:
      - "7199"
    networks:
      php:
        ipv4_address: 10.10.5.4
networks:
  php:
    name: php
    driver: bridge
    ipam:
      config:
        - subnet: 10.10.5.0/24
          gateway: 10.10.5.1
    driver_opts:
      com.docker.network.bridge.name: php