Notes:Extract[Elasticsearch, Logstash, Kibana, DevOps, Fluentd, Monitoring, Logging, Data Cache, Others]:20180115

gistfile1.txt

===========================================================
Java (Elasticsearch Prerequisite)
===========================================================

1. Navigate to the tmp directory
  $ cd /tmp

2. Download the RPM installer
  $ curl --insecure --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" -L "http://download.oracle.com/otn-pub/java/jdk/8u77-b02/jdk-8u77-linux-x64.rpm" > jdk-8u77-linux-x64.rpm

3. Install the Java JDK
  $ yum -y localinstall jdk-8u73-linux-x64.rpm


===========================================================
Elasticsearch 6.x - Via RPM
===========================================================

Prerequisite : Install Java

1. Add the Elasticsearch key
  $ rrpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2. Create a file named elasticsearch.repo in the /etc/yum.repos.d/ directory. And add the following content
  [elasticsearch-6.x]
  name=Elasticsearch repository for 6.x packages
  baseurl=https://artifacts.elastic.co/packages/6.x/yum
  gpgcheck=1
  gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
  enabled=1
  autorefresh=1
  type=rpm-md

3. Install Elasticsearch
  $ sudo yum install elasticsearch

4. Configure elasticsearch (Optional)
  $ sudo vi /etc/elasticsearch/elasticsearch.yml
    network.host : <private-ip-of-elasticsearch-server>
    path :
      logs : /<project-name>/shared<efk>elasticsearch/logs
      data : /<project-name>/shared<efk>elasticsearch/data
    cluster.name : <project-name>-cluster-elk-eh-<project-name>-prod
    node.name : <project-name>-node-elk-eh-<project-name>-prod-001
  Note : Create the following directories if not yet existing
    $ sudo mkdir -p /<project-name>/shared<efk>elasticsearch/logs
    $ sudo mkdir -p /<project-name>/shared<efk>elasticsearch/data
    $ sudo chown -R elasticsearch:elasticsearch /<project-name>/shared<efk>elasticsearch

5. Check if elasticsearch is running
  curl -XGET '<elasticsearch-private-ip>:9200/?pretty'


===========================================================
Fluentd - Prerequisites
===========================================================

Set Up NTP
-----------------------------------------------------------

$ sudo apt-get update
$ sudo apt-get install ntp
$ sudo service ntp restart

||

Install NTP
$ yum install ntp

Start the service
$ systemctl start ntpd
$ systemctl enable ntpd

Check the status
$ systemctl status ntpd

-----------------------------------------------------------

1. Check the current limit.
  $ ulimit -n

2. If it ouputs 1024, it is insuficcient. Update the setting to 65536
  $ nano /etc/security/limits.conf
  Add or update to the following
  root soft nofile 65536
  root hard nofile 65536
  * soft nofile 65536
  * hard nofile 65536

3. Reboot the machine

4. Check the updated limit
  $ ulimit -n

Optimize Network Kernel Parameters
-----------------------------------------------------------
1. Add the following parameters to the /etc/sysctl.conf
  net.core.somaxconn = 1024
  net.core.netdev_max_backlog = 5000
  net.core.rmem_max = 16777216
  net.core.wmem_max = 16777216
  net.ipv4.tcp_wmem = 4096 12582912 16777216
  net.ipv4.tcp_rmem = 4096 12582912 16777216
  net.ipv4.tcp_max_syn_backlog = 8096
  net.ipv4.tcp_slow_start_after_idle = 0
  net.ipv4.tcp_tw_reuse = 1
  net.ipv4.ip_local_port_range = 10240 65535

2. Execute the following or reboot your node to have the changes take effect.
  $ sysctl -p


===========================================================
Fluentd
===========================================================

1. Download the file install-redhat-td-agent2.sh using curl
  $ curl -L https://toolbelt.treasuredata.com/sh/install-redhat-td-agent2.sh | sh

2. Execute the installer
  $ sh install-redhat-td-agent2.sh

3. Launch the background service / Daemon
  $ /etc/init.d/td-agent start 
  Other comands:
  # check the status
  $ /etc/init.d/td-agent status
  # stop
  $ /etc/init.d/td-agent stop
  # restart
  $ /etc/init.d/td-agent restart
  Note: Make sure your configuration file is located at /etc/td-agent/td-agent.conf.

4. Test the installation by sending sample logs
  $ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test


===========================================================
Kibana 6.x via RPM
===========================================================

1. Download the and install the public signing key
  $ rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

2. Create a file called kibana.repo in the /etc/yum.repos.d/. Add the following content
  [kibana-6.x]
  name=Kibana repository for 6.x packages
  baseurl=https://artifacts.elastic.co/packages/6.x/yum
  gpgcheck=1
  gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
  enabled=1
  autorefresh=1
  type=rpm-md

3. Install Kibana
  $ sudo yum install kibana

4. Update the configuration (Optional)
  $ sudo nano /etc/kibana/kibana.yml
    server.host: "0.0.0.0"
    server.basePath : "/kibana"
    elasticsearch.url: "http://<elasticsearch-ip>:<elasticsearch-port>"
    logging.dest : /<project-name>/shared<efk>kibana/logs/kibana.log
    Note : Create the directories if not yet existing.
    $ sudo mkdir -p /<project-name>/shared<efk>kibana/logs
    $ sudo chown -R kibana:kibana /<project-name>/shared<efk>kibana/logs

5. Start Kibana
  $ sudo -i service kibana start

6. Stop Kibana using the following
  $ sudo -i service kibana stop