gzholder

/** @scratch /configuration/config.js/1
 *
 * == Configuration
 * config.js is where you will find the core Kibana configuration. This file contains parameter that
 * must be set before kibana is run for the first time.
 */
define(['settings'],
function (Settings) {

gzholder

Here I will go over how to setup Logstash, Kibana, Redis, and Elasticsearch in an EC2 environment behind a public Load Balancer. The setup I'll be doing will have:

1) One server for Redis to act as the broker/buffer to receive logs.
2) One server using Logstash receive logs from Redis and parse/index them over to Elasticsearch.
3) One server for Elasticsearch to receive logs and Kibana to view them in a browser.
4) One server to send the logs using logstash.
5) One public Load Balancer.

This may seem like a lot but follow these steps and you'll get the hang of it :)
What you will need:

gzholder

 cloud:
    aws:
      access_key: ACCESS_KEY
      secret_key: SECRET_KEY

 discovery:
   type: ec2
   ec2:
     groups: SECURITY_GROUP-NAME

gzholder

#!/bin/sh
#
# chkconfig: 2345 70 40
# description: logstash startup script
#
LOGSTASH=/opt/logstash/logstash-1.3.3-flatjar.jar
CONF=/opt/logstash/shipper.conf
#TMPDIR=/dev/shm
#export TMPDIR

gzholder

 [root:~]$ df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        1.9G     0  1.9G   0% /dev
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           1.9G  1.1M  1.9G   1% /run
/dev/xvda1      9.8G  1.5G  7.8G  17% /
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/xvda2      394G  150M  374G   1% /data

gzholder

input {
  redis  {
    host => "10.110.65.91"
    data_type => "list"
    key => "logstash"
    codec => json
  }
}

output {

gzholder

input {
  file {
    sincedb_path => "/opt/logstash/"
    path => "/var/log/messages"
    path => "/var/log/secure"
    type => "MESSAGES"
  }
}

filter {