Skip to content

Instantly share code, notes, and snippets.

Gabriel Holder gzion7

Block or report user

Report or block gzion7

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View kibana
/** @scratch /configuration/config.js/1
*
* == Configuration
* config.js is where you will find the core Kibana configuration. This file contains parameter that
* must be set before kibana is run for the first time.
*/
define(['settings'],
function (Settings) {
@gzion7
gzion7 / Logstash, Elasticsearch in an EC2_AWS enviroment
Created Mar 11, 2014
Logstash, Elasticsearch in an EC2_AWS enviroment
View Logstash, Elasticsearch in an EC2_AWS enviroment
Here I will go over how to setup Logstash, Kibana, Redis, and Elasticsearch in an EC2 environment behind a public Load Balancer. The setup I'll be doing will have:
1) One server for Redis to act as the broker/buffer to receive logs.
2) One server using Logstash receive logs from Redis and parse/index them over to Elasticsearch.
3) One server for Elasticsearch to receive logs and Kibana to view them in a browser.
4) One server to send the logs using logstash.
5) One public Load Balancer.
This may seem like a lot but follow these steps and you'll get the hang of it :)
What you will need:
View Elastic_Config
cloud:
aws:
access_key: ACCESS_KEY
secret_key: SECRET_KEY
discovery:
type: ec2
ec2:
groups: SECURITY_GROUP-NAME
View Logstash Init
#!/bin/sh
#
# chkconfig: 2345 70 40
# description: logstash startup script
#
LOGSTASH=/opt/logstash/logstash-1.3.3-flatjar.jar
CONF=/opt/logstash/shipper.conf
#TMPDIR=/dev/shm
#export TMPDIR
View df -h
[root:~]$ df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.9G 0 1.9G 0% /dev
tmpfs 1.9G 0 1.9G 0% /dev/shm
tmpfs 1.9G 1.1M 1.9G 1% /run
/dev/xvda1 9.8G 1.5G 7.8G 17% /
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/xvda2 394G 150M 374G 1% /data
View Indexer.conf
input {
redis {
host => "10.110.65.91"
data_type => "list"
key => "logstash"
codec => json
}
}
output {
View Shipper.conf
input {
file {
sincedb_path => "/opt/logstash/"
path => "/var/log/messages"
path => "/var/log/secure"
type => "MESSAGES"
}
}
filter {
You can’t perform that action at this time.