Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Installing logstash server and client

Installing logstash

Server install


Install the Java prerequisite:

sudo aptitude install openjdk-7-jre-headless

Set up the package repository so that we can use aptitude (docs):

wget -qO - | sudo apt-key add -
echo "deb stable main" | sudo tee -a /etc/apt/sources.list

Install logstash itself:

sudo aptitude install logstash

Generate SSL certificates


Create the needed directories:

sudo mkdir -p /etc/pki/tls/{certs,private}

Create FQDN (DNS) based certificates (change CN as appropriate):

cd /etc/pki/tls; sudo openssl req -subj '/CN=*' -x509 -days 3650 -batch -nodes -newkey rsa:4096 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt


Lumberjack input

Choose a port.

  • Replace the 5000 example below.
  • Add the port to the inbound rules for the security group.

Create and edit a configuration file for the logstash-forwarder:

sudo vim /etc/logstash/conf.d/01-lumberjack-input.conf

with the following contents:

input {
  lumberjack {
    port => 5000
    type => "logs"
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"


Create the file:

sudo vim /etc/logstash/conf.d/10-syslog.conf

with the following contents:

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]

Debugging output


sudo vim /etc/logstash/conf.d/30-output-stdout.conf


output {
  stdout { codec => json }

S3 output


sudo vim /etc/logstash/conf.d/30-syslog-output.conf


output {
  if [type] == "syslog" {
    s3 {
      region => "us-west-2"
      bucket => "YOUR_BUCKET_NAME_HERE"
      canned_acl => "private"
      codec => "json_lines"
      size_file => 10485760
      time_file => 60

      prefix => "syslog/"

Start it

Start or restart logstash:

sudo service logstash restart


Check /var/log/logstash/ for the daemon's output

Client install


Set up the package repository:

wget -O - | sudo apt-key add -
echo 'deb stable main' | sudo tee /etc/apt/sources.list.d/logstashforwarder.list

Install logstash-forwarder itself:

sudo aptitude update
sudo aptitude install logstash-forwarder

Set up SSL certificates

Copy the Logstash server's SSL certificate:

sudo mkdir -p /etc/pki/tls/certs
# Either copy/paste or use `scp`.
sudo vim /etc/pki/tls/certs/logstash-forwarder.crt


Edit the configuration file:

sudo vim /etc/logstash-forwarder.conf

Under the network section add the following with the proper host and port:

    "servers": [ "logstash_server_private_address:5000" ],
    "timeout": 15,
    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"

Under the files section add:

      "paths": [
      "fields": { "type": "syslog" }

Start it

Start or restart logstash-forwarder:

sudo service logstash-forwarder restart

Useful tips

The logstash-forwarder keeps track of files it has already sent in:

Copy link

spemble commented Sep 15, 2015

I'm having trouble getting this to work - I think it may be due to new versions of the packages. The package repository cannot find any package named or described as "logstash-forwarder".

Copy link

kpcool commented Dec 11, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment