Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
Installing logstash server and client

Installing logstash

Server install


Install the Java prerequisite:

sudo aptitude install openjdk-7-jre-headless

Set up the package repository so that we can use aptitude (docs):

wget -qO - | sudo apt-key add -
echo "deb stable main" | sudo tee -a /etc/apt/sources.list

Install logstash itself:

sudo aptitude install logstash

Generate SSL certificates


Create the needed directories:

sudo mkdir -p /etc/pki/tls/{certs,private}

Create FQDN (DNS) based certificates (change CN as appropriate):

cd /etc/pki/tls; sudo openssl req -subj '/CN=*' -x509 -days 3650 -batch -nodes -newkey rsa:4096 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt


Lumberjack input

Choose a port.

  • Replace the 5000 example below.
  • Add the port to the inbound rules for the security group.

Create and edit a configuration file for the logstash-forwarder:

sudo vim /etc/logstash/conf.d/01-lumberjack-input.conf

with the following contents:

input {
  lumberjack {
    port => 5000
    type => "logs"
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"


Create the file:

sudo vim /etc/logstash/conf.d/10-syslog.conf

with the following contents:

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]

Debugging output


sudo vim /etc/logstash/conf.d/30-output-stdout.conf


output {
  stdout { codec => json }

S3 output


sudo vim /etc/logstash/conf.d/30-syslog-output.conf


output {
  if [type] == "syslog" {
    s3 {
      region => "us-west-2"
      bucket => "YOUR_BUCKET_NAME_HERE"
      canned_acl => "private"
      codec => "json_lines"
      size_file => 10485760
      time_file => 60

      prefix => "syslog/"

Start it

Start or restart logstash:

sudo service logstash restart


Check /var/log/logstash/ for the daemon's output

Client install


Set up the package repository:

wget -O - | sudo apt-key add -
echo 'deb stable main' | sudo tee /etc/apt/sources.list.d/logstashforwarder.list

Install logstash-forwarder itself:

sudo aptitude update
sudo aptitude install logstash-forwarder

Set up SSL certificates

Copy the Logstash server's SSL certificate:

sudo mkdir -p /etc/pki/tls/certs
# Either copy/paste or use `scp`.
sudo vim /etc/pki/tls/certs/logstash-forwarder.crt


Edit the configuration file:

sudo vim /etc/logstash-forwarder.conf

Under the network section add the following with the proper host and port:

    "servers": [ "logstash_server_private_address:5000" ],
    "timeout": 15,
    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"

Under the files section add:

      "paths": [
      "fields": { "type": "syslog" }

Start it

Start or restart logstash-forwarder:

sudo service logstash-forwarder restart

Useful tips

The logstash-forwarder keeps track of files it has already sent in:


This comment has been minimized.

Copy link

@spemble spemble commented Sep 15, 2015

I'm having trouble getting this to work - I think it may be due to new versions of the packages. The package repository cannot find any package named or described as "logstash-forwarder".


This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.