Testing your logstash filters

1.4.2 template.rb

# encoding: utf-8
require "test_utils"

describe "Logstash filters test suite" do
  extend LogStash::RSpec
  
  describe "Logstash filters for my_type inputs" do
    #You are loading your real configuration, no repetition
    config File.new("/etc/logstash/filters.conf").read

    #Using the type method, you do not have to repeat it in each sample
    type "my_type"
    
    #When passing a string, it is used as the `message` event field
    sample "message string" do
      #Do you testing here
      insist { subject["message"] } == "message string"
    end 
    
    #When passing a hash, it is used to fill the event fields
    sample({"custom_field" => "value", "message" => "message string"}) do 
      #Do you testing here
      insist { subject["message"] } == "message string"
      insist { subject["custom_field"] } == "value"
    end
    
    #When passing an array, you can test configuration containing a multiline filter
    sample [
      "2013-09-30 13:31:47,230 ERROR lu.intrasoft.xtnet.web.mainxtnet.SetupDimSelectionAction     - m.warn.noobjectinsession",
      "java.lang.Exception: m.warn.noobjectinsession",
      "  at lu.intrasoft.xtnet.web.utils.QueryDatasetInitialiser.<init>(QueryDatasetInitialiser.java:65)",
      "truncated for test"
    ] do
      #Do you testing here
    end
  end
end

blabla.asciidoc

Using Logstash, I found very useful to separate my configuration files between inputs, filters and outputs. For example, the easiest would be to have 3 files :

• /etc/logstash/inputs.conf

• /etc/logstash/filters.conf

• /etc/logstash/outputs.conf

The direct benefit is that you can now easily write test for the filters part using rspec and run it with logstash bin/logstash rspec spec/filters_spec.rb to validate your configuration after changes

How to build samples

When you do not know the event format produced by your inputs, a good solution to build samples is to use a specific config with no filters and a single debug input, like

input {
  #add the input config to analyse
}
output {
  stdout {
    codec => rubydebug
  }
}

TODOs

• 1.5 template