Fix Year Timestamps in Logstash

_README.md

Problem

Ingesting last year's syslog loglines will use this year's year since syslog format doesn't have a year in it (SYSLOGTIMESTAMP %{MONTH} +%{MONTHDAY} %{TIME}). I'm reading in the syslog lines from the file input and am still playing around with ES config, so I'm constantly DELETEing indexes, so I need to keep re-importing. This isn't a production-style solution.

Extra Info

I'm currently processing the timestamp pulled out with the syslog pattern using the date filter

    date {
        match => [ "timestamp", "MMM dd HH:mm:ss",
                                "MMM  d HH:mm:ss"]
        timezone => "Etc/UTC"
    }

Hack Solution

If the @timestamp field is > now, subtract 1 year from it. Our syslog is always in Etc/UTC so I just blindly reassign it to -00:00.

fixyear.rb

# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"

# This filter looks at the date, and if it's in the future, subtracts a year
#
# This assumes that your datetimes are already in UTC and just assigns them
# that way, ignoring your timezone in the @timestamp
# @TODO: Respect Timestamps
#
# The config looks like this:
#
#     filter {
#       fixyear { }
#     }
#

class LogStash::Filters::FixYear < LogStash::Filters::Base

  config_name "fixyear"
  milestone 1

  public
  def initialize(config = {})
    super

    # @threadsafe = false
  end # def initialize

  public
  def register
    # @cache = ThreadSafe::Cache.new

  end # def register

  public
  def filter(event)
    return unless filter?(event)

    return if event['@timestamp'].nil?
    if event['@timestamp'] > Time.new()
        year = event['@timestamp'].strftime('%Y').to_i - 1
        datestring = event['@timestamp'].strftime("#{year}-%m-%dT%T-00:00")
        event['@timestamp'] = Time.strptime(datestring, '%FT%T%:z')
    end
  end
end