Odoo Logstash Filter

filter

filter {
  if [type] == "odoo" {
    grok {
      match => { "message" => "%{ODOOLOG}" }
    }
    date {
      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
    }
  }
}

input

input {
  file {
    type => "odoo"
    path => "/path/to/server.log"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601} "
      negate => true
      what => previous
    }
  }
}

pattern

WERKZEUGREQUESTTIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR} %{TIME}
WERKZEUGREQUEST %{IPORHOST:client} %{USER:ident} %{USER:auth} \[%{WERKZEUGREQUESTTIMESTAMP:request_timestamp}\] "%{WORD:request_type} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} -
ODOOMISC %{GREEDYDATA}
ODOOLOG %{TIMESTAMP_ISO8601:timestamp} %{POSINT:pid} %{LOGLEVEL:level} (?:%{USERNAME:user}|\?) %{PROG:module}: (?:%{WERKZEUGREQUEST}|%{ODOOMISC:misc})

If using multiline filter rather than codec, the date filter should come before grok, otherwise grok will fail to parse, and confusingly only present the last line as the message.

e.g.

filter {
  if [type] == "odoo" {
    multiline {
      patterns_dir => "/home/elk/current/logstash/patterns"
      negate => 'true'
      pattern => "^%{TIMESTAMP_ISO8601} "
      what => 'previous'
    }
    date {
      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
    }
    grok {
      patterns_dir => "/home/elk/current/logstash/patterns"
      match => { "message" => "%{ODOOLOG}" }
    }
  }
}

It could be interesting to consider the case of http error response code with stack trace separately. This append with only a specific module name : openerp.addons.website.models.ir_http followed by answer code and code name, and then stack trace.

HTTPCODE [1-5][0-9][0-9]
HTTPDESCRIPTION [a-zA-Z0-9\s_-]*
HTTPMODULE openerp.addons.website.models.ir_http: %{HTTPCODE:http_code} %{HTTPDESCRIPTION:http_description}:%{GREEDYDATA:stacktrace}
WERKZEUGREQUESTTIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR} %{TIME}
WERKZEUGREQUEST %{IPORHOST:client} %{USER:ident} %{USER:auth} [%{WERKZEUGREQUESTTIMESTAMP:request_timestamp}] "%{WORD:request_type} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} -
ODOOMISC %{GREEDYDATA}
OTHERMODULE %{PROG:module}: (?:%{WERKZEUGREQUEST}|%{ODOOMISC:misc})
ODOOLOG %{TIMESTAMP_ISO8601:timestamp} %{POSINT:pid} %{LOGLEVEL:level} (?:%{USERNAME:user}|?) (?:%{HTTPMODULE}|%{OTHERMODULE})

Not yet tested, just beginning to look after odoo log analytics

In combination with Filebeat :
filebeat.yml : (using regex cause filebeat doesn't know TIMESTAMP_ISO8601 grok pattern)

  - {
    document_type: odoo,
    paths: ['{{ odoo_config_logfile }}'],
    multiline: {
      pattern: "^(?>\\d\\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[T ](?:2[0123]|[01]?[0-9]):?(?:[0-5][0-9])(?::?(?:(?:[0-5][0-9]|60)(?:[:.,][0-9]+)?))?(?:Z|[+-](?:2[0123]|[01]?[0-9])(?::?(?:[0-5][0-9])))?",
      negate: true,
      match: after
      }
    }

Logstash filter :

filter {
  if [type] == "odoo" {
    date {
      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
    }
    grok {
      match => { "message" => "%{ODOOLOG}" }
    }

  }
}