Skip to content

Instantly share code, notes, and snippets.

@nofxx
Last active December 24, 2015 13:49
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nofxx/6807671 to your computer and use it in GitHub Desktop.
Save nofxx/6807671 to your computer and use it in GitHub Desktop.
Centralized Ruby/Rails/MongoDB/Redis/Sidekiq/Elasticsearch/Haproxy/PG/OS -> syslog-ng -> logstash -> elasticsearch -> kibana =D
#
# Logstash conf!
#
input {
tcp {
host => "0.0.0.0"
port => 514
type => syslog
}
udp {
host => "0.0.0.0"
port => 514
type => syslog
}
}
filter {
grok {
match => { "message" => "(?:<%{INT:priority}>)?%{SYSLOGBASE2} (?:\s?%{LOGLEVEL:log_level} )?(?:\s?%{WORD:log_format}: )?%{GREEDYDATA:syslog_message}" }
}
syslog_pri { }
if ("_grokparsefailure" in [tags]) {
mutate {
replace => ["@message", "TOPARSE: %{message}"]
}
} else if [log_format] == "json" {
mutate {
gsub => ["syslog_message", "@timestamp", "syslog_timestamp"]
}
json {
source => "syslog_message"
}
mutate {
replace => ["@message", "%{message}"]
}
} else {
mutate {
replace => [ "@message", "%{syslog_message}" ]
}
}
mutate {
remove_field => [
"syslog_hostname", "syslog_message", "syslog_timestamp",
"syslog_severity_code", "syslog_facility_code",
"message" #facility_label", "severity_label"
]
}
}
output {
stdout {
debug => true
}
elasticsearch {
# bind_host => "foo"
host => "localhost"
embedded => false
}
}
require 'log4r'
require 'log4r/yamlconfigurator'
require 'log4r/outputter/syslogoutputter'
module App
class Application < Rails::Application
log4r_config= YAML.load_file(File.join(File.dirname(__FILE__),"log4r.yml"))
Log4r::YamlConfigurator.decode_yaml( log4r_config['log4r_config'] )
config.logger = Log4r::Logger['rails']
config.log_level = Log4r::DEBUG
Mongoid.logger = Log4r::Logger['mongoid']
Moped.logger = Log4r::Logger['mongoid']
...
# Lograge
config.lograge.enabled = true
config.lograge.formatter = Lograge::Formatters::Logstash.new
destination logstash { udp("logs.xxx.com"); };
log { source(src); destination(logstash); };
@nofxx
Copy link
Author

nofxx commented Oct 3, 2013

Syslog parameters correctly parsed: LOGLEVEL (INFO|WARN...), LOGTITLE (logtitle: some log...)

@anlek
Copy link

anlek commented Aug 16, 2014

How does your log4r.yml look like?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment