Skip to content

Instantly share code, notes, and snippets.

@untergeek
Last active November 3, 2021 16:56
Show Gist options
  • Star 14 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save untergeek/0373ee85a41d03ae1b78 to your computer and use it in GitHub Desktop.
Save untergeek/0373ee85a41d03ae1b78 to your computer and use it in GitHub Desktop.
JSON rsyslog output for Logstash
template(name="ls_json"
type="list"
option.json="on") {
constant(value="{")
constant(value="\"@timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"@version\":\"1")
constant(value="\",\"message\":\"") property(name="msg")
constant(value="\",\"host\":\"") property(name="hostname")
constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
constant(value="\",\"facility\":\"") property(name="syslogfacility-text")
constant(value="\",\"programname\":\"") property(name="programname")
constant(value="\",\"procid\":\"") property(name="procid")
constant(value="\"}\n")
}
input {
udp {
port => 10514
codec => "json"
type => "syslog"
}
}
filter {
# This replaces the host field (UDP source) with the host that generated the message (sysloghost)
if [sysloghost] {
mutate {
replace => [ "host", "%{sysloghost}" ]
remove_field => "sysloghost" # prune the field after successfully replacing "host"
}
}
}
output {
# whatever
}
# Other config goes above here...
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
# Set the target host here by replacing 127.0.0.1
*.* @127.0.0.1:10514;ls_json
@Thinkscape
Copy link

rsyslog doesn't understand that template... is it -ng or some other variant ?

@untergeek
Copy link
Author

You probably have an rsyslog version older than 6 if this does not work.

@cvaldess
Copy link

Hi,
don´t work for rsyslog-7.4.7 and logstash:5.4.0

any working version ??

@DFazeli
Copy link

DFazeli commented Sep 10, 2018

I use this tutorial but get Error.
all my config :
** rsyslog side**
: /etc/rsylog.conf

*.*  action(type="omfwd" target="192.168.163.41" port="514" protocol="udp"
            action.resumeRetryCount="100"
            queue.type="linkedList" queue.size="10000" template="ls_json")

/etc/rsyslog.d/json.conf

template(name="ls_json"
         type="list"
         option.json="on") {
           constant(value="{")
             constant(value="\"@timestamp\":\"")     property(name="timereported" dateFormat="rfc3339")
             constant(value="\",\"@version\":\"1")
             constant(value="\",\"message\":\"")     property(name="msg")
             constant(value="\",\"host\":\"")        property(name="hostname")
             constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")
             constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")
             constant(value="\",\"programname\":\"") property(name="programname")
             constant(value="\",\"procid\":\"")      property(name="procid")
             constant(value="\"}\n")
         }

Logstash Side

logstash.conf

input {
    udp {
        port => 10514
        host => "192.168.163.41"
        tags => "syslog"
        codec => "json"
        }
}
output {
  if "rsyslog" in [tags] {
    gelf {
        host => "192.168.163.163"
        sender => "192.168.163.41"
       }
     }
}

I get Error when recive log from rsyslog to logstash

Sep  9 19:39:46 logread logstash: [2018-09-09T19:39:46,244][ERROR][logstash.codecs.json     ] JSON parse error, original data now in message field {:error=>#<LogStash::Json::ParserError: Unexpected character ('@' (code 64)): was expecting comma to separate Object entries
Sep  9 19:39:46 logread logstash: at [Source: (String)"{"@timestamp":"2018-09-09T19:39:46.239257-04:00","@version":"1","message":"\"2018-09-09T19:39:14.023499-04:00\",\"@version\":\"1\",\"message\":\"ddddddddddd\",\"host\":\"kafka1\",\"severity\":\"notice\",\"facility\":\"user\",\"programname\":\"root\",\"procid\":\"-\"}","sysloghost":"192.168.163.37","severity":"notice","facility":"user","programname":"{"@timestamp"","procid":"-"}
Sep  9 19:39:46 logread logstash: "; line: 1, column: 356]>, :data=>"{\"@timestamp\":\"2018-09-09T19:39:46.239257-04:00\",\"@version\":\"1\",\"message\":\"\\\"2018-09-09T19:39:14.023499-04:00\\\",\\\"@version\\\":\\\"1\\\",\\\"message\\\":\\\"ddddddddddd\\\",\\\"host\\\":\\\"kafka1\\\",\\\"severity\\\":\\\"notice\\\",\\\"facility\\\":\\\"user\\\",\\\"programname\\\":\\\"root\\\",\\\"procid\\\":\\\"-\\\"}\",\"sysloghost\":\"192.168.163.37\",\"severity\":\"notice\",\"facility\":\"user\",\"programname\":\"{\"@timestamp\"\",\"procid\":\"-\"}\n"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment