Skip to content

Instantly share code, notes, and snippets.

@LVLAaron
Last active November 16, 2016 22:06
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save LVLAaron/8085923 to your computer and use it in GitHub Desktop.
Save LVLAaron/8085923 to your computer and use it in GitHub Desktop.
Config files used to send Windows Eventlog data with NXLOG to Logstash
{
"_index": "logstash-2013.12.23",
"_type": "logs",
"_id": "eMsyB2q4TA6sH0bW3kCBZA",
"_score": null,
"_source": {
"EventTime": "2013-12-22 22:45:27",
"Hostname": "aapc",
"Keywords": -9187343239835812000,
"EventType": "INFO",
"SeverityValue": 2,
"Severity": "INFO",
"EventID": 7036,
"SourceName": "Service Control Manager",
"ProviderGuid": "{555908D1-A6D7-4695-8E1E-26931D2012F4}",
"Version": 0,
"Task": 0,
"OpcodeValue": 0,
"RecordNumber": 57035,
"ProcessID": 632,
"ThreadID": 5772,
"Channel": "System",
"Message": "The nxlog service entered the stopped state.",
"param1": "nxlog",
"param2": "stopped",
"EventReceivedTime": "2013-12-22 22:45:29",
"SourceModuleName": "in",
"SourceModuleType": "im_msvistalog",
"@timestamp": "2013-12-23T03:45:40.544Z",
"@version": "1",
"type": "WindowsEventLog",
"host": "10.10.1.2",
"tags": [
"_grokparsefailure"
],
"priority": 13,
"severity": 5,
"facility": 1,
"facility_label": "user-level",
"severity_label": "Notice"
},
"sort": [
1387770340544
]
}
define ROOT C:\Program Files (x86)\nxlog
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
<Extension syslog>
Module xm_syslog
</Extension>
<Extension json>
Module xm_json
</Extension>
<Input in>
Module im_msvistalog
Exec $raw_event = to_json();
</Input>
<Processor buffer>
Module pm_buffer
MaxSize 1024
Type Mem
WarnLimit 512
</Processor>
<Output out>
Module om_tcp
Host 10.1.150.59
Port 1514
</Output>
<Route r>
Path in => buffer => out
</Route>
input {
syslog {
type => "WindowsEventLog"
port => 1514
codec => json {
charset => [ "CP1252" ]
}
}
}
filter {
if [type] == "WindowsEventLog" {
mutate { remove_field => [ "Keywords","SourceModuleType","facility","facility_label","host","priority","severity","severity_label","SourceModuleName","RecordNumber","ProviderGuid","Task","ThreadID","ProcessID","Severity","SeverityValue","OpcodeValue","Opcode","ThreadID" ] }
}
}
output {
elasticsearch {
cluster => "Mint16ES"
}
}
@dantheautomator
Copy link

I don't have access to a windows machine, so I have not tried it with im_msvistalog, but I do similar with im_file. If you want to drop some fields inside nxlog and not send them to logstash at all try:

Module im_msvistalog Exec delete($Keywords); delete($SourceModuleType); to_json();

I know the below config works for me on linux (partial of my whole config)

Module im_file File "/var/log/messages" SavePos TRUE Exec parse_syslog_bsd(); Exec delete($EventReceivedTime); Exec to_json();

See my whole config for nxlog and logstash here: https://gist.github.com/dantheautomator/9255555

@colegatron
Copy link

It would be better to do the field remove in the NXlog side, to avoid to transfer everything all over the network only to drop it on the logstash side. It is what I am looking for right now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment