Skip to content

Instantly share code, notes, and snippets.

@jovimon
Created September 23, 2016 10:45
Show Gist options
  • Save jovimon/045361974e74b0661f56f225d9099987 to your computer and use it in GitHub Desktop.
Save jovimon/045361974e74b0661f56f225d9099987 to your computer and use it in GitHub Desktop.
Logstash config to parse json files resulting from plaso processed evtx files
# Plaso is great but elasticsearch output module does not always work, so there's
# a config to parse json_line plaso output into ELK
#
# Usage (tested on Ubuntu 14.04 w/ ELK packages and Plaso git version 1.5.1_20160922):
# for i in $(ls *.evtx); do name=${i::-5}; echo $name; log2timeline.py $name.plaso $i; psort.py -o json_line -w json/$name.json $name.plaso; done
#
# Note:
# To wipe elasticsearch data: $ sudo rm -r /var/lib/elasticsearch/elasticsearch/*
# To remove logstash sincedb files so input files can be reread: $ sudo rm /var/lib/logstash/.sincedb_*
input {
file {
path => "/path/to/dir/json/*.json"
codec => json
start_position => beginning
}
}
filter {
xml {
source => "xml_string"
target => "xml"
remove_namespaces => true
xpath => [ '//Data[@Name="SubjectUserSid"]/text()', 'SubjectUserSid']
xpath => [ '//Data[@Name="SubjectUserName"]/text()', 'SubjectUserName']
xpath => [ '//Data[@Name="SubjectDomainName"]/text()', 'SubjectDomainName']
xpath => [ '//Data[@Name="SubjectLogonId"]/text()', 'SubjectLogonId']
xpath => [ '//Data[@Name="AuditSourceName"]/text()', 'AuditSourceName']
xpath => [ '//Data[@Name="EventSourceId"]/text()', 'EventSourceId']
xpath => [ '//Data[@Name="ProcessId"]/text()', 'ProcessId']
xpath => [ '//Data[@Name="ProcessName"]/text()', 'ProcessName']
# Add more xpath entries for other xml eventdata elements as they don't get through correctly.
}
# mutate {
# remove_field => [ "xml_string", "message", "host", "path", "xml.EventData", "pathspec" , "filename", "display_name", "data_type" ]
# }
}
output {
elasticsearch { hosts => localhost }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment