Skip to content

Instantly share code, notes, and snippets.

View andrewkroh's full-sized avatar

Andrew Kroh andrewkroh

View GitHub Profile
@andrewkroh
andrewkroh / access.log
Created May 5, 2020 19:07
Filebeat Squid Proxy Access Log Parsing
1348870236.160 0 192.168.0.35 TCP_DENIED/403 3293 GET http://armdl.adobe.com/pub/adobe/acrobat/win/9.x/9.5.2/misc/AcrobatUpd952_all_incr.msp - NONE/- text/html
1348870236.273 0 192.168.0.35 TCP_DENIED/403 3274 GET http://armdl.adobe.com/pub/adobe/acrobat/win/9.x/9.5.2/misc/AcrobatUpd952_all_incr.msp - NONE/- text/html
1348870236.386 0 192.168.0.35 TCP_DENIED/403 3274 GET http://armdl.adobe.com/pub/adobe/acrobat/win/9.x/9.5.2/misc/AcrobatUpd952_all_incr.msp - NONE/- text/html
1348870236.499 0 192.168.0.35 TCP_DENIED/403 3274 GET http://armdl.adobe.com/pub/adobe/acrobat/win/9.x/9.5.2/misc/AcrobatUpd952_all_incr.msp - NONE/- text/html
1348870237.550 0 192.168.0.35 TCP_DENIED/403 3269 GET http://armdl.adobe.com/pub/adobe/acrobat/win/9.x/9.4.6/misc/AcrobatUpd946_all_incr.msp - NONE/- text/html
1348870274.248 59875 192.168.0.35 TCP_MISS/503 0 CONNECT client84.dropbox.com:443 - DIRECT/- -
1348870284.249 59872 192.168.0.35 TCP_MISS/503 0 CONNECT client62.dropbox.com:443 - DIRECT/- -
1348870
@andrewkroh
andrewkroh / msobjs.c
Created July 23, 2019 20:26
Extact the msobjs.dll message table
#include <windows.h>
#include <stdio.h>
int ProcessBlock(MESSAGE_RESOURCE_DATA* data, MESSAGE_RESOURCE_BLOCK* block)
{
MESSAGE_RESOURCE_ENTRY* entry = (MESSAGE_RESOURCE_ENTRY*) ((unsigned char*)data + block->OffsetToEntries);
for (DWORD id = block->LowId; id <= block->HighId; id++)
{
if (entry->Flags == 0x0001) // wide char
printf("%d, %ls", id, entry->Text);
@andrewkroh
andrewkroh / filebeat.filestream-evtx_dump.yml
Last active November 1, 2023 01:17
Ingest Windows event logs to Elasticsearch on Linux using evtx_dump and Filebeat
---
filebeat.inputs:
# Consume output from
# evtx_dump --dont-show-record-number -o xml <file.evtx> > /tmp/samples/file.evtx.xml
# See https://github.com/omerbenamram/evtx.
- type: filestream
id: evtx_dump_xml
parsers:
- multiline:
@andrewkroh
andrewkroh / filebeat.udp-cef-extrahop.yml
Last active October 24, 2023 20:41
ExtraHop CEF logging to Filebeat
filebeat.inputs:
- host: localhost:9514
id: udp-extrahop-cef-9514
type: udp
processors:
- convert:
mode: copy
fields:
- { from: "message", to: "event.original" }
@andrewkroh
andrewkroh / filebeat.cel.yml
Created October 1, 2023 19:32
Filebeat CEL input - ingest complete config file when it changes
---
filebeat.inputs:
- type: cel
id: config-123-watcher
interval: 1m
resource:
url: file:///etc/conf.d/foo.conf
program: |
file(state.url).as(content, content.sha256().hex().as(hash, {
@andrewkroh
andrewkroh / filebeat.journald-kubelet.yml
Last active September 22, 2023 20:29
Filebeat - processing kubelet json logs read from journald
filebeat.inputs:
- type: journald
processors:
# For https://kubernetes.io/docs/concepts/cluster-administration/system-logs/#json-log-format
- if:
and:
- equals.journald.process.name: kubelet
- regexp.message: '^{'
then:
# 'kubelet' should be mapped as a flattened field in ES because
@andrewkroh
andrewkroh / beats-logstash-tls.md
Last active August 30, 2023 06:40
Using TLS between Beats and Logstash

Using TLS between Beats and Logstash

Beats to Logstash over TLS

The purpose of this document is to help with configuring and troubleshooting using TLS on the connection between Beats and Logstash.

Configuration

You must configure TLS on both the client and server to make this work. This

@andrewkroh
andrewkroh / howto.txt
Last active May 5, 2023 10:07
Microsoft-Windows-Windows Defender Event Log Message Resources
800, AntiVirus
801, AntiSpyware
802, Antimalware
803, Full
804, Delta
805, Full Scan
806, Quick Scan
807, Custom Scan
808, Remove
809, Quarantine
@andrewkroh
andrewkroh / beat.yml
Last active March 10, 2023 15:04
Beat script processor to filter out IPv6
processors:
- script:
# This uses a Beat script processor to include only ipv4 addresses
# in the host.ip field. This would need to placed after the add_host_metadata
# processor.
#
# It would be a lot more efficient to have add_host_metadata allow controlling
# what addresses were included because this has to execute for every event.
#
# References:
@andrewkroh
andrewkroh / instructions.md
Last active March 6, 2023 19:09
Adding event.ingested and lag calculations to Winlogbeat events

Adding event.ingested and lag calculations to Winlogbeat events

Create an Ingest Pipeline that will add four fields:

  • event.ingested - Time when the event was processed by Elasticsearch.
  • event.lag.read - Time difference in milliseconds between @timestamp and event.created. This measures how long it took for Winlogbeat read the event from the event log (for WEC this includes the delivery time from forwarder to collector).
  • event.lag.ingest - Time difference in milliseconds between event.created and event.ingested. This measures the time between Winlogbeat reading the event (time when it "created" the document) to when it was written to Elasticsearch.