Skip to content

Instantly share code, notes, and snippets.

@martinseener
martinseener / gist:5238576
Last active March 23, 2021 01:20
Grok ESXi 5.x Pattern (for Logstash) (including puppet format with special escaping!)
filter {
grok {
pattern => ['(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGHOST:logsource}) (?:%{SYSLOGPROG}): (?<messagebody>(?:\[(?<esxi_thread_id>[0-9A-Z]{8,8}) %{DATA:esxi_loglevel} \'%{DATA:esxi_service}\'\] %{GREEDYDATA:esxi_message}|%{GREEDYDATA}))']
type => "esxi"
}
}
# Puppet format with escaping
pattern => [ "(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:.* (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}|(?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG}): (?:(?:\[[0-9A-Z]{8,8}) (?:%{GREEDYDATA:esxi_loglevel}) \\\'(?:%{GREEDYDATA:esxi_service})\\\'] (?:%{GREEDYDATA:message})|(?:%{GREEDYDATA:message}))" ],
@martinseener
martinseener / gist:5247292
Last active July 1, 2019 21:38
Grok Sophos UTM 9.x Pattern (for logstash) (Simple)
filter {
grok {
pattern => ['(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGHOST:logsource}) (?:%{YEAR}): (?:%{MONTHNUM}):(?:%{MONTHDAY})-(?:%{HOUR}):(?:%{MINUTE}):(?:%{SECOND}) (?:%{SYSLOGHOST}) (?:%{SYSLOGPROG}): (?<messagebody>(?:id=\"%{INT:utm_id}\" severity=\"%{LOGLEVEL:utm_severity}\" sys=\"%{DATA:utm_sys}\" sub=\"%{DATA:utm_sub}\" name=\"%{DATA:utm_name}\" action=\"%{DATA:utm_action}\" fwrule=\"%{INT:utm_ulogd_fwrule}\" initf=\"%{DATA:utm_ulogd_initf}\" outitf=\"%{DATA:utm_ulogd_outif}\" (?:srcmac=\"%{GREEDYDATA:utm_ulogd_srcmac}\" dstmac=\"%{GREEDYDATA:utm_ulogd_dstmac}\"|srcmac=\"%{GREEDYDATA:utm_ulogd_srcmac}\") srcip=\"%{IP:utm_srcip}\" dstip=\"%{IP:utm_dstip}\" proto=\"%{INT:utm_protocol}\" length=\"%{INT:utm_ulogd_pkglength}\" tos=\"%{DATA:utm_ulogd_tos}\" prec=\"%{DATA:utm_ulogd_prec}\" ttl=\"%{INT:utm_ulogd_ttl}\" srcport=\"%{INT:utm_srcport}\" dstport=\"%{INT:utm_dstport}\" tcpflags=\"%{DATA:utm_ulogd_tcpflags}\"|id=\"%{INT:utm_id}\" severity=\"%{LOGLEVEL:utm
@martinseener
martinseener / gist:5318215
Last active December 15, 2015 20:19
Grok Apache2 Custom-Pattern (streamlined Custom-Log to comply Error Log and enhanced Custom Log by adding more information)
filter {
grok {
pattern => ['(?:%{SYSLOGTIMESTAMP:timestamp}|%{TIMESTAMP_ISO8601:timestamp8601}) (?:%{SYSLOGHOST:logsource}) (?:%{SYSLOGPROG}): (?<messagebody>\[%{DAY} %{MONTH} %{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{YEAR}\] \[%{LOGLEVEL:severity}\] \[client %{IP:a2_client_ip}\] (?:Request: \"%{GREEDYDATA:a2_request}\" %{INT:a2_http_code} (?:%{GREEDYDATA:a2_sent_bytes}|%{INT:a2_sent_bytes}) %{INT:a2_response_time} %{GREEDYDATA:a2_referer}\" \"%{GREEDYDATA:a2_user_agent}\" \"%{GREEDYDATA:a2_ssl_protocol}\" \"%{GREEDYDATA:a2_ssl_cipher}\"|Request: \"%{GREEDYDATA:a2_request}\" %{INT:a2_http_code} (?:%{GREEDYDATA:a2_sent_bytes}|%{INT:a2_sent_bytes}) %{INT:a2_response_time} \"%{GREEDYDATA:a2_referer}\" \"%{GREEDYDATA:a2_user_agent}\")|%{GREEDYDATA})']
type => "apache2"
}
}
# Apache 2 Logformat for customlog with SSL/TLS Logging (last part "rsysloghostnames" can be changed to anything else)
LogFormat "[%{%a %b %d %H:%M:%S %Y}t] [info] [client %h] Request: \"%m http(s)://%{Host}i%U%q %H\" %>s %b %D \"%{
@martinseener
martinseener / getOldestFile.sh
Created July 22, 2013 10:15
Bash Funcion which returns the oldest file for specified path (recursive)
getOldestFile(){
# Returns oldest file in path specified by $1
if [ -z "$1" ]; then
echo "No Path argument given - exiting."
exit 1
fi
find $1 -type f -printf '%T+ %p\n' | sort | head -1 | cut -d' ' -f2
}
@martinseener
martinseener / gist:6865383
Last active December 24, 2015 21:29
Grok Advanced Syslog Pattern wich adds postfix/smtp support for outgoing mails
%{SYSLOGBASE2} (?<messagebody>(?:%{DATA:mail_message_id}: to=<%{DATA:mail_recipient}>, relay=%{DATA:mail_relay_url}\[%{IP:mail_relay_ip}\]:%{INT:mail_relay_port}, delay=%{BASE10NUM:mail_processing_time}, delays=%{GREEDYDATA:mail_processing_delays}, dsn=%{DATA:mail_delivery_status_notification}, status=%{GREEDYDATA:mail_relay_status}|%{GREEDYDATA}))
@martinseener
martinseener / gist:9411739
Last active August 29, 2015 13:57
Shrinking Virtual Machine Boxes

Shrinking Virtual Boxes

  • Remove manuals

rm -rf /usr/share/doc

  • Remove VirtualBox OSE

rm -rf /usr/src/vboxguest*
rm -rf /usr/src/virtualbox-ose-guest*

#! /bin/bash
#
# This script needs "fpm". If you dont have it,
# run "gem install fpm"
#
# You also need to "apt-get install python-setuptools" (otherwise fpm fails)
clean() {
rm -rf whisper-0.9.9 carbon-0.9.9 graphite-web-0.9.9
@martinseener
martinseener / carbon-cache.sh
Last active August 29, 2015 14:01
Debian init script for Graphite's carbon-cache daemon
#!/bin/sh
### BEGIN INIT INFO
# Provides: carbon-cache
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: carbon-cache init script
# Description: An init script for Graphite's carbon-cache daemon.
### END INIT INFO
@martinseener
martinseener / modx-revolution-backup.php
Created May 13, 2014 10:04
MODX Revolution Quick File and DB Backup PHP-CLI Script
<?php
if ( !shell_exec("type type")) { echo "Weak your PHP powers are, Luke."; die; }
$dir = "../backup"; /* Verzeichnis außerhalb des Webroot */
$configFile = "./core/config/config.inc.php";
if (file_exists($configFile)) {
include($configFile);
$date = date("Ymd-His");
$targetSql = "$dir/{$date}_mysql.sql";
$targetTar = "$dir/{$date}_files.tar";
system("mkdir $dir");
@martinseener
martinseener / modx-revolution-upgrade.php
Created May 13, 2014 10:07
MODX Revolution Quick Upgrade PHP-CLI Script
<?php
if ( !shell_exec("type type")) { echo "Weak your PHP powers are, Luke."; die; }
exec('wget -O latest.zip http://modx.com/download/latest;');
exec('unzip latest.zip; rm latest.zip;');
exec('cd modx-*; cp -r ./* ../; cd ..; rm -R modx-*;');
exec('chmod 660 ./core/config/config.inc.php');
echo("Done. Now call /setup of your MODX installation to complete the Upgrade!");
?>