Skip to content

Instantly share code, notes, and snippets.

View holybit's full-sized avatar

Joseph Crotty holybit

5 followers · 0 following
View GitHub Profile
@holybit
holybit / bash_try
Last active August 29, 2015 14:13
rsyslog filter/template problem rules work OK under 7.x but not 8.x
$ /sbin/rsyslogd -N1
rsyslogd: error during parsing file /etc/rsyslog.d/central.conf, on or before line 105: invalid character '�' in expression - is there an invalid escape sequence somewhere? [try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.d/central.conf, on or before line 105: invalid character '�' in expression - is there an invalid escape sequence somewhere? [try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.d/central.conf, on or before line 105: invalid character '�' in expression - is there an invalid escape sequence somewhere? [try http://www.rsyslog.com/e/2207 ]
rsyslogd: error during parsing file /etc/rsyslog.d/central.conf, on or before line 105: syntax error on token '-' [try http://www.rsyslog.com/e/2207 ]
rsyslogd: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2207 ]
rsyslogd: run failed with error -2207 (see rsyslog.h or try http://www.rsyslog.com/e/2207 to learn wh
@holybit
holybit / gist:a5559faed6db86bc327a
Last active August 29, 2015 14:13
rsyslog property replacer
# Logs example below
# I am interested in the APP-NAME (i.e., apache-access_log-foo_bar) of the HEADER
Jan 12 11:19:36 hostx apache-access_log-foo_bar: site=postmaster.dev.returnpath.net ip=10.0.2.81 datetime="2015-01-12 11:19:36 -0700" timestamp=1421086776 host=postmaster.dev.returnpath.net request="GET /assets/font/fontawesome-webfont.woff HTTP/1.1"
# I want to be able to write out the syslog above to a file such that APP-NAME is used for the file name
/tmp/apache-acces_log-foo_bar
# Template
$template APACHE-ACCESS,"/tmp/%SYSLOGTAG%.log"
@holybit
holybit / cli.sh
Last active August 29, 2015 14:12
Email::Address->parse hangs indefinetley
$ perl -v
This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-ld
$ perl -MEmail::Simple -e 'print "$Email::Simple::VERSION\n"'
2.202
$ perl -MEmail::Address -e 'print "$Email::Address::VERSION\n"'
1.905
@holybit
holybit / rsyslog.conf
Last active June 16, 2020 08:23
rsyslog splitting large messages on local socket with logger
# rsyslog configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
# The imjournal module bellow is now used as a message source instead of imuxsock.
$MaxMessageSize 12k
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
@holybit
holybit / gist:a387a88ae4f131c4f327
Last active August 29, 2015 14:11
Log Layout Design Questions

Current Apache Log Format

Old school key=value pairs. Apache httpd.conf snippet follows:

LogFormat "site=%{site_name}e ip=%h datetime="%{%F %H:%M:%S %z}t" timestamp=%{%s}t host=%V request="%r" status=%>s response_size=%b response_time=%>D referer="%{Referer}i" user_agent="%{User-Agent}i" filename=%f session_id=%{rp_session_id}n tracking_id=%{RPID}C user_id=%{user_id}n realm_id=%{realm_id}n superuser_id=%{superuser_id}n" custom_log

Log Storage

Logs are currently parsed by a Perl script and then stored in Hadoop Hive for a number of business critical use cases.

@holybit
holybit / gist:31436bbb8043e252b4e9
Last active August 29, 2015 14:10
Oracle SQL odd date format results GROUP BY week (i.e., IW)
--------------------------------------------------------
-- DDL for Table ABUSE_CUSTID_INST_ROLLUP
--------------------------------------------------------
CREATE TABLE "FBL_OWNER"."ABUSE_CUSTID_INST_ROLLUP"
( "ROLLUP_DT" DATE,
"COMPANY_ID" NUMBER,
"INST_COUNT" NUMBER DEFAULT 0
);
--------------------------------------------------------
-- DDL for Index ABUSE_CUSTID_INST_ROLLUP_PK
@holybit
holybit / data.tsv
Last active August 29, 2015 14:10
Line Chart Inline Data Day/Week/Year
We can make this file beautiful and searchable if this error is corrected: No tabs found in this TSV file in line 0.
date close
1-Jan-14 122
1-Feb-12 435
@holybit
holybit / gist:9296900
Last active August 29, 2015 13:56
Perl DBD::Oracle client sniffer
#!/usr/bin/env bash
# Compare Oracle client version used to compile the installed
# DBD::Oracle against the currently installed Oracle client, if they
# differ then remove DBD::Oracle so it can be re-installed
# DBD::Oracle always treats Oracle client version numbers as four
# numbers separated by dots starting at the front of version string.
# self exec dir
SELF_DIR="$( cd -P "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
@holybit
holybit / gist:7148906
Last active December 26, 2015 12:09
README.md attempt
# rpm-builds
Collection of scripts for building CentOS RPMs. Once created RPMs are stored on kickstart server for use by IT automation software that helps system administrators manage infrastructure throughout its lifecycle.
## RPM List
By no means exhaustive but a few of the better known software packages provided herein.
* Git
* Perl
@holybit
holybit / gist:6836230
Last active December 24, 2015 17:29
Git compile/RPM questions. Working on compiling git and then rolling an RPM for it. Configured as below. The Perl used was installed from an RPM.
./configure --prefix="/usr/local/git-1.8.4"\
--with-perl=/usr/local/perls/perl-5.18.1-nt/bin/perl\
--with-libpcre
Questions
1. Does the RPM Perl used by --with-perl need to be on the target machine for the git runtime?
2. Which git executables should I run ldd against to determine the runtime required shared libs?