Skip to content

Instantly share code, notes, and snippets.

@vmadman
vmadman / logstash-shipper
Created April 27, 2013 06:43
A configuration for a logstash agent. This logstash agent collects logs from apache access, apache error, and syslog via UDP.
input {
tcp {
port => 5000
type => "syslog"
host => "127.0.0.1"
}
udp {
port => 5000
type => "syslog"
host => "127.0.0.1"
@vmadman
vmadman / logstash-indexer
Created April 27, 2013 06:46
A configuration for a logstash agent. This agent accepts input from REDIS and outputs to Graylog2 using GELF. An additional STATSD output sends throughput stats to statsd, which outputs to graphite.
input {
redis {
host => "###REDIS_IP###"
type => "redis-input"
# these settings should match the output of the agent
data_type => "list"
key => "logstash"
# We use json_event here since the sender is a logstash agent
format => "json_event"
@vmadman
vmadman / logstash-supervisord
Created April 27, 2013 06:48
An example SupervisorD configuration for all three logstash components. Some of it might look obvious, but it took a ton of tweaking to figure it out. (but I might just be dumb)
[program:lss]
process_name=Shipper
command=java -jar /usr/local/logstash/bin/logstash-1.1.9-monolithic.jar agent --config /usr/local/logstash/conf/shipper.conf --log /usr/local/logstash/log/shipper.log
user=logstash
startretries=3
redirect_stderr=true
std_out_logfile=NONE
startsecs=3
environment=HOME="/usr/local/logstash/"
@vmadman
vmadman / apache-perl-udp-pipe
Created April 27, 2013 06:57
A PERL UDP client. This client serves (in my usage) as a pipe target for apache logs. After reception, logs are forwarded to logstash. I found this at: http://untergeek.com/2012/10/11/getting-apache-to-output-json-for-logstash/
#!/usr/bin/perl
#udpclient.pl
use IO::Socket::INET;
my $host = $ARGV[0];
my $port = $ARGV[1];
# flush after every write
$| = 1;
my ($socket,$logdata);
@vmadman
vmadman / apache-json-log-format
Created April 27, 2013 06:59
An apache log format that allow access logs (but not error logs) to be output in JSON format. I found this here: http://untergeek.com/2012/10/11/getting-apache-to-output-json-for-logstash/ -- but modified it for my purposes a good bit.
# Access Logs
LogFormat "{ \
\"@vips\":[\"%v\"], \
\"@source\":\"%v%U%q\", \
\"@source_host\": \"%v\", \
\"@source_path\": \"%f\", \
\"@tags\":[\"Apache\",\"Access\"], \
\"@message\": \"%h %l %u %t \\\"%r\\\" %>s %b\", \
\"@fields\": { \
\"timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \
@vmadman
vmadman / apache-virtualhost-log-cfg
Created April 27, 2013 07:03
This is just the log settings I put on each of our virtual hosts so that they pipe logs to LogStash via a UDP proxy. In my setup the error UDP port increments for each virtual host, which allows my LogStash shipper to apply hostnames.
# Error logging to LogStash via UDP client pipe
ErrorLog "||/usr/local/bin/udpclient.pl 127.0.0.1 5002"
LogLevel warn
# Access logging to LogStash via UDP client pipe
CustomLog "||/usr/local/bin/udpclient.pl 127.0.0.1 5001" ls_apache_json
@vmadman
vmadman / logstash-rsyslog-conf
Created April 27, 2013 07:09
This is the UDP output settings I added to /etc/rsyslog.conf so that it would output system events to my LogStash shipper agent. (shipper is the name I took from the LogStash docs, but basically its the "client", not the "server", as loosely as those words apply to logstash)
# Send syslog to LogStash
*.* @@127.0.0.1:5000
@vmadman
vmadman / logio.rb
Last active December 28, 2015 12:49
Log.io output plugin for Logstash -- logstash / lib / logstash / outputs / logio.rb
require "logstash/outputs/base"
require "logstash/namespace"
require "socket"
# Log.io Output
#
# Sends events to a Log.io server over TCP.
#
# Plugin is fault tolerant. If the plugin is unable to connect to the server,
# or writes to a broken socket, it will attempt to reconnect indefinitely.
@vmadman
vmadman / tmpSugarFix.ts
Created December 19, 2013 04:13
This is *my* current temporary fix to get SugarJS's static method additions working in typescript.
/// <reference path="../path/to/sugar.d.ts" />
/**
* Usage Examples:
* sgObject.isArray([1]); // -> true
*/
declare var sgObject:ObjectStatic;
declare var sgNumber:NumberStatic;
declare var sgDate:DateStatic;
@vmadman
vmadman / issue_comments_suggestion.bmml
Created February 22, 2015 15:55
Balsamiq Mockup for Nodebug.me Issue Detail Proposal
<mockup version="1.0" skin="sketch" fontFace="Balsamiq Sans" measuredW="1322" measuredH="912" mockupW="861" mockupH="861">
<controls>
<control controlID="0" controlTypeID="com.balsamiq.mockups::Label" x="531" y="73" w="-1" h="-1" measuredW="520" measuredH="48" zOrder="3" locked="false" isInGroup="-1">
<controlProperties>
<bold>true</bold>
<color>15658734</color>
<size>40</size>
<text>Triaging%20%26joyent/node%20%231234%26</text>
</controlProperties>
</control>