Skip to content

Instantly share code, notes, and snippets.

Avatar

Spencer Owen spuder

View GitHub Profile
@spuder
spuder / logstash.md
Last active Aug 29, 2015
logstash trouble
View logstash.md

Why is logstash not parsing my logs?

logstash 1.3 & elasticsearch 0.90 installed form deb packages

sudo service logstash start
sudo service elasticsearch start

ps aux |grep logstash

/usr/bin/java -Xmx256m -Djava.io.tmpdir=/var/lib/logstash/ -jar /opt/logstash/logstash.jar agent -f /etc/logstash/conf.d --log /var/log/logstash/logstash.log

View deb
If you export to .deb then you can not use -p otherwise you get an error
#Fail
fpm -p /tmp -s dir -m sowen@foo.com -C /tmp/installdir -t deb -n redis-6390 -v 2.8.6 bin/ etc/ var/
Process failed: ar failed (exit code 1). Full command was:["ar", "-qc", "/tmp", "debian-binary", "control.tar.gz", "/tmp/package-deb-build20140304-28142-nfr1q3/data.tar.gz"] {:level=>:error}
#Success
fpm -p /tmp -s dir -m sowen@foo.com -C /tmp/installdir -t rpm -n redis-6390 -v 2.8.6 bin/ etc/ var/
@spuder
spuder / 6390.conf
Created Mar 6, 2014
redis_6390 config
View 6390.conf
## Generated by install_server.sh ##
# Redis configuration file example
# Note on units: when memory size is needed, it is possible to specify
# it in the usual form of 1k 5GB 4M and so forth:
#
# 1k => 1000 bytes
# 1kb => 1024 bytes
# 1m => 1000000 bytes
# 1mb => 1024*1024 bytes
@spuder
spuder / redis_6390_ubuntu
Last active Aug 29, 2015
redis-6390 init script
View redis_6390_ubuntu
#! /bin/sh
### BEGIN INIT INFO
# Provides: redis-server
# Required-Start: $syslog $remote_fs
# Required-Stop: $syslog $remote_fs
# Should-Start: $local_fs
# Should-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: redis-server - Persistent key-value db
View fibinachi.rb
# Array returned by this question http://stackoverflow.com/questions/22752407/why-cant-i-append-to-the-end-of-an-array-recursively-in-ruby
2
3
2
4
2
3
5
2
3
@spuder
spuder / gist:10210424
Last active Aug 29, 2015
logstash not sending to elasticsearch
View gist:10210424
I have logstash 1.4 configured to take everything frim /tmp/access_log and stdin and send it to both elasticsearch and stdout
Documentation page
http://logstash.net/docs/1.4.0/tutorials/getting-started-with-logstash
input {
@spuder
spuder / logstash-torque-error
Last active Aug 29, 2015
logstash toruqe failure
View logstash-torque-error
Error: Expected one of #, => at line 14, column 26 (byte 328) after filter {
grok {
#04/04/2014 18:26:34;0040;PBS_Server.28407;Req;set_nodes;allocating nodes for job 25.interactive.novalocal with node expression 'node02'
match => { "message"
The config file
@spuder
spuder / redis exec
Last active Aug 29, 2015
Puppet redis exec and install
View redis exec
# The case statment is running too late, it needs to run earlier
case $::osfamily {
redhat:{
$redis_6390_version = 'redis-6390-2.8.7-1.x86_64.rpm'
notify {"Installing redhat redis from /tmp/${redis_6390_vesion}":}
}
debian:{
$redis_6390_version = 'redis-6390_2.8.7_amd64.deb'
View puppet-ensure-packages
# How would one guarantee that 'fpm' is installed after 'ruby-devel' ?
$required_packages=['rubygems','ruby-devel', 'gcc', 'gcc-c++', 'rpm-build']
ensure_packages($required_packages)
package { 'fpm' :
ensure => latest,
provider => gem,
@spuder
spuder / logstash
Last active Aug 29, 2015
Troubleshoot logstash grok logs
View logstash
#This config does not work, it never enters the if [tag] == derp block
input {
stdin {}
}
filter {
grok {
#match => [ "message", ".*" ]
match => [ "message", "%{GREEDYDATA}" ]
add_tag => [ "derp" ]