Skip to content

Instantly share code, notes, and snippets.

View electrical's full-sized avatar

Richard Pijnenburg electrical

View GitHub Profile
diff --git a/logstash.gemspec b/logstash.gemspec
index dbc88ed..ec7cf22 100644
--- a/logstash.gemspec
+++ b/logstash.gemspec
@@ -39,6 +39,7 @@ Gem::Specification.new do |gem|
# gem.add_runtime_dependency "jar-dependencies", [">= 0.1.2"] #(MIT license)
gem.add_runtime_dependency "ruby-maven" #(EPL license)
+ gem.add_runtime_dependency "maven-tools", "= 1.0.5"
gem.add_runtime_dependency "minitar"
hist = log --all --graph --pretty='[%C(cyan)%h%Creset]%C(bold cyan)%d%Creset %s'
@colinsurprenant
colinsurprenant / docker_clean.sh
Created February 25, 2014 20:21
clean docker repositoty
# remove stopped containers
docker rm $(docker ps -a -q)
# remove untagged images
docker rmi $(docker images | grep "^<none>" | awk "{print $3}")
@garthk
garthk / elastic_template.pp
Created February 6, 2013 04:30
Puppet module to define elastic_template resource type. Doesn't ensure that the index is correct, but does ensure it's present.
define elastic_template($host='localhost', $port='9200', $prefix='', $require=undef) {
$es_templ = "${name}-template.json"
file { "/tmp/${es_templ}":
ensure => present,
source => "puppet:///modules/${module_name}/${prefix}${es_templ}",
require => $require,
}
$es_locn = "http://${host}:${port}/_template/${name}"
exec { "curl -s -XPUT ${es_locn} -d @/tmp/${es_templ}":
@louiszuckerman
louiszuckerman / gist:4658921
Last active December 11, 2015 20:59
kibana on passenger in a subdirectory of an existing apache virtual host
# These instructions are derived from the Passenger documentation,
# http://www.modrails.com/documentation/Users%20guide%20Apache.html#deploying_rack_to_sub_uri
# And tested with kibana git master on 1/28/13
# First, create a symlink from the existing virtualhost's
# document root to your kibana installation's public/static
# directory. for example,
# ln -s /path/to/kibana/public /var/www/kibana
# kibana 0.2.0 and earlier will have a /static directory
# instead of /public
@paulczar
paulczar / logstash-perf.md
Last active May 14, 2018 03:13
logstash performance deets

Logstash Performance Testing

Server Details

HP BL460

  • 48 Gb Memory
  • 2 x X5675 @ 3.07GHz
  • 2 x 10 gbps NIC
  • 2tb NetApp NFS volume for ES data
#!/usr/bin/ruby
queue_list = `rabbitmqctl list_queues`.split("\n")
queue_list.pop
queue_list.shift
queues = Hash.new
queue_list.each do |queue|
queue = queue.strip.split(/\s+/).collect(&:strip)
queue_stat = queue.pop
queue_name = queue.pop
@robinsmidsrod
robinsmidsrod / _json-logger.md
Created December 18, 2012 10:10
json-logger: A practical script to capture multiline output from e.g. cron jobs into a single logstash JSON event

Can be used in crontab like this:

SHELL=/bin/bash
BASH_ENV=~/.bash_profile
# m h  dom mon dow   command
0    * * * * /home/portfolio/portfolio/bin/session.pl                2> >(json-logger -ep pf_session)                | json-logger -p pf_session
0    2 * * * /home/portfolio/portfolio/bin/purge_deleted_messages.pl 2> >(json-logger -ep pf_purge_deleted_messages) | json-logger -p pf_purge_deleted_messages
0,30 * * * * /home/portfolio/portfolio/bin/expire_page_views.pl      2> >(json-logger -ep pf_expire_page_views)      | json-logger -p pf_expire_page_views

I would suggest you put it in /usr/bin to avoid having to setup your PATH variable.

input {
tcp {
type => 'json-logger'
port => 3517
format => "json_event"
}
}
filter {
mutate {
# Convert hostname => @source_host, because it's overwritten
anonymous
anonymous / elasticsearch.yml
Created December 17, 2012 13:59
ElasticSearch configuration for LogStash
cluster:
name: elasticsearch
network:
host: 127.0.0.1
path:
logs: /var/log/elasticsearch
data: /var/lib/elasticsearch
conf: /etc/elasticsearch