jordansissel

parsing mysql query logs with logstash

The problem is that some lines in the file are missing timestamps when they aren't continuations of any previous line. It's dumb, really.

The mysql query log is seriously bullshit format, but nothing logstash can't unscrew.

The main goal here is to show how we can fix the 'missing timestamp' problem.

% ruby bin/logstash agent -e '

juanje

input {
  tcp {
    type => "linux-syslog"
    port => 3333
  }

  file {
    type => "linux-syslog"
    path => [ "/var/log/auth.log" ]
  }

danielsdeleo

#!/usr/bin/env ruby

recipe_path = ARGV[0]

if recipe_path.nil?
  STDERR.puts "usage: chef-apply RECIPE_FILE"
  exit 1
end

recipe_path = File.expand_path(recipe_path)

erikh

#!/usr/bin/env sh

##
# This is script with usefull tips taken from:
#   https://github.com/mathiasbynens/dotfiles/blob/master/.osx
#
# install it:
#   curl -sL https://raw.github.com/gist/2108403/hack.sh | sh
#

DAddYE

#!/bin/sh

##
# This is a script with usefull tips taken from:
#   https://github.com/mathiasbynens/dotfiles/blob/master/.osx
#
# Run in interactive mode with:
#   $ sh -c "$(curl -sL https://raw.github.com/gist/2108403/hack.sh)"
#
# or run it without prompt questions:

jordansissel

% ruby  multiple-to-rpm.rb thin sinatra cabin json                    
Trying to download {"gem":"thin","version":null,"level":"info"}
Using gem file {"path":"/home/jls/.rvm/gems/ruby-1.9.3-p0/cache/thin-1.3.1.gem","level":"info"}
Building native extensions.  This could take a while...
Successfully installed thin-1.3.1
1 gem installed
Trying to download {"gem":"sinatra","version":null,"level":"info"}
Using gem file {"path":"/home/jls/.rvm/gems/ruby-1.9.3-p0/cache/sinatra-1.3.2.gem","level":"info"}
Successfully installed sinatra-1.3.2
1 gem installed

karussell

Several times in a month there pop up questions regarding query structure on the ElasticSearch user group.

Although there are good docs explaining this in depth probably the bird view of the Query DSL is necessary to
understand what is written there. There is even already some good external documentation available:
http://www.elasticsearch.org/tutorials/2011/08/28/query-dsl-explained.html

And there were attempts to define a schema:

http://groups.google.com/group/json-schema/browse_thread/thread/ae498ee818155d50
https://gist.github.com/8887766ca0e7052814b0

jimbojsb

Step 0:

Get Homebrew installed on your mac if you don't already have it

Step 1:

Install highlight. "brew install highlight". (This brings down Lua and Boost as well)

Step 2:

fnichol

#!/usr/bin/env bash
set -e

# # Mac OS X 10.7/10.8/10.9/10.10 Bootstrapping
#
# ## Pre-requisites
#
# 1.  Set your hostname: In **System Preferences** go to **Sharing** and enter
#     the name in **Computer Name**
# 2.  Run **Software Update** and reboot if necessary

jtimberman

# Use Chef resources in an application via solo mode.
# Could also be configured in client mode, and then use a server.

require 'rubygems'
require 'chef'
require 'chef/client'
require 'chef/run_context'

Chef::Config[:solo] = true
Chef::Config[:log_level] = :info