Skip to content

Instantly share code, notes, and snippets.

@kiyoto
kiyoto / default.json
Created July 17, 2014 19:49
EFK Kibana Starter Dashboard
{
"title": "EFK Starter Dashboard",
"services": {
"query": {
"list": {
"0": {
"query": "severity=info",
"alias": "",
"color": "#7EB26D",
"id": 0,
\[(?<time>(?:(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))))\] (?:Warning:\s*(?<nagios_message>.*)|(?<nagios_type>CURRENT SERVICE STATE): (?<nagios_hostname>.*?);(?<nagios_service>.*?);(?<nagios_state>.*?);(?<nagios_statetype>.*?);(?<nagios_statecode>.*?);(?<nagios_message>.*)|(?<nagios_type>CURRENT HOST STATE): (?<nagios_hostname>.*?);(?<nagios_state>.*?);(?<nagios_statetype>.*?);(?<nagios_statecode>.*?);(?<nagios_message>.*)|(?<nagios_type>SERVICE NOTIFICATION): (?<nagios_notifyname>.*?);(?<nagios_hostname>.*?);(?<nagios_service>.*?);(?<nagios_state>.*?);(?<nagios_contact>.*?);(?<nagios_message>.*)|HOST NOTIFICATION: (?<nagios_notifyname>.*?);(?<nagios_hostname>.*?);(?<nagios_state>.*?);(?<nagios_contact>.*?);(?<nagios_message>.*)|(?<nagios_type>SERVICE ALERT): (?<nagios_hostname>.*?);(?<nagios_service>.*?);(?<nagios_state>.*?);(?<nagios_statelevel>.*?);(?<nagios_attempt>(?:(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))));(?<nagios_message>.*)|(?<nagios_type>HOST ALERT): (?<
@kiyoto
kiyoto / fluentd-etymology.markdown
Last active November 18, 2020 01:50
The Etymology of Fluentd

#"Why is it called Fluentd?"

My day (and night) job is to answer questions about Fluentd, and enough people have asked me why it's called that. It is not the easiest to pronounce. It is not mono- or bysyllabic. It's not the end of the world that it is trisyllabic, but you need to inunciate reasonably well to be heard correctly ("Wait, what is it? Flu..what?").

Yes, your guess is as good as anyone else's. It is pronounced as fluent-dee. Like "fluent" with a "d" tacked onto it.

Fluentd is called Fluentd because it wasn't called Fluentd on day one. Its initial name was "Fluent" (hence our repo owner's name), with the intent that this software should make streaming data from sources to sinks more "fluent". Less hassle, more extensibility, more reliability.

However, the author realized (or rather did not realize at first) that if you search for "fluent", you would get bazillion links to Rosetta Stone apps and their siblings. Fluency

<section>
<section>
<h1>PostGIS</h1>
<h2>in the Open Cloud</h2>
<p>writing portable open source mapping applications</p>
<p>with</p>
<h2>Leaflet</h2>
<p>and</p>
<h2>OpenShift</h2>
<br/>
@kiyoto
kiyoto / repo_names.txt
Created June 8, 2014 02:59
docker repo names
0718c7bf/fbtrack
0718c7bf/tomcat6java8
0x42/base
0xahmed/netcat
0xfa1a/centos-ssh
0xfa1a/dockerfiles
0xffea/raring-server-cloudimg-amd64
0xffea/saucy-server-cloudimg-amd64
0xffea/saucy-server-existdb-amd64
1uptalent/docker-registry
@kiyoto
kiyoto / in_steam_friends_online.rb
Last active August 29, 2015 14:01
Steam friend online status plugin
module Fluent
class SteamFriendsOnlineInput < Input
Plugin.register_input('steam_friends_online', self)
config_param :steam_id, :integer
config_param :tag, :string, default:"steam_stats"
config_param :friend_nicknames, :string
config_param :stats_interval, :time, default:300
def initialize
@kiyoto
kiyoto / example.conf
Last active August 29, 2015 14:01
Selecting a particular field for Fluentd to process further
<source>
type forward
</source>
<match test.**>
type select_fields
add_tag_prefix fields_selected
selected_field foo
</match>
@kiyoto
kiyoto / hn.rb
Last active August 29, 2015 14:00
hacker news top page scraper
require 'nokogiri'
require 'open-uri'
require 'json'
class HNParseError < RuntimeError; end
$stdout.sync = true
url = "https://news.ycombinator.com"
@kiyoto
kiyoto / fluentd_philosophy.md
Last active August 29, 2015 14:00
Kiyoto's take on Fluentd's philosophy

Buried in code, comments and issues is philosophy. Philosophy is important, for it informs judgment, which in turn informs decisions. Everything boils down to philosophy. Here is my take on it.

Fluentd's Philosophy (as I see it)

  • Keep the core lean and mean: we should strive to keep Fluentd itself simple and small. Whenever we feel like adding a new feature, let's remember the programming adage "the least buggy code is the code that's not written." Let's not be afraid to say "no" for feature requests into the core, especially if the end goal can be achieved with plugins.
  • Plugins should be easy to write and even easier to reuse: it's cool when someone can write their own plugin to fulfill their needs. It's even cooler if many more people can re-use that plugin. So, make it easy to write plugins but make it even easier to find and use existing plugins.
  • Be sincere and helpful to everyone: helping others and documenting features/use cases are just as important to the project as writing code
@kiyoto
kiyoto / forest-example.conf
Created April 20, 2014 00:47
fluent-plugin-forest example
<source>
type forward
</source>
<match nodelog.*>
type forest
subtype file
<template>
path test/nodelog.__TAG__
flush_interval 3s