With this config, if your hostname is koala
and if you have files like
log/t1.log
/t2.log
/...
They are sent to
------------------------------------------------------------------- | |
test_in_exec.rb | |
------------------------------------------------------------------- | |
Run options: | |
# Running tests: | |
...... | |
Finished tests in 8.276824s, 0.7249 tests/s, 1.8123 assertions/s. |
<source> | |
type tail | |
tag apache.access | |
format apache2 | |
path /path/to/your/log | |
read_from_head true #過去ログ用。ファイルの先頭から読む | |
</source> | |
<match apache.access> | |
type elasticsearch |
With this config, if your hostname is koala
and if you have files like
log/t1.log
/t2.log
/...
They are sent to
require 'fluent-logger' | |
# specify the host and port to forward data to | |
logger = Fluent::Logger.new(nil, :host => "localhost", :port => 24224) | |
fn = ARGV[0] | |
fh = File.new(fn) | |
msgpack_unpacker = MessagePack::Unpacker.new(fh) |
module Fluent | |
class IfFilter < Filter | |
Fluent::Plugin.register_filter('if', self) | |
config_param :if, :string | |
def configure(conf) | |
super | |
@placeholder_expander = RubyPlaceholderExpander.new(log) | |
<source> | |
type tail | |
format json | |
path /path/to/file | |
tag aoi.yu | |
</source> |
As Fluentd starts rolling out new major uprates (v0.12 & v0.14) and inches towards the v1 release, I want to re-organize and version the docs website
Here is my plan.
https://docs.fluentd.org/<LANGUAGE>/<VERSION>/<PAGE_NAME>
. Ex. https://docs.fluentd.org/ja/v0-12/config-file
. The pages that have version info in the URL are called Versioned Pages.require 'nokogiri' | |
require 'open-uri' | |
require 'json' | |
BASE_URL = 'http://www.tomtunguz.com' | |
class URLFormatError < RuntimeError; end | |
def fetch_categories(base_url) | |
html = open(base_url).read |
This is a little note about how I would go about updating fluent-plugin-flume to resolve this issue.
The current fluent-plugin-flume supports 0.9 protocol. However, most curernt deployments are on the 1.5 branch. This causes a compatibility issue because the Thrift definition for Flume changed between 0.9 and 1.5.
lib/fluent/plugin/thrift
)# sender td-agent.conf | |
#--------------------------------------------------- | |
<source> | |
type forward | |
</source> | |
<match apache.*> | |
type secure_forward | |
shared_key somekey |