Fluentdは JSONストリームとしてログを扱うログ収集デーモンです。
これまでのところ、最大のユーザーではピーク時で
100台以上のサーバ、650GB daily 、70,000msgs/sec のログを収集しています。
Changes:
-
this version includes backport of Greg Price's patch for speedup startup http://bugs.ruby-lang.org/issues/7158 .
ruby-core prefers his way to do thing, so that I abandon cached-lp and sorted-lf patches of mine.
-
this version integrates 'array as queue' patch, which improves performance when push/shift pattern is heavily used on Array.
This patch is accepted into trunk for Ruby 2.0 and last possible bug is found by Yui Naruse. It is used in production* for a couple of months without issues even with this bug.
-
Jubatus サーバのタイムアウト機能を使用したくないユーザ (
--timeout 0
) の救済方法を検討する- 解決策: クライアントから close できるインタフェースを用意する (Jubatus クライアントを修正)
-
Jubatus サーバのタイムアウト機能を使用したいユーザの救済方法を検討する
- サーバから timeout で自動切断(サーバから TCP FIN パケット送信)された後に RPC メソッドを呼んだ際に RPC エラーが起きるのが不親切 (C++/Python/Ruby のみ)
-
解決策の案(松): サーバから受け取った FIN リクエストを正しくハンドリングするように修正 (msgpack-rpc ライブラリを修正)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# -*- coding:utf-8 -*- | |
## why define like a this? | |
def _get_getter(o, attribute_string): | |
return lambda : _getter(o, attribute_string) | |
def _getter(o, attribute_string): | |
for k in attribute_string.split("."): | |
o = getattr(o, k) | |
return o |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
前提 | |
----------- | |
- docker 0.9.1 | |
- lxcを入れること | |
- linux 3.11で確認。 3.8以上じゃないとだめっぽい。 | |
- ubuntu image (docker pull ubuntuで取ってきたもので試してみた) | |
- imageにはpython2を入れておくこと | |
- /usr/bin/tee がimageにあること |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<? | |
function hash_value($obj, $keys){ | |
if(empty($keys)) return $obj; | |
$key = array_shift($keys); | |
if(isset($obj[$key])){ | |
return hash_value($obj[$key], $keys); | |
}else{ | |
return ''; | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# -*- coding: utf-8 -*- | |
# | |
# usage: swift-demangle-filter.py [-h] [-o <outfile>] [<file> <options>] | |
# | |
# Read swift assembly, demangle Swift Symbol and output it. | |
# | |
# positional arguments: | |
# <file> <options> If <file> has .swift, compile it with <options>; or read | |
# (<file> or <stdin>) as assembly. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python3 | |
import datetime | |
import sys | |
import math | |
import numpy as np | |
from argparse import ArgumentParser | |
from collections import defaultdict | |
from chainer import FunctionSet, Variable, functions, optimizers |