Create a gist now

Instantly share code, notes, and snippets.

@sonots /sigdump.log Secret
Last active Jan 4, 2016

What would you like to do?
fluentd does not die at `TCPSocket.new`
Sigdump at 2014-01-29 14:49:19 +0900 process 2422 (/usr/lib64/fluent/ruby/bin/fluentd)
Thread #<Thread:0x00000000e7bff8> status=run priority=0
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:38:in `dump_backtrace'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:24:in `block in dump_all_thread_backtrace'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:23:in `each'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:23:in `dump_all_thread_backtrace'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:16:in `block in dump'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:107:in `open'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:107:in `_open_dump_path'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:14:in `dump'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:7:in `block in setup'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:266:in `call'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:266:in `join'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:266:in `block in shutdown'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:256:in `each'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:256:in `shutdown'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:222:in `run'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:386:in `run_engine'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:94:in `block in start'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:198:in `call'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:198:in `main_process'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:173:in `block in supervise'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:172:in `fork'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:172:in `supervise'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:87:in `start'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/command/fluentd.rb:146:in `<top (required)>'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/bin/fluentd:6:in `require'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/bin/fluentd:6:in `<top (required)>'
/usr/lib64/fluent/ruby/bin/fluentd:23:in `load'
/usr/lib64/fluent/ruby/bin/fluentd:23:in `<main>'
Thread #<Thread:0x0000000257a5d8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e3a468> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e38b18> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e3db90> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e3bdb8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e409f8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e4a138> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e486f8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e4e670> status=sleep priority=0
Thread #<Thread:0x00000000e486f8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e4e670> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e4d180> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e516b8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e50858> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e57e00> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e56938> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e54f98> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e5fa88> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e5df80> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000e61568> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001016e30> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000000f56180> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x0000000108b500> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001119ad0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001118e50> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000011182e8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001117ac8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000014bcea8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000014bba08> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x0000000148ceb0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x0000000146aef0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001446eb0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000014236e0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000013ede00> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000013d0b70> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x0000000122ffc8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001068c30> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001129b88> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001131978> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001170c68> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x0000000117bbe0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001189e70> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000011d0078> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000011e3538> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001367be8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001366e50> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001317990> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001316a68> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000011a72e0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001231378> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001230658> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000015518a0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001556a30> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000015547a8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001ae5910> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001ae46c8> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x00000001525c50> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
Thread #<Thread:0x000000019b6418> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:184:in `log_event_loop'
Thread #<Thread:0x000000017ff7a0> status=sleep priority=0
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:58:in `open'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:58:in `shutdown'
/usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:259:in `block (2 levels) in shutdown'
Built-in objects:
327,865: TOTAL
109,396: FREE
60,721: T_STRING
56,494: T_ARRAY
47,732: T_DATA
29,480: T_FLOAT
7,948: T_OBJECT
7,149: T_NODE
3,088: T_CLASS
2,838: T_HASH
2,290: T_REGEXP
239: T_BIGNUM
215: T_ICLASS
166: T_MODULE
70: T_STRUCT
20: T_RATIONAL
15: T_FILE
3: T_MATCH
1: T_COMPLEX
All objects:
60,007: String
50,328: Array
35,041: Time
29,480: Float
6,066: RubyVM::InstructionSequence
2,467: Proc
2,457: RubyVM::Env
2,290: Regexp
1,667: Gem::Version
1,544: Fluent::Config::Element
1,536: Gem::Requirement
1,215: Hash
1,004: Gem::Dependency
936: Class
872: Thread
623: Fluent::Match
623: Fluent::GlobMatchPattern
619: Fluent::CopyOutput
574: Fluent::GrepOutput
400: Mutex
260: Method
239: Bignum
236: Gem::Specification
166: Module
121: Fluent::ForwardOutput::Node
121: Fluent::OutputThread
121: Fluent::ForwardOutput::FailureDetector
121: ConditionVariable
99: Encoding
70: Range
62: Fluent::DataCounterOutput
58: Bundler::Dependency
53: Fluent::RecordReformerOutput
52: Bundler::LazySpecification
44: Fluent::StatsOutput
41: Fluent::RecordReformerOutput::PlaceholderExpander
41: Fluent::ParserOutput
41: FluentExt::TextParser
25: MatchData
23: FluentExt::TextParser::RegexpParser
20: Rational
18: FluentExt::TextParser::LabeledTSVParser
18: Coolio::IO::Watcher
12: Fluent::RecordReformerOutput::RubyPlaceholderExpander
12: Fluent::MemoryBufferChunk
11: OptionParser::Switch::RequiredArgument
9: IO::Buffer
8: OptionParser::Switch::NoArgument
8: Coolio::Loop
8: Fluent::GrepCounterOutput
7: UDPSocket
7: Gem::Platform
6: Fluent::StdoutOutput
6: Fluent::MemoryBuffer
6: Fluent::ForwardOutput::HeartbeatRequestTimer
6: OptionParser::OptionMap
6: Fluent::ForwardOutput::HeartbeatHandler
5: Fluent::HashForwardOutput
4: Object
4: UUIDTools::UUID
4: Pathname
3: Bundler::SpecSet
3: IO
3: OptionParser::List
3: Bundler::Index
3: JSON::Ext::Generator::State
3: TCPSocket
2: Fluent::ForwardInput::Handler
2: Process::Status
2: Fluent::RewriteOutput
2: OptionParser::Switch::OptionalArgument
2: UnboundMethod
2: Enumerator
2: Fluent::OneEventStream
1: Fluent::ReemitOutput
1: fatal
1: Data
1: OptionParser
1: ARGF.class
1: URI::Parser
1: Fluent::Supervisor::LoggerInitializer
1: Random
1: OptionParser::Switch::PlacedArgument
1: OptionParser::CompletingHash
1: Fluent::ConfigExpanderInput
1: URI::HTTPS
1: Bundler::Source::Rubygems
1: Fluent::EngineClass
1: SystemStackError
1: Bundler::Definition
1: NoMemoryError
1: RubyVM
1: Fluent::KeepForwardOutput
1: Binding
1: Fluent::ElapsedTimeOutput
1: OpenSSL::X509::Store
1: Resolv::DNS::Config
1: Resolv::DNS
1: Resolv::Hosts
1: Resolv
1: Fluent::Log
1: Fluent::FlowCounterOutput
1: IOError
1: ThreadGroup
1: Bundler::Settings
1: Fluent::Supervisor
1: TCPServer
1: Coolio::TCPServer
1: Coolio::TimerWatcher
1: Fluent::ForwardInput::HeartbeatRequestHandler
1: Complex
1: Fluent::NullOutputChain
1: Gem::ConfigFile
1: Bundler::Runtime
1: Bundler::RubygemsIntegration::Modern
1: Fluent::NullOutput
1: WEBrick::HTTPVersion
1: Fluent::StatusClass
1: Fluent::PingMessageInput
1: Fiber
1: Fluent::PluginClass
1: YAML::EngineManager
1: Fluent::ForwardInput
1: Fluent::OutputChain
1: RuntimeError
1: File
1:
1: MessagePack::Buffer
1: MessagePack::Unpacker
String 2,004,096 bytes
Array 2 elements
Hash 16 pairs
Owner

sonots commented Jan 29, 2014

 Thread #<Thread:0x000000017ff7a0> status=sleep priority=0
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:58:in `open'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/in_forward.rb:58:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:259:in `block (2 levels) in shutdown'

looks weird. At the concerned line, fluentd calls TCPSocket.new at shutdown, and gets stuck.

Owner

sonots commented Jan 31, 2014

I got same situation again. However, this time, fluentd is stopping at httpclient in out_yohoushi plugin. However, again, the line calls TCPSocket.new (at /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:803)

Sigdump at 2014-01-31 15:49:03 +0900 process 12322 (/usr/lib64/fluent/ruby/bin/fluentd)
  Thread #<Thread:0x00000000cbeff8> status=run priority=0
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:38:in `dump_backtrace'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:24:in `block in dump_all_thread_backtrace'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:23:in `each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:23:in `dump_all_thread_backtrace'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:16:in `block in dump'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:107:in `open'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:107:in `_open_dump_path'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:14:in `dump'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/sigdump-0.2.2/lib/sigdump.rb:7:in `block in setup'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:266:in `call'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:266:in `join'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:266:in `block in shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:256:in `each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:256:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:222:in `run'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:386:in `run_engine'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:94:in `block in start'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:198:in `call'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:198:in `main_process'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:173:in `block in supervise'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:172:in `fork'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:172:in `supervise'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/supervisor.rb:87:in `start'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/command/fluentd.rb:146:in `<top (required)>'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/bin/fluentd:6:in `require'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/bin/fluentd:6:in `<top (required)>'
      /usr/lib64/fluent/ruby/bin/fluentd:23:in `load'
      /usr/lib64/fluent/ruby/bin/fluentd:23:in `<main>'
  Thread #<Thread:0x000000025d5640> status=sleep priority=0
      /usr/lib64/fluent/ruby/lib/ruby/1.9.1/thread.rb:71:in `wait'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:156:in `cond_wait'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:140:in `run'
  Thread #<Thread:0x0000000337f160> status=sleep priority=0
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:803:in `new'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:803:in `create_socket'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:752:in `block in connect'
      /usr/lib64/fluent/ruby/lib/ruby/1.9.1/timeout.rb:68:in `timeout'
      /usr/lib64/fluent/ruby/lib/ruby/1.9.1/timeout.rb:99:in `timeout'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:751:in `connect'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:609:in `query'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient/session.rb:164:in `query'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:1083:in `do_get_block'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:887:in `block in do_request'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:981:in `protect_keep_alive_disconnected'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:886:in `do_request'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:774:in `request'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/httpclient-2.3.4.1/lib/httpclient.rb:684:in `post'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/growthforecast-client-0.62.3/lib/growthforecast/client.rb:64:in `post_query'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/growthforecast-client-0.62.3/lib/growthforecast/client.rb:164:in `post_graph'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/multiforecast-client-0.72.0.1/lib/multiforecast/client.rb:153:in `post_graph'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-yohoushi-0.1.1/lib/fluent/plugin/out_yohoushi.rb:111:in `post'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-yohoushi-0.1.1/lib/fluent/plugin/out_yohoushi.rb:141:in `block (2 levels) in write'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-yohoushi-0.1.1/lib/fluent/plugin/out_yohoushi.rb:137:in `each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-yohoushi-0.1.1/lib/fluent/plugin/out_yohoushi.rb:137:in `block in write'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/buf_memory.rb:62:in `feed_each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/buf_memory.rb:62:in `msgpack_each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-yohoushi-0.1.1/lib/fluent/plugin/out_yohoushi.rb:124:in `write'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/buffer.rb:296:in `write_chunk'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/buffer.rb:276:in `pop'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:306:in `try_flush'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:131:in `run'
  Thread #<Thread:0x00000044f6a178> status=sleep priority=0
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:184:in `log_event_loop'
  Thread #<Thread:0x00000003115438> status=sleep priority=0
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:110:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:229:in `block in shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:229:in `each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/output.rb:229:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluent-plugin-yohoushi-0.1.1/lib/fluent/plugin/out_yohoushi.rb:106:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/out_copy.rb:56:in `block in shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/out_copy.rb:55:in `each'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/plugin/out_copy.rb:55:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/match.rb:44:in `shutdown'
      /usr/lib64/fluent/ruby/lib/ruby/gems/1.9.1/gems/fluentd-0.10.42/lib/fluent/engine.rb:259:in `block (2 levels) in shutdown'
  Thread #<Thread:0x0000000306fc18> status=sleep priority=0
      /usr/lib64/fluent/ruby/lib/ruby/1.9.1/timeout.rb:61:in `block in timeout'
  Built-in objects:
   460,330: TOTAL
   254,185: T_STRING
    58,814: T_ARRAY
    42,527: T_DATA
    24,834: T_NODE
    21,995: T_HASH
    19,659: FREE
    18,812: T_OBJECT
     7,067: T_FLOAT
     5,699: T_CLASS
     3,171: T_REGEXP
     1,890: T_MATCH
       993: T_ICLASS
       260: T_STRUCT
       198: T_FILE
       196: T_MODULE
        28: T_BIGNUM
         1: T_COMPLEX
         1: T_RATIONAL
  All objects:
   244,398: String
    49,244: Array
    17,056: Hash
    14,334: Proc
    10,664: RubyVM::Env
     7,595: RubyVM::InstructionSequence
     7,055: Float
     4,726: Fluent::Config::Element
     3,147: Regexp
     3,043: Thread
     2,378: Mutex
     1,859: URI::HTTP
     1,787: MatchData
     1,699: Gem::Version
     1,568: Gem::Requirement
     1,343: Time
     1,299: Fluent::GlobMatchPattern
     1,299: Fluent::Match
     1,294: Fluent::CopyOutput
     1,281: Fluent::StdoutOutput
     1,095: Method
     1,093: Class
     1,028: Gem::Dependency
       554: Fluent::MailOutput
       552: Fluent::IkachanOutput
       537: Pathname
       522: Fluent::GrepCounterOutput
       378: HTTP::Message::Body
       378: HTTP::Message
       378: HTTP::Message::Headers
       275: Fluent::OutputThread
       275: Fluent::MemoryBuffer
       275: ConditionVariable
       274: GrowthForecast::Client
       274: MultiForecast::Client
       274: Fluent::YohoushiOutput
       243: Range
       240: Gem::Specification
       226: Fluent::YohoushiOutput::RubyPlaceholderExpander
       204: HTTPClient::Site
       196: Module
       195: 
       179: TCPSocket
       177: Binding
       177: HTTPClient::Session
       176: HTTPClient::Connection
       176: Queue
       176: JSON::Ext::Parser
       141: Fluent::StatsOutput
       117: Enumerator
       104: HTTPClient::NegotiateAuth
        99: Encoding
        74: Fluent::StatsNotifierOutput
        58: Bundler::Dependency
        56: Fluent::ChangeFinder
        52: Bundler::LazySpecification
        52: HTTPClient::SSPINegotiateAuth
        48: Fluent::YohoushiOutput::PlaceholderExpander
        28: Bignum
        27: OpenSSL::X509::Store
        26: HTTPClient
        26: HTTPClient::ProxyDigestAuth
        26: WebAgent::CookieManager
        26: HTTPClient::SSLConfig
        26: HTTPClient::WWWAuth
        26: HTTPClient::ProxyAuth
        26: HTTPClient::SessionManager
        26: HTTPClient::OAuth
        26: HTTPClient::DigestAuth
        26: HTTPClient::BasicAuth
        26: HTTPClient::ProxyBasicAuth
        19: Fluent::AnomalyDetectOutput
        11: OptionParser::Switch::RequiredArgument
         8: OptionParser::Switch::NoArgument
         7: Gem::Platform
         6: Coolio::IO::Watcher
         6: OptionParser::OptionMap
         6: Thor::Option
         4: Object
         4: Fluent::MemoryBufferChunk
         4: UUIDTools::UUID
         4: Thor::Command
         3: IO
         3: OptionParser::List
         3: IO::Buffer
         3: Coolio::Loop
         3: Bundler::Index
         3: JSON::Ext::Generator::State
         3: Bundler::SpecSet
         2: UnboundMethod
         2: Fluent::ReemitOutput
         2: Fluent::ElapsedTimeOutput
         2: Fluent::FlowCounterOutput
         2: UDPSocket
         2: OptionParser::Switch::OptionalArgument
         2: MessagePack::Buffer
         2: Thor::CoreExt::OrderedHash
         2: Process::Status
         2: MessagePack::Unpacker
         2: Fluent::RewriteOutput
         1: Bundler::Settings
         1: Resolv::Hosts
         1: Fluent::Log
         1: Fluent::NullOutput
         1: Fiber
         1: Fluent::PingMessageInput
         1: Resolv::DNS
         1: Resolv::DNS::Config
         1: WEBrick::HTTPVersion
         1: Fluent::ForwardInput
         1: Fluent::ConfigExpanderInput
         1: Resolv
         1: Fluent::KeepForwardOutput
         1: Fluent::PluginClass
         1: OptionParser
         1: Fluent::Supervisor
         1: Fluent::Supervisor::LoggerInitializer
         1: Rational
         1: Fluent::ForwardOutput::Node
         1: Bundler::RubygemsIntegration::Modern
         1: Complex
         1: YAML::EngineManager
         1: ThreadGroup
         1: Fluent::StatusClass
         1: Gem::ConfigFile
         1: IOError
         1: fatal
         1: Data
         1: OptionParser::Switch::PlacedArgument
         1: Fluent::ForwardOutput::HeartbeatRequestTimer
         1: Bundler::Runtime
         1: Bundler::Definition
         1: Fluent::ForwardOutput::HeartbeatHandler
         1: Bundler::Source::Rubygems
         1: ARGF.class
         1: Random
         1: URI::HTTPS
         1: Fluent::NullOutputChain
         1: Fluent::ForwardOutput::FailureDetector
         1: SystemStackError
         1: NoMemoryError
         1: RubyVM
         1: File
         1: Coolio::TimerWatcher
         1: Fluent::ForwardInput::HeartbeatRequestHandler
         1: TCPServer
         1: Coolio::TCPServer
         1: Fluent::EngineClass
         1: OptionParser::CompletingHash
         1: URI::Parser
         1: Fluent::ForwardInput::Handler
  String 56,191,345 bytes
   Array 3 elements
    Hash 0 pairs
Owner

sonots commented Jan 31, 2014

This bug ticket of ruby looks related with this problem https://bugs.ruby-lang.org/issues/5101

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment