Skip to content

Instantly share code, notes, and snippets.

@guilhem

guilhem/osd stacktrace

Created January 17, 2013 10:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save guilhem/4555135 to your computer and use it in GitHub Desktop.
Save guilhem/4555135 to your computer and use it in GitHub Desktop.
Download ZIP
Ceph #2843
Raw
osd stacktrace
2013-01-17 11:25:43.017199 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is supported and appears to work
2013-01-17 11:25:43.017236 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
2013-01-17 11:25:43.018036 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount did NOT detect btrfs
2013-01-17 11:25:43.043559 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount syncfs(2) syscall fully supported (by glibc and kernel)
2013-01-17 11:25:43.044050 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount found snaps <>
2013-01-17 11:25:43.055493 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount: enabling WRITEAHEAD journal mode: btrfs not detected
2013-01-17 11:25:43.148956 7fec1ff6e780 1 journal _open /dev/vdb fd 24: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
2013-01-17 11:25:43.154841 7fec1ff6e780 1 journal _open /dev/vdb fd 24: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
2013-01-17 11:25:43.157524 7fec1ff6e780 1 journal close /dev/vdb
2013-01-17 11:25:43.162119 7fec1ff6e780 2 osd.2 0 mounting /var/lib/ceph/osd/ceph-2 /dev/vdb
2013-01-17 11:25:43.166832 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is supported and appears to work
2013-01-17 11:25:43.166864 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
2013-01-17 11:25:43.167333 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount did NOT detect btrfs
2013-01-17 11:25:43.191820 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount syncfs(2) syscall fully supported (by glibc and kernel)
2013-01-17 11:25:43.192272 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount found snaps <>
2013-01-17 11:25:43.203591 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount: enabling WRITEAHEAD journal mode: btrfs not detected
2013-01-17 11:25:43.259918 7fec1ff6e780 1 journal _open /dev/vdb fd 32: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
2013-01-17 11:25:43.264955 7fec1ff6e780 1 journal _open /dev/vdb fd 32: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
2013-01-17 11:25:43.266060 7fec1ff6e780 2 osd.2 0 boot
2013-01-17 11:25:43.266352 7fec1ff6e780 10 osd.2 0 read_superblock sb(bc769874-9103-45a3-9d25-5f0743ae6ed0 osd.2 c371d180-7297-4dfc-bd53-6f0fbf6bb3af e8036 [6111,8036] lci=[6125,8036])
2013-01-17 11:25:43.266446 7fec1ff6e780 20 osd.2 0 get_map 8036 - loading and decoding 0x3030340
2013-01-17 11:25:43.266523 7fec1ff6e780 10 osd.2 0 add_map_bl 8036 0 bytes
2013-01-17 11:25:43.269516 7fec1ff6e780 -1 *** Caught signal (Aborted) **
in thread 7fec1ff6e780
ceph version 0.56.1 (e4a541624df62ef353e754391cbbb707f54b16f7)
1: ceph-osd() [0x834e00]
2: (()+0xfcb0) [0x7fec1f408cb0]
3: (gsignal()+0x35) [0x7fec1ddc0425]
4: (abort()+0x17b) [0x7fec1ddc3b8b]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fec1e6bbe2d]
6: (()+0x5ef26) [0x7fec1e6b9f26]
7: (()+0x5ef53) [0x7fec1e6b9f53]
8: (()+0x5f17e) [0x7fec1e6ba17e]
9: (ceph::buffer::list::iterator::copy(unsigned int, char*)+0x127) [0x8dc8b7]
10: (OSDMap::decode(ceph::buffer::list::iterator&)+0x3c) [0x8b0f8c]
11: (OSDMap::decode(ceph::buffer::list&)+0x3e) [0x8b192e]
12: (OSDService::get_map(unsigned int)+0x31e) [0x6d17fe]
13: (OSD::init()+0x7da) [0x6dbb5a]
14: (main()+0x205c) [0x61620c]
15: (__libc_start_main()+0xed) [0x7fec1ddab76d]
16: ceph-osd() [0x618599]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
--- begin dump of recent events ---
-63> 2013-01-17 11:25:43.003450 7fec1ff6e780 5 asok(0x2fd4000) register_command perfcounters_dump hook 0x2fbc010
-62> 2013-01-17 11:25:43.003570 7fec1ff6e780 5 asok(0x2fd4000) register_command 1 hook 0x2fbc010
-61> 2013-01-17 11:25:43.003578 7fec1ff6e780 5 asok(0x2fd4000) register_command perf dump hook 0x2fbc010
-60> 2013-01-17 11:25:43.003639 7fec1ff6e780 5 asok(0x2fd4000) register_command perfcounters_schema hook 0x2fbc010
-59> 2013-01-17 11:25:43.003645 7fec1ff6e780 5 asok(0x2fd4000) register_command 2 hook 0x2fbc010
-58> 2013-01-17 11:25:43.003652 7fec1ff6e780 5 asok(0x2fd4000) register_command perf schema hook 0x2fbc010
-57> 2013-01-17 11:25:43.003657 7fec1ff6e780 5 asok(0x2fd4000) register_command config show hook 0x2fbc010
-56> 2013-01-17 11:25:43.003657 7fec1ff6e780 5 asok(0x2fd4000) register_command config set hook 0x2fbc010
-55> 2013-01-17 11:25:43.003663 7fec1ff6e780 5 asok(0x2fd4000) register_command log flush hook 0x2fbc010
-54> 2013-01-17 11:25:43.003664 7fec1ff6e780 5 asok(0x2fd4000) register_command log dump hook 0x2fbc010
-53> 2013-01-17 11:25:43.003664 7fec1ff6e780 5 asok(0x2fd4000) register_command log reopen hook 0x2fbc010
-52> 2013-01-17 11:25:43.005789 7fec1ff6e780 0 ceph version 0.56.1 (e4a541624df62ef353e754391cbbb707f54b16f7), process ceph-osd, pid 21032
-51> 2013-01-17 11:25:43.006352 7fec1ff6e780 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6800/21032 need_addr=1
-50> 2013-01-17 11:25:43.006409 7fec1ff6e780 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6801/21032 need_addr=1
-49> 2013-01-17 11:25:43.006427 7fec1ff6e780 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6802/21032 need_addr=1
-48> 2013-01-17 11:25:43.009951 7fec1ff6e780 5 asok(0x2fd4000) init /var/run/ceph/ceph-osd.2.asok
-47> 2013-01-17 11:25:43.010000 7fec1ff6e780 5 asok(0x2fd4000) bind_and_listen /var/run/ceph/ceph-osd.2.asok
-46> 2013-01-17 11:25:43.010056 7fec1ff6e780 5 asok(0x2fd4000) register_command 0 hook 0x2fba0b0
-45> 2013-01-17 11:25:43.010068 7fec1ff6e780 5 asok(0x2fd4000) register_command version hook 0x2fba0b0
-44> 2013-01-17 11:25:43.010073 7fec1ff6e780 5 asok(0x2fd4000) register_command git_version hook 0x2fba0b0
-43> 2013-01-17 11:25:43.010076 7fec1ff6e780 5 asok(0x2fd4000) register_command help hook 0x2fbc0c0
-42> 2013-01-17 11:25:43.010270 7fec1bdf3700 5 asok(0x2fd4000) entry start
-41> 2013-01-17 11:25:43.017199 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is supported and appears to work
-40> 2013-01-17 11:25:43.017236 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
-39> 2013-01-17 11:25:43.018036 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount did NOT detect btrfs
-38> 2013-01-17 11:25:43.043559 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount syncfs(2) syscall fully supported (by glibc and kernel)
-37> 2013-01-17 11:25:43.044050 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount found snaps <>
-36> 2013-01-17 11:25:43.055493 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount: enabling WRITEAHEAD journal mode: btrfs not detected
-35> 2013-01-17 11:25:43.145030 7fec1ff6e780 2 journal open /dev/vdb fsid c371d180-7297-4dfc-bd53-6f0fbf6bb3af fs_op_seq 24718019
-34> 2013-01-17 11:25:43.148956 7fec1ff6e780 1 journal _open /dev/vdb fd 24: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
-33> 2013-01-17 11:25:43.150665 7fec1ff6e780 2 journal read_entry 3747028992 : seq 24718019 775 bytes
-32> 2013-01-17 11:25:43.151624 7fec1ff6e780 2 journal read_entry 3747033088 : bad header magic, end of journal
-31> 2013-01-17 11:25:43.151660 7fec1ff6e780 2 journal read_entry 3747033088 : bad header magic, end of journal
-30> 2013-01-17 11:25:43.151677 7fec1ff6e780 3 journal journal_replay: end of journal, done.
-29> 2013-01-17 11:25:43.154841 7fec1ff6e780 1 journal _open /dev/vdb fd 24: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
-28> 2013-01-17 11:25:43.156904 7fec18ded700 1 FileStore::op_tp worker finish
-27> 2013-01-17 11:25:43.157036 7fec195ee700 1 FileStore::op_tp worker finish
-26> 2013-01-17 11:25:43.157524 7fec1ff6e780 1 journal close /dev/vdb
-25> 2013-01-17 11:25:43.161312 7fec1ff6e780 10 monclient(hunting): build_initial_monmap
-24> 2013-01-17 11:25:43.161408 7fec1ff6e780 5 adding auth protocol: cephx
-23> 2013-01-17 11:25:43.161429 7fec1ff6e780 5 adding auth protocol: cephx
-22> 2013-01-17 11:25:43.161557 7fec1ff6e780 1 -- 0.0.0.0:6800/21032 messenger.start
-21> 2013-01-17 11:25:43.161633 7fec1ff6e780 1 -- :/0 messenger.start
-20> 2013-01-17 11:25:43.161718 7fec1ff6e780 1 -- 0.0.0.0:6802/21032 messenger.start
-19> 2013-01-17 11:25:43.161801 7fec1ff6e780 1 -- 0.0.0.0:6801/21032 messenger.start
-18> 2013-01-17 11:25:43.162119 7fec1ff6e780 2 osd.2 0 mounting /var/lib/ceph/osd/ceph-2 /dev/vdb
-17> 2013-01-17 11:25:43.166832 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is supported and appears to work
-16> 2013-01-17 11:25:43.166864 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount FIEMAP ioctl is disabled via 'filestore fiemap' config option
-15> 2013-01-17 11:25:43.167333 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount did NOT detect btrfs
-14> 2013-01-17 11:25:43.191820 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount syncfs(2) syscall fully supported (by glibc and kernel)
-13> 2013-01-17 11:25:43.192272 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount found snaps <>
-12> 2013-01-17 11:25:43.203591 7fec1ff6e780 0 filestore(/var/lib/ceph/osd/ceph-2) mount: enabling WRITEAHEAD journal mode: btrfs not detected
-11> 2013-01-17 11:25:43.256832 7fec1ff6e780 2 journal open /dev/vdb fsid c371d180-7297-4dfc-bd53-6f0fbf6bb3af fs_op_seq 24718019
-10> 2013-01-17 11:25:43.259918 7fec1ff6e780 1 journal _open /dev/vdb fd 32: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
-9> 2013-01-17 11:25:43.261290 7fec1ff6e780 2 journal read_entry 3747028992 : seq 24718019 775 bytes
-8> 2013-01-17 11:25:43.262081 7fec1ff6e780 2 journal read_entry 3747033088 : bad header magic, end of journal
-7> 2013-01-17 11:25:43.262107 7fec1ff6e780 2 journal read_entry 3747033088 : bad header magic, end of journal
-6> 2013-01-17 11:25:43.262115 7fec1ff6e780 3 journal journal_replay: end of journal, done.
-5> 2013-01-17 11:25:43.264955 7fec1ff6e780 1 journal _open /dev/vdb fd 32: 10485760000 bytes, block size 4096 bytes, directio = 1, aio = 0
-4> 2013-01-17 11:25:43.266060 7fec1ff6e780 2 osd.2 0 boot
-3> 2013-01-17 11:25:43.266352 7fec1ff6e780 10 osd.2 0 read_superblock sb(bc769874-9103-45a3-9d25-5f0743ae6ed0 osd.2 c371d180-7297-4dfc-bd53-6f0fbf6bb3af e8036 [6111,8036] lci=[6125,8036])
-2> 2013-01-17 11:25:43.266446 7fec1ff6e780 20 osd.2 0 get_map 8036 - loading and decoding 0x3030340
-1> 2013-01-17 11:25:43.266523 7fec1ff6e780 10 osd.2 0 add_map_bl 8036 0 bytes
0> 2013-01-17 11:25:43.269516 7fec1ff6e780 -1 *** Caught signal (Aborted) **
in thread 7fec1ff6e780
ceph version 0.56.1 (e4a541624df62ef353e754391cbbb707f54b16f7)
1: ceph-osd() [0x834e00]
2: (()+0xfcb0) [0x7fec1f408cb0]
3: (gsignal()+0x35) [0x7fec1ddc0425]
4: (abort()+0x17b) [0x7fec1ddc3b8b]
5: (__gnu_cxx::__verbose_terminate_handler()+0x11d) [0x7fec1e6bbe2d]
6: (()+0x5ef26) [0x7fec1e6b9f26]
7: (()+0x5ef53) [0x7fec1e6b9f53]
8: (()+0x5f17e) [0x7fec1e6ba17e]
9: (ceph::buffer::list::iterator::copy(unsigned int, char*)+0x127) [0x8dc8b7]
10: (OSDMap::decode(ceph::buffer::list::iterator&)+0x3c) [0x8b0f8c]
11: (OSDMap::decode(ceph::buffer::list&)+0x3e) [0x8b192e]
12: (OSDService::get_map(unsigned int)+0x31e) [0x6d17fe]
13: (OSD::init()+0x7da) [0x6dbb5a]
14: (main()+0x205c) [0x61620c]
15: (__libc_start_main()+0xed) [0x7fec1ddab76d]
16: ceph-osd() [0x618599]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
30/30 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
0/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 hadoop
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 100000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment