Skip to content

Instantly share code, notes, and snippets.

@pvanheus
Created January 15, 2023 06:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pvanheus/5c57455cacdc91afc9ce27fd489cae25 to your computer and use it in GitHub Desktop.
Save pvanheus/5c57455cacdc91afc9ce27fd489cae25 to your computer and use it in GitHub Desktop.
# ceph-bluestore-tool fsck --path /var/lib/ceph/ed7b2c16-b053-45e2-a1fe-bf3474f90508/osd.27/
2023-01-15T08:38:04.289+0200 7f2a2a03c540 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file)
/build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)' thread 7f2a2a03c540 time 2023-01-15T08:39:31.304968+0200
/build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: 18968: FAILED ceph_assert(collection_ref)
2023-01-15T08:39:31.298+0200 7f2a2a03c540 -1 bluestore::NCB::read_allocation_from_onodes::stray object 2#55:ffffffff:::2000055f327.00002287:head# not owned by any collection
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14f) [0x7f2a2acc07c6]
2: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
3: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
4: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
5: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
6: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
7: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
8: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
9: main()
10: __libc_start_main()
11: _start()
*** Caught signal (Aborted) **
in thread 7f2a2a03c540 thread_name:ceph-bluestore-
2023-01-15T08:39:31.306+0200 7f2a2a03c540 -1 /build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)' thread 7f2a2a03c540 time 2023-01-15T08:39:31.304968+0200
/build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: 18968: FAILED ceph_assert(collection_ref)
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14f) [0x7f2a2acc07c6]
2: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
3: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
4: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
5: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
6: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
7: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
8: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
9: main()
10: __libc_start_main()
11: _start()
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f2a2a75d420]
2: gsignal()
3: abort()
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f2a2acc082e]
5: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
6: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
7: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
8: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
9: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
10: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
11: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
12: main()
13: __libc_start_main()
14: _start()
2023-01-15T08:39:31.314+0200 7f2a2a03c540 -1 *** Caught signal (Aborted) **
in thread 7f2a2a03c540 thread_name:ceph-bluestore-
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f2a2a75d420]
2: gsignal()
3: abort()
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f2a2acc082e]
5: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
6: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
7: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
8: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
9: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
10: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
11: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
12: main()
13: __libc_start_main()
14: _start()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
-309> 2023-01-15T08:38:04.289+0200 7f2a2a03c540 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file)
-308> 2023-01-15T08:39:31.298+0200 7f2a2a03c540 -1 bluestore::NCB::read_allocation_from_onodes::stray object 2#55:ffffffff:::2000055f327.00002287:head# not owned by any collection
-307> 2023-01-15T08:39:31.306+0200 7f2a2a03c540 -1 /build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)' thread 7f2a2a03c540 time 2023-01-15T08:39:31.304968+0200
/build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: 18968: FAILED ceph_assert(collection_ref)
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14f) [0x7f2a2acc07c6]
2: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
3: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
4: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
5: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
6: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
7: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
8: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
9: main()
10: __libc_start_main()
11: _start()
-306> 2023-01-15T08:39:31.314+0200 7f2a2a03c540 -1 *** Caught signal (Aborted) **
in thread 7f2a2a03c540 thread_name:ceph-bluestore-
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f2a2a75d420]
2: gsignal()
3: abort()
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f2a2acc082e]
5: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
6: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
7: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
8: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
9: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
10: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
11: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
12: main()
13: __libc_start_main()
14: _start()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
-10> 2023-01-15T08:38:04.289+0200 7f2a2a03c540 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file)
-2> 2023-01-15T08:39:31.298+0200 7f2a2a03c540 -1 bluestore::NCB::read_allocation_from_onodes::stray object 2#55:ffffffff:::2000055f327.00002287:head# not owned by any collection
-1> 2023-01-15T08:39:31.306+0200 7f2a2a03c540 -1 /build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: In function 'int BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)' thread 7f2a2a03c540 time 2023-01-15T08:39:31.304968+0200
/build/ceph-17.2.5/src/os/bluestore/BlueStore.cc: 18968: FAILED ceph_assert(collection_ref)
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14f) [0x7f2a2acc07c6]
2: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
3: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
4: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
5: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
6: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
7: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
8: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
9: main()
10: __libc_start_main()
11: _start()
0> 2023-01-15T08:39:31.314+0200 7f2a2a03c540 -1 *** Caught signal (Aborted) **
in thread 7f2a2a03c540 thread_name:ceph-bluestore-
ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)
1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f2a2a75d420]
2: gsignal()
3: abort()
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1b7) [0x7f2a2acc082e]
5: /usr/lib/ceph/libceph-common.so.2(+0x27c9d8) [0x7f2a2acc09d8]
6: (BlueStore::read_allocation_from_onodes(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0xa24) [0x560d6baf5754]
7: (BlueStore::reconstruct_allocations(SimpleBitmap*, BlueStore::read_alloc_stats_t&)+0x5f) [0x560d6baf66ff]
8: (BlueStore::read_allocation_from_drive_on_startup()+0x99) [0x560d6baf68b9]
9: (BlueStore::_init_alloc(std::map<unsigned long, unsigned long, std::less<unsigned long>, std::allocator<std::pair<unsigned long const, unsigned long> > >*)+0xaca) [0x560d6bb0c15a]
10: (BlueStore::_open_db_and_around(bool, bool)+0x35c) [0x560d6bb380dc]
11: (BlueStore::_fsck(BlueStore::FSCKDepth, bool)+0x250) [0x560d6bb3a8c0]
12: main()
13: __libc_start_main()
14: _start()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Aborted (core dumped)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment