Shawn Edwards lesserevil
lesserevil
/ gist:4e6df0845518d47611335fbd862585e8
Created
April 3, 2019 20:16
partial osd log for broken secrets
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2019-04-03 13:15:48.483 7fcacf30c700 0 auth: could not find secret_id=41688 | |
| 2019-04-03 13:15:48.483 7fcacf30c700 0 cephx: verify_authorizer could not get service secret for service osd secret_id=41688 | |
| 2019-04-03 13:15:48.483 7fcacf30c700 0 --1- v1:10.36.9.28:6805/9853 >> v1:10.36.9.28:6830/8422 conn(0x136a26400 0x147eed000 :6805 s=ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0).handle_connect_message_2: got bad authorizer, auth_reply_len=0 | |
| 2019-04-03 13:15:48.663 7fcaceb0b700 0 auth: could not find secret_id=41688 | |
| 2019-04-03 13:15:48.663 7fcaceb0b700 0 cephx: verify_authorizer could not get service secret for service osd secret_id=41688 | |
| 2019-04-03 13:15:48.663 7fcaceb0b700 0 --1- v1:10.36.9.28:6805/9853 >> v1:10.36.9.22:6833/8017 conn(0x148b66880 0x14c9b7000 :6805 s=ACCEPTING_WAIT_CONNECT_MSG_AUTH pgs=0 cs=0 l=0).handle_connect_message_2: got bad authorizer, auth_reply_len=0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| WHO MASK LEVEL OPTION VALUE RO | |
| global advanced auth_client_required cephx * | |
| global advanced auth_cluster_required cephx * | |
| global advanced auth_service_required cephx * | |
| global advanced auth_service_ticket_ttl 3600.000000 | |
| global advanced cephx_sign_messages false | |
| global basic device_failure_prediction_mode local | |
| global advanced mon_max_pg_per_osd 500 | |
| global advanced mon_max_pg_per_osd 1000 | |
| global advanced ms_bind_msgr1 true |
lesserevil
/ gist:b805e99b40ef52d77968f86207bcffa1
Created
April 4, 2019 16:30
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cluster: | |
| id: 3f390b5e-2b1d-4a2f-ba00-64ec34377619 | |
| health: HEALTH_ERR | |
| 1 filesystem is degraded | |
| 1 MDSs report slow metadata IOs | |
| Reduced data availability: 6365 pgs inactive, 1213 pgs down, 5063 pgs peering, 260 pgs stale | |
| Degraded data redundancy: 169551/124029696 objects degraded (0.137%), 37 pgs degraded, 44 pgs undersized | |
| 564 pgs not deep-scrubbed in time | |
| 25716 slow requests are blocked > 32 sec | |
| 10091 stuck requests are blocked > 4096 sec |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| root@tyr-ceph-mon0:~# ceph osd df | |
| ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS | |
| 12 hdd 7.27539 1.00000 7.3 TiB 5.1 TiB 2.1 TiB 81 MiB 0 B 2.1 TiB 70.70 1.03 101 up | |
| 13 hdd 7.27539 1.00000 0 B 0 B 0 B 0 B 0 B 0 B 0 0 42 up | |
| 14 hdd 3.63689 1.00000 3.6 TiB 2.7 TiB 929 GiB 51 MiB 0 B 929 GiB 75.05 1.09 60 up | |
| 15 hdd 3.63689 1.00000 3.6 TiB 2.8 TiB 878 GiB 87 MiB 0 B 878 GiB 76.42 1.11 38 up | |
| 16 hdd 7.27539 1.00000 0 B 0 B 0 B 0 B 0 B 0 B 0 0 52 up | |
| 17 hdd 3.63689 1.00000 3.6 TiB 2.8 TiB 862 GiB 57 MiB 0 B 862 GiB 76.86 1.12 48 up | |
| 18 hdd 7.27539 1.00000 7.3 TiB 5.5 TiB 1.8 TiB 191 MiB 0 B 1.8 TiB 75.57 1.10 103 up | |
| 19 hdd 3.63689 1.00000 0 B 0 B 0 B 0 B 0 B 0 B 0 0 24 up |