Skip to content

Instantly share code, notes, and snippets.

@rzarzynski
Last active February 7, 2023 15:35
Show Gist options
  • Save rzarzynski/b38134b301d05637404ec0e96adae2d1 to your computer and use it in GitHub Desktop.
Save rzarzynski/b38134b301d05637404ec0e96adae2d1 to your computer and use it in GitHub Desktop.
20230202_zeroed_startevent_of_historic_op.md

The zeroed StartEvent problem

After running some client IO:

[rzarzynski@o06 build]$ bin/ceph tell osd.2 dump_historic_slow_ops
...
            {
                "type": "client_request",
                "id": 216614,
                "detail": {
                    "time_event": {
                        "name": "crimson::osd::TrackableOperationT<crimson::osd::ClientRequest>::StartEvent",                                                                                                     
                        "initiated_at": "0.000000"
                    },
                    "blocking_event": {
                        "name": "crimson::osd::ConnectionPipeline::AwaitActive",
                        "initiated_at": "2023-02-02T01:08:59.502627+0000"
                    },
                    "blocking_event": {
                        "name": "crimson::osd::ConnectionPipeline::AwaitMap",
                        "initiated_at": "2023-02-02T01:08:59.502658+0000"
                    },
                    "blocking_event": {
                        "name": "crimson::osd::OSDMapGate<(crimson::osd::OSDMapGateType)0>::OSDMapBlocker",                                                                                                       
                        "initiated_at": "0.000000"
                    },
                    "blocking_event": {
                        "name": "crimson::osd::ConnectionPipeline::GetPG",
                        "initiated_at": "2023-02-02T01:08:59.502683+0000"
                    },
                    "blocking_event": {
                        "name": "crimson::osd::PGMap::PGCreationState",
                        "initiated_at": "0.000000"
                    },
                    "time_event": {
                        "name": "crimson::osd::TrackableOperationT<crimson::osd::ClientRequest>::CompletionEvent",                                                                                                
                        "initiated_at": "2023-02-02T01:08:59.502908+0000"
                    }
                }
            }
        ]
    },

@rzarzynski
Copy link
Author

The root cause for zeroed StartEvent:

580b5e24c6ad7f9cbf36fecd4be6af2c7ad053e2 has removed the call to track_event<StartEvent>. I think the rationale was that e.g. RepRequest is not trackable to the new start_pg_operation() is shared with it as well.

@rzarzynski
Copy link
Author

diff --git a/src/crimson/osd/osd_operations/client_request.cc b/src/crimson/osd/osd_operations/client_request.cc
index 87f28a482d2..0d22fe47e35 100644
--- a/src/crimson/osd/osd_operations/client_request.cc
+++ b/src/crimson/osd/osd_operations/client_request.cc
@@ -55,7 +55,9 @@ ClientRequest::ClientRequest(
     conn(std::move(conn)),
     m(std::move(m)),
     instance_handle(new instance_handle_t)
-{}
+{
+  track_event<StartEvent>();
+}
 
 ClientRequest::~ClientRequest()
 {

@rzarzynski
Copy link
Author

Fix PR: ceph/ceph#50019.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment