Skip to content

Instantly share code, notes, and snippets.

@ncdc

ncdc/kcp.log

Created January 26, 2022 15:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ncdc/e87f5a47b7a423d3d201e581efe3d272 to your computer and use it in GitHub Desktop.
Save ncdc/e87f5a47b7a423d3d201e581efe3d272 to your computer and use it in GitHub Desktop.
Download ZIP
kcp ingress source log file - e2e test run
Raw
kcp.log
I0126 10:33:12.395799 96588 etcd.go:57] Creating embedded etcd server
{"level":"warn","ts":1643211193.659682,"caller":"fileutil/fileutil.go:57","msg":"check file permission","error":"directory \"/Users/angoldst/go/src/github.com/kcp-dev/kcp/testout/TestIngressController/create_an_ingress_that_points_to_a_valid_service_and_expect_the_Ingress_to_be_scheduled_to_the_same_cluster/2345490025/data/kcp/source\" exist, but the permission is \"drwxr-xr-x\". The recommended permission is \"-rwx------\" to prevent possible unprivileged access to the data"}
{"level":"warn","ts":"2022-01-26T10:33:16.624-0500","caller":"etcdserver/metrics.go:224","msg":"failed to get file descriptor usage","error":"cannot get FDUsage on darwin"}
{"level":"warn","ts":"2022-01-26T10:33:18.036-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"265.457663ms","expected-duration":"100ms","prefix":"","request":"ID:4097570423066461443 Method:\"PUT\" Path:\"/0/version\" Val:\"3.5.0\" ","response":""}
{"level":"warn","ts":"2022-01-26T10:33:18.036-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:17.553-0500","time spent":"482.957407ms","remote":"127.0.0.1:63708","response type":"/etcdserverpb.Cluster/MemberList","request count":-1,"request size":-1,"response count":-1,"response size":-1,"request content":""}
Connected to etcd 7466225816993020125 default
I0126 10:33:19.510133 96588 serving.go:341] Generated self-signed cert (/Users/angoldst/go/src/github.com/kcp-dev/kcp/testout/TestIngressController/create_an_ingress_that_points_to_a_valid_service_and_expect_the_Ingress_to_be_scheduled_to_the_same_cluster/2345490025/data/kcp/source/apiserver.crt, /Users/angoldst/go/src/github.com/kcp-dev/kcp/testout/TestIngressController/create_an_ingress_that_points_to_a_valid_service_and_expect_the_Ingress_to_be_scheduled_to_the_same_cluster/2345490025/data/kcp/source/apiserver.key)
I0126 10:33:19.510156 96588 server.go:399] external host was not specified, using 192.168.4.118
I0126 10:33:19.511766 96588 plugins.go:158] Loaded 1 mutating admission controller(s) successfully in the following order: NamespaceLifecycle.
W0126 10:33:20.045374 96588 genericapiserver.go:454] Skipping API authentication.k8s.io/v1beta1 because it has no resources.
W0126 10:33:20.046059 96588 genericapiserver.go:454] Skipping API authorization.k8s.io/v1beta1 because it has no resources.
W0126 10:33:20.046924 96588 genericapiserver.go:454] Skipping API certificates.k8s.io/v1beta1 because it has no resources.
W0126 10:33:20.047342 96588 genericapiserver.go:454] Skipping API coordination.k8s.io/v1beta1 because it has no resources.
W0126 10:33:20.048359 96588 genericapiserver.go:454] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources.
W0126 10:33:20.048365 96588 genericapiserver.go:454] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
W0126 10:33:20.049130 96588 genericapiserver.go:454] Skipping API flowcontrol.apiserver.k8s.io/v1alpha1 because it has no resources.
W0126 10:33:20.049591 96588 genericapiserver.go:454] Skipping API events.k8s.io/v1beta1 because it has no resources.
I0126 10:33:20.049880 96588 plugins.go:158] Loaded 1 mutating admission controller(s) successfully in the following order: NamespaceLifecycle.
I0126 10:33:20.597453 96588 dynamic_serving_content.go:129] "Starting controller" name="serving-cert::/Users/angoldst/go/src/github.com/kcp-dev/kcp/testout/TestIngressController/create_an_ingress_that_points_to_a_valid_service_and_expect_the_Ingress_to_be_scheduled_to_the_same_cluster/2345490025/data/kcp/source/apiserver.crt::/Users/angoldst/go/src/github.com/kcp-dev/kcp/testout/TestIngressController/create_an_ingress_that_points_to_a_valid_service_and_expect_the_Ingress_to_be_scheduled_to_the_same_cluster/2345490025/data/kcp/source/apiserver.key"
I0126 10:33:20.597586 96588 secure_serving.go:256] Serving securely on [::]:62939
I0126 10:33:20.597636 96588 tlsconfig.go:240] "Starting DynamicServingCertificateController"
I0126 10:33:20.597764 96588 establishing_controller.go:77] Starting EstablishingController
I0126 10:33:20.597768 96588 controller.go:130] Starting OpenAPI controller
I0126 10:33:20.597801 96588 naming_controller.go:296] Starting NamingConditionController
I0126 10:33:20.597804 96588 crd_finalizer.go:269] Starting CRDFinalizer
I0126 10:33:20.597811 96588 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
I0126 10:33:20.597841 96588 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
I0126 10:33:20.598056 96588 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
I0126 10:33:20.598066 96588 shared_informer.go:240] Waiting for caches to sync for cluster_authentication_trust_controller
I0126 10:33:20.598113 96588 namespace_controller.go:200] Starting namespace controller
I0126 10:33:20.598120 96588 shared_informer.go:240] Waiting for caches to sync for namespace
I0126 10:33:20.698806 96588 shared_informer.go:247] Caches are synced for namespace
I0126 10:33:20.698812 96588 shared_informer.go:247] Caches are synced for cluster_authentication_trust_controller
I0126 10:33:20.699058 96588 bootstrap.go:76] bootstrapping clusters.cluster.example.dev
I0126 10:33:20.699058 96588 bootstrap.go:76] bootstrapping apiresourceimports.apiresource.kcp.dev
I0126 10:33:20.699074 96588 bootstrap.go:76] bootstrapping negotiatedapiresources.apiresource.kcp.dev
{"level":"warn","ts":"2022-01-26T10:33:20.905-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"291.307938ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:20.905-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"201.383667ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:20.905-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"201.342822ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:20.905-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"299.745331ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-node-high\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:20.905-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"291.299903ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:20.905-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"201.278407ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:21.572-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"663.61189ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/probes\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:21.572-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:20.908-0500","time spent":"663.700125ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":65,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/probes\" "}
{"level":"warn","ts":"2022-01-26T10:33:21.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"336.332572ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461472 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" mod_revision:0 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" value_size:3378 >> failure:<>>","response":"size:14"}
I0126 10:33:21.573382 96588 trace.go:205] Trace[989900470]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/probes,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:ffac27cc-6ae1-4d47-824f-82b5216a14e8,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:20.908) (total time: 664ms):
Trace[989900470]: [664.533107ms] [664.533107ms] END
{"level":"warn","ts":"2022-01-26T10:33:21.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"551.851594ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:21.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:20.909-0500","time spent":"664.390925ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":7518,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" mod_revision:0 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" value_size:7409 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:21.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:20.909-0500","time spent":"664.395163ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":7336,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" mod_revision:0 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" value_size:7223 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:21.574-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:20.909-0500","time spent":"664.647546ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":3477,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" mod_revision:0 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" value_size:3378 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:21.574-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"552.018973ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:21.574-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.022-0500","time spent":"552.051634ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:21.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.021-0500","time spent":"551.90295ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:33:21.577007 96588 trace.go:205] Trace[1947116174]: "Create" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:391e2d94-efe8-4a79-bc66-904165fd0e5d,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:20.907) (total time: 669ms):
Trace[1947116174]: ---"Object stored in database" 669ms (10:33:21.576)
Trace[1947116174]: [669.902606ms] [669.902606ms] END
I0126 10:33:21.577154 96588 trace.go:205] Trace[1268371454]: "Create" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:87f86b8f-ff1e-42f3-8cd6-32780dcf8015,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:20.907) (total time: 669ms):
Trace[1268371454]: ---"Object stored in database" 669ms (10:33:21.576)
Trace[1268371454]: [669.897722ms] [669.897722ms] END
I0126 10:33:21.577499 96588 trace.go:205] Trace[1328406908]: "Create" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:8679ec9d-31c8-4ed7-9d27-7964be1c37f7,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:20.907) (total time: 670ms):
Trace[1328406908]: ---"Object stored in database" 669ms (10:33:21.577)
Trace[1328406908]: [670.262519ms] [670.262519ms] END
{"level":"warn","ts":"2022-01-26T10:33:22.077-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461476,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:22.578-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461476,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:23.078-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461476,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:23.318-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.740980915s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3489"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.577-0500","time spent":"1.741185984s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":93,"response count":1,"response size":3512,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.741406364s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-leader-election\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.63346282s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.740606457s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7348"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.741005742s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7530"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.633830817s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.740578037s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3489"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.578-0500","time spent":"1.741050372s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":103,"response count":1,"response size":7553,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.719701558s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/\" range_end:\"/registry/rbac.authorization.k8s.io/clusterroles/admin0\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.685-0500","time spent":"1.633871102s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.685-0500","time spent":"1.633500677s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.578-0500","time spent":"1.740679673s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":107,"response count":1,"response size":7371,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.577-0500","time spent":"1.74148401s","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-leader-election\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.578-0500","time spent":"1.740685066s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":93,"response count":1,"response size":3512,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:23.319-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:21.599-0500","time spent":"1.719862333s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":114,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/\" range_end:\"/registry/rbac.authorization.k8s.io/clusterroles/admin0\" "}
I0126 10:33:23.319587 96588 trace.go:205] Trace[1650951669]: "List etcd3" key:/rbac.authorization.k8s.io/clusterroles/admin,resourceVersion:,resourceVersionMatch:,limit:0,continue: (26-Jan-2022 10:33:21.598) (total time: 1720ms):
Trace[1650951669]: [1.720530449s] [1.720530449s] END
I0126 10:33:23.319595 96588 trace.go:205] Trace[1012237941]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/system-leader-election,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:9d2fbb13-4141-46c9-bba2-5f0763d803e0,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:21.577) (total time: 1742ms):
Trace[1012237941]: [1.742110116s] [1.742110116s] END
I0126 10:33:23.319874 96588 trace.go:205] Trace[1597583239]: "List" url:/apis/rbac.authorization.k8s.io/v1/clusterroles,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:469dcaef-bdb6-46f4-9bc9-e8c3c1c60a92,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:21.598) (total time: 1720ms):
Trace[1597583239]: ---"Listing from storage done" 1720ms (10:33:23.319)
Trace[1597583239]: [1.720823301s] [1.720823301s] END
I0126 10:33:23.320439 96588 trace.go:205] Trace[1421827037]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:e92b1088-c170-4366-a942-1d46810338e0,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:21.577) (total time: 1742ms):
Trace[1421827037]: ---"About to write a response" 1742ms (10:33:23.320)
Trace[1421827037]: [1.742479516s] [1.742479516s] END
I0126 10:33:23.320803 96588 trace.go:205] Trace[2075143048]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:85e3685d-c3ea-4c56-9676-d468ba46a760,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:21.577) (total time: 1742ms):
Trace[2075143048]: ---"About to write a response" 1742ms (10:33:23.320)
Trace[2075143048]: [1.742864945s] [1.742864945s] END
I0126 10:33:23.321030 96588 trace.go:205] Trace[73858329]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:f22ccf3d-315b-4daa-8b3d-1a020af2e58f,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:21.578) (total time: 1742ms):
Trace[73858329]: ---"About to write a response" 1741ms (10:33:23.320)
Trace[73858329]: [1.742701789s] [1.742701789s] END
I0126 10:33:23.323086 96588 trace.go:205] Trace[161894676]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:21.578) (total time: 1744ms):
Trace[161894676]: ---"initial value restored" 1741ms (10:33:23.320)
Trace[161894676]: [1.744734557s] [1.744734557s] END
I0126 10:33:23.323406 96588 trace.go:205] Trace[173077444]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:50c5ace6-9b48-4d2f-bc8a-4aadefa62910,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:21.577) (total time: 1745ms):
Trace[173077444]: ---"Object stored in database" 1744ms (10:33:23.323)
Trace[173077444]: [1.745365078s] [1.745365078s] END
{"level":"warn","ts":"2022-01-26T10:33:23.529-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"205.286517ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-admin\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"189.22133ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"189.243314ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:4400"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"189.266576ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7348"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"188.878043ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"286.423149ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:4400"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"286.051812ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7530"}
{"level":"warn","ts":"2022-01-26T10:33:23.611-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"189.456624ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7530"}
{"level":"warn","ts":"2022-01-26T10:33:24.114-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461486,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:24.614-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461486,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:25.115-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461486,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:25.615-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461486,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:25.728-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000585804s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:25.728-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000639952s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:25.728-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.727-0500","time spent":"2.000645182s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:25.728-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.727-0500","time spent":"2.000689225s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:26.116-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461486,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:26.617-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461486,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:26.718-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.104758482s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/edit\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:26.718-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.613-0500","time spent":"3.104854645s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/edit\" "}
{"level":"warn","ts":"2022-01-26T10:33:26.718-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.615-0500","time spent":"3.10353173s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8469,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" mod_revision:5 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" value_size:8360 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:26.718-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.614-0500","time spent":"3.104346925s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4397,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" mod_revision:8 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" value_size:4298 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" > >"}
I0126 10:33:26.719030 96588 trace.go:205] Trace[1338742162]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/edit,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:6284caec-b53e-4af1-a848-f23f91532b9f,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.613) (total time: 3105ms):
Trace[1338742162]: [3.105316985s] [3.105316985s] END
I0126 10:33:26.719445 96588 trace.go:205] Trace[238089926]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:23.324) (total time: 3394ms):
Trace[238089926]: ---"initial value restored" 287ms (10:33:23.612)
Trace[238089926]: ---"Transaction committed" 3105ms (10:33:26.719)
Trace[238089926]: [3.394551987s] [3.394551987s] END
I0126 10:33:26.719591 96588 trace.go:205] Trace[1330486984]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:23.325) (total time: 3394ms):
Trace[1330486984]: ---"initial value restored" 287ms (10:33:23.612)
Trace[1330486984]: ---"Transaction committed" 3104ms (10:33:26.719)
Trace[1330486984]: [3.3941605s] [3.3941605s] END
I0126 10:33:26.719756 96588 trace.go:205] Trace[809585463]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:bce79d91-0d63-4039-822d-616d767aa11d,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.324) (total time: 3395ms):
Trace[809585463]: ---"Object stored in database" 3394ms (10:33:26.719)
Trace[809585463]: [3.395285271s] [3.395285271s] END
I0126 10:33:26.719952 96588 trace.go:205] Trace[949781206]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:5cbf42ef-96e9-4609-9436-600ac0f81ec5,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.324) (total time: 3395ms):
Trace[949781206]: ---"Object stored in database" 3394ms (10:33:26.719)
Trace[949781206]: [3.395116515s] [3.395116515s] END
{"level":"warn","ts":"2022-01-26T10:33:27.267-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"546.054308ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7348"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.6454764s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:4409"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"547.021544ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-view\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.653422245s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-controller-manager\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:26.721-0500","time spent":"546.154969ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":107,"response count":1,"response size":7371,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.622-0500","time spent":"3.645553224s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":93,"response count":1,"response size":4432,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.614-0500","time spent":"3.653559173s","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-controller-manager\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:26.720-0500","time spent":"547.126922ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-view\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.430130317s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.645508183s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8481"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.430266549s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:25.837-0500","time spent":"1.430395844s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"544.576811ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8481"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:25.837-0500","time spent":"1.430437597s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.622-0500","time spent":"3.645660845s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":103,"response count":1,"response size":8504,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.645346382s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7348"}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:26.723-0500","time spent":"544.687632ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":103,"response count":1,"response size":8504,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:27.268-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:23.622-0500","time spent":"3.645753514s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":107,"response count":1,"response size":7371,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" "}
I0126 10:33:27.269000 96588 trace.go:205] Trace[1052363088]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:aggregate-to-view,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:e0a07872-af01-46d9-8927-adb2d57ec4b0,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:26.720) (total time: 548ms):
Trace[1052363088]: [548.151371ms] [548.151371ms] END
I0126 10:33:27.269019 96588 trace.go:205] Trace[1706669448]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/kube-controller-manager,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:2013fcfa-0a5d-47f5-bcae-68bda8a4b559,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.614) (total time: 3654ms):
Trace[1706669448]: [3.654643072s] [3.654643072s] END
I0126 10:33:27.270128 96588 trace.go:205] Trace[1725052766]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:fcb35ca9-2060-47db-a748-a68b3072a5b9,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.622) (total time: 3647ms):
Trace[1725052766]: ---"About to write a response" 3647ms (10:33:27.269)
Trace[1725052766]: [3.647809544s] [3.647809544s] END
I0126 10:33:27.270513 96588 bootstrap.go:78] bootstrapped clusters.cluster.example.dev after 6.571266184s
I0126 10:33:27.270817 96588 trace.go:205] Trace[906929637]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:6ffde85e-7a1f-4047-ac93-79a6221720df,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.622) (total time: 3648ms):
Trace[906929637]: ---"About to write a response" 3647ms (10:33:27.270)
Trace[906929637]: [3.648394416s] [3.648394416s] END
I0126 10:33:27.270945 96588 trace.go:205] Trace[978305459]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:37084961-361a-406a-aa7f-22486083a1b4,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:23.622) (total time: 3648ms):
Trace[978305459]: ---"About to write a response" 3648ms (10:33:27.270)
Trace[978305459]: [3.648667175s] [3.648667175s] END
{"level":"warn","ts":"2022-01-26T10:33:27.772-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461493,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:28.272-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461493,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:28.539-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.272-0500","time spent":"1.266864472s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8303,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" mod_revision:6 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" value_size:8190 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:28.539-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.272-0500","time spent":"1.266150319s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8478,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" mod_revision:12 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" value_size:8369 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:28.539-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.266932525s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterrolebindings/admin/system:discovery\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:28.539-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.272-0500","time spent":"1.267029818s","remote":"127.0.0.1:63962","response type":"/etcdserverpb.KV/Range","request count":0,"request size":80,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterrolebindings/admin/system:discovery\" "}
I0126 10:33:28.539923 96588 trace.go:205] Trace[1227808691]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterrolebindings/system:discovery,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:abad1e7f-1c5b-460b-840c-d349edc5028e,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:27.272) (total time: 1267ms):
Trace[1227808691]: [1.267815066s] [1.267815066s] END
I0126 10:33:28.540269 96588 trace.go:205] Trace[1463746058]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:26.721) (total time: 1818ms):
Trace[1463746058]: ---"initial value restored" 548ms (10:33:27.269)
Trace[1463746058]: ---"Transaction committed" 1267ms (10:33:28.539)
Trace[1463746058]: [1.818480199s] [1.818480199s] END
I0126 10:33:28.540554 96588 trace.go:205] Trace[1259963111]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:26.723) (total time: 1816ms):
Trace[1259963111]: ---"initial value restored" 546ms (10:33:27.270)
Trace[1259963111]: ---"Transaction committed" 1267ms (10:33:28.540)
Trace[1259963111]: [1.81695301s] [1.81695301s] END
I0126 10:33:28.540891 96588 trace.go:205] Trace[373890729]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:29abab16-776b-45f0-a2c6-c5c45bd3bf1c,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:26.721) (total time: 1819ms):
Trace[373890729]: ---"Object stored in database" 1818ms (10:33:28.540)
Trace[373890729]: [1.819491239s] [1.819491239s] END
I0126 10:33:28.540992 96588 trace.go:205] Trace[1745031006]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:f075b617-962d-41aa-bd9a-38502ac3c7ef,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:26.723) (total time: 1817ms):
Trace[1745031006]: ---"Object stored in database" 1817ms (10:33:28.540)
Trace[1745031006]: [1.817893007s] [1.817893007s] END
{"level":"warn","ts":"2022-01-26T10:33:28.728-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.343407308s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.456309636s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-scheduler\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.385-0500","time spent":"1.343518964s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"185.189345ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/cluster-admin\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.272-0500","time spent":"1.456377191s","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":73,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-scheduler\" "}
{"level":"warn","ts":"2022-01-26T10:33:28.728-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.406088514s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8490"}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.406243603s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8315"}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.343543176s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.322-0500","time spent":"1.406321223s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":107,"response count":1,"response size":8338,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.322-0500","time spent":"1.406276128s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":103,"response count":1,"response size":8513,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:27.385-0500","time spent":"1.343665092s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:28.729-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"183.415816ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8315"}
I0126 10:33:28.730214 96588 trace.go:205] Trace[35385381]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/kube-scheduler,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:386c1a32-26c9-4b88-8875-9eff8c6e38c8,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:27.272) (total time: 1457ms):
Trace[35385381]: [1.457599727s] [1.457599727s] END
I0126 10:33:28.731664 96588 trace.go:205] Trace[636860343]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:73c5f8eb-700e-4439-895e-bdec1586d581,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:27.322) (total time: 1409ms):
Trace[636860343]: ---"About to write a response" 1408ms (10:33:28.731)
Trace[636860343]: [1.409115044s] [1.409115044s] END
I0126 10:33:28.732089 96588 trace.go:205] Trace[279590545]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:80dd7fdc-6ba0-4d19-b91f-a2acfe7f2176,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:27.322) (total time: 1409ms):
Trace[279590545]: ---"About to write a response" 1408ms (10:33:28.731)
Trace[279590545]: [1.40957589s] [1.40957589s] END
I0126 10:33:28.732751 96588 bootstrap.go:78] bootstrapped apiresourceimports.apiresource.kcp.dev after 8.033454228s
{"level":"warn","ts":"2022-01-26T10:33:29.234-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461499,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:29.735-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461499,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:30.235-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461499,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:30.735-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461499,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:30.756-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:28.732-0500","time spent":"2.023421847s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":563,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/cluster-admin\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/cluster-admin\" value_size:487 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:30.756-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.022458741s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-system-service-accounts\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:30.756-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:28.733-0500","time spent":"2.022338558s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8312,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" mod_revision:14 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" value_size:8199 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:30.756-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:28.733-0500","time spent":"2.02250744s","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":87,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-system-service-accounts\" "}
I0126 10:33:30.756567 96588 trace.go:205] Trace[2048671883]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/kube-system-service-accounts,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:feda9ad8-f688-43cb-b6f1-2130bba9dee6,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:28.733) (total time: 2022ms):
Trace[2048671883]: [2.022954961s] [2.022954961s] END
I0126 10:33:30.756744 96588 trace.go:205] Trace[518332200]: "Create" url:/apis/rbac.authorization.k8s.io/v1/clusterroles,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:e356c110-c9f2-460e-baa9-5a2ff3ea79b0,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:28.731) (total time: 2025ms):
Trace[518332200]: ---"Object stored in database" 2024ms (10:33:30.756)
Trace[518332200]: [2.025348934s] [2.025348934s] END
I0126 10:33:30.756939 96588 trace.go:205] Trace[948290110]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:28.545) (total time: 2211ms):
Trace[948290110]: ---"initial value restored" 185ms (10:33:28.731)
Trace[948290110]: ---"Transaction committed" 2022ms (10:33:30.756)
Trace[948290110]: [2.211345269s] [2.211345269s] END
I0126 10:33:30.757375 96588 trace.go:205] Trace[788286486]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:15e71513-b4bc-4700-8709-0480838be5ea,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:28.544) (total time: 2212ms):
Trace[788286486]: ---"Object stored in database" 2211ms (10:33:30.756)
Trace[788286486]: [2.212428563s] [2.212428563s] END
{"level":"warn","ts":"2022-01-26T10:33:30.834-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000702429s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:30.834-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000780282s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:30.834-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:28.833-0500","time spent":"2.000826949s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:30.834-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:28.833-0500","time spent":"2.00077434s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
W0126 10:33:30.834142 96588 server.go:1299] [core] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
{"level":"warn","ts":"2022-01-26T10:33:31.244-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.421439647s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8324"}
{"level":"warn","ts":"2022-01-26T10:33:31.244-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:28.822-0500","time spent":"2.421510332s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":107,"response count":1,"response size":8347,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:31.244-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:30.757-0500","time spent":"486.486978ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":739,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-system-service-accounts\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-system-service-accounts\" value_size:646 >> failure:<>"}
I0126 10:33:31.246128 96588 trace.go:205] Trace[1544821229]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:b86fcfef-edfc-437a-9614-e8fb92dce311,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:28.822) (total time: 2423ms):
Trace[1544821229]: ---"About to write a response" 2423ms (10:33:31.245)
Trace[1544821229]: [2.423371683s] [2.423371683s] END
I0126 10:33:31.246691 96588 bootstrap.go:78] bootstrapped negotiatedapiresources.apiresource.kcp.dev after 10.547300937s
I0126 10:33:31.246824 96588 bootstrap.go:76] bootstrapping clusters.cluster.example.dev
I0126 10:33:31.246830 96588 bootstrap.go:76] bootstrapping apiresourceimports.apiresource.kcp.dev
I0126 10:33:31.246854 96588 bootstrap.go:76] bootstrapping negotiatedapiresources.apiresource.kcp.dev
{"level":"warn","ts":"2022-01-26T10:33:31.745-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461505,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:32.245-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461505,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:32.745-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461505,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:32.936-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.00066651s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:32.936-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000721883s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:32.936-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:30.935-0500","time spent":"2.000797432s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:32.936-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:30.935-0500","time spent":"2.00080342s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:33.246-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461505,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:33.747-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461505,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:34.248-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461505,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.558527177s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:discovery\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.069759227s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/service-accounts\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:30.757-0500","time spent":"3.558611339s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":73,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:discovery\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.068270601s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.068286784s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.264943862s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:31.246-0500","time spent":"3.069822302s","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":75,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/service-accounts\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.265024972s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:31.248-0500","time spent":"3.06832399s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":0,"response size":27,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"3.068730744s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:33.051-0500","time spent":"1.265040548s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:31.248-0500","time spent":"3.068346799s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":0,"response size":27,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:31.247-0500","time spent":"3.068817273s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":0,"response size":27,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.316-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:33.051-0500","time spent":"1.2650902s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:33:34.316924 96588 trace.go:205] Trace[1201623134]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:discovery,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:edea3912-6fb8-4c3c-ae71-f3580573b02d,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:30.757) (total time: 3559ms):
Trace[1201623134]: [3.559247082s] [3.559247082s] END
I0126 10:33:34.317056 96588 trace.go:205] Trace[1951324233]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:7daa3c01-b491-4830-bd1d-4df5c89299ad,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:31.247) (total time: 3068ms):
Trace[1951324233]: [3.068961114s] [3.068961114s] END
I0126 10:33:34.317067 96588 trace.go:205] Trace[1752978979]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/service-accounts,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:3653aa52-540f-404b-a047-b2c368132e37,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:31.246) (total time: 3070ms):
Trace[1752978979]: [3.070545269s] [3.070545269s] END
I0126 10:33:34.317121 96588 trace.go:205] Trace[2124898625]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:70b12f65-0b96-4fe8-aae1-cd58f1768536,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:31.247) (total time: 3069ms):
Trace[2124898625]: [3.069483114s] [3.069483114s] END
I0126 10:33:34.317216 96588 trace.go:205] Trace[245110695]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:7f8191dd-9bae-45f0-b208-edb326daf996,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:31.247) (total time: 3069ms):
Trace[245110695]: [3.069140378s] [3.069140378s] END
{"level":"warn","ts":"2022-01-26T10:33:34.576-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"146.147812ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461509 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:discovery\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:discovery\" value_size:578 >> failure:<>>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:34.577-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"257.530151ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/global-default\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.868-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"436.300487ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.868-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.431-0500","time spent":"436.398079ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:34.868-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"436.467026ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:34.868-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.431-0500","time spent":"436.507383ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.369-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461515,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"951.54639ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.870-0500","time spent":"951.605439ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.240738025s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7347"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.242326046s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3488"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.241501535s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7347"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.581-0500","time spent":"1.240789606s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":7370,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.579-0500","time spent":"1.242370019s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":3511,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.580-0500","time spent":"1.241546591s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":7370,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"841.080103ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"841.912446ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.981-0500","time spent":"841.118041ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.980-0500","time spent":"841.952187ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.241998281s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7529"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.580-0500","time spent":"1.242303516s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":7552,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.242731162s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:monitoring\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:35.822-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:34.579-0500","time spent":"1.242779318s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":74,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:monitoring\" "}
I0126 10:33:35.822825 96588 trace.go:205] Trace[1200570570]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/system,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:89bad7a7-79d7-47f9-93d7-cf5ab08ddf09,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:34.870) (total time: 952ms):
Trace[1200570570]: [952.733873ms] [952.733873ms] END
I0126 10:33:35.823240 96588 trace.go:205] Trace[1713323349]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:8c69c29a-9a1e-4e29-bd2b-8c705560e7f4,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:34.579) (total time: 1243ms):
Trace[1713323349]: ---"About to write a response" 1243ms (10:33:35.823)
Trace[1713323349]: [1.243909816s] [1.243909816s] END
I0126 10:33:35.823387 96588 trace.go:205] Trace[1867658632]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:monitoring,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:7fff81c6-cb17-4f89-a0b5-ab3e56eab883,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:34.578) (total time: 1244ms):
Trace[1867658632]: [1.244372176s] [1.244372176s] END
I0126 10:33:35.823592 96588 trace.go:205] Trace[2054308085]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:3245f8a5-2656-435a-b303-13af86f40987,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:34.580) (total time: 1243ms):
Trace[2054308085]: ---"About to write a response" 1242ms (10:33:35.823)
Trace[2054308085]: [1.243069463s] [1.243069463s] END
I0126 10:33:35.824132 96588 trace.go:205] Trace[1631636854]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:c701bf52-8708-4a7a-88d7-7c6d1e0af33c,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:34.579) (total time: 1244ms):
Trace[1631636854]: ---"About to write a response" 1243ms (10:33:35.823)
Trace[1631636854]: [1.244243399s] [1.244243399s] END
{"level":"warn","ts":"2022-01-26T10:33:36.196-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"116.900868ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461519 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" mod_revision:22 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" value_size:8190 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" > >>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:36.196-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"369.678335ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:basic-user\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:36.196-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.825-0500","time spent":"370.421786ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":710,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" value_size:623 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:36.196-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.825-0500","time spent":"370.587966ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8302,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" mod_revision:22 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" value_size:8190 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:36.196-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.826-0500","time spent":"369.713911ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":74,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:basic-user\" "}
I0126 10:33:36.197250 96588 trace.go:205] Trace[522779536]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:34.581) (total time: 1615ms):
Trace[522779536]: ---"initial value restored" 1242ms (10:33:35.823)
Trace[522779536]: ---"Transaction committed" 371ms (10:33:36.196)
Trace[522779536]: [1.615998347s] [1.615998347s] END
I0126 10:33:36.197533 96588 trace.go:205] Trace[320290492]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:f1737fb0-3a93-4045-a135-ba7c5a6fd597,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:34.580) (total time: 1616ms):
Trace[320290492]: ---"Object stored in database" 1616ms (10:33:36.197)
Trace[320290492]: [1.61662173s] [1.61662173s] END
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"539.272712ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"539.152176ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"547.305769ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8314"}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.933-0500","time spent":"539.347316ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"547.246458ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7529"}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.933-0500","time spent":"539.250821ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"547.931685ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3488"}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.925-0500","time spent":"547.349618ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":7552,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.925-0500","time spent":"547.393078ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":8337,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:36.472-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:35.924-0500","time spent":"548.030198ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":3511,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
I0126 10:33:36.473873 96588 trace.go:205] Trace[1960623078]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:405a5ad9-8157-4c85-8b84-2295cadb630a,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:35.924) (total time: 549ms):
Trace[1960623078]: ---"About to write a response" 549ms (10:33:36.473)
Trace[1960623078]: [549.674649ms] [549.674649ms] END
I0126 10:33:36.474207 96588 trace.go:205] Trace[877853708]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:35a3e006-aed3-431f-80da-c3bfe93bf1d8,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:35.924) (total time: 549ms):
Trace[877853708]: ---"About to write a response" 548ms (10:33:36.473)
Trace[877853708]: [549.384129ms] [549.384129ms] END
I0126 10:33:36.474216 96588 trace.go:205] Trace[1007670217]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:8da05cf0-8524-442a-80df-fc7bf308070f,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:35.924) (total time: 549ms):
Trace[1007670217]: ---"About to write a response" 548ms (10:33:36.473)
Trace[1007670217]: [549.323859ms] [549.323859ms] END
{"level":"warn","ts":"2022-01-26T10:33:36.972-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461523,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:37.472-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461523,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:37.973-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461523,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:38.473-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461523,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:38.589-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000697488s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:38.589-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000832494s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:38.589-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.588-0500","time spent":"2.000896322s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.589-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.588-0500","time spent":"2.000803777s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.709036567s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7529"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"216.229009ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"216.736486ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.710021616s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/node-high\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.198-0500","time spent":"2.709111374s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":7552,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.197-0500","time spent":"2.71012191s","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":84,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/node-high\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.383115189s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:7529"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.383164458s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8314"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.709132305s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8314"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.524-0500","time spent":"2.383256911s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":8337,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.525-0500","time spent":"2.383157516s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":7552,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.198-0500","time spent":"2.709316029s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":8337,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.433552259s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:public-info-viewer\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.474-0500","time spent":"2.433829844s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:public-info-viewer\" "}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.383789346s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3488"}
{"level":"warn","ts":"2022-01-26T10:33:38.908-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:36.524-0500","time spent":"2.384063378s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":3511,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
I0126 10:33:38.908600 96588 trace.go:205] Trace[1464301063]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/node-high,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:b779851c-6646-4d5a-9776-0bf627ff6ced,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.197) (total time: 2710ms):
Trace[1464301063]: [2.710731914s] [2.710731914s] END
I0126 10:33:38.908825 96588 trace.go:205] Trace[331507708]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:public-info-viewer,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:22568371-9de0-4386-a292-2a282b6dfd3a,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.474) (total time: 2434ms):
Trace[331507708]: [2.434482372s] [2.434482372s] END
I0126 10:33:38.909429 96588 trace.go:205] Trace[239907654]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:03d46b14-1c99-4c20-b159-c8fb966b6801,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.524) (total time: 2385ms):
Trace[239907654]: ---"About to write a response" 2385ms (10:33:38.909)
Trace[239907654]: [2.385228349s] [2.385228349s] END
I0126 10:33:38.909746 96588 trace.go:205] Trace[835244016]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:af0b39a9-17f0-439c-8f5d-9b9645b8b200,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.524) (total time: 2384ms):
Trace[835244016]: ---"About to write a response" 2384ms (10:33:38.909)
Trace[835244016]: [2.384774875s] [2.384774875s] END
I0126 10:33:38.909802 96588 trace.go:205] Trace[832918113]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:b325b96b-80b6-4e6a-b8b6-3b0ae40f0925,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.524) (total time: 2384ms):
Trace[832918113]: ---"About to write a response" 2384ms (10:33:38.909)
Trace[832918113]: [2.384958562s] [2.384958562s] END
{"level":"warn","ts":"2022-01-26T10:33:39.524-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"240.456724ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461527 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" mod_revision:23 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" value_size:8360 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" > >>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:39.524-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.911-0500","time spent":"613.02849ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8311,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" mod_revision:27 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" value_size:8199 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:39.524-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.911-0500","time spent":"613.262469ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8468,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" mod_revision:23 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" value_size:8360 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:39.524-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"613.385617ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/admin\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:39.524-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.911-0500","time spent":"613.43551ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":62,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/admin\" "}
{"level":"warn","ts":"2022-01-26T10:33:39.524-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.911-0500","time spent":"613.131531ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":716,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/node-high\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/node-high\" value_size:626 >> failure:<>"}
I0126 10:33:39.525898 96588 trace.go:205] Trace[671994541]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/admin,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:61a6f216-67c3-4bb3-8c7b-d0fc361cadc3,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:38.911) (total time: 614ms):
Trace[671994541]: [614.756796ms] [614.756796ms] END
I0126 10:33:39.526103 96588 trace.go:205] Trace[479186890]: "Create" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:e4c41499-a744-4933-8a2e-a5cc8ac31f3f,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:38.911) (total time: 614ms):
Trace[479186890]: ---"Object stored in database" 614ms (10:33:39.525)
Trace[479186890]: [614.935317ms] [614.935317ms] END
I0126 10:33:39.526134 96588 trace.go:205] Trace[203481991]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:36.198) (total time: 3327ms):
Trace[203481991]: ---"initial value restored" 2710ms (10:33:38.909)
Trace[203481991]: ---"Transaction committed" 614ms (10:33:39.525)
Trace[203481991]: [3.32727986s] [3.32727986s] END
I0126 10:33:39.526273 96588 trace.go:205] Trace[1951211968]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:36.198) (total time: 3327ms):
Trace[1951211968]: ---"initial value restored" 2710ms (10:33:38.909)
Trace[1951211968]: ---"Transaction committed" 614ms (10:33:39.525)
Trace[1951211968]: [3.327329223s] [3.327329223s] END
I0126 10:33:39.526467 96588 trace.go:205] Trace[1771345528]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:48bd3d3a-b5ae-4a48-a0ba-3e9ca604ac46,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.198) (total time: 3327ms):
Trace[1771345528]: ---"Object stored in database" 3327ms (10:33:39.526)
Trace[1771345528]: [3.327931197s] [3.327931197s] END
I0126 10:33:39.526774 96588 trace.go:205] Trace[275720234]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:ecd9ccf9-953a-4fd3-a6a7-428106dc3388,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:36.198) (total time: 3328ms):
Trace[275720234]: ---"Object stored in database" 3327ms (10:33:39.526)
Trace[275720234]: [3.328177544s] [3.328177544s] END
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.084424913s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3488"}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.924-0500","time spent":"1.084514482s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":3511,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.084166623s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8323"}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.08418638s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8480"}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.925-0500","time spent":"1.08423837s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":106,"response count":1,"response size":8346,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/negotiatedapiresources.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"985.51706ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"985.481769ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:39.023-0500","time spent":"985.86057ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:39.023-0500","time spent":"985.900671ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:40.009-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:38.925-0500","time spent":"1.084331359s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":8503,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:40.010-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"353.065071ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461531 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/admin\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/admin\" value_size:562 >> failure:<>>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:40.010-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:39.528-0500","time spent":"482.141242ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":630,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/admin\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/admin\" value_size:562 >> failure:<>"}
I0126 10:33:40.011084 96588 trace.go:205] Trace[1916555789]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:1412f213-a326-4bb6-905d-dcf548247ddb,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:38.924) (total time: 1086ms):
Trace[1916555789]: ---"About to write a response" 1086ms (10:33:40.010)
Trace[1916555789]: [1.086572688s] [1.086572688s] END
I0126 10:33:40.011576 96588 trace.go:205] Trace[1228112955]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:06ab3c11-433c-4460-9dea-111ef80129d2,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:38.924) (total time: 1086ms):
Trace[1228112955]: ---"About to write a response" 1086ms (10:33:40.011)
Trace[1228112955]: [1.086557485s] [1.086557485s] END
I0126 10:33:40.011621 96588 trace.go:205] Trace[1046052379]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/negotiatedapiresources.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:c783d893-eb7c-46cc-a8d7-6f3362dce674,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:38.924) (total time: 1086ms):
Trace[1046052379]: ---"About to write a response" 1086ms (10:33:40.011)
Trace[1046052379]: [1.086636698s] [1.086636698s] END
I0126 10:33:40.012240 96588 bootstrap.go:78] bootstrapped negotiatedapiresources.apiresource.kcp.dev after 8.765126511s
{"level":"warn","ts":"2022-01-26T10:33:40.157-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:40.657-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:41.158-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:41.659-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:42.112-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.999967114s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:42.112-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:40.112-0500","time spent":"2.000018679s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
W0126 10:33:42.112682 96588 server.go:1299] [core] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
{"level":"warn","ts":"2022-01-26T10:33:42.112-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.999984911s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context canceled"}
{"level":"warn","ts":"2022-01-26T10:33:42.112-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:40.112-0500","time spent":"2.000023696s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
W0126 10:33:42.112823 96588 server.go:1299] [core] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
{"level":"warn","ts":"2022-01-26T10:33:42.160-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:42.660-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:43.160-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:43.661-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:44.162-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:44.219-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000628579s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:44.219-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:42.218-0500","time spent":"2.00074089s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:44.219-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000699969s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:44.219-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:42.218-0500","time spent":"2.000797097s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:44.662-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:45.162-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:45.662-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:46.163-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461532,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:46.345-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000619469s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:46.345-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:44.345-0500","time spent":"2.000741158s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:46.345-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000714985s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:46.345-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:44.345-0500","time spent":"2.000790875s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"etcdserver/v3_server.go:827","msg":"timed out waiting for read index response (local node might have slow network)","timeout":"7s"}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"7.12835756s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"","error":"etcdserver: request timed out"}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"7.129518765s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/leader-election\" ","response":"","error":"etcdserver: request timed out"}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"7.129060425s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"","error":"etcdserver: request timed out"}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:39.528-0500","time spent":"7.128427999s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":0,"response size":0,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:39.527-0500","time spent":"7.129587449s","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":90,"response count":0,"response size":0,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/leader-election\" "}
{"level":"warn","ts":"2022-01-26T10:33:46.657-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:39.528-0500","time spent":"7.129130084s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":0,"response size":0,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
I0126 10:33:46.657838 96588 trace.go:205] Trace[586004889]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:39.527) (total time: 7129ms):
Trace[586004889]: [7.129663384s] [7.129663384s] END
E0126 10:33:46.657878 96588 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out
E0126 10:33:46.657882 96588 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out
I0126 10:33:46.657904 96588 trace.go:205] Trace[1381813657]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:39.528) (total time: 7128ms):
Trace[1381813657]: [7.128993765s] [7.128993765s] END
E0126 10:33:46.657933 96588 status.go:71] apiserver received an error that is not an metav1.Status: rpctypes.EtcdError{code:0xe, desc:"etcdserver: request timed out"}: etcdserver: request timed out
I0126 10:33:46.658151 96588 trace.go:205] Trace[182556415]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/leader-election,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:87a7b695-4398-4e3d-a8d0-59aee665f6f1,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:39.527) (total time: 7130ms):
Trace[182556415]: [7.130476112s] [7.130476112s] END
E0126 10:33:46.658509 96588 storage_flowcontrol.go:120] "APF bootstrap ensurer ran into error, will retry later" err="failed ensuring suggested settings - failed to retrieve PriorityLevelConfiguration type=suggested name=\"leader-election\" error=etcdserver: request timed out"
I0126 10:33:46.659132 96588 trace.go:205] Trace[2100345544]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:03739bf5-6185-4ae3-bcd4-23332c288ba7,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:39.527) (total time: 7131ms):
Trace[2100345544]: [7.131204173s] [7.131204173s] END
I0126 10:33:46.660275 96588 trace.go:205] Trace[317090039]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:9adad44d-8591-43a1-8950-67c471753994,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:39.528) (total time: 7131ms):
Trace[317090039]: [7.131879962s] [7.131879962s] END
E0126 10:33:46.660478 96588 establishing_controller.go:110] user#$#apiresourceimports.apiresource.kcp.dev failed with: etcdserver: request timed out
E0126 10:33:46.662329 96588 naming_controller.go:328] user#$#clusters.cluster.example.dev failed with: etcdserver: request timed out
{"level":"warn","ts":"2022-01-26T10:33:47.157-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461533,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:47.658-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461533,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:48.159-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461533,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.73814168s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.518682698s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8480"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"8.174108556s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/edit\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:46.668-0500","time spent":"1.518756321s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":8503,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:40.012-0500","time spent":"8.174189641s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/edit\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"8.161377786s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8480"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.737709353s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:46.449-0500","time spent":"1.737764979s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:40.025-0500","time spent":"8.161443903s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":8503,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.187-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"8.161905753s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3488"}
{"level":"warn","ts":"2022-01-26T10:33:48.187-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"525.901417ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-nodes\" ","response":"range_response_count:1 size:696"}
{"level":"warn","ts":"2022-01-26T10:33:48.186-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:46.448-0500","time spent":"1.738213631s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.187-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:40.024-0500","time spent":"8.161949538s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":3511,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.187-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:47.661-0500","time spent":"525.950313ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":71,"response count":1,"response size":719,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-nodes\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.187-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.51886043s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:3488"}
{"level":"warn","ts":"2022-01-26T10:33:48.187-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:46.668-0500","time spent":"1.518898225s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":3511,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
I0126 10:33:48.187269 96588 trace.go:205] Trace[1111665254]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/edit,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:cb0d5e1f-e4e5-4142-9252-70ea64b5388b,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:40.012) (total time: 8174ms):
Trace[1111665254]: [8.174842785s] [8.174842785s] END
I0126 10:33:48.187507 96588 trace.go:205] Trace[270096289]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/system-nodes,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:cdb1311e-72a7-400d-b393-5ffc84223374,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:47.660) (total time: 526ms):
Trace[270096289]: ---"About to write a response" 526ms (10:33:48.187)
Trace[270096289]: [526.803501ms] [526.803501ms] END
I0126 10:33:48.189365 96588 trace.go:205] Trace[1642932816]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:80a40e17-594d-4175-8f96-0f0e1949448a,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:40.024) (total time: 8164ms):
Trace[1642932816]: ---"About to write a response" 8164ms (10:33:48.188)
Trace[1642932816]: [8.164529715s] [8.164529715s] END
I0126 10:33:48.189747 96588 trace.go:205] Trace[1817634496]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:5f59985b-282f-46c0-881f-ea24c10c59c3,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:40.025) (total time: 8164ms):
Trace[1817634496]: ---"About to write a response" 8162ms (10:33:48.187)
Trace[1817634496]: [8.164413812s] [8.164413812s] END
{"level":"warn","ts":"2022-01-26T10:33:48.495-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"303.46376ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/probes\" ","response":"range_response_count:1 size:681"}
{"level":"warn","ts":"2022-01-26T10:33:48.495-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.191-0500","time spent":"303.552676ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":65,"response count":1,"response size":704,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/probes\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.495-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.191-0500","time spent":"303.276888ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4387,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" mod_revision:24 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" value_size:4289 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:48.495-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.193-0500","time spent":"302.163632ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":8477,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" mod_revision:31 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" value_size:8369 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" > >"}
I0126 10:33:48.496887 96588 trace.go:205] Trace[1781027774]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:46.667) (total time: 1828ms):
Trace[1781027774]: ---"initial value restored" 1521ms (10:33:48.189)
Trace[1781027774]: ---"Transaction committed" 303ms (10:33:48.496)
Trace[1781027774]: [1.828952189s] [1.828952189s] END
I0126 10:33:48.497268 96588 trace.go:205] Trace[544292268]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:a9c40574-3fdc-47de-97d6-1610998c48ce,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:46.667) (total time: 1829ms):
Trace[544292268]: ---"Object stored in database" 1829ms (10:33:48.496)
Trace[544292268]: [1.829723521s] [1.829723521s] END
I0126 10:33:48.497313 96588 trace.go:205] Trace[1758733207]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:46.668) (total time: 1829ms):
Trace[1758733207]: ---"initial value restored" 1520ms (10:33:48.189)
Trace[1758733207]: ---"Transaction committed" 305ms (10:33:48.497)
Trace[1758733207]: [1.829159878s] [1.829159878s] END
I0126 10:33:48.497570 96588 trace.go:205] Trace[597460255]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:55d1518a-1ec3-491a-bab2-36713493b412,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:46.667) (total time: 1829ms):
Trace[597460255]: ---"Object stored in database" 1829ms (10:33:48.497)
Trace[597460255]: [1.829625852s] [1.829625852s] END
{"level":"warn","ts":"2022-01-26T10:33:48.804-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"305.22541ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:4399"}
{"level":"warn","ts":"2022-01-26T10:33:48.804-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.499-0500","time spent":"305.301113ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":4422,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.804-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"515.217297ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:48.804-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.289-0500","time spent":"515.271838ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.804-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"515.728548ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.289-0500","time spent":"515.836719ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"579.476179ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" ","response":"range_response_count:1 size:8489"}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.225-0500","time spent":"579.519438ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":1,"response size":8512,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/apiresourceimports.apiresource.kcp.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"612.713446ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/view\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.192-0500","time spent":"612.787001ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":61,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/view\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"579.676127ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:4399"}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.225-0500","time spent":"579.808391ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":4422,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"307.948844ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-leader-election\" ","response":"range_response_count:1 size:908"}
{"level":"warn","ts":"2022-01-26T10:33:48.805-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.497-0500","time spent":"308.014298ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":1,"response size":931,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-leader-election\" "}
I0126 10:33:48.805744 96588 trace.go:205] Trace[1877522848]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/view,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:7857882a-83bd-491e-a034-d86e63dde89e,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.191) (total time: 613ms):
Trace[1877522848]: [613.840091ms] [613.840091ms] END
I0126 10:33:48.806392 96588 trace.go:205] Trace[323517896]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:8ac846ac-4d28-4fb6-8e8b-6bc0e00194d2,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.225) (total time: 581ms):
Trace[323517896]: ---"About to write a response" 580ms (10:33:48.806)
Trace[323517896]: [581.210181ms] [581.210181ms] END
I0126 10:33:48.807254 96588 trace.go:205] Trace[1642213596]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/apiresourceimports.apiresource.kcp.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:8bfc9b11-c654-4973-a719-548d54c7c80b,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.225) (total time: 581ms):
Trace[1642213596]: ---"About to write a response" 581ms (10:33:48.806)
Trace[1642213596]: [581.791512ms] [581.791512ms] END
I0126 10:33:48.807794 96588 bootstrap.go:78] bootstrapped apiresourceimports.apiresource.kcp.dev after 17.560439217s
{"level":"warn","ts":"2022-01-26T10:33:49.309-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461543,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:49.650-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"841.646585ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-controller-manager\" ","response":"range_response_count:1 size:747"}
{"level":"warn","ts":"2022-01-26T10:33:49.650-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.807-0500","time spent":"842.756888ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":4396,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" mod_revision:36 > success:<request_put:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" value_size:4298 >> failure:<request_range:<key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" > >"}
{"level":"warn","ts":"2022-01-26T10:33:49.650-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.809-0500","time spent":"841.841561ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":1,"response size":770,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-controller-manager\" "}
I0126 10:33:49.652306 96588 trace.go:205] Trace[1877484028]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/kube-controller-manager,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:6bbb3388-4ed1-4c37-a56e-6ef4fd84916d,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.808) (total time: 843ms):
Trace[1877484028]: ---"About to write a response" 843ms (10:33:49.652)
Trace[1877484028]: [843.874292ms] [843.874292ms] END
I0126 10:33:49.652990 96588 trace.go:205] Trace[1940273362]: "GuaranteedUpdate etcd3" type:*apiextensions.CustomResourceDefinition (26-Jan-2022 10:33:48.499) (total time: 1153ms):
Trace[1940273362]: ---"initial value restored" 306ms (10:33:48.805)
Trace[1940273362]: ---"Transaction committed" 844ms (10:33:49.652)
Trace[1940273362]: [1.153496503s] [1.153496503s] END
I0126 10:33:49.653483 96588 trace.go:205] Trace[894654115]: "Update" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev/status,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:2cb209d4-f01c-4a47-80df-d82a944cce55,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.499) (total time: 1154ms):
Trace[894654115]: ---"Object stored in database" 1153ms (10:33:49.653)
Trace[894654115]: [1.154199582s] [1.154199582s] END
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"494.080672ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-scheduler\" ","response":"range_response_count:1 size:720"}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.323614587s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" ","response":"range_response_count:1 size:4408"}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:49.654-0500","time spent":"494.176756ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":73,"response count":1,"response size":743,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-scheduler\" "}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.824-0500","time spent":"1.323669122s","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":92,"response count":1,"response size":4431,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/user/clusters.cluster.example.dev\" "}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.227828909s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.338462823s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-admin\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.920-0500","time spent":"1.227924849s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.810-0500","time spent":"1.338505036s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":82,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-admin\" "}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.227981049s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.148-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:48.920-0500","time spent":"1.228021693s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:33:50.149995 96588 trace.go:205] Trace[291424097]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:aggregate-to-admin,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:767a7c5a-b69e-4985-82b9-4c86467321a3,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.809) (total time: 1340ms):
Trace[291424097]: [1.340425365s] [1.340425365s] END
I0126 10:33:50.150183 96588 trace.go:205] Trace[663635187]: "Get" url:/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.cluster.example.dev,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:bbb47955-4b0b-4bd8-8815-e5804d25e962,client:192.168.4.118,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:48.824) (total time: 1325ms):
Trace[663635187]: ---"About to write a response" 1325ms (10:33:50.149)
Trace[663635187]: [1.325484281s] [1.325484281s] END
I0126 10:33:50.150546 96588 bootstrap.go:78] bootstrapped clusters.cluster.example.dev after 18.90315689s
I0126 10:33:50.157125 96588 controller.go:301] Starting APIResource controller
I0126 10:33:50.157134 96588 shared_informer.go:240] Waiting for caches to sync for apiresource
I0126 10:33:50.157144 96588 controller.go:219] Starting Cluster controller
I0126 10:33:50.157152 96588 shared_informer.go:240] Waiting for caches to sync for cluster
{"level":"warn","ts":"2022-01-26T10:33:50.496-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"235.712648ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.496-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"344.415738ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/service-accounts\" ","response":"range_response_count:1 size:720"}
{"level":"warn","ts":"2022-01-26T10:33:50.496-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"289.264753ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiresource.kcp.dev/negotiatedapiresources/\" range_end:\"/registry/apiresource.kcp.dev/negotiatedapiresources0\" count_only:true ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.496-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.152-0500","time spent":"344.47392ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":75,"response count":1,"response size":743,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/service-accounts\" "}
{"level":"warn","ts":"2022-01-26T10:33:50.496-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"288.932154ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiresource.kcp.dev/negotiatedapiresources/admin/\" range_end:\"/registry/apiresource.kcp.dev/negotiatedapiresources/admin0\" limit:500 ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"247.039489ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/cluster.example.dev/clusters/admin/\" range_end:\"/registry/cluster.example.dev/clusters/admin0\" limit:500 ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"335.703833ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin0\" limit:500 ","response":"range_response_count:3 size:21211"}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"203.874876ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiresource.kcp.dev/apiresourceimports/admin/\" range_end:\"/registry/apiresource.kcp.dev/apiresourceimports/admin0\" limit:500 ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"247.523919ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/cluster.example.dev/clusters/\" range_end:\"/registry/cluster.example.dev/clusters0\" count_only:true ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.161-0500","time spent":"335.795794ms","remote":"127.0.0.1:63902","response type":"/etcdserverpb.KV/Range","request count":0,"request size":133,"response count":3,"response size":21234,"request content":"key:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin/\" range_end:\"/registry/apiextensions.k8s.io/customresourcedefinitions/admin0\" limit:500 "}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"204.614204ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/apiresource.kcp.dev/apiresourceimports/\" range_end:\"/registry/apiresource.kcp.dev/apiresourceimports0\" count_only:true ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.497-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"236.51877ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.496-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"343.483084ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-edit\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:50.498-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.153-0500","time spent":"344.836285ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-edit\" "}
I0126 10:33:50.557321 96588 shared_informer.go:247] Caches are synced for apiresource
I0126 10:33:50.557435 96588 shared_informer.go:247] Caches are synced for cluster
{"level":"warn","ts":"2022-01-26T10:33:51.001-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461553,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:51.501-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461553,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:52.002-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461553,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.417994766s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.528188934s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" ","response":"range_response_count:1 size:722"}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.611-0500","time spent":"1.418061451s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.417968093s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.500-0500","time spent":"1.528273087s","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":1,"response size":745,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.527095084s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-view\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.611-0500","time spent":"1.418080918s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.029-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:50.501-0500","time spent":"1.527252072s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:aggregate-to-view\" "}
I0126 10:33:52.029644 96588 trace.go:205] Trace[98133585]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/system,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:45b4d2fb-58c8-40cd-a11e-97fc3c83bfca,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:50.500) (total time: 1529ms):
Trace[98133585]: ---"About to write a response" 1528ms (10:33:52.029)
Trace[98133585]: [1.529015288s] [1.529015288s] END
I0126 10:33:52.029682 96588 trace.go:205] Trace[423580178]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:aggregate-to-view,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:9d39ee4f-3324-4a34-ac4d-5ca4a7a23869,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:50.501) (total time: 1527ms):
Trace[423580178]: [1.527821663s] [1.527821663s] END
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"389.076878ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461560 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/leader-election\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/leader-election\" value_size:632 >> failure:<>>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.033-0500","time spent":"459.41253ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":728,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/leader-election\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/leader-election\" value_size:632 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"361.004227ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"459.256639ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:heapster\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"360.381162ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.131-0500","time spent":"361.053632ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.033-0500","time spent":"459.308521ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":72,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:heapster\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.492-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.132-0500","time spent":"360.431379ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"254.770755ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"367.825698ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"254.796311ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"368.654087ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-high\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:52.863-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.495-0500","time spent":"367.876981ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":68,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node\" "}
{"level":"warn","ts":"2022-01-26T10:33:52.863-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.494-0500","time spent":"368.741257ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":88,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-high\" "}
{"level":"warn","ts":"2022-01-26T10:33:53.175-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.865-0500","time spent":"309.808239ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":1496,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node\" value_size:1422 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:53.175-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"308.783144ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-low\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:53.175-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.866-0500","time spent":"308.823703ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":87,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-low\" "}
{"level":"warn","ts":"2022-01-26T10:33:53.471-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"491.742407ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:53.471-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"491.792921ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:53.471-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.979-0500","time spent":"491.808691ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:53.471-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:52.979-0500","time spent":"491.870235ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:53.471-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"160.966465ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461569 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-low\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-low\" value_size:630 >> failure:<>>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:53.811-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461570,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:54.311-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461570,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:54.811-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461570,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:55.311-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461570,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:55.586-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000429827s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:55.586-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:53.585-0500","time spent":"2.000609428s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:55.586-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000547107s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:55.586-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:53.585-0500","time spent":"2.000773919s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:55.627-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.154550541s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/global-default\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:55.627-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:53.472-0500","time spent":"2.15467418s","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":89,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/global-default\" "}
{"level":"warn","ts":"2022-01-26T10:33:55.627-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.450860179s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-problem-detector\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:55.627-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:53.176-0500","time spent":"2.450937107s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":85,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-problem-detector\" "}
I0126 10:33:55.629765 96588 trace.go:205] Trace[709349490]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:node-problem-detector,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:cedc1f63-1f53-40e1-9cb9-270da9c9188c,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:53.176) (total time: 2453ms):
Trace[709349490]: [2.453099339s] [2.453099339s] END
I0126 10:33:55.629785 96588 trace.go:205] Trace[760707349]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/global-default,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:4f880c1a-047f-4a8a-9ef0-1fe23e6905c9,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:53.472) (total time: 2157ms):
Trace[760707349]: [2.157222859s] [2.157222859s] END
{"level":"warn","ts":"2022-01-26T10:33:56.405-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"591.721836ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461574 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-problem-detector\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-problem-detector\" value_size:574 >> failure:<>>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:56.405-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:55.632-0500","time spent":"773.127539ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":665,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-problem-detector\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-problem-detector\" value_size:574 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:56.406-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"702.664291ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.406-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:55.703-0500","time spent":"702.74494ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:56.406-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"773.827126ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/exempt\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.406-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"702.900335ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.406-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:55.632-0500","time spent":"773.894114ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":65,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/exempt\" "}
{"level":"warn","ts":"2022-01-26T10:33:56.406-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:55.703-0500","time spent":"702.992582ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:33:56.406761 96588 trace.go:205] Trace[808614183]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/exempt,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:001a536a-64fe-4f5f-af85-c9bc53f2e12f,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:55.632) (total time: 774ms):
Trace[808614183]: [774.440508ms] [774.440508ms] END
I0126 10:33:56.406775 96588 trace.go:205] Trace[253039537]: "Create" url:/apis/rbac.authorization.k8s.io/v1/clusterroles,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:993133f1-fd3c-4f01-93ee-635453ad782e,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:55.631) (total time: 774ms):
Trace[253039537]: ---"Object stored in database" 774ms (10:33:56.406)
Trace[253039537]: [774.773833ms] [774.773833ms] END
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"454.751808ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kubelet-api-admin\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"338.78427ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"338.821363ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:56.524-0500","time spent":"338.840309ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:56.408-0500","time spent":"454.806578ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kubelet-api-admin\" "}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:56.524-0500","time spent":"338.874022ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"453.637613ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/catch-all\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:56.863-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:56.409-0500","time spent":"453.735856ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":68,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/catch-all\" "}
{"level":"warn","ts":"2022-01-26T10:33:57.103-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"144.726476ms","expected-duration":"100ms","prefix":"","request":"header:<ID:4097570423066461581 auth_revision:1 > txn:<compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kubelet-api-admin\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kubelet-api-admin\" value_size:597 >> failure:<>>","response":"size:14"}
{"level":"warn","ts":"2022-01-26T10:33:57.103-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"124.606566ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:57.103-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"237.977358ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/catch-all\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:57.103-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"124.484642ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:57.607-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461585,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:57.626-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:57.107-0500","time spent":"519.570628ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":643,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-bootstrapper\" mod_revision:0 > success:<request_put:<key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-bootstrapper\" value_size:556 >> failure:<>"}
{"level":"warn","ts":"2022-01-26T10:33:57.626-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"519.702201ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/exempt\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:33:57.627-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:57.107-0500","time spent":"519.814999ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":0,"response size":27,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/exempt\" "}
I0126 10:33:57.627873 96588 trace.go:205] Trace[2029505859]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/exempt,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:9788d97a-38d5-4977-b4ee-b5fce3f9d8cb,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:57.107) (total time: 520ms):
Trace[2029505859]: [520.556525ms] [520.556525ms] END
I0126 10:33:57.627930 96588 trace.go:205] Trace[969587000]: "Create" url:/apis/rbac.authorization.k8s.io/v1/clusterroles,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:fac338c8-41a5-46f6-931d-448ef07d4429,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:57.106) (total time: 520ms):
Trace[969587000]: ---"Object stored in database" 520ms (10:33:57.627)
Trace[969587000]: [520.961821ms] [520.961821ms] END
{"level":"warn","ts":"2022-01-26T10:33:58.127-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:58.627-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:59.127-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:33:59.222-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.00038927s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:59.222-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:57.221-0500","time spent":"2.000545528s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:33:59.222-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000691771s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:33:59.222-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:57.221-0500","time spent":"2.000825654s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
W0126 10:33:59.222753 96588 server.go:1299] [core] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
{"level":"warn","ts":"2022-01-26T10:33:59.628-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:00.128-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:00.628-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:01.129-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:01.325-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000258044s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:34:01.325-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000253395s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:34:01.325-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:59.325-0500","time spent":"2.000328101s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:01.325-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:59.325-0500","time spent":"2.000346481s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:01.629-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461587,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:01.710-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:57.629-0500","time spent":"4.080125013s","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Txn","request count":1,"request size":519,"response count":0,"response size":37,"request content":"compare:<target:MOD key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/exempt\" mod_revision:0 > success:<request_put:<key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/exempt\" value_size:432 >> failure:<>"}
I0126 10:34:01.710707 96588 trace.go:205] Trace[1023717918]: "Create" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:6daa9acd-041f-4b33-898f-285ccfb1fe66,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:57.629) (total time: 4081ms):
Trace[1023717918]: ---"Object stored in database" 4081ms (10:34:01.710)
Trace[1023717918]: [4.081205128s] [4.081205128s] END
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"4.769530804s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:auth-delegator\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"965.633703ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:33:57.629-0500","time spent":"4.769731971s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":78,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:auth-delegator\" "}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"965.729999ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"685.872261ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/\" range_end:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin0\" ","response":"range_response_count:12 size:8857"}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:01.433-0500","time spent":"965.772242ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:01.433-0500","time spent":"965.86062ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:02.399-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:01.713-0500","time spent":"685.988246ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":118,"response count":12,"response size":8880,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/\" range_end:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin0\" "}
I0126 10:34:02.400665 96588 trace.go:205] Trace[418941843]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:auth-delegator,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:13eaaa07-f314-434c-b3eb-6e824da7254b,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:33:57.629) (total time: 4771ms):
Trace[418941843]: [4.771071843s] [4.771071843s] END
I0126 10:34:02.400860 96588 trace.go:205] Trace[958360366]: "List etcd3" key:/flowcontrol.apiserver.k8s.io/flowschemas/admin,resourceVersion:,resourceVersionMatch:,limit:0,continue: (26-Jan-2022 10:34:01.713) (total time: 687ms):
Trace[958360366]: [687.515108ms] [687.515108ms] END
I0126 10:34:02.401448 96588 trace.go:205] Trace[635480785]: "List" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:b555fcf1-a65e-481f-aaf7-3326bf37491f,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:01.713) (total time: 688ms):
Trace[635480785]: ---"Listing from storage done" 687ms (10:34:02.400)
Trace[635480785]: [688.094166ms] [688.094166ms] END
{"level":"warn","ts":"2022-01-26T10:34:02.676-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"159.956612ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:02.676-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"271.086771ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-aggregator\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:02.676-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"160.010769ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:02.676-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"270.702159ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/system-nodes\" ","response":"range_response_count:1 size:696"}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"460.697459ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-controller-manager\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"361.191168ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"361.214499ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"460.832502ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/probes\" ","response":"range_response_count:1 size:681"}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:02.780-0500","time spent":"361.252749ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:02.780-0500","time spent":"361.289788ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:02.680-0500","time spent":"460.778601ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":87,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-controller-manager\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.141-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:02.680-0500","time spent":"460.91001ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":65,"response count":1,"response size":704,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/probes\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"492.432498ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/workload-leader-election\" ","response":"range_response_count:1 size:828"}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"378.595087ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.144-0500","time spent":"492.504988ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":83,"response count":1,"response size":851,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/workload-leader-election\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"378.60419ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.258-0500","time spent":"378.693175ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"491.769591ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-dns\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.258-0500","time spent":"378.757399ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:03.636-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.144-0500","time spent":"491.990012ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":72,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-dns\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"472.279235ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"585.507948ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-scheduler\" ","response":"range_response_count:1 size:720"}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"585.165963ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:persistent-volume-provisioner\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.640-0500","time spent":"585.565648ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":73,"response count":1,"response size":743,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/kube-scheduler\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.753-0500","time spent":"472.344138ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"472.278957ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:04.225-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.640-0500","time spent":"585.274425ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":93,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:persistent-volume-provisioner\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.226-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:03.753-0500","time spent":"472.429451ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:34:04.226755 96588 trace.go:205] Trace[920221167]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:persistent-volume-provisioner,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:90ed3515-1d13-498e-a377-fa8273c9be1b,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:03.640) (total time: 586ms):
Trace[920221167]: [586.207551ms] [586.207551ms] END
I0126 10:34:04.226798 96588 trace.go:205] Trace[894389854]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/kube-scheduler,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:a283a313-e3ee-4734-8ba2-d906faedf9cc,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:03.640) (total time: 586ms):
Trace[894389854]: ---"About to write a response" 586ms (10:34:04.226)
Trace[894389854]: [586.629838ms] [586.629838ms] END
{"level":"warn","ts":"2022-01-26T10:34:04.728-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461610,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:04.784-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"443.433415ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.341-0500","time spent":"443.513254ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"555.919285ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:certificatesigningrequests:nodeclient\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.229-0500","time spent":"556.090257ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":121,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:certificatesigningrequests:nodeclient\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"556.717739ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/service-accounts\" ","response":"range_response_count:1 size:720"}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.228-0500","time spent":"557.007827ms","remote":"127.0.0.1:63968","response type":"/etcdserverpb.KV/Range","request count":0,"request size":75,"response count":1,"response size":743,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/service-accounts\" "}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"444.289434ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:04.785-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.341-0500","time spent":"444.342394ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:34:04.786196 96588 trace.go:205] Trace[1759405362]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:certificates.k8s.io:certificatesigningrequests:nodeclient,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:494db391-042b-4e62-bc76-f4d39abe28de,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:04.228) (total time: 557ms):
Trace[1759405362]: [557.305735ms] [557.305735ms] END
I0126 10:34:04.786210 96588 trace.go:205] Trace[1486326099]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/flowschemas/service-accounts,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:564fd746-7d08-40b5-b4f2-03d9a5a4e9e4,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:04.228) (total time: 557ms):
Trace[1486326099]: ---"About to write a response" 557ms (10:34:04.786)
Trace[1486326099]: [557.867526ms] [557.867526ms] END
{"level":"warn","ts":"2022-01-26T10:34:05.289-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461614,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:05.789-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461614,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:06.290-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461614,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:06.790-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461614,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:06.889-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.999984567s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:34:06.889-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.000244293s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"","error":"context deadline exceeded"}
{"level":"warn","ts":"2022-01-26T10:34:06.889-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.889-0500","time spent":"2.000272704s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:06.889-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.889-0500","time spent":"2.000357384s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":0,"request content":"key:\"/registry/health\" "}
W0126 10:34:06.889958 96588 server.go:1299] [core] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
W0126 10:34:06.890007 96588 server.go:1299] [core] grpc: Server.processUnaryRPC failed to write status: connection error: desc = "transport is closing"
{"level":"warn","ts":"2022-01-26T10:34:07.290-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461614,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.614026973s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"411.543239ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"412.540859ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.789-0500","time spent":"2.614095067s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":125,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient\" "}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:06.992-0500","time spent":"411.601632ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.61521683s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" ","response":"range_response_count:1 size:722"}
{"level":"warn","ts":"2022-01-26T10:34:07.403-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:06.991-0500","time spent":"412.594442ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:07.404-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:04.788-0500","time spent":"2.615356836s","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":1,"response size":745,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/system\" "}
I0126 10:34:07.404303 96588 trace.go:205] Trace[947971918]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:97a42b58-9d15-4225-823e-3a597e35b50a,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:04.789) (total time: 2614ms):
Trace[947971918]: [2.614606458s] [2.614606458s] END
I0126 10:34:07.404596 96588 trace.go:205] Trace[1969752889]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/system,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:87557b49-a635-4b4d-9fcf-900bc3f1ca56,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:04.788) (total time: 2616ms):
Trace[1969752889]: ---"About to write a response" 2616ms (10:34:07.404)
Trace[1969752889]: [2.616230074s] [2.616230074s] END
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"359.562927ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"457.901514ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-high\" ","response":"range_response_count:1 size:737"}
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"359.775531ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"457.283586ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:volume-scheduler\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.407-0500","time spent":"457.961137ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":88,"response count":1,"response size":760,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/workload-high\" "}
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.506-0500","time spent":"359.623802ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:07.865-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.506-0500","time spent":"359.818968ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:07.866-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.408-0500","time spent":"457.378775ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":80,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:volume-scheduler\" "}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"409.089759ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"512.449017ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:legacy-unknown-approver\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.972-0500","time spent":"409.238304ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.868-0500","time spent":"512.523276ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":107,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:legacy-unknown-approver\" "}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"409.710921ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.972-0500","time spent":"409.76399ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:08.381-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"513.418802ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/global-default\" ","response":"range_response_count:1 size:739"}
{"level":"warn","ts":"2022-01-26T10:34:08.382-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:07.868-0500","time spent":"513.661447ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":89,"response count":1,"response size":762,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/global-default\" "}
I0126 10:34:08.382346 96588 trace.go:205] Trace[368324386]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:certificates.k8s.io:legacy-unknown-approver,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:d57b891d-3541-4f7b-84e0-2eda62c56f12,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:07.868) (total time: 513ms):
Trace[368324386]: [513.607357ms] [513.607357ms] END
I0126 10:34:08.382625 96588 trace.go:205] Trace[651827070]: "Get" url:/apis/flowcontrol.apiserver.k8s.io/v1beta1/prioritylevelconfigurations/global-default,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:230bc4ca-3c41-40c6-a943-240491167356,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:07.868) (total time: 514ms):
Trace[651827070]: ---"About to write a response" 514ms (10:34:08.382)
Trace[651827070]: [514.332534ms] [514.332534ms] END
{"level":"warn","ts":"2022-01-26T10:34:08.658-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"160.56575ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:08.658-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"273.515523ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/flowschemas/admin/catch-all\" ","response":"range_response_count:1 size:736"}
{"level":"warn","ts":"2022-01-26T10:34:08.658-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"273.292609ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:kubelet-serving-approver\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:08.658-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"160.764754ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.023-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"360.541267ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:kube-apiserver-client-approver\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.023-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"252.786821ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.023-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:08.662-0500","time spent":"360.608486ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":114,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:kube-apiserver-client-approver\" "}
{"level":"warn","ts":"2022-01-26T10:34:09.023-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"360.775665ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/exempt\" ","response":"range_response_count:1 size:531"}
{"level":"warn","ts":"2022-01-26T10:34:09.023-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"252.745932ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.023-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:08.662-0500","time spent":"360.856689ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":81,"response count":1,"response size":554,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/exempt\" "}
{"level":"warn","ts":"2022-01-26T10:34:09.340-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"200.073763ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.340-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"312.795059ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/\" range_end:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin0\" ","response":"range_response_count:8 size:5536"}
{"level":"warn","ts":"2022-01-26T10:34:09.340-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"312.918081ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:kube-apiserver-client-kubelet-approver\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.340-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:09.027-0500","time spent":"312.869984ms","remote":"127.0.0.1:63978","response type":"/etcdserverpb.KV/Range","request count":0,"request size":150,"response count":8,"response size":5559,"request content":"key:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin/\" range_end:\"/registry/flowcontrol.apiserver.k8s.io/prioritylevelconfigurations/admin0\" "}
{"level":"warn","ts":"2022-01-26T10:34:09.340-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"200.184424ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.340-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:09.027-0500","time spent":"313.016066ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":122,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:certificates.k8s.io:kube-apiserver-client-kubelet-approver\" "}
{"level":"warn","ts":"2022-01-26T10:34:09.934-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"474.385617ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.934-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"589.367171ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:service-account-issuer-discovery\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.934-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"474.349352ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:09.934-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:09.344-0500","time spent":"589.457729ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":96,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:service-account-issuer-discovery\" "}
{"level":"warn","ts":"2022-01-26T10:34:09.934-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:09.459-0500","time spent":"474.48663ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:09.934-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:09.459-0500","time spent":"474.523704ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:34:09.935216 96588 trace.go:205] Trace[1792875269]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:service-account-issuer-discovery,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:0ac38e7f-13d4-4bc8-bc3f-feacf6f86fa2,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:09.344) (total time: 590ms):
Trace[1792875269]: [590.450012ms] [590.450012ms] END
{"level":"warn","ts":"2022-01-26T10:34:10.439-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461641,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:10.939-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461641,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:11.440-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461641,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:11.940-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461641,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:11.951-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.900565649s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:11.951-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.900454932s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:11.951-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:10.051-0500","time spent":"1.900629064s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:11.951-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:10.051-0500","time spent":"1.900512482s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:11.951-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"2.013343872s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-proxier\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:11.951-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:09.938-0500","time spent":"2.013415731s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":76,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:node-proxier\" "}
I0126 10:34:11.952342 96588 trace.go:205] Trace[229649477]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:node-proxier,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:0a855686-ddbf-4931-b385-fbfd2317fb43,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:09.938) (total time: 2014ms):
Trace[229649477]: [2.014104317s] [2.014104317s] END
{"level":"warn","ts":"2022-01-26T10:34:12.401-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"346.394199ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.401-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"446.211016ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-scheduler\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.401-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:12.055-0500","time spent":"346.571504ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:12.401-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:11.955-0500","time spent":"446.284303ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":78,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:kube-scheduler\" "}
{"level":"warn","ts":"2022-01-26T10:34:12.402-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"346.914369ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.402-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:12.055-0500","time spent":"347.010175ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:12.698-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"179.446795ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.699-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"179.459571ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.698-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"200.887932ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:clusterrole-aggregation-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.934-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"119.509989ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.934-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"119.517036ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:12.934-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"232.413232ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:cronjob-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:13.437-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461655,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:13.646-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"608.769287ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:13.646-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"709.144997ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:daemon-set-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:13.646-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:13.038-0500","time spent":"608.814688ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:13.646-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"608.780428ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:13.646-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:12.937-0500","time spent":"709.224346ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":96,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:daemon-set-controller\" "}
{"level":"warn","ts":"2022-01-26T10:34:13.646-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:13.038-0500","time spent":"608.899847ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:34:13.647262 96588 trace.go:205] Trace[824422720]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:controller:daemon-set-controller,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:296ba8af-1c63-4d79-b23c-0510ffbeba13,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:12.937) (total time: 709ms):
Trace[824422720]: [709.666039ms] [709.666039ms] END
{"level":"warn","ts":"2022-01-26T10:34:14.149-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461658,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:14.650-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461658,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:15.015-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.266601161s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.015-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.366206907s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:deployment-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.015-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:13.749-0500","time spent":"1.266664211s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:15.015-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:13.649-0500","time spent":"1.366259156s","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":96,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:deployment-controller\" "}
{"level":"warn","ts":"2022-01-26T10:34:15.015-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"1.266673359s","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.015-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:13.749-0500","time spent":"1.266773786s","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
I0126 10:34:15.016146 96588 trace.go:205] Trace[1331156573]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:controller:deployment-controller,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:e63fe9a6-e071-48a9-ae4e-eb0b85f63a28,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:13.649) (total time: 1366ms):
Trace[1331156573]: [1.366779891s] [1.366779891s] END
{"level":"warn","ts":"2022-01-26T10:34:15.330-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"310.791204ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:disruption-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.330-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"199.196844ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.330-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"199.207062ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.330-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:15.019-0500","time spent":"310.850111ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":96,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:disruption-controller\" "}
{"level":"warn","ts":"2022-01-26T10:34:15.597-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"164.672181ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.597-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"164.696146ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:15.597-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"263.850776ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:endpoint-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.047-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"334.251009ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.047-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"447.570957ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:endpointslice-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.047-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"334.279163ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.047-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:15.599-0500","time spent":"447.661485ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":99,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:endpointslice-controller\" "}
{"level":"warn","ts":"2022-01-26T10:34:16.047-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:15.712-0500","time spent":"334.436927ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:16.047-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:15.712-0500","time spent":"334.338759ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:16.448-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"285.685857ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.448-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"285.832816ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.449-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"397.94246ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:endpointslicemirroring-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.449-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:16.051-0500","time spent":"398.114095ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":108,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:endpointslicemirroring-controller\" "}
{"level":"warn","ts":"2022-01-26T10:34:16.678-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"111.493992ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.678-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"224.761508ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:expand-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:16.678-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"111.53184ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.182-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461677,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:17.423-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"640.497106ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.423-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"640.473207ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.423-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"741.377835ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:ephemeral-volume-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.423-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:16.782-0500","time spent":"640.599722ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:17.423-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:16.782-0500","time spent":"640.543282ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:17.423-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:16.681-0500","time spent":"741.514606ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":102,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:ephemeral-volume-controller\" "}
I0126 10:34:17.423956 96588 trace.go:205] Trace[1674426686]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:controller:ephemeral-volume-controller,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:d34a7bad-c789-469a-9802-8015b5f83b46,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:16.681) (total time: 742ms):
Trace[1674426686]: [742.117973ms] [742.117973ms] END
{"level":"warn","ts":"2022-01-26T10:34:17.625-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"198.110276ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:generic-garbage-collector\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.986-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"258.053709ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.986-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"258.205451ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:17.986-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"266.3079ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:job-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.329-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"249.373365ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:node-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.329-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"226.050743ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.329-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"226.029254ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.651-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"216.990486ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.651-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"217.016924ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.651-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"317.579949ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:persistent-volume-binder\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:18.651-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:18.333-0500","time spent":"317.689044ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":99,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:persistent-volume-binder\" "}
{"level":"warn","ts":"2022-01-26T10:34:19.050-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"296.72146ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.050-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"396.586665ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:pod-garbage-collector\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.050-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"296.726749ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.050-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:18.654-0500","time spent":"396.654001ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":96,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:pod-garbage-collector\" "}
{"level":"warn","ts":"2022-01-26T10:34:19.195-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"141.607274ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:replicaset-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.473-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"178.378726ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:resourcequota-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.473-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"174.84132ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.473-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"174.872224ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.712-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"126.072408ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.712-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"146.975522ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:service-account-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:19.712-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"126.080024ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.031-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"204.153387ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.031-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"316.659078ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:service-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.031-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:19.715-0500","time spent":"316.721164ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":93,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:service-controller\" "}
{"level":"warn","ts":"2022-01-26T10:34:20.031-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"204.160939ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.535-0500","caller":"etcdserver/v3_server.go:815","msg":"waiting for ReadIndex response took too long, retrying","sent-request-id":4097570423066461716,"retry-timeout":"500ms"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"423.277885ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"116.703199ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:20.150-0500","time spent":"423.357636ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"376.926867ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/roles/\" range_end:\"/registry/rbac.authorization.k8s.io/roles0\" count_only:true ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"116.748983ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"423.343307ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/health\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:20.196-0500","time spent":"376.987397ms","remote":"127.0.0.1:63946","response type":"/etcdserverpb.KV/Range","request count":0,"request size":90,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/roles/\" range_end:\"/registry/rbac.authorization.k8s.io/roles0\" count_only:true "}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"538.351971ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:statefulset-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:20.150-0500","time spent":"423.438717ms","remote":"127.0.0.1:64038","response type":"/etcdserverpb.KV/Range","request count":0,"request size":18,"response count":0,"response size":27,"request content":"key:\"/registry/health\" "}
{"level":"warn","ts":"2022-01-26T10:34:20.573-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:20.035-0500","time spent":"538.537749ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:statefulset-controller\" "}
I0126 10:34:20.574034 96588 trace.go:205] Trace[511951154]: "Get" url:/apis/rbac.authorization.k8s.io/v1/clusterroles/system:controller:statefulset-controller,user-agent:kcp/v0.0.0 (darwin/amd64) kubernetes/$Format,audit-id:3ac174d9-6588-4a37-be4e-8b1fdac8c38f,client:::1,accept:application/json, */*,protocol:HTTP/2.0 (26-Jan-2022 10:34:20.034) (total time: 539ms):
Trace[511951154]: [539.143346ms] [539.143346ms] END
E0126 10:34:20.574424 96588 timeout.go:135] post-timeout activity - time-elapsed: 118.881154ms, GET "/livez" result: Header called after Handler finished
goroutine 8774 [running]:
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1.1()
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/timeout.go:103 +0xb0
panic({0x305d0e0, 0x395e1b0})
/usr/local/Cellar/go/1.17.6/libexec/src/runtime/panic.go:1038 +0x215
golang.org/x/net/http2.(*responseWriter).Header(0x5086100)
/Users/angoldst/go/pkg/mod/golang.org/x/net@v0.0.0-20210520170846-37e1c6afe023/http2/server.go:2611 +0x71
net/http.Error({0x39a5880, 0xc0005e1578}, {0xc0036602c0, 0x297}, 0x2)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2057 +0x46
k8s.io/apiserver/pkg/server/healthz.handleRootHealth.func1({0x39b0290, 0xc003556fc8}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/healthz/healthz.go:258 +0x8f2
k8s.io/apiserver/pkg/endpoints/metrics.InstrumentHandlerFunc.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/metrics/metrics.go:509 +0x2cc
net/http.HandlerFunc.ServeHTTP(0xc0037a07e0, {0x10087c60, 0xc003556fc0}, 0x6)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP(0xc0017e8600, {0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/mux/pathrecorder.go:242 +0x250
k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP(0xc0037a07e0, {0x10087c60, 0xc003556fc0}, 0xc00483e7e0)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/mux/pathrecorder.go:235 +0x70
k8s.io/apiserver/pkg/server.director.ServeHTTP({{0x34fbdfc, 0xc002288af8}, 0xc0006d6990, 0xc00170dc00}, {0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/handler.go:161 +0x82a
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc001029428, {0x10087c60, 0xc003556fc0}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filters/authorization.go:64 +0x47d
net/http.HandlerFunc.ServeHTTP(0x5102138, {0x10087c60, 0xc003556fc0}, 0x3f747ae147ae147b)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:79 +0x178
net/http.HandlerFunc.ServeHTTP(0x508eae0, {0x10087c60, 0xc003556fc0}, 0x3)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/maxinflight.go:187 +0x29d
net/http.HandlerFunc.ServeHTTP(0x0, {0x10087c60, 0xc003556fc0}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc0030c92c0, {0x10087c60, 0xc003556fc0}, 0x10953f0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filters/impersonation.go:50 +0x21c
net/http.HandlerFunc.ServeHTTP(0x5102138, {0x10087c60, 0xc003556fc0}, 0x1)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:79 +0x178
net/http.HandlerFunc.ServeHTTP(0x0, {0x10087c60, 0xc003556fc0}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0x5102138, {0x10087c60, 0xc003556fc0}, 0x1)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:79 +0x178
net/http.HandlerFunc.ServeHTTP(0x0, {0x10087c60, 0xc003556fc0}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0x39be9b8, {0x10087c60, 0xc003556fc0}, 0x3640888)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filters.withAuthentication.func1({0x10087c60, 0xc003556fc0}, 0xc003a72f00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filters/authentication.go:80 +0x83c
net/http.HandlerFunc.ServeHTTP(0x39be980, {0x10087c60, 0xc003556fc0}, 0x3952488)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc003556fc0}, 0xc003a72d00)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:88 +0x46b
net/http.HandlerFunc.ServeHTTP(0x347, {0x10087c60, 0xc003556fc0}, 0xc0045fe500)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1()
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/timeout.go:108 +0x99
created by k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/timeout.go:94 +0x2d1
E0126 10:34:20.574617 96588 timeout.go:135] post-timeout activity - time-elapsed: 118.941585ms, GET "/readyz" result: Header called after Handler finished
goroutine 8890 [running]:
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1.1()
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/timeout.go:103 +0xb0
panic({0x305d0e0, 0x395e1b0})
/usr/local/Cellar/go/1.17.6/libexec/src/runtime/panic.go:1038 +0x215
golang.org/x/net/http2.(*responseWriter).Header(0x5086100)
/Users/angoldst/go/pkg/mod/golang.org/x/net@v0.0.0-20210520170846-37e1c6afe023/http2/server.go:2611 +0x71
net/http.Error({0x39a5880, 0xc0005e1570}, {0xc00450e2c0, 0x2bb}, 0x2)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2057 +0x46
k8s.io/apiserver/pkg/server/healthz.handleRootHealth.func1({0x39b0290, 0xc0005e1588}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/healthz/healthz.go:258 +0x8f2
k8s.io/apiserver/pkg/endpoints/metrics.InstrumentHandlerFunc.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/metrics/metrics.go:509 +0x2cc
net/http.HandlerFunc.ServeHTTP(0xc003452d50, {0x10087c60, 0xc0005e1580}, 0x7)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP(0xc0017e8600, {0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/mux/pathrecorder.go:242 +0x250
k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP(0xc003452d50, {0x10087c60, 0xc0005e1580}, 0xc004520120)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/mux/pathrecorder.go:235 +0x70
k8s.io/apiserver/pkg/server.director.ServeHTTP({{0x34fbdfc, 0xc003c2aaf8}, 0xc0006d6990, 0xc00170dc00}, {0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/handler.go:161 +0x82a
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0xc001029428, {0x10087c60, 0xc0005e1580}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filters/authorization.go:64 +0x47d
net/http.HandlerFunc.ServeHTTP(0x5102138, {0x10087c60, 0xc0005e1580}, 0x3f647ae147ae147b)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:79 +0x178
net/http.HandlerFunc.ServeHTTP(0x508eae0, {0x10087c60, 0xc0005e1580}, 0x3)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/maxinflight.go:187 +0x29d
net/http.HandlerFunc.ServeHTTP(0x0, {0x10087c60, 0xc0005e1580}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0x1, {0x10087c60, 0xc0005e1580}, 0x50cee00)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filters/impersonation.go:50 +0x21c
net/http.HandlerFunc.ServeHTTP(0x5102138, {0x10087c60, 0xc0005e1580}, 0x1)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:79 +0x178
net/http.HandlerFunc.ServeHTTP(0x0, {0x10087c60, 0xc0005e1580}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0x5102138, {0x10087c60, 0xc0005e1580}, 0x1)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:79 +0x178
net/http.HandlerFunc.ServeHTTP(0x0, {0x10087c60, 0xc0005e1580}, 0x0)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackCompleted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:103 +0x1a5
net/http.HandlerFunc.ServeHTTP(0x39be9b8, {0x10087c60, 0xc0005e1580}, 0x3640888)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filters.withAuthentication.func1({0x10087c60, 0xc0005e1580}, 0xc00451c500)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filters/authentication.go:80 +0x83c
net/http.HandlerFunc.ServeHTTP(0x39be980, {0x10087c60, 0xc0005e1580}, 0x3952488)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/endpoints/filterlatency.trackStarted.func1({0x10087c60, 0xc0005e1580}, 0xc00451c300)
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/endpoints/filterlatency/filterlatency.go:88 +0x46b
net/http.HandlerFunc.ServeHTTP(0xc0005e1570, {0x10087c60, 0xc0005e1580}, 0xc00335e778)
/usr/local/Cellar/go/1.17.6/libexec/src/net/http/server.go:2047 +0x2f
k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1()
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/timeout.go:108 +0x99
created by k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP
/Users/angoldst/go/pkg/mod/github.com/kcp-dev/kubernetes/staging/src/k8s.io/apiserver@v0.0.0-20220112171016-760c05c1a7d0/pkg/server/filters/timeout.go:94 +0x2d1
{"level":"warn","ts":"2022-01-26T10:34:20.784-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"207.921605ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:ttl-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:21.063-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"180.848408ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:pvc-protection-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:21.313-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"165.085669ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:ttl-after-finished-controller\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:21.671-0500","caller":"etcdserver/util.go:166","msg":"apply request took too long","took":"354.15346ms","expected-duration":"100ms","prefix":"read-only range ","request":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:root-ca-cert-publisher\" ","response":"range_response_count:0 size:4"}
{"level":"warn","ts":"2022-01-26T10:34:21.671-0500","caller":"v3rpc/interceptor.go:197","msg":"request stats","start time":"2022-01-26T10:34:21.317-0500","time spent":"354.319828ms","remote":"127.0.0.1:63954","response type":"/etcdserverpb.KV/Range","request count":0,"request size":97,"response count":0,"response size":27,"request content":"key:\"/registry/rbac.authorization.k8s.io/clusterroles/admin/system:controller:root-ca-cert-publisher\" "}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment