Skip to content

Instantly share code, notes, and snippets.

@tghartland
Created September 25, 2019 14:14
Show Gist options
  • Save tghartland/68c97dbaf6515a99c855b000b49a69cf to your computer and use it in GitHub Desktop.
Save tghartland/68c97dbaf6515a99c855b000b49a69cf to your computer and use it in GitHub Desktop.
I0925 13:57:40.944011 1 magnum_manager_resize.go:227] Returning node instances:
[
{
"Id": "openstack:///9c097488-1c6a-446b-a878-9abd3e0ded95",
"Status": {
"State": 1,
"ErrorInfo": null
}
},
{
"Id": "414",
"Status": {
"State": 2,
"ErrorInfo": {
"ErrorClass": 1,
"ErrorCode": "",
"ErrorMessage": "Forbidden: resources[414].resources.kube-minion: Quota exceeded for cores, instances, ram: Requested 2, 1, 3750, but already used 10, 5, 18750 of 10, 5, 20480 cores, instances, ram (HTTP 403) (Request-ID: req-84907180-ea16-412f-abba-5205ae6c617d)"
}
}
},
{
"Id": "openstack:///97666314-01c2-4c46-a257-c74822dd1307",
"Status": {
"State": 1,
"ErrorInfo": null
}
},
{
"Id": "412",
"Status": {
"State": 2,
"ErrorInfo": {
"ErrorClass": 1,
"ErrorCode": "",
"ErrorMessage": "Forbidden: resources[412].resources.kube-minion: Quota exceeded for cores, instances, ram: Requested 2, 1, 3750, but already used 10, 5, 18750 of 10, 5, 20480 cores, instances, ram (HTTP 403) (Request-ID: req-5b8f16d4-f302-4c95-8c52-dd649d54d62c)"
}
}
},
{
"Id": "413",
"Status": {
"State": 2,
"ErrorInfo": {
"ErrorClass": 1,
"ErrorCode": "",
"ErrorMessage": "Forbidden: resources[413].resources.kube-minion: Quota exceeded for cores, instances, ram: Requested 2, 1, 3750, but already used 10, 5, 18750 of 10, 5, 20480 cores, instances, ram (HTTP 403) (Request-ID: req-64818114-f12a-4667-8e67-f835892bbfae)"
}
}
},
{
"Id": "openstack:///979d7eed-b85e-4595-9e1d-41ce1559b64b",
"Status": {
"State": 1,
"ErrorInfo": null
}
},
{
"Id": "openstack:///84081f5f-cdbc-4469-a127-7d622a8915ea",
"Status": {
"State": 1,
"ErrorInfo": null
}
}
]
I0925 13:57:40.944019 1 magnum_manager_resize.go:228] Cluster status is "UPDATE_FAILED", cluster is changing = false
W0925 13:57:40.944193 1 clusterstate.go:281] Disabling scale-up for node group default-worker until 2019-09-25 14:02:39.126799482 +0000 UTC m=+843.623349073; errorClass=OutOfResource; errorCode=
I0925 13:57:40.944445 1 magnum_manager_resize.go:246] Deleting fake node 414
I0925 13:57:40.944450 1 magnum_manager_resize.go:246] Deleting fake node 412
I0925 13:57:40.944454 1 magnum_manager_resize.go:246] Deleting fake node 413
I0925 13:57:40.944459 1 magnum_manager_resize.go:260] resizeOpts: node_count=4, remove=[414 412 413]
I0925 13:57:56.581870 1 magnum_cloud_provider.go:135] default-worker min=1 max=7 target=4
I0925 13:57:56.582518 1 magnum_manager_resize.go:246] Deleting fake node 414
I0925 13:57:56.582523 1 magnum_manager_resize.go:246] Deleting fake node 412
I0925 13:57:56.582527 1 magnum_manager_resize.go:246] Deleting fake node 413
I0925 13:57:56.582537 1 magnum_manager_resize.go:260] resizeOpts: node_count=1, remove=[414 412 413]
W0925 13:58:02.129589 1 scale_up.go:334] Node group default-worker is not ready for scaleup - backoff
I0925 13:58:12.135285 1 magnum_cloud_provider.go:135] default-worker min=1 max=7 target=1
W0925 13:58:12.135871 1 static_autoscaler.go:525] Error while trying to delete nodes from default-worker: size decrease too large, desired:-2 min:1
W0925 13:58:12.136599 1 scale_up.go:334] Node group default-worker is not ready for scaleup - backoff
I0925 13:58:22.144259 1 magnum_cloud_provider.go:135] default-worker min=1 max=7 target=1
W0925 13:58:22.146342 1 static_autoscaler.go:525] Error while trying to delete nodes from default-worker: size decrease too large, desired:-2 min:1
W0925 13:58:22.149330 1 scale_up.go:334] Node group default-worker is not ready for scaleup - backoff
I0925 13:58:32.162658 1 magnum_cloud_provider.go:135] default-worker min=1 max=7 target=1
W0925 13:58:32.163344 1 static_autoscaler.go:525] Error while trying to delete nodes from default-worker: size decrease too large, desired:-2 min:1
W0925 13:58:32.164377 1 scale_up.go:334] Node group default-worker is not ready for scaleup - backoff
I0925 13:58:42.170759 1 magnum_cloud_provider.go:135] default-worker min=1 max=7 target=1
W0925 13:58:42.171420 1 static_autoscaler.go:525] Error while trying to delete nodes from default-worker: size decrease too large, desired:-2 min:1
I0925 13:58:52.178132 1 magnum_cloud_provider.go:135] default-worker min=1 max=7 target=1
W0925 13:58:52.178939 1 static_autoscaler.go:525] Error while trying to delete nodes from default-worker: size decrease too large, desired:-2 min:1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment