Created
October 11, 2017 21:19
-
-
Save hobochili/762f8ebade8288a7322f81ab7931cf1d to your computer and use it in GitHub Desktop.
Nomad auto_revert failure server logs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2017/10/11 21:18:10.529927 [DEBUG] http: Request /v1/jobs?region=global (830.125µs) | |
| 2017/10/11 21:18:10.530330 [DEBUG] worker: dequeued evaluation 3dea4483-f550-1d1a-a052-253472757fa3 | |
| 2017/10/11 21:18:10.531010 [DEBUG] http: Request /v1/evaluation/3dea4483-f550-1d1a-a052-253472757fa3?region=global (142.696µs) | |
| 2017/10/11 21:18:10.531025 [DEBUG] sched: <Eval "3dea4483-f550-1d1a-a052-253472757fa3" JobID: "test" Namespace: "default">: Total changes: (place 1) (destructive 0) (inplace 0) (stop 1) | |
| Created Deployment: "1b0f1af6-1911-adaa-32f7-d6f264ab1801" | |
| Desired Changes for "pass": (place 0) (inplace 0) (destructive 0) (stop 1) (migrate 0) (ignore 0) (canary 0) | |
| Desired Changes for "fail": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0) | |
| 2017/10/11 21:18:10.532028 [DEBUG] worker: submitted plan for evaluation 3dea4483-f550-1d1a-a052-253472757fa3 | |
| 2017/10/11 21:18:10.532122 [DEBUG] sched: <Eval "3dea4483-f550-1d1a-a052-253472757fa3" JobID: "test" Namespace: "default">: setting status to complete | |
| 2017/10/11 21:18:10.532395 [DEBUG] worker: updated evaluation <Eval "3dea4483-f550-1d1a-a052-253472757fa3" JobID: "test" Namespace: "default"> | |
| 2017/10/11 21:18:10.532485 [DEBUG] worker: ack for evaluation 3dea4483-f550-1d1a-a052-253472757fa3 | |
| 2017/10/11 21:18:10.533020 [DEBUG] client: updated allocations at index 2834 (total 260) (pulled 2) (filtered 258) | |
| 2017/10/11 21:18:10.533257 [DEBUG] client: allocs: (added 1) (removed 0) (updated 1) (ignore 258) | |
| 2017/10/11 21:18:10.536262 [DEBUG] http: Request /v1/evaluation/3dea4483-f550-1d1a-a052-253472757fa3/allocations?region=global (176.522µs) | |
| 2017/10/11 21:18:10.549605 [DEBUG] client: starting task runners for alloc '3d1e439d-bd09-e253-4769-da87340278e1' | |
| 2017/10/11 21:18:10.550340 [DEBUG] client.alloc_watcher: deadline (10s) for alloc "3d1e439d-bd09-e253-4769-da87340278e1" is at 2017-10-11 21:18:20.550317327 +0000 UTC m=+2402.335620918 | |
| 2017/10/11 21:18:10.551106 [DEBUG] client: starting task context for 'tail' (alloc '3d1e439d-bd09-e253-4769-da87340278e1') | |
| 2017-10-11T21:18:10.556Z [DEBUG] plugin: plugin process exited: path=/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad | |
| 2017/10/11 21:18:10.557180 [INFO] client: marking allocation 3f0c43fb-d459-87a5-1d4b-b40abdd6ac9c for GC | |
| 2017/10/11 21:18:10.557528 [INFO] client: marking allocation 3f0c43fb-d459-87a5-1d4b-b40abdd6ac9c for GC | |
| 2017/10/11 21:18:10.740472 [DEBUG] client: updated allocations at index 2836 (total 260) (pulled 0) (filtered 260) | |
| 2017/10/11 21:18:10.740948 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 260) | |
| 2017/10/11 21:18:11.554141 [DEBUG] http: Request /v1/evaluation/3dea4483-f550-1d1a-a052-253472757fa3?region=global (15.074641ms) | |
| 2017/10/11 21:18:11.559339 [DEBUG] http: Request /v1/evaluation/3dea4483-f550-1d1a-a052-253472757fa3/allocations?region=global (307.971µs) | |
| 2017-10-11T21:18:12.541Z [DEBUG] plugin: starting plugin: path=/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad args="[/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad executor {"LogFile":"/tmp/NomadClient446480596/3d1e439d-bd09-e253-4769-da87340278e1/tail/executor.out","LogLevel":"DEBUG"}]" | |
| 2017-10-11T21:18:12.543Z [DEBUG] plugin: waiting for RPC address: path=/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad | |
| 2017-10-11T21:18:12.667Z [DEBUG] plugin.nomad: plugin address: timestamp=2017-10-11T21:18:12.666Z address=/tmp/plugin757659969 network=unix | |
| 2017/10/11 21:18:12.708044 [DEBUG] driver.exec: started process via plugin with pid: 25807 | |
| 2017-10-11T21:18:12.715Z [DEBUG] plugin: plugin process exited: path=/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad | |
| 2017/10/11 21:18:12.723643 [INFO] client: task "tail" for alloc "3d1e439d-bd09-e253-4769-da87340278e1" failed: Wait returned exit code 1, signal 0, and error <nil> | |
| 2017/10/11 21:18:12.723667 [INFO] client: Restarting task "tail" for alloc "3d1e439d-bd09-e253-4769-da87340278e1" in 18.064244619s | |
| 2017/10/11 21:18:12.943953 [DEBUG] client: updated allocations at index 2837 (total 260) (pulled 0) (filtered 260) | |
| 2017/10/11 21:18:12.944445 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 260) | |
| 2017/10/11 21:18:18.329401 [INFO] client: garbage collecting allocation 3f0c43fb-d459-87a5-1d4b-b40abdd6ac9c due to number of allocations is over the limit (50) | |
| 2017/10/11 21:18:19.287359 [DEBUG] client: terminating runner for alloc '3f0c43fb-d459-87a5-1d4b-b40abdd6ac9c' | |
| 2017/10/11 21:18:19.287427 [DEBUG] client: garbage collected "3f0c43fb-d459-87a5-1d4b-b40abdd6ac9c" | |
| 2017/10/11 21:18:19.289167 [WARN] client: garbage collection due to number of allocations is over the limit (50) skipped because no terminal allocations | |
| 2017/10/11 21:18:20.033225 [DEBUG] worker: dequeued evaluation 099353f2-6b5b-84f5-936a-5147d97ba79b | |
| 2017/10/11 21:18:20.033327 [DEBUG] sched.core: eval GC: scanning before index 0 (1h0m0s) | |
| 2017/10/11 21:18:20.033660 [DEBUG] worker: ack for evaluation 099353f2-6b5b-84f5-936a-5147d97ba79b | |
| 2017/10/11 21:18:20.033769 [DEBUG] worker: dequeued evaluation 24012ca1-2728-27f3-8f6c-70a49a28c540 | |
| 2017/10/11 21:18:20.033788 [DEBUG] sched.core: deployment GC: scanning before index 0 (1h0m0s) | |
| 2017/10/11 21:18:20.033915 [DEBUG] worker: ack for evaluation 24012ca1-2728-27f3-8f6c-70a49a28c540 | |
| 2017/10/11 21:18:20.033941 [DEBUG] worker: dequeued evaluation 025cec00-d262-e250-0998-4731641eaa7b | |
| 2017/10/11 21:18:20.033955 [DEBUG] sched.core: node GC: scanning before index 0 (24h0m0s) | |
| 2017/10/11 21:18:20.033969 [DEBUG] worker: ack for evaluation 025cec00-d262-e250-0998-4731641eaa7b | |
| 2017/10/11 21:18:20.033990 [DEBUG] worker: dequeued evaluation 5d1cecfd-916c-e123-a8ce-8fe745d73993 | |
| 2017/10/11 21:18:20.034114 [DEBUG] sched.core: job GC: scanning before index 0 (4h0m0s) | |
| 2017/10/11 21:18:20.034187 [DEBUG] worker: ack for evaluation 5d1cecfd-916c-e123-a8ce-8fe745d73993 | |
| 2017/10/11 21:18:20.585535 [DEBUG] client: event from "Alloc Unhealthy" for task "tail" in alloc "3d1e439d-bd09-e253-4769-da87340278e1": Task not running by deadline | |
| 2017/10/11 21:18:20.743837 [DEBUG] client: updated allocations at index 2839 (total 260) (pulled 0) (filtered 260) | |
| 2017/10/11 21:18:20.744258 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 260) | |
| 2017/10/11 21:18:20.744921 [DEBUG] worker: dequeued evaluation 42d325d8-0868-40f9-2b40-a92bdc096e31 | |
| 2017/10/11 21:18:20.745648 [DEBUG] sched: <Eval "42d325d8-0868-40f9-2b40-a92bdc096e31" JobID: "test" Namespace: "default">: Total changes: (place 1) (destructive 0) (inplace 0) (stop 1) | |
| Created Deployment: "2ae5f5ca-af34-6fdd-5d18-a77fa4effb4c" | |
| Desired Changes for "fail": (place 0) (inplace 0) (destructive 0) (stop 1) (migrate 0) (ignore 0) (canary 0) | |
| Desired Changes for "pass": (place 1) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 0) (canary 0) | |
| 2017/10/11 21:18:20.746574 [DEBUG] worker: submitted plan for evaluation 42d325d8-0868-40f9-2b40-a92bdc096e31 | |
| 2017/10/11 21:18:20.746666 [DEBUG] sched: <Eval "42d325d8-0868-40f9-2b40-a92bdc096e31" JobID: "test" Namespace: "default">: setting status to complete | |
| 2017/10/11 21:18:20.746943 [DEBUG] worker: updated evaluation <Eval "42d325d8-0868-40f9-2b40-a92bdc096e31" JobID: "test" Namespace: "default"> | |
| 2017/10/11 21:18:20.746996 [DEBUG] worker: ack for evaluation 42d325d8-0868-40f9-2b40-a92bdc096e31 | |
| 2017/10/11 21:18:20.747204 [DEBUG] client: updated allocations at index 2841 (total 261) (pulled 2) (filtered 259) | |
| 2017/10/11 21:18:20.747489 [DEBUG] client: allocs: (added 1) (removed 0) (updated 1) (ignore 259) | |
| 2017/10/11 21:18:20.747600 [DEBUG] client: Not restarting task: tail because it has been destroyed | |
| 2017/10/11 21:18:20.760118 [INFO] client: marking allocation 3d1e439d-bd09-e253-4769-da87340278e1 for GC | |
| 2017/10/11 21:18:20.760259 [INFO] client: marking allocation 3d1e439d-bd09-e253-4769-da87340278e1 for GC | |
| 2017/10/11 21:18:20.760689 [DEBUG] client: starting task runners for alloc '044221f1-5aa1-e5b7-b1c7-202a4552837a' | |
| 2017/10/11 21:18:20.760829 [DEBUG] client: starting task context for 'tail_255' (alloc '044221f1-5aa1-e5b7-b1c7-202a4552837a') | |
| 2017/10/11 21:18:20.760977 [DEBUG] client.alloc_watcher: deadline (10s) for alloc "044221f1-5aa1-e5b7-b1c7-202a4552837a" is at 2017-10-11 21:18:30.760951849 +0000 UTC m=+2412.546255423 | |
| 2017/10/11 21:18:20.772046 [INFO] client: marking allocation 3d1e439d-bd09-e253-4769-da87340278e1 for GC | |
| 2017/10/11 21:18:20.939066 [DEBUG] client: updated allocations at index 2843 (total 261) (pulled 0) (filtered 261) | |
| 2017/10/11 21:18:20.939378 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 261) | |
| 2017/10/11 21:18:21.119218 [DEBUG] http: Request /v1/deployments (2.711035ms) | |
| 2017-10-11T21:18:23.280Z [DEBUG] plugin: starting plugin: path=/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad args="[/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad executor {"LogFile":"/tmp/NomadClient446480596/044221f1-5aa1-e5b7-b1c7-202a4552837a/tail_255/executor.out","LogLevel":"DEBUG"}]" | |
| 2017-10-11T21:18:23.282Z [DEBUG] plugin: waiting for RPC address: path=/opt/gopath/src/github.com/hashicorp/nomad/bin/nomad | |
| 2017-10-11T21:18:23.401Z [DEBUG] plugin.nomad: plugin address: timestamp=2017-10-11T21:18:23.401Z address=/tmp/plugin478676677 network=unix | |
| 2017/10/11 21:18:23.437765 [DEBUG] driver.exec: started process via plugin with pid: 25841 | |
| 2017/10/11 21:18:23.539042 [DEBUG] client: updated allocations at index 2844 (total 261) (pulled 0) (filtered 261) | |
| 2017/10/11 21:18:23.539605 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 261) | |
| 2017/10/11 21:18:24.538648 [DEBUG] client: updated allocations at index 2845 (total 261) (pulled 0) (filtered 261) | |
| 2017/10/11 21:18:24.539030 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 261) | |
| 2017/10/11 21:18:25.790936 [DEBUG] worker: dequeued evaluation b7f00aca-ce2b-1fec-e662-7a3bf2523625 | |
| 2017/10/11 21:18:25.792239 [DEBUG] sched: <Eval "b7f00aca-ce2b-1fec-e662-7a3bf2523625" JobID: "test" Namespace: "default">: Total changes: (place 0) (destructive 0) (inplace 0) (stop 0) | |
| Deployment Update for ID "2ae5f5ca-af34-6fdd-5d18-a77fa4effb4c": Status "successful"; Description "Deployment completed successfully" | |
| Desired Changes for "pass": (place 0) (inplace 0) (destructive 0) (stop 0) (migrate 0) (ignore 1) (canary 0) | |
| 2017/10/11 21:18:25.793330 [DEBUG] worker: submitted plan for evaluation b7f00aca-ce2b-1fec-e662-7a3bf2523625 | |
| 2017/10/11 21:18:25.793384 [DEBUG] sched: <Eval "b7f00aca-ce2b-1fec-e662-7a3bf2523625" JobID: "test" Namespace: "default">: setting status to complete | |
| 2017/10/11 21:18:25.793600 [DEBUG] worker: updated evaluation <Eval "b7f00aca-ce2b-1fec-e662-7a3bf2523625" JobID: "test" Namespace: "default"> | |
| 2017/10/11 21:18:25.793639 [DEBUG] worker: ack for evaluation b7f00aca-ce2b-1fec-e662-7a3bf2523625 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment