Skip to content

Instantly share code, notes, and snippets.

@gegere

gegere/gist:73c069ef2c9e9845a3c8

Created December 16, 2014 23:44
Show Gist options
  • Save gegere/73c069ef2c9e9845a3c8 to your computer and use it in GitHub Desktop.
Save gegere/73c069ef2c9e9845a3c8 to your computer and use it in GitHub Desktop.
Download ZIP
Helpful logs around the time of etcd failures...
Raw
gistfile1.txt
Dec 14 08:42:04 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit mysqld.service stop: job=999801
Dec 14 08:42:04 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit mysqld.service
Dec 14 08:42:04 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:42:04 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=mysqld.service reason="unit loaded but not sche
Dec 14 08:42:09 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit mysqld-discovery.service stop: job=999805
Dec 14 08:42:09 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit mysqld-discovery.service
Dec 14 08:42:09 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:42:09 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=mysqld-discovery.service reason="unit loaded bu
Dec 14 08:44:48 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit mysqld.service (535b)
Dec 14 08:44:48 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:44:49 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit mysqld-discovery.service (339b)
Dec 14 08:44:49 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=mysqld.service reason="unit scheduled here but no
Dec 14 08:44:49 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:44:49 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=mysqld-discovery.service reason="unit scheduled h
Dec 14 08:44:49 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit mysqld-discovery.service start: job=1000195
Dec 14 08:44:49 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=mysqld-discovery.service reason="unit currently
Dec 14 08:44:57 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit mysqld.service start: job=1000358
Dec 14 08:44:57 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=mysqld.service reason="unit currently loaded but
Dec 14 08:45:47 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit mysqld.service stop: job=1000593
Dec 14 08:45:47 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit mysqld.service
Dec 14 08:45:47 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:45:48 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=mysqld.service reason="unit loaded but not sche
Dec 14 08:45:52 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit mysqld-discovery.service stop: job=1000595
Dec 14 08:45:52 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit mysqld-discovery.service
Dec 14 08:45:52 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:45:52 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=mysqld-discovery.service reason="unit loaded bu
Dec 14 08:46:17 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit mysqld.service (533b)
Dec 14 08:46:17 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:46:17 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=mysqld.service reason="unit scheduled here but no
Dec 14 08:46:19 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit mysqld-discovery.service (339b)
Dec 14 08:46:19 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:46:19 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=mysqld-discovery.service reason="unit scheduled h
Dec 14 08:46:19 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit mysqld-discovery.service start: job=1000596
Dec 14 08:46:19 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=mysqld-discovery.service reason="unit currently
Dec 14 08:46:27 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit mysqld.service start: job=1000757
Dec 14 08:46:27 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=mysqld.service reason="unit currently loaded but
Dec 14 08:47:32 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit ApacheLink@1.service start: job=1000992
Dec 14 08:47:32 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=ApacheLink@1.service reason="unit currently load
Dec 14 08:51:44 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit mysqld.service stop: job=1001622
Dec 14 08:51:44 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit mysqld.service
Dec 14 08:51:44 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:51:44 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=mysqld.service reason="unit loaded but not sche
Dec 14 08:51:49 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit mysqld-discovery.service stop: job=1001625
Dec 14 08:51:49 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit mysqld-discovery.service
Dec 14 08:51:49 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:51:49 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=mysqld-discovery.service reason="unit loaded bu
Dec 14 08:58:01 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit mysqld.service (533b)
Dec 14 08:58:01 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:58:01 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=mysqld.service reason="unit scheduled here but no
Dec 14 08:58:01 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit mysqld.service start: job=1002487
Dec 14 08:58:01 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=mysqld.service reason="unit currently loaded but
Dec 14 08:58:06 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit mysqld-discovery.service (339b)
Dec 14 08:58:06 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 08:58:06 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=mysqld-discovery.service reason="unit scheduled h
Dec 14 08:58:06 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit mysqld-discovery.service start: job=1002570
Dec 14 08:58:06 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=mysqld-discovery.service reason="unit currently
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit ApacheLink@1.service stop: job=1003116
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit ApacheLink@1.service
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:89: Triggered systemd unit ApacheLinkDiscovery@1.service stop: job=1003117
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:231: Removing systemd unit ApacheLinkDiscovery@1.service
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=ApacheLink@1.service reason="unit loaded but no
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 09:01:41 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=UnloadUnit job=ApacheLinkDiscovery@1.service reason="unit load
Dec 14 09:02:00 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit ApacheLink@1.service (719b)
Dec 14 09:02:00 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 09:02:00 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:218: Writing systemd unit ApacheLinkDiscovery@1.service (366b)
Dec 14 09:02:00 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=ApacheLink@1.service reason="unit scheduled here
Dec 14 09:02:00 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:142: Instructing systemd to reload units
Dec 14 09:02:00 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=LoadUnit job=ApacheLinkDiscovery@1.service reason="unit schedu
Dec 14 09:02:18 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit ApacheLinkDiscovery@1.service start: job=1003118
Dec 14 09:02:18 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=ApacheLinkDiscovery@1.service reason="unit curre
Dec 14 09:02:18 coreos63.htmlgraphic.com fleetd[14289]: INFO manager.go:78: Triggered systemd unit ApacheLink@1.service start: job=1003195
Dec 14 09:02:18 coreos63.htmlgraphic.com fleetd[14289]: INFO reconcile.go:274: AgentReconciler completed task: type=StartUnit job=ApacheLink@1.service reason="unit currently load
Dec 14 11:47:03 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: too many redirects
Dec 14 11:47:03 coreos63.htmlgraphic.com fleetd[14289]: ERROR client.go:200: Unable to get result for {Get /_coreos.com/fleet/job}, retrying in 100ms
Dec 14 11:47:03 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: too many redirects
Dec 14 11:47:03 coreos63.htmlgraphic.com fleetd[14289]: ERROR client.go:200: Unable to get result for {Get /_coreos.com/fleet/job}, retrying in 200ms
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: too many redirects
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: too many redirects
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: ERROR client.go:200: Unable to get result for {Get /_coreos.com/fleet/engine/version}, retrying in 200ms
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: too many redirects
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: ERROR client.go:200: Unable to get result for {Get /_coreos.com/fleet/job}, retrying in 400ms
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: too many redirects
Dec 14 11:47:04 coreos63.htmlgraphic.com fleetd[14289]: ERROR client.go:200: Unable to get result for {Get /_coreos.com/fleet/engine/version}, retrying in 400ms
Dec 15 14:38:07 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: cancelled
Dec 15 14:38:07 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: cancelled
Dec 15 14:38:07 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: cancelled
Dec 15 14:38:07 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: cancelled
Dec 15 14:38:07 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: cancelled
Dec 15 14:38:08 coreos63.htmlgraphic.com fleetd[14289]: INFO client.go:278: Failed getting response from http://localhost:4001/: cancelled
Dec 15 16:01:06 coreos63.htmlgraphic.com systemd[1]: fleet.service holdoff time over, scheduling restart.
Dec 15 16:01:06 coreos63.htmlgraphic.com systemd[1]: fleet.service failed to schedule restart job: Transaction is destructive.
Dec 15 16:01:06 coreos63.htmlgraphic.com systemd[1]: Unit fleet.service entered failed state.
Dec 15 16:25:04 coreos63.htmlgraphic.com systemd[1]: Stopped fleet daemon.
-- Reboot --
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment