Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save perfecto25/dcbb50a81b91956065217b76804df866 to your computer and use it in GitHub Desktop.
Save perfecto25/dcbb50a81b91956065217b76804df866 to your computer and use it in GitHub Desktop.
PuppetServer shutdown on FileSync
2016-05-16 16:50:08,802 INFO [main] [o.e.j.u.log] Logging initialized @79620ms
2016-05-16 16:50:22,699 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Initializing web server(s).
2016-05-16 16:50:23,031 INFO [async-dispatch-2] [p.e.s.f.file-sync-storage-core] Initializing file sync server data dir: /opt/puppetlabs/server/data/puppetserver/filesync/storage
2016-05-16 16:50:23,043 INFO [async-dispatch-2] [p.e.s.f.file-sync-storage-core] Initializing Git repository at /opt/puppetlabs/server/data/puppetserver/filesync/storage/puppet-code.git
2016-05-16 16:50:23,321 INFO [async-dispatch-2] [p.e.s.f.file-sync-storage-service] File sync storage service mounting repositories at /file-sync-git
2016-05-16 16:50:23,706 INFO [async-dispatch-2] [p.e.s.f.file-sync-storage-service] Registering file sync storage HTTP API
2016-05-16 16:50:23,713 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-storage-service service
2016-05-16 16:50:23,747 INFO [async-dispatch-2] [p.c.services] Starting code-manager v1 API...
2016-05-16 16:50:23,802 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for code-manager-service service
2016-05-16 16:50:24,791 INFO [async-dispatch-2] [p.c.config] Writing r10k config to "/opt/puppetlabs/server/data/code-manager/r10k.yaml"
2016-05-16 16:50:25,013 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-puppet-profiler service
2016-05-16 16:50:25,057 INFO [async-dispatch-2] [p.s.j.jruby-puppet-service] Initializing the JRuby service
2016-05-16 16:50:25,117 INFO [async-dispatch-2] [p.s.p.puppet-admin-service] Starting Puppet Admin web app
2016-05-16 16:50:25,191 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 1.
2016-05-16 16:50:55,437 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master
2016-05-16 16:50:59,308 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 1 of 3
2016-05-16 16:50:59,309 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 2.
2016-05-16 16:50:59,399 INFO [async-dispatch-2] [p.s.c.puppet-server-config-core] Not overriding webserver settings with values from core Puppet
2016-05-16 16:50:59,439 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Initializing file sync client service
2016-05-16 16:50:59,467 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for file-sync-client-service service
2016-05-16 16:50:59,500 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Registering file sync client HTTP API
2016-05-16 16:51:00,453 WARN [async-dispatch-2] [p.e.s.f.file-sync-client-core] Repo ':puppet-code' at commit '65e76fe0e5c5b1b66ccf991effd8aeaf5c6b542c' is dirty. Detailed repo status follows.
{:clean false,
:permissions-modified [],
:modified [],
:missing
[".gitmodules"
"environments/development"
"environments/production"
"environments/testing"
"hiera.yaml"],
:untracked [],
:dirty-submodules {}}
2016-05-16 16:51:00,644 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-jruby-metrics service
2016-05-16 16:51:00,911 WARN [async-dispatch-2] [o.e.j.s.h.ContextHandler] Empty contextPath
2016-05-16 16:51:00,941 INFO [async-dispatch-2] [p.p.certificate-authority] CA already initialized for SSL
2016-05-16 16:51:00,968 INFO [async-dispatch-2] [p.s.c.certificate-authority-service] CA Service adding a ring handler
2016-05-16 16:51:02,159 INFO [async-dispatch-2] [p.p.certificate-authority] Master already initialized for SSL
2016-05-16 16:51:02,161 INFO [async-dispatch-2] [p.e.s.m.master-service] Master Service adding a ring handler
2016-05-16 16:51:02,167 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for pe-master service
2016-05-16 16:51:02,187 INFO [async-dispatch-2] [p.t.s.w.jetty9-service] Starting web server(s).
2016-05-16 16:51:02,999 INFO [async-dispatch-2] [p.t.s.w.jetty9-core] Starting web server.
2016-05-16 16:51:03,024 INFO [async-dispatch-2] [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT
2016-05-16 16:51:03,400 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@601e203c{/code-manager/v1,null,AVAILABLE}
2016-05-16 16:51:03,950 INFO [async-dispatch-2] [o.e.j.s.ServerConnector] Started ServerConnector@4b13d4ef{SSL-HTTP/1.1}{0.0.0.0:8170}
2016-05-16 16:51:03,951 INFO [async-dispatch-2] [o.e.j.s.Server] Started @134779ms
2016-05-16 16:51:04,093 INFO [async-dispatch-2] [p.t.s.w.jetty9-core] Starting web server.
2016-05-16 16:51:04,095 INFO [async-dispatch-2] [o.e.j.s.Server] jetty-9.2.z-SNAPSHOT
2016-05-16 16:51:04,452 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.ServletContextHandler@25078adb{/packages,file:/opt/puppetlabs/server/data/packages/public/,AVAILABLE}
2016-05-16 16:51:04,657 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.ServletContextHandler@44428b1d{/file-sync-git,null,AVAILABLE}
2016-05-16 16:51:04,659 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@3669d5a{/puppet-admin-api,null,AVAILABLE}
2016-05-16 16:51:04,661 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@1aae9ebf{/,null,AVAILABLE}
2016-05-16 16:51:04,662 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@4ae4f494{/puppet-ca,null,AVAILABLE}
2016-05-16 16:51:04,674 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@21a36e78{/puppet,null,AVAILABLE}
2016-05-16 16:51:04,754 INFO [async-dispatch-2] [o.e.j.s.ServerConnector] Started ServerConnector@721ef68a{SSL-HTTP/1.1}{0.0.0.0:8140}
2016-05-16 16:51:04,755 INFO [async-dispatch-2] [o.e.j.s.Server] Started @135584ms
2016-05-16 16:51:04,773 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status callback function for status-service service
2016-05-16 16:51:04,774 INFO [async-dispatch-2] [p.t.s.s.status-service] Registering status service HTTP API at /status
2016-05-16 16:51:04,863 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@2d1fb977{/status,null,AVAILABLE}
2016-05-16 16:51:04,867 INFO [async-dispatch-2] [p.e.s.f.file-sync-web-service] Registering file sync HTTP API at /file-sync
2016-05-16 16:51:04,946 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@27419d35{/file-sync,null,AVAILABLE}
2016-05-16 16:51:04,998 INFO [async-dispatch-2] [p.e.s.f.file-sync-client-service] Starting file sync client service
2016-05-16 16:51:05,585 INFO [async-dispatch-2] [p.s.l.legacy-routes-service] The legacy routing service has successfully started and is now ready to handle requests
2016-05-16 16:51:05,600 INFO [async-dispatch-2] [p.e.s.m.master-service] Puppet Server has successfully started and is now ready to handle requests
2016-05-16 16:51:09,952 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fetching ':testing' to 2e585e239336a2254bce929f87f837f719dc4ab2
2016-05-16 16:51:12,443 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] fetch of ':testing' into /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code/testing.git successful. New latest commit: 2e585e239336a2254bce929f87f837f719dc4ab2
2016-05-16 16:51:12,459 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fetching ':production' to e74f4df1f94e0bb90bd217e139f6086d1047674a
2016-05-16 16:51:13,182 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] fetch of ':production' into /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code/production.git successful. New latest commit: e74f4df1f94e0bb90bd217e139f6086d1047674a
2016-05-16 16:51:13,213 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fetching ':puppet-code' to 02aeff610b9e1a6272f63a73190e2563ae51d155
2016-05-16 16:51:13,688 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] fetch of ':puppet-code' into /opt/puppetlabs/server/data/puppetserver/filesync/client/puppet-code.git successful. New latest commit: 02aeff610b9e1a6272f63a73190e2563ae51d155
2016-05-16 16:51:13,879 INFO [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Forcefully syncing live directory at /etc/puppetlabs/code for repository :puppet-code
2016-05-16 16:51:15,946 ERROR [clojure-agent-send-off-pool-0] [p.e.s.f.file-sync-client-core] Fatal error during file sync, requesting shutdown.
org.eclipse.jgit.api.errors.JGitInternalException: Destination path "production" already exists and is not an empty directory
at org.eclipse.jgit.api.CloneCommand.init(CloneCommand.java:162) ~[puppet-server-release.jar:na]
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:133) ~[puppet-server-release.jar:na]
at org.eclipse.jgit.api.SubmoduleUpdateCommand.call(SubmoduleUpdateCommand.java:170) ~[puppet-server-release.jar:na]
at puppetlabs.enterprise.jgit_utils$eval31193$submodule_update__31198$fn__31199.invoke(jgit_utils.clj:532) ~[na:na]
at puppetlabs.enterprise.jgit_utils$eval31193$submodule_update__31198.invoke(jgit_utils.clj:511) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33530$do_sync_BANG___33535$fn__33536.invoke(file_sync_client_core.clj:1086) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33530$do_sync_BANG___33535.invoke(file_sync_client_core.clj:1052) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33559$do_forced_sync_BANG___33564$fn__33565.invoke(file_sync_client_core.clj:1106) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33559$do_forced_sync_BANG___33564.invoke(file_sync_client_core.clj:1088) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33662$sync_live_dir_BANG___33667$fn__33671$fn__33674.invoke(file_sync_client_core.clj:1183) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33662$sync_live_dir_BANG___33667$fn__33671.invoke(file_sync_client_core.clj:1181) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33662$sync_live_dir_BANG___33667.invoke(file_sync_client_core.clj:1153) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_service$reify__33985$service_fnk__5254__auto___positional$reify__34004.sync_live_dir_BANG_(file_sync_client_service.clj:130) ~[na:na]
at puppetlabs.enterprise.services.protocols.file_sync_client$eval33896$fn__33912$G__33888__33918.invoke(file_sync_client.clj:3) ~[na:na]
at puppetlabs.enterprise.services.protocols.file_sync_client$eval33896$fn__33912$G__33887__33925.invoke(file_sync_client.clj:3) ~[na:na]
at clojure.core$partial$fn__4527.invoke(core.clj:2494) ~[puppet-server-release.jar:na]
at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__37962$service_fnk__5254__auto___positional$reify__37973$sync_live_code__37978.invoke(file_sync_versioned_code_service.clj:48) ~[na:na]
at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__37962$service_fnk__5254__auto___positional$reify__37973$fn__37984$fn__37995.invoke(file_sync_versioned_code_service.clj:80) ~[na:na]
at puppetlabs.enterprise.services.file_sync.file_sync_versioned_code_service$reify__37962$service_fnk__5254__auto___positional$reify__37973$fn__37984.invoke(file_sync_versioned_code_service.clj:72) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33011$process_callbacks_BANG___33016$fn__33017.invoke(file_sync_client_core.clj:606) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33011$process_callbacks_BANG___33016.invoke(file_sync_client_core.clj:587) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33070$process_repos_for_updates__33075$fn__33076.invoke(file_sync_client_core.clj:621) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33070$process_repos_for_updates__33075.invoke(file_sync_client_core.clj:608) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33097$sync_on_agent__33102$fn__33103.invoke(file_sync_client_core.clj:642) ~[na:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33097$sync_on_agent__33102.invoke(file_sync_client_core.clj:624) ~[na:na]
at clojure.lang.AFn.applyToHelper(AFn.java:165) ~[puppet-server-release.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:144) ~[puppet-server-release.jar:na]
at clojure.core$apply.invoke(core.clj:630) ~[puppet-server-release.jar:na]
at puppetlabs.enterprise.services.file_sync_client.file_sync_client_core$eval33372$start_periodic_sync_process_BANG___33377$fn__33378$periodic_sync__33379.doInvoke(file_sync_client_core.clj:896) ~[na:na]
at clojure.lang.RestFn.applyTo(RestFn.java:137) ~[puppet-server-release.jar:na]
at clojure.core$apply.invoke(core.clj:636) ~[puppet-server-release.jar:na]
at clojure.core$binding_conveyor_fn$fn__4444.doInvoke(core.clj:1928) ~[puppet-server-release.jar:na]
at clojure.lang.RestFn.applyTo(RestFn.java:146) ~[puppet-server-release.jar:na]
at clojure.lang.Agent$Action.doRun(Agent.java:114) [puppet-server-release.jar:na]
at clojure.lang.Agent$Action.run(Agent.java:163) [puppet-server-release.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
2016-05-16 16:51:15,974 INFO [main] [p.t.internal] Beginning shutdown sequence
2016-05-16 16:51:16,060 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopping metrics sampler job
2016-05-16 16:51:16,072 INFO [async-dispatch-3] [p.e.s.j.pe-jruby-metrics-service] PE JRuby Metrics Service: stopped metrics sampler job
2016-05-16 16:51:16,100 INFO [async-dispatch-3] [p.e.s.f.file-sync-client-service] Stopping file sync client service
2016-05-16 16:51:21,664 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master
2016-05-16 16:51:23,676 INFO [clojure-agent-send-pool-0] [puppet-server] /File[/etc/puppetlabs/code/environments/production]/ensure created
2016-05-16 16:51:23,975 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 2 of 3
2016-05-16 16:51:23,976 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-internal] Creating JRuby instance with id 3.
2016-05-16 16:51:35,765 INFO [clojure-agent-send-pool-0] [puppet-server] Puppet Puppet settings initialized; run mode: master
2016-05-16 16:51:37,606 INFO [clojure-agent-send-pool-0] [p.s.j.jruby-puppet-agents] Finished creating JRubyPuppet instance 3 of 3
2016-05-16 16:51:37,618 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Flush request received; creating new JRuby pool.
2016-05-16 16:51:37,637 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Replacing old JRuby pool with new instance.
2016-05-16 16:51:37,638 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-agents] Swapped JRuby pools, beginning cleanup of old pool.
2016-05-16 16:51:37,687 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 1.
2016-05-16 16:51:37,705 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 2.
2016-05-16 16:51:37,715 INFO [clojure-agent-send-pool-1] [p.s.j.jruby-puppet-internal] Cleaned up old JRuby instance with id 3.
2016-05-16 16:51:37,727 INFO [async-dispatch-3] [p.c.services] Shutting down code-manager...
2016-05-16 16:51:37,730 INFO [async-dispatch-3] [p.c.shell-workers] Attempting to shut down deploy-pool workers...
2016-05-16 16:51:37,736 INFO [async-dispatch-3] [p.c.shell-workers] deploy-pool workers sucessfully shut down.
2016-05-16 16:51:37,736 INFO [async-dispatch-3] [p.c.services] code-manager shut down.
2016-05-16 16:51:37,741 INFO [async-dispatch-3] [p.e.s.f.file-sync-storage-service] Closing all client status channels
2016-05-16 16:51:37,768 INFO [async-dispatch-3] [p.t.s.w.jetty9-service] Shutting down web server(s).
2016-05-16 16:51:37,801 INFO [async-dispatch-3] [p.t.s.w.jetty9-core] Shutting down web server.
2016-05-16 16:51:37,824 INFO [async-dispatch-3] [o.e.j.s.ServerConnector] Stopped ServerConnector@4b13d4ef{SSL-HTTP/1.1}{0.0.0.0:8170}
2016-05-16 16:51:37,828 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@601e203c{/code-manager/v1,null,UNAVAILABLE}
2016-05-16 16:51:37,860 INFO [async-dispatch-3] [p.t.s.w.jetty9-core] Web server shutdown
2016-05-16 16:51:37,864 INFO [async-dispatch-3] [p.t.s.w.jetty9-core] Shutting down web server.
2016-05-16 16:51:37,880 INFO [async-dispatch-3] [o.e.j.s.ServerConnector] Stopped ServerConnector@721ef68a{SSL-HTTP/1.1}{0.0.0.0:8140}
2016-05-16 16:51:37,881 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@27419d35{/file-sync,null,UNAVAILABLE}
2016-05-16 16:51:37,882 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@2d1fb977{/status,null,UNAVAILABLE}
2016-05-16 16:51:37,882 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@21a36e78{/puppet,null,UNAVAILABLE}
2016-05-16 16:51:37,883 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@4ae4f494{/puppet-ca,null,UNAVAILABLE}
2016-05-16 16:51:37,883 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@1aae9ebf{/,null,UNAVAILABLE}
2016-05-16 16:51:37,883 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.h.ContextHandler@3669d5a{/puppet-admin-api,null,UNAVAILABLE}
2016-05-16 16:51:37,901 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.ServletContextHandler@44428b1d{/file-sync-git,null,UNAVAILABLE}
2016-05-16 16:51:37,902 INFO [async-dispatch-3] [o.e.j.s.h.ContextHandler] Stopped o.e.j.s.ServletContextHandler@25078adb{/packages,file:/opt/puppetlabs/server/data/packages/public/,UNAVAILABLE}
2016-05-16 16:51:37,911 INFO [async-dispatch-3] [p.t.s.w.jetty9-core] Web server shutdown
2016-05-16 16:51:37,944 INFO [main] [p.t.internal] Finished shutdown sequence
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment