Skip to content

Instantly share code, notes, and snippets.

@clehene

clehene/mesos-master.log Secret

Created April 4, 2016 20:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save clehene/09a9ddfe49b92a5deb4c1b421f63479e to your computer and use it in GitHub Desktop.
Save clehene/09a9ddfe49b92a5deb4c1b421f63479e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
mesos-master.log
Apr 04 13:00:06 ip-10-27-9-130.REDACTED systemd[1]: Started Mesos master.
Apr 04 13:00:06 ip-10-27-9-130.REDACTED systemd[1]: Starting Mesos master...
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.606253 30865 logging.cpp:188] INFO level logging started!
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.606497 30865 main.cpp:237] Build: 2016-03-17 17:47:25 by root
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.606506 30865 main.cpp:239] Version: 0.28.0
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.606511 30865 main.cpp:242] Git tag: 0.28.0
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.606513 30865 main.cpp:246] Git SHA: 961edbd82e691a619a4c171a7aadc9c32957fa73
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.606603 30865 main.cpp:260] Using 'HierarchicalDRF' allocator
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.610090 30865 leveldb.cpp:174] Opened db in 3.411084ms
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611100 30865 leveldb.cpp:181] Compacted db in 978301ns
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611122 30865 leveldb.cpp:196] Created db iterator in 7604ns
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611127 30865 leveldb.cpp:202] Seeked to beginning of db in 527ns
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611130 30865 leveldb.cpp:271] Iterated through 0 keys in the db in 163ns
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611156 30865 replica.cpp:779] Replica recovered with log positions 0 -> 0 with 1 holes and 0 unlearned
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-27-9-130.REDACTED
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-229.el7.x86_64
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Fri Mar 6 11:36:42 UTC 2015
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611580 30867 log.cpp:236] Attempting to join replica to ZooKeeper group
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00c146700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00c146700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-27-9-130.REDACTED
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00c146700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00c146700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-229.el7.x86_64
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00c146700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Fri Mar 6 11:36:42 UTC 2015
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611737 30868 recover.cpp:447] Starting replica recovery
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00a943700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00a943700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-27-9-130.REDACTED
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00a943700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00a943700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-229.el7.x86_64
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00a943700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Fri Mar 6 11:36:42 UTC 2015
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.611790 30865 main.cpp:471] Starting Mesos master
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00e14a700):ZOO_INFO@log_env@712: Client environment:zookeeper.version=zookeeper C client 3.4.5
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00e14a700):ZOO_INFO@log_env@716: Client environment:host.name=ip-10-27-9-130.REDACTED
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00e14a700):ZOO_INFO@log_env@723: Client environment:os.name=Linux
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00e14a700):ZOO_INFO@log_env@724: Client environment:os.arch=3.10.0-229.el7.x86_64
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00e14a700):ZOO_INFO@log_env@725: Client environment:os.version=#1 SMP Fri Mar 6 11:36:42 UTC 2015
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@741: Client environment:user.home=/root
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@log_env@753: Client environment:user.dir=/
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,611:30865(0x7ff00b144700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=ip-10-27-9-127.REDACTED:2181,ip-10-27-9-130.REDACTED:2181,ip-10-27-15-203.REDACTED:2181 sessionTimeout=10000 watcher=0x7ff0157f8280 sessionId=0 sessionPasswd=<null> context=0x7fefe4001750 flags=0
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00a943700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00a943700):ZOO_INFO@log_env@741: Client environment:user.home=/root
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00a943700):ZOO_INFO@log_env@753: Client environment:user.dir=/
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00a943700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=ip-10-27-9-127.REDACTED:2181,ip-10-27-9-130.REDACTED:2181,ip-10-27-15-203.REDACTED:2181 sessionTimeout=10000 watcher=0x7ff0157f8280 sessionId=0 sessionPasswd=<null> context=0x7feff4003320 flags=0
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00c146700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00c146700):ZOO_INFO@log_env@741: Client environment:user.home=/root
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00c146700):ZOO_INFO@log_env@753: Client environment:user.dir=/
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00c146700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=ip-10-27-9-127.REDACTED:2181,ip-10-27-9-130.REDACTED:2181,ip-10-27-15-203.REDACTED:2181 sessionTimeout=10000 watcher=0x7ff0157f8280 sessionId=0 sessionPasswd=<null> context=0x7ff000000f40 flags=0
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00e14a700):ZOO_INFO@log_env@733: Client environment:user.name=(null)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00e14a700):ZOO_INFO@log_env@741: Client environment:user.home=/root
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00e14a700):ZOO_INFO@log_env@753: Client environment:user.dir=/
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,612:30865(0x7ff00e14a700):ZOO_INFO@zookeeper_init@786: Initiating client connection, host=ip-10-27-9-127.REDACTED:2181,ip-10-27-9-130.REDACTED:2181,ip-10-27-15-203.REDACTED:2181 sessionTimeout=10000 watcher=0x7ff0157f8280 sessionId=0 sessionPasswd=<null> context=0x7fefec001400 flags=0
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.613394 30872 recover.cpp:473] Replica is in EMPTY status
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,614:30865(0x7feffbfff700):ZOO_INFO@check_events@1703: initiated connection to server [10.27.9.130:2181]
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,614:30865(0x7ff009140700):ZOO_INFO@check_events@1703: initiated connection to server [10.27.9.130:2181]
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,615:30865(0x7feffbfff700):ZOO_INFO@check_events@1750: session establishment complete on server [10.27.9.130:2181], sessionId=0x253e224e631019f, negotiated timeout=10000
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,615:30865(0x7ff009140700):ZOO_INFO@check_events@1750: session establishment complete on server [10.27.9.130:2181], sessionId=0x253e224e63101a0, negotiated timeout=10000
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.614385 30865 master.cpp:375] Master 3f6595b7-9c79-43d0-ac18-33bfc9d8ba1c (ip-10-27-9-130.REDACTED) started on 10.27.9.130:5050
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,621:30865(0x7feffa7fc700):ZOO_INFO@check_events@1703: initiated connection to server [10.27.9.130:2181]
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,621:30865(0x7feffa7fc700):ZOO_INFO@check_events@1750: session establishment complete on server [10.27.9.130:2181], sessionId=0x253e224e63101a1, negotiated timeout=10000
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,621:30865(0x7feffb7fe700):ZOO_INFO@check_events@1703: initiated connection to server [10.27.9.130:2181]
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: 2016-04-04 13:00:06,621:30865(0x7feffb7fe700):ZOO_INFO@check_events@1750: session establishment complete on server [10.27.9.130:2181], sessionId=0x253e224e63101a2, negotiated timeout=10000
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621036 30865 master.cpp:377] Flags at startup: --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate="false" --authenticate_http="false" --authenticate_slaves="false" --authenticators="crammd5" --authorizers="local" --cluster="mesos" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="1" --logging_level="INFO" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5" --port="5050" --quiet="false" --quorum="32752" --recovery_slave_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1mins" --registry_store_timeout="20secs" --registry_strict="false" --root_submissions="true" --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins" --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui" --work_dir="/var/lib/mesos/master" --zk="zk://ip-10-27-9-127.REDACTED:2181,ip-10-27-9-130.REDACTED:2181,ip-10-27-15-203.REDACTED:2181/mesos" --zk_session_timeout="10secs"
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621373 30865 master.cpp:424] Master allowing unauthenticated frameworks to register
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621381 30865 master.cpp:429] Master allowing unauthenticated slaves to register
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621387 30865 master.cpp:467] Using default 'crammd5' authenticator
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621387 30872 group.cpp:349] Group process (group(4)@10.27.9.130:5050) connected to ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: W0404 13:00:06.621395 30865 authenticator.cpp:511] No credentials provided, authentication requests will be refused
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621407 30872 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621417 30866 group.cpp:349] Group process (group(3)@10.27.9.130:5050) connected to ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621537 30872 group.cpp:427] Trying to create path '/mesos' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621428 30873 group.cpp:349] Group process (group(2)@10.27.9.130:5050) connected to ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621446 30870 group.cpp:349] Group process (group(1)@10.27.9.130:5050) connected to ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621567 30873 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (1, 0, 0)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621575 30873 group.cpp:427] Trying to create path '/mesos/log_replicas' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621520 30865 authenticator.cpp:518] Initializing server SASL
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621580 30870 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621593 30870 group.cpp:427] Trying to create path '/mesos/log_replicas' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621556 30866 group.cpp:831] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0)
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.621635 30866 group.cpp:427] Trying to create path '/mesos' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.622973 30869 detector.cpp:152] Detected a new leader: (id='8')
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.623044 30866 network.hpp:413] ZooKeeper group memberships changed
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.623067 30869 group.cpp:700] Trying to get '/mesos/json.info_0000000008' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.623153 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000007' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.623538 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000008' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.623836 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000009' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.624119 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000010' in ZooKeeper
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.624351 30873 detector.cpp:479] A new leading master (UPID=master@10.27.9.130:5050) is detected
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.624568 30868 network.hpp:461] ZooKeeper group PIDs: { log-replica(1)@10.27.9.127:5050, log-replica(1)@10.27.9.130:5050, log-replica(1)@10.27.15.203:5050 }
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.625387 30871 master.cpp:1650] Successfully attached file '/var/log/mesos/mesos-master.INFO'
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.625407 30871 master.cpp:1711] The newly elected leader is master@10.27.9.130:5050 with id a028a424-9ab0-40a9-b386-c4cf7f41dffd
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.625433 30872 contender.cpp:147] Joining the ZK group
Apr 04 13:00:06 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:06.626327 30872 contender.cpp:263] New candidate (id='11') has entered the contest for leadership
Apr 04 13:00:09 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:09.153292 30873 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59565 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.002646 30871 detector.cpp:152] Detected a new leader: (id='9')
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.002734 30871 group.cpp:700] Trying to get '/mesos/json.info_0000000009' in ZooKeeper
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.002873 30867 network.hpp:413] ZooKeeper group memberships changed
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.002934 30869 group.cpp:700] Trying to get '/mesos/log_replicas/0000000008' in ZooKeeper
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.003218 30869 group.cpp:700] Trying to get '/mesos/log_replicas/0000000009' in ZooKeeper
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.003259 30871 detector.cpp:479] A new leading master (UPID=master@10.27.15.203:5050) is detected
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.003305 30871 master.cpp:1711] The newly elected leader is master@10.27.15.203:5050 with id 33dccda1-6c0a-493d-a348-88ceaae3dda8
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.003486 30869 group.cpp:700] Trying to get '/mesos/log_replicas/0000000010' in ZooKeeper
Apr 04 13:00:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:14.003921 30871 network.hpp:461] ZooKeeper group PIDs: { log-replica(1)@10.27.9.127:5050, log-replica(1)@10.27.9.130:5050, log-replica(1)@10.27.15.203:5050 }
Apr 04 13:00:16 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:16.621575 30874 recover.cpp:109] Unable to finish the recover protocol in 10secs, retrying
Apr 04 13:00:19 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:00:19.185187 30871 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59565 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:01:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:14.554059 30869 network.hpp:413] ZooKeeper group memberships changed
Apr 04 13:01:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:14.554153 30867 group.cpp:700] Trying to get '/mesos/log_replicas/0000000008' in ZooKeeper
Apr 04 13:01:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:14.554445 30867 group.cpp:700] Trying to get '/mesos/log_replicas/0000000009' in ZooKeeper
Apr 04 13:01:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:14.554800 30867 group.cpp:700] Trying to get '/mesos/log_replicas/0000000010' in ZooKeeper
Apr 04 13:01:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:14.555308 30867 group.cpp:700] Trying to get '/mesos/log_replicas/0000000011' in ZooKeeper
Apr 04 13:01:14 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:14.555726 30873 network.hpp:461] ZooKeeper group PIDs: { log-replica(1)@10.27.9.127:5050, log-replica(1)@10.27.9.130:5050, log-replica(1)@10.27.15.203:5050 }
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.002324 30873 detector.cpp:152] Detected a new leader: (id='10')
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.002398 30867 group.cpp:700] Trying to get '/mesos/json.info_0000000010' in ZooKeeper
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.002516 30871 network.hpp:413] ZooKeeper group memberships changed
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.002591 30870 group.cpp:700] Trying to get '/mesos/log_replicas/0000000009' in ZooKeeper
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.002924 30872 detector.cpp:479] A new leading master (UPID=master@10.27.9.127:5050) is detected
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.002965 30872 master.cpp:1711] The newly elected leader is master@10.27.9.127:5050 with id 38c604b3-491f-4add-8881-5d1c52a54b2c
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.003057 30870 group.cpp:700] Trying to get '/mesos/log_replicas/0000000010' in ZooKeeper
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.003408 30870 group.cpp:700] Trying to get '/mesos/log_replicas/0000000011' in ZooKeeper
Apr 04 13:01:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:01:22.003998 30871 network.hpp:461] ZooKeeper group PIDs: { log-replica(1)@10.27.9.127:5050, log-replica(1)@10.27.9.130:5050, log-replica(1)@10.27.15.203:5050 }
Apr 04 13:02:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:22.338366 30866 network.hpp:413] ZooKeeper group memberships changed
Apr 04 13:02:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:22.338454 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000009' in ZooKeeper
Apr 04 13:02:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:22.338796 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000010' in ZooKeeper
Apr 04 13:02:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:22.339066 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000011' in ZooKeeper
Apr 04 13:02:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:22.339352 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000012' in ZooKeeper
Apr 04 13:02:22 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:22.339740 30872 network.hpp:461] ZooKeeper group PIDs: { log-replica(1)@10.27.9.127:5050, log-replica(1)@10.27.9.130:5050, log-replica(1)@10.27.15.203:5050 }
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002310 30868 detector.cpp:152] Detected a new leader: (id='11')
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002403 30869 group.cpp:700] Trying to get '/mesos/json.info_0000000011' in ZooKeeper
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002549 30870 network.hpp:413] ZooKeeper group memberships changed
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002585 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000010' in ZooKeeper
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002903 30869 detector.cpp:479] A new leading master (UPID=master@10.27.9.130:5050) is detected
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002957 30869 master.cpp:1711] The newly elected leader is master@10.27.9.130:5050 with id 3f6595b7-9c79-43d0-ac18-33bfc9d8ba1c
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002969 30869 master.cpp:1724] Elected as the leading master!
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.002977 30869 master.cpp:1469] Recovering from registrar
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.003119 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000011' in ZooKeeper
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.003202 30866 registrar.cpp:307] Recovering registrar
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.003556 30871 group.cpp:700] Trying to get '/mesos/log_replicas/0000000012' in ZooKeeper
Apr 04 13:02:30 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:30.003983 30871 network.hpp:461] ZooKeeper group PIDs: { log-replica(1)@10.27.9.127:5050, log-replica(1)@10.27.9.130:5050, log-replica(1)@10.27.15.203:5050 }
Apr 04 13:02:39 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:39.019999 30871 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59908 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:02:49 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:49.047642 30866 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59908 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:02:59 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:02:59.067562 30866 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59908 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:03:09 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:03:09.093200 30866 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59908 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:03:19 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:03:19.119928 30872 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59908 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:03:29 ip-10-27-9-130.REDACTED mesos-master[30865]: I0404 13:03:29.143762 30873 http.cpp:312] HTTP GET for /master/state from 10.30.86.132:59908 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0'
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: F0404 13:03:30.004353 30868 master.cpp:1458] Recovery failed: Failed to recover registrar: Failed to perform fetch within 1mins
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: *** Check failure stack trace: ***
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015bdfa5d google::LogMessage::Fail()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015be179c google::LogMessage::SendToLog()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015bdf64c google::LogMessage::Flush()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015be2099 google::LogMessageFatal::~LogMessageFatal()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff01540159c mesos::internal::master::fail()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0154410e0 _ZNSt17_Function_handlerIFvRKSsEZNK7process6FutureI7NothingE8onFailedISt5_BindIFPFvS1_S1_EPKcSt12_PlaceholderILi1EEEEvEERKS6_OT_NS6_6PreferEEUlS1_E_E9_M_invokeERKSt9_Any_dataS1_
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x4280fe process::Future<>::fail()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015466b85 process::internal::thenf<>()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0154b8776 _ZN7process8internal3runISt8functionIFvRKNS_6FutureIN5mesos8internal8RegistryEEEEEJRS7_EEEvRKSt6vectorIT_SaISE_EEDpOT0_
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0154c8ef6 process::Future<>::fail()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0152b3046 process::internal::run<>()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0154c8ee3 process::Future<>::fail()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0154ad5bc mesos::internal::master::RegistrarProcess::_recover()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015b8d4b1 process::ProcessManager::resume()
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff015b8d7b7 _ZNSt6thread5_ImplISt12_Bind_simpleIFSt5_BindIFZN7process14ProcessManager12init_threadsEvEUlRKSt11atomic_boolE_St17reference_wrapperIS6_EEEvEEE6_M_runEv
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff014499220 (unknown)
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff0146f3dc5 start_thread
Apr 04 13:03:30 ip-10-27-9-130.REDACTED mesos-master[30865]: @ 0x7ff013c0128d __clone
Apr 04 13:03:30 ip-10-27-9-130.REDACTED systemd[1]: mesos-master.service: main process exited, code=killed, status=6/ABRT
Apr 04 13:03:30 ip-10-27-9-130.REDACTED systemd[1]: Unit mesos-master.service entered failed state.
Apr 04 13:03:30 ip-10-27-9-130.REDACTED systemd[1]: mesos-master.service failed.
A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment