Skip to content

Instantly share code, notes, and snippets.

@sgoings

sgoings/gist:536c12009fb2beb4cdc3

Created October 23, 2015 19:36
Show Gist options
  • Save sgoings/536c12009fb2beb4cdc3 to your computer and use it in GitHub Desktop.
Save sgoings/536c12009fb2beb4cdc3 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
Installing Deis...
Storage subsystem...
deis-store-gateway@1.service: loaded
deis-store-volume.service: loadeddeis-store-metadata.service: loaded
deis-store-monitor.service: loadeddeis-store-daemon.service: loaded
Logging subsystem...deis-logspout.service: loaded
deis-logger.service: loadedControl plane...
deis-registry@1.service: loadeddeis-builder.service: loaded
deis-controller.service: loaded
deis-database.service: loadedData plane...
deis-publisher.service: loadedRouter mesh...
deis-router@1.service: loadeddeis-router@2.service: loaded
deis-router@3.service: loaded
Done.
Please run `deisctl start platform` to boot up Deis.
Starting Deis...
Storage subsystem...
deis-store-monitor.service: active/running
deis-store-daemon.service: active/running
deis-store-metadata.service: active/running
deis-store-gateway@1.service: active/running
deis-store-volume.service: active/running
Logging subsystem...
deis-logger.service: active/running
deis-logspout.service: active/running
Control plane...
deis-database.service: active/running
deis-registry@1.service: active/running
Could not find unit: deis-controller.service
Could not find unit: deis-builder.servicest
Data plane...
deis-publisher.service: active/running
Router mesh...
deis-router@3.service: active/running
deis-router@1.service: active/running
Could not find unit: deis-router@2.service
Done.
Please set up an administrative account. See 'deis help register'
Checking infrastructure on azure...
Warning: Permanently added '104.210.35.150' (RSA) to the list of known hosts.
==> etcdctl cluster-health
member 2e2f3d8c82c8c1f is healthy: got healthy result from http://10.0.0.4:2379
member ea33324a29df5c84 is unhealthy: got unhealthy result from http://10.0.0.5:2379
member f824c65b43bd39b9 is healthy: got healthy result from http://10.0.0.6:2379
cluster is healthy
==> fleetctl list-machines
MACHINE IP METADATA
2b6828f5... 104.210.32.83 controlPlane=true,dataPlane=true,routerMesh=true
2c69cbd3... 104.210.32.83 controlPlane=true,dataPlane=true,routerMesh=true
c78cebe2... 104.210.32.83 controlPlane=true,dataPlane=true,routerMesh=true
==> fleetctl list-units
UNIT MACHINE ACTIVE SUB
deis-database.service 2c69cbd3.../104.210.32.83 active running
deis-logger.service 2c69cbd3.../104.210.32.83 active running
deis-logspout.service 2b6828f5.../104.210.32.83 active running
deis-logspout.service 2c69cbd3.../104.210.32.83 active running
deis-publisher.service 2b6828f5.../104.210.32.83 active running
deis-publisher.service 2c69cbd3.../104.210.32.83 active running
deis-router@1.service 2b6828f5.../104.210.32.83 active running
deis-router@3.service 2c69cbd3.../104.210.32.83 active running
deis-store-daemon.service 2b6828f5.../104.210.32.83 active running
deis-store-daemon.service 2c69cbd3.../104.210.32.83 active running
deis-store-gateway@1.service 2c69cbd3.../104.210.32.83 active running
deis-store-metadata.service 2b6828f5.../104.210.32.83 active running
deis-store-metadata.service 2c69cbd3.../104.210.32.83 active running
deis-store-monitor.service 2b6828f5.../104.210.32.83 active running
deis-store-monitor.service 2c69cbd3.../104.210.32.83 active running
deis-store-volume.service 2b6828f5.../104.210.32.83 active running
deis-store-volume.service 2c69cbd3.../104.210.32.83 active running
==> journalctl -u deis-store-monitor
-- Logs begin at Fri 2015-10-23 16:24:26 UTC, end at Fri 2015-10-23 17:13:47 UTC. --
Oct 23 16:59:34 deisNode0 systemd[1]: Starting deis-store-monitor...
Oct 23 16:59:40 deisNode0 sh[1561]: 3.2: Pulling from alpine
Oct 23 16:59:40 deisNode0 sh[1561]: f4fddc471ec2: Pulling fs layer
Oct 23 16:59:42 deisNode0 sh[1561]: f4fddc471ec2: Verifying Checksum
Oct 23 16:59:42 deisNode0 sh[1561]: f4fddc471ec2: Download complete
Oct 23 16:59:42 deisNode0 sh[1561]: f4fddc471ec2: Pull complete
Oct 23 16:59:42 deisNode0 sh[1561]: Digest: sha256:b4769592c47ebc65f85999c61cf33dcdd5f04a7660c4225b454f875c57ef79fd
Oct 23 16:59:42 deisNode0 sh[1561]: Status: Downloaded newer image for alpine:3.2
Oct 23 16:59:46 deisNode0 sh[1687]: git-dbd178f: Pulling from deisci/store-monitor
Oct 23 16:59:46 deisNode0 sh[1687]: 095996d36002: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: cb54a9ba0f34: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: 57b9b827cb78: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: af67df0a9c47: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: bcdbb77dad24: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: 62633914cc0b: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: 9038dc68bd75: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: ef3ecec48a62: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: 834dedadfcfa: Pulling fs layer
Oct 23 16:59:46 deisNode0 sh[1687]: f8ca1e93cc8b: Pulling fs layer
Oct 23 16:59:47 deisNode0 sh[1687]: cb54a9ba0f34: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: cb54a9ba0f34: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: 57b9b827cb78: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: 57b9b827cb78: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: 9038dc68bd75: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: 9038dc68bd75: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: ef3ecec48a62: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: ef3ecec48a62: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: 62633914cc0b: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: 62633914cc0b: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: af67df0a9c47: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: af67df0a9c47: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: f8ca1e93cc8b: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: f8ca1e93cc8b: Download complete
Oct 23 16:59:47 deisNode0 sh[1687]: 834dedadfcfa: Verifying Checksum
Oct 23 16:59:47 deisNode0 sh[1687]: 834dedadfcfa: Download complete
Oct 23 16:59:50 deisNode0 sh[1687]: 095996d36002: Verifying Checksum
Oct 23 16:59:50 deisNode0 sh[1687]: 095996d36002: Download complete
Oct 23 16:59:54 deisNode0 sh[1687]: 095996d36002: Pull complete
Oct 23 16:59:54 deisNode0 sh[1687]: cb54a9ba0f34: Pull complete
Oct 23 16:59:55 deisNode0 sh[1687]: 57b9b827cb78: Pull complete
Oct 23 16:59:55 deisNode0 sh[1687]: af67df0a9c47: Pull complete
Oct 23 17:00:27 deisNode0 sh[1687]: bcdbb77dad24: Verifying Checksum
Oct 23 17:00:27 deisNode0 sh[1687]: bcdbb77dad24: Download complete
Oct 23 17:00:37 deisNode0 sh[1687]: bcdbb77dad24: Pull complete
Oct 23 17:00:41 deisNode0 sh[1687]: 62633914cc0b: Pull complete
Oct 23 17:00:41 deisNode0 sh[1687]: 9038dc68bd75: Pull complete
Oct 23 17:00:42 deisNode0 sh[1687]: ef3ecec48a62: Pull complete
Oct 23 17:00:42 deisNode0 sh[1687]: 834dedadfcfa: Pull complete
Oct 23 17:00:43 deisNode0 sh[1687]: f8ca1e93cc8b: Pull complete
Oct 23 17:00:43 deisNode0 sh[1687]: Digest: sha256:fa64170007a937f08f497abbcb77282185469c91bfc16f95aa138cd066d6051e
Oct 23 17:00:43 deisNode0 sh[1687]: Status: Downloaded newer image for deisci/store-monitor:git-dbd178f
Oct 23 17:00:47 deisNode0 systemd[1]: Started deis-store-monitor.
Oct 23 17:00:53 deisNode0 sh[1782]: 2015-10-23 17:00:53.486984 7f7808285700 0 -- :/1000055 >> 10.0.0.6:6789/0 pipe(0x7f7800064010 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f780005c560).fault
Oct 23 17:00:53 deisNode0 sh[1782]: 2015-10-23 17:00:53.486984 7f7808285700 0 -- :/1000055 >> 10.0.0.6:6789/0 pipe(0x7f7800064010 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f780005c560).fault
Oct 23 17:00:56 deisNode0 sh[1782]: got monmap epoch 1
Oct 23 17:00:56 deisNode0 sh[1782]: creating /tmp/ceph.mon.keyring
Oct 23 17:00:56 deisNode0 sh[1782]: importing contents of /etc/ceph/ceph.client.admin.keyring into /tmp/ceph.mon.keyring
Oct 23 17:00:56 deisNode0 sh[1782]: importing contents of /etc/ceph/ceph.mon.keyring into /tmp/ceph.mon.keyring
Oct 23 17:00:56 deisNode0 sh[1782]: ceph-mon: set fsid to 09a4a2c2-fc49-457b-bad7-605312ac3221
Oct 23 17:00:57 deisNode0 sh[1782]: ceph-mon: created monfs at /var/lib/ceph/mon/ceph-deisNode0 for mon.deisNode0
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.082111 7f33f8a568c0 0 ceph version 0.94.4 (95292699291242794510b39ffde3f4df67898d3a), process ceph-mon, pid 1
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.208901 7f33f8a568c0 0 mon.deisNode0 does not exist in monmap, will attempt to join an existing cluster
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.209379 7f33f8a568c0 0 using public_addr 10.0.0.4:6789/0 -> 10.0.0.4:6789/0
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.209450 7f33f8a568c0 0 starting mon.deisNode0 rank -1 at 10.0.0.4:6789/0 mon_data /var/lib/ceph/mon/ceph-deisNode0 fsid 09a4a2c2-fc49-457b-bad7-605312ac3221
Oct 23 17:00:57 deisNode0 sh[1782]: starting mon.deisNode0 rank -1 at 10.0.0.4:6789/0 mon_data /var/lib/ceph/mon/ceph-deisNode0 fsid 09a4a2c2-fc49-457b-bad7-605312ac3221
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.209789 7f33f8a568c0 1 mon.deisNode0@-1(probing) e0 preinit fsid 09a4a2c2-fc49-457b-bad7-605312ac3221
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.209875 7f33f8a568c0 1 mon.deisNode0@-1(probing) e0 initial_members deisNode1, filtering seed monmap
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.376898 7f33ef24e700 0 -- 10.0.0.4:6789/0 >> 10.0.0.5:6789/0 pipe(0x5284000 sd=11 :51643 s=2 pgs=2 cs=1 l=0 c=0x50df4a0).fault, initiating reconnect
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.379846 7f33ef14d700 0 -- 10.0.0.4:6789/0 >> 10.0.0.5:6789/0 pipe(0x5289000 sd=20 :6789 s=0 pgs=0 cs=0 l=0 c=0x50e07e0).accept connect_seq 0 vs existing 2 state connecting
Oct 23 17:00:57 deisNode0 sh[1782]: 2015-10-23 17:00:57.379881 7f33ef14d700 0 -- 10.0.0.4:6789/0 >> 10.0.0.5:6789/0 pipe(0x5289000 sd=20 :6789 s=0 pgs=0 cs=0 l=0 c=0x50e07e0).accept peer reset, then tried to connect to us, replacing
Oct 23 17:00:59 deisNode0 sh[1782]: 2015-10-23 17:00:59.258283 7f33f1252700 0 mon.deisNode0@-1(probing) e2 my rank is now 0 (was -1)
Oct 23 17:00:59 deisNode0 sh[1782]: 2015-10-23 17:00:59.261540 7f33ef14d700 0 -- 10.0.0.4:6789/0 >> 10.0.0.5:6789/0 pipe(0x5289000 sd=20 :6789 s=0 pgs=0 cs=0 l=0 c=0x50e0940).accept connect_seq 2 vs existing 0 state connecting
Oct 23 17:00:59 deisNode0 sh[1782]: 2015-10-23 17:00:59.261575 7f33ef14d700 0 -- 10.0.0.4:6789/0 >> 10.0.0.5:6789/0 pipe(0x5289000 sd=20 :6789 s=0 pgs=0 cs=0 l=0 c=0x50e0940).accept we reset (peer sent cseq 2, 0x5284000.cseq = 0), sending RESETSESSION
Oct 23 17:00:59 deisNode0 sh[1782]: 2015-10-23 17:00:59.303004 7f33ef14d700 0 -- 10.0.0.4:6789/0 >> 10.0.0.5:6789/0 pipe(0x5284000 sd=11 :51654 s=2 pgs=7 cs=1 l=0 c=0x50e07e0).reader missed message? skipped from seq 0 to 1369843742
Oct 23 17:00:59 deisNode0 sh[1782]: 2015-10-23 17:00:59.303366 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:00:59 deisNode0 sh[1782]: 2015-10-23 17:00:59.303538 7f33f1252700 1 mon.deisNode0@0(electing).elector(1) init, last seen epoch 1
Oct 23 17:01:02 deisNode0 sh[1782]: 2015-10-23 17:01:02.524521 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:01:02 deisNode0 sh[1782]: 2015-10-23 17:01:02.524674 7f33f1252700 1 mon.deisNode0@0(electing).elector(3) init, last seen epoch 3
Oct 23 17:01:02 deisNode0 sh[1782]: 2015-10-23 17:01:02.610006 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,1
Oct 23 17:01:02 deisNode0 sh[1782]: 2015-10-23 17:01:02.664853 7f33f1252700 1 mon.deisNode0@0(leader) e2 apply_quorum_to_compatset_features enabling new quorum features: compat={},rocompat={},incompat={4=support erasure code pools,5=new-style osdmap encoding,6=support isa/lrc erasure code}
Oct 23 17:01:02 deisNode0 sh[1782]: 2015-10-23 17:01:02.813205 7f33f1252700 0 log_channel(cluster) log [INF] : HEALTH_ERR; no osds
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.180571 7f33f1252700 1 mon.deisNode0@0(leader).paxosservice(pgmap 1..2) refresh upgraded, format 0 -> 1
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.180595 7f33f1252700 1 mon.deisNode0@0(leader).pg v0 on_upgrade discarding in-core PGMap
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.181323 7f33f1252700 0 mon.deisNode0@0(leader).mds e1 print_map
Oct 23 17:01:03 deisNode0 sh[1782]: epoch 1
Oct 23 17:01:03 deisNode0 sh[1782]: flags 0
Oct 23 17:01:03 deisNode0 sh[1782]: created 0.000000
Oct 23 17:01:03 deisNode0 sh[1782]: modified 2015-10-23 17:00:53.341228
Oct 23 17:01:03 deisNode0 sh[1782]: tableserver 0
Oct 23 17:01:03 deisNode0 sh[1782]: root 0
Oct 23 17:01:03 deisNode0 sh[1782]: session_timeout 0
Oct 23 17:01:03 deisNode0 sh[1782]: session_autoclose 0
Oct 23 17:01:03 deisNode0 sh[1782]: max_file_size 0
Oct 23 17:01:03 deisNode0 sh[1782]: last_failure 0
Oct 23 17:01:03 deisNode0 sh[1782]: last_failure_osd_epoch 0
Oct 23 17:01:03 deisNode0 sh[1782]: compat compat={},rocompat={},incompat={}
Oct 23 17:01:03 deisNode0 sh[1782]: max_mds 0
Oct 23 17:01:03 deisNode0 sh[1782]: in
Oct 23 17:01:03 deisNode0 sh[1782]: up {}
Oct 23 17:01:03 deisNode0 sh[1782]: failed
Oct 23 17:01:03 deisNode0 sh[1782]: stopped
Oct 23 17:01:03 deisNode0 sh[1782]: data_pools
Oct 23 17:01:03 deisNode0 sh[1782]: metadata_pool 0
Oct 23 17:01:03 deisNode0 sh[1782]: inline_data disabled
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.181481 7f33f1252700 1 mon.deisNode0@0(leader).osd e1 e1: 0 osds: 0 up, 0 in
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.232345 7f33f1252700 0 mon.deisNode0@0(leader).osd e1 crush map has features 1107558400, adjusting msgr requires
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.232384 7f33f1252700 0 mon.deisNode0@0(leader).osd e1 crush map has features 1107558400, adjusting msgr requires
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.232391 7f33f1252700 0 mon.deisNode0@0(leader).osd e1 crush map has features 1107558400, adjusting msgr requires
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.232395 7f33f1252700 0 mon.deisNode0@0(leader).osd e1 crush map has features 1107558400, adjusting msgr requires
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.232892 7f33f1252700 1 mon.deisNode0@0(leader).paxosservice(auth 1..2) refresh upgraded, format 0 -> 1
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.309710 7f33f1252700 0 log_channel(cluster) log [INF] : monmap e2: 2 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0}
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.309876 7f33f1252700 0 log_channel(cluster) log [INF] : pgmap v2: 64 pgs: 64 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.309947 7f33f1252700 0 log_channel(cluster) log [INF] : mdsmap e1: 0/0/0 up
Oct 23 17:01:03 deisNode0 sh[1782]: 2015-10-23 17:01:03.310070 7f33f1252700 0 log_channel(cluster) log [INF] : osdmap e1: 0 osds: 0 up, 0 in
Oct 23 17:01:13 deisNode0 sh[1782]: 2015-10-23 17:01:13.923816 7f33f1252700 0 mon.deisNode0@0(leader) e2 handle_command mon_command({"prefix": "osd create"} v 0) v1
Oct 23 17:01:13 deisNode0 sh[1782]: 2015-10-23 17:01:13.923916 7f33f1252700 0 log_channel(audit) log [INF] : from='client.4101 :/0' entity='client.admin' cmd=[{"prefix": "osd create"}]: dispatch
Oct 23 17:01:14 deisNode0 sh[1782]: 2015-10-23 17:01:14.058541 7f33f1252700 1 mon.deisNode0@0(leader) e2 adding peer 10.0.0.6:6789/0 to list of hints
Oct 23 17:01:14 deisNode0 sh[1782]: 2015-10-23 17:01:14.059502 7f33f1252700 1 mon.deisNode0@0(leader) e2 adding peer 10.0.0.6:6789/0 to list of hints
Oct 23 17:01:14 deisNode0 sh[1782]: 2015-10-23 17:01:14.059607 7f33f1252700 1 mon.deisNode0@0(leader) e2 adding peer 10.0.0.6:6789/0 to list of hints
Oct 23 17:01:14 deisNode0 sh[1782]: 2015-10-23 17:01:14.186238 7f33f2a55700 1 mon.deisNode0@0(leader).osd e2 e2: 1 osds: 0 up, 0 in
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.392309 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.4101 :/0' entity='client.admin' cmd='[{"prefix": "osd create"}]': finished
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.432634 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e2: 1 osds: 0 up, 0 in
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.432832 7f33f1252700 1 mon.deisNode0@0(leader) e2 adding peer 10.0.0.6:6789/0 to list of hints
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.432896 7f33f1252700 1 mon.deisNode0@0(leader) e2 adding peer 10.0.0.6:6789/0 to list of hints
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.433007 7f33f1252700 0 mon.deisNode0@0(leader).monmap v2 adding/updating deisNode2 at 10.0.0.6:6789/0 to monitor cluster
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.510898 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v3: 64 pgs: 64 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.746162 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:01:15 deisNode0 sh[1782]: 2015-10-23 17:01:15.746347 7f33f1252700 1 mon.deisNode0@0(electing).elector(4) init, last seen epoch 4
Oct 23 17:01:16 deisNode0 sh[1782]: 2015-10-23 17:01:16.265604 7f33ef04c700 0 -- 10.0.0.4:6789/0 >> 10.0.0.6:6789/0 pipe(0x5289000 sd=20 :6789 s=2 pgs=2 cs=1 l=0 c=0x50e15a0).fault, initiating reconnect
Oct 23 17:01:16 deisNode0 sh[1782]: 2015-10-23 17:01:16.268460 7f33eef4b700 0 -- 10.0.0.4:6789/0 >> 10.0.0.6:6789/0 pipe(0x5369000 sd=21 :6789 s=0 pgs=0 cs=0 l=0 c=0x50e1440).accept connect_seq 0 vs existing 2 state connecting
Oct 23 17:01:16 deisNode0 sh[1782]: 2015-10-23 17:01:16.268530 7f33eef4b700 0 -- 10.0.0.4:6789/0 >> 10.0.0.6:6789/0 pipe(0x5369000 sd=21 :6789 s=0 pgs=0 cs=0 l=0 c=0x50e1440).accept peer reset, then tried to connect to us, replacing
Oct 23 17:01:20 deisNode0 sh[1782]: 2015-10-23 17:01:20.917181 7f33f1a53700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,1
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:20.982708 7f33f1a53700 0 log_channel(cluster) log [INF] : HEALTH_WARN; 64 pgs stuck inactive; 64 pgs stuck unclean; 1 mons down, quorum 0,1 deisNode0,deisNode1
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.071778 7f33f1252700 0 log_channel(cluster) log [INF] : monmap e3: 3 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0,deisNode2=10.0.0.6:6789/0}
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.071906 7f33f1252700 0 log_channel(cluster) log [INF] : pgmap v3: 64 pgs: 64 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.071972 7f33f1252700 0 log_channel(cluster) log [INF] : mdsmap e1: 0/0/0 up
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.072036 7f33f1252700 0 log_channel(cluster) log [INF] : osdmap e2: 1 osds: 0 up, 0 in
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.105310 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd create"} v 0) v1
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.105389 7f33f1252700 0 log_channel(audit) log [INF] : from='client.4103 :/0' entity='client.admin' cmd=[{"prefix": "osd create"}]: dispatch
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.268886 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd create"} v 0) v1
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.268934 7f33f1252700 0 log_channel(audit) log [INF] : from='client.4103 :/0' entity='client.admin' cmd=[{"prefix": "osd create"}]: dispatch
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.270311 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.270483 7f33f1252700 1 mon.deisNode0@0(electing).elector(6) init, last seen epoch 6
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.411230 7f33f1252700 0 mon.deisNode0@0(electing) e3 handle_command mon_command({"prefix": "auth get-or-create", "entity": "osd.0", "caps": ["osd", "allow *", "mon", "allow profile osd"]} v 0) v1
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.411283 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.5:0/1000076' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "osd.0", "caps": ["osd", "allow *", "mon", "allow profile osd"]}]: dispatch
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.479475 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,1,2
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.573606 7f33f1252700 0 log_channel(cluster) log [INF] : HEALTH_WARN; 64 pgs stuck inactive; 64 pgs stuck unclean
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.574009 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd create"} v 0) v1
Oct 23 17:01:21 deisNode0 sh[1782]: 2015-10-23 17:01:21.574058 7f33f1252700 0 log_channel(audit) log [INF] : from='client.4103 :/0' entity='unknown.' cmd=[{"prefix": "osd create"}]: dispatch
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.453036 7f33f2a55700 1 mon.deisNode0@0(leader).osd e3 e3: 2 osds: 0 up, 0 in
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.495707 7f33f2a55700 0 log_channel(cluster) log [INF] : monmap e3: 3 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0,deisNode2=10.0.0.6:6789/0}
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.541966 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v3: 64 pgs: 64 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.542065 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e1: 0/0/0 up
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.542139 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e3: 2 osds: 0 up, 0 in
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.754164 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd lspools"} v 0) v1
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.754213 7f33f1252700 0 log_channel(audit) log [DBG] : from='client.? 10.0.0.4:0/1000023' entity='client.admin' cmd=[{"prefix": "osd lspools"}]: dispatch
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.857580 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v4: 64 pgs: 64 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.902448 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd create"} v 0) v1
Oct 23 17:01:22 deisNode0 sh[1782]: 2015-10-23 17:01:22.902500 7f33f1252700 0 log_channel(audit) log [INF] : from='client.4105 :/0' entity='client.admin' cmd=[{"prefix": "osd create"}]: dispatch
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.010298 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.5:0/1000076' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "osd.0", "caps": ["osd", "allow *", "mon", "allow profile osd"]}]': finished
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.235167 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd pool create", "pg_num": 64, "pool": "data"} v 0) v1
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.235218 7f33f1252700 0 log_channel(audit) log [INF] : from='client.4108 :/0' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 64, "pool": "data"}]: dispatch
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.385909 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd crush add", "args": ["root=default", "host=deisNode1"], "id": 0, "weight": 1.0} v 0) v1
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.385998 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.5:0/1000107' entity='client.admin' cmd=[{"prefix": "osd crush add", "args": ["root=default", "host=deisNode1"], "id": 0, "weight": 1.0}]: dispatch
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.700450 7f33f2a55700 1 mon.deisNode0@0(leader).osd e4 e4: 4 osds: 0 up, 0 in
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.748374 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.4103 :/0' entity='unknown.' cmd='[{"prefix": "osd create"}]': finished
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.748578 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.4105 :/0' entity='client.admin' cmd='[{"prefix": "osd create"}]': finished
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.748727 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.4108 :/0' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 64, "pool": "data"}]': finished
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.748851 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.5:0/1000107' entity='client.admin' cmd='[{"prefix": "osd crush add", "args": ["root=default", "host=deisNode1"], "id": 0, "weight": 1.0}]': finished
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.777645 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e4: 4 osds: 0 up, 0 in
Oct 23 17:01:23 deisNode0 sh[1782]: 2015-10-23 17:01:23.909914 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v5: 128 pgs: 128 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:24 deisNode0 sh[1782]: 2015-10-23 17:01:24.523847 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd pool create", "pg_num": 64, "pool": "metadata"} v 0) v1
Oct 23 17:01:24 deisNode0 sh[1782]: 2015-10-23 17:01:24.523915 7f33f1252700 0 log_channel(audit) log [INF] : from='client.14102 :/0' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 64, "pool": "metadata"}]: dispatch
Oct 23 17:01:25 deisNode0 sh[1782]: 2015-10-23 17:01:25.194145 7f33f2a55700 1 mon.deisNode0@0(leader).osd e5 e5: 4 osds: 0 up, 0 in
Oct 23 17:01:25 deisNode0 sh[1782]: 2015-10-23 17:01:25.630019 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.14102 :/0' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 64, "pool": "metadata"}]': finished
Oct 23 17:01:25 deisNode0 sh[1782]: 2015-10-23 17:01:25.669300 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e5: 4 osds: 0 up, 0 in
Oct 23 17:01:26 deisNode0 sh[1782]: 2015-10-23 17:01:26.667402 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "auth get-or-create", "entity": "osd.2", "caps": ["osd", "allow *", "mon", "allow profile osd"]} v 0) v1
Oct 23 17:01:26 deisNode0 sh[1782]: 2015-10-23 17:01:26.667457 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.4:0/1000076' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "osd.2", "caps": ["osd", "allow *", "mon", "allow profile osd"]}]: dispatch
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.444976 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v6: 192 pgs: 192 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.610383 7f33f2a55700 1 mon.deisNode0@0(leader).osd e6 e6: 4 osds: 1 up, 1 in
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.644672 7f33f2a55700 0 log_channel(cluster) log [INF] : osd.0 10.0.0.5:6800/1 boot
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.673350 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e6: 4 osds: 1 up, 1 in
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.673441 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.4:0/1000076' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "osd.2", "caps": ["osd", "allow *", "mon", "allow profile osd"]}]': finished
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.799351 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v7: 192 pgs: 192 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.966676 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd crush add", "args": ["root=default", "host=deisNode0"], "id": 2, "weight": 1.0} v 0) v1
Oct 23 17:01:27 deisNode0 sh[1782]: 2015-10-23 17:01:27.966788 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.4:0/1000107' entity='client.admin' cmd=[{"prefix": "osd crush add", "args": ["root=default", "host=deisNode0"], "id": 2, "weight": 1.0}]: dispatch
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.291774 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "fs new", "data": "data", "fs_name": "deis", "metadata": "metadata"} v 0) v1
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.291835 7f33f1252700 0 log_channel(audit) log [INF] : from='client.14108 :/0' entity='client.admin' cmd=[{"prefix": "fs new", "data": "data", "fs_name": "deis", "metadata": "metadata"}]: dispatch
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.440290 7f33f2a55700 1 mon.deisNode0@0(leader).osd e7 e7: 4 osds: 1 up, 1 in
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.475826 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.4:0/1000107' entity='client.admin' cmd='[{"prefix": "osd crush add", "args": ["root=default", "host=deisNode0"], "id": 2, "weight": 1.0}]': finished
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.502148 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e7: 4 osds: 1 up, 1 in
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.627816 7f33f2a55700 0 mon.deisNode0@0(leader).mds e2 print_map
Oct 23 17:01:28 deisNode0 sh[1782]: epoch 2
Oct 23 17:01:28 deisNode0 sh[1782]: flags 0
Oct 23 17:01:28 deisNode0 sh[1782]: created 2015-10-23 17:01:28.362135
Oct 23 17:01:28 deisNode0 sh[1782]: modified 2015-10-23 17:01:28.362185
Oct 23 17:01:28 deisNode0 sh[1782]: tableserver 0
Oct 23 17:01:28 deisNode0 sh[1782]: root 0
Oct 23 17:01:28 deisNode0 sh[1782]: session_timeout 60
Oct 23 17:01:28 deisNode0 sh[1782]: session_autoclose 300
Oct 23 17:01:28 deisNode0 sh[1782]: max_file_size 1099511627776
Oct 23 17:01:28 deisNode0 sh[1782]: last_failure 0
Oct 23 17:01:28 deisNode0 sh[1782]: last_failure_osd_epoch 0
Oct 23 17:01:28 deisNode0 sh[1782]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
Oct 23 17:01:28 deisNode0 sh[1782]: max_mds 1
Oct 23 17:01:28 deisNode0 sh[1782]: in
Oct 23 17:01:28 deisNode0 sh[1782]: up {}
Oct 23 17:01:28 deisNode0 sh[1782]: failed
Oct 23 17:01:28 deisNode0 sh[1782]: stopped
Oct 23 17:01:28 deisNode0 sh[1782]: data_pools 1
Oct 23 17:01:28 deisNode0 sh[1782]: metadata_pool 2
Oct 23 17:01:28 deisNode0 sh[1782]: inline_data disabled
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.628012 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.14108 :/0' entity='client.admin' cmd='[{"prefix": "fs new", "data": "data", "fs_name": "deis", "metadata": "metadata"}]': finished
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.628089 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e2: 0/0/1 up
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.628155 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v8: 192 pgs: 192 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.674562 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "auth get-or-create", "entity": "osd.3", "caps": ["osd", "allow *", "mon", "allow profile osd"]} v 0) v1
Oct 23 17:01:28 deisNode0 sh[1782]: 2015-10-23 17:01:28.674613 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.6:0/1000081' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "osd.3", "caps": ["osd", "allow *", "mon", "allow profile osd"]}]: dispatch
Oct 23 17:01:29 deisNode0 sh[1782]: 2015-10-23 17:01:29.002536 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.6:0/1000081' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "osd.3", "caps": ["osd", "allow *", "mon", "allow profile osd"]}]': finished
Oct 23 17:01:29 deisNode0 sh[1782]: 2015-10-23 17:01:29.210338 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "auth get-or-create", "entity": "mds.deisNode0", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]} v 0) v1
Oct 23 17:01:29 deisNode0 sh[1782]: 2015-10-23 17:01:29.210425 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.4:0/1000219' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "mds.deisNode0", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]}]: dispatch
Oct 23 17:01:29 deisNode0 sh[1782]: 2015-10-23 17:01:29.346118 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.4:0/1000219' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "mds.deisNode0", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]}]': finished
Oct 23 17:01:29 deisNode0 sh[1782]: 2015-10-23 17:01:29.410722 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "osd crush add", "args": ["root=default", "host=deisNode2"], "id": 3, "weight": 1.0} v 0) v1
Oct 23 17:01:29 deisNode0 sh[1782]: 2015-10-23 17:01:29.410872 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.6:0/1000112' entity='client.admin' cmd=[{"prefix": "osd crush add", "args": ["root=default", "host=deisNode2"], "id": 3, "weight": 1.0}]: dispatch
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.143127 7f33f2a55700 1 mon.deisNode0@0(leader).osd e8 e8: 4 osds: 2 up, 2 in
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.178113 7f33f2a55700 0 log_channel(cluster) log [INF] : osd.2 10.0.0.4:6800/1 boot
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.178208 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.6:0/1000112' entity='client.admin' cmd='[{"prefix": "osd crush add", "args": ["root=default", "host=deisNode2"], "id": 3, "weight": 1.0}]': finished
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.206452 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e8: 4 osds: 2 up, 2 in
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.246469 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "auth get-or-create", "entity": "mds.deisNode2", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]} v 0) v1
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.246505 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.6:0/1000033' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "mds.deisNode2", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]}]: dispatch
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.346088 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v9: 192 pgs: 192 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:30 deisNode0 sh[1782]: 2015-10-23 17:01:30.955719 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.6:0/1000033' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "mds.deisNode2", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]}]': finished
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.143058 7f33f2a55700 0 mon.deisNode0@0(leader).mds e3 print_map
Oct 23 17:01:31 deisNode0 sh[1782]: epoch 3
Oct 23 17:01:31 deisNode0 sh[1782]: flags 0
Oct 23 17:01:31 deisNode0 sh[1782]: created 2015-10-23 17:01:28.362135
Oct 23 17:01:31 deisNode0 sh[1782]: modified 2015-10-23 17:01:30.392818
Oct 23 17:01:31 deisNode0 sh[1782]: tableserver 0
Oct 23 17:01:31 deisNode0 sh[1782]: root 0
Oct 23 17:01:31 deisNode0 sh[1782]: session_timeout 60
Oct 23 17:01:31 deisNode0 sh[1782]: session_autoclose 300
Oct 23 17:01:31 deisNode0 sh[1782]: max_file_size 1099511627776
Oct 23 17:01:31 deisNode0 sh[1782]: last_failure 0
Oct 23 17:01:31 deisNode0 sh[1782]: last_failure_osd_epoch 0
Oct 23 17:01:31 deisNode0 sh[1782]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
Oct 23 17:01:31 deisNode0 sh[1782]: max_mds 1
Oct 23 17:01:31 deisNode0 sh[1782]: in
Oct 23 17:01:31 deisNode0 sh[1782]: up {}
Oct 23 17:01:31 deisNode0 sh[1782]: failed
Oct 23 17:01:31 deisNode0 sh[1782]: stopped
Oct 23 17:01:31 deisNode0 sh[1782]: data_pools 1
Oct 23 17:01:31 deisNode0 sh[1782]: metadata_pool 2
Oct 23 17:01:31 deisNode0 sh[1782]: inline_data disabled
Oct 23 17:01:31 deisNode0 sh[1782]: 14127: 10.0.0.4:6804/1 'deisNode0' mds.-1.0 up:standby seq 1
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.143368 7f33f2a55700 0 log_channel(cluster) log [INF] : mds.? 10.0.0.4:6804/1 up:boot
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.143465 7f33f2a55700 1 mon.deisNode0@0(leader).mds e3 adding standby 10.0.0.6:6804/1 as mds.0
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.174338 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e3: 0/0/1 up, 1 up:standby
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.580690 7f33f2a55700 0 mon.deisNode0@0(leader).mds e4 print_map
Oct 23 17:01:31 deisNode0 sh[1782]: epoch 4
Oct 23 17:01:31 deisNode0 sh[1782]: flags 0
Oct 23 17:01:31 deisNode0 sh[1782]: created 2015-10-23 17:01:28.362135
Oct 23 17:01:31 deisNode0 sh[1782]: modified 2015-10-23 17:01:31.143489
Oct 23 17:01:31 deisNode0 sh[1782]: tableserver 0
Oct 23 17:01:31 deisNode0 sh[1782]: root 0
Oct 23 17:01:31 deisNode0 sh[1782]: session_timeout 60
Oct 23 17:01:31 deisNode0 sh[1782]: session_autoclose 300
Oct 23 17:01:31 deisNode0 sh[1782]: max_file_size 1099511627776
Oct 23 17:01:31 deisNode0 sh[1782]: last_failure 0
Oct 23 17:01:31 deisNode0 sh[1782]: last_failure_osd_epoch 0
Oct 23 17:01:31 deisNode0 sh[1782]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
Oct 23 17:01:31 deisNode0 sh[1782]: max_mds 1
Oct 23 17:01:31 deisNode0 sh[1782]: in 0
Oct 23 17:01:31 deisNode0 sh[1782]: up {0=14136}
Oct 23 17:01:31 deisNode0 sh[1782]: failed
Oct 23 17:01:31 deisNode0 sh[1782]: stopped
Oct 23 17:01:31 deisNode0 sh[1782]: data_pools 1
Oct 23 17:01:31 deisNode0 sh[1782]: metadata_pool 2
Oct 23 17:01:31 deisNode0 sh[1782]: inline_data disabled
Oct 23 17:01:31 deisNode0 sh[1782]: 14127: 10.0.0.4:6804/1 'deisNode0' mds.-1.0 up:standby seq 1
Oct 23 17:01:31 deisNode0 sh[1782]: 14136: 10.0.0.6:6804/1 'deisNode2' mds.0.1 up:creating seq 1
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.581034 7f33f2a55700 0 log_channel(cluster) log [INF] : mds.? 10.0.0.6:6804/1 up:boot
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.581109 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e4: 1/1/1 up {0=deisNode2=up:creating}, 1 up:standby
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.799341 7f33f2a55700 1 mon.deisNode0@0(leader).osd e9 e9: 4 osds: 3 up, 3 in
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.834838 7f33f2a55700 0 log_channel(cluster) log [INF] : osd.3 10.0.0.6:6800/1 boot
Oct 23 17:01:31 deisNode0 sh[1782]: 2015-10-23 17:01:31.861710 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e9: 4 osds: 3 up, 3 in
Oct 23 17:01:32 deisNode0 sh[1782]: 2015-10-23 17:01:32.066587 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v10: 192 pgs: 192 creating; 0 bytes data, 0 kB used, 0 kB / 0 kB avail
Oct 23 17:01:33 deisNode0 sh[1782]: 2015-10-23 17:01:33.037813 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "auth get-or-create", "entity": "mds.deisNode1", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]} v 0) v1
Oct 23 17:01:33 deisNode0 sh[1782]: 2015-10-23 17:01:33.037865 7f33f1252700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.5:0/1000038' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "mds.deisNode1", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]}]: dispatch
Oct 23 17:01:33 deisNode0 sh[1782]: 2015-10-23 17:01:33.737373 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.? 10.0.0.5:0/1000038' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "mds.deisNode1", "caps": ["mds", "allow", "osd", "allow *", "mon", "allow profile mds"]}]': finished
Oct 23 17:01:34 deisNode0 sh[1782]: 2015-10-23 17:01:34.851005 7f33f2a55700 0 mon.deisNode0@0(leader).mds e5 print_map
Oct 23 17:01:34 deisNode0 sh[1782]: epoch 5
Oct 23 17:01:34 deisNode0 sh[1782]: flags 0
Oct 23 17:01:34 deisNode0 sh[1782]: created 2015-10-23 17:01:28.362135
Oct 23 17:01:34 deisNode0 sh[1782]: modified 2015-10-23 17:01:34.737191
Oct 23 17:01:34 deisNode0 sh[1782]: tableserver 0
Oct 23 17:01:34 deisNode0 sh[1782]: root 0
Oct 23 17:01:34 deisNode0 sh[1782]: session_timeout 60
Oct 23 17:01:34 deisNode0 sh[1782]: session_autoclose 300
Oct 23 17:01:34 deisNode0 sh[1782]: max_file_size 1099511627776
Oct 23 17:01:34 deisNode0 sh[1782]: last_failure 0
Oct 23 17:01:34 deisNode0 sh[1782]: last_failure_osd_epoch 0
Oct 23 17:01:34 deisNode0 sh[1782]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
Oct 23 17:01:34 deisNode0 sh[1782]: max_mds 1
Oct 23 17:01:34 deisNode0 sh[1782]: in 0
Oct 23 17:01:34 deisNode0 sh[1782]: up {0=14136}
Oct 23 17:01:34 deisNode0 sh[1782]: failed
Oct 23 17:01:34 deisNode0 sh[1782]: stopped
Oct 23 17:01:34 deisNode0 sh[1782]: data_pools 1
Oct 23 17:01:34 deisNode0 sh[1782]: metadata_pool 2
Oct 23 17:01:34 deisNode0 sh[1782]: inline_data disabled
Oct 23 17:01:34 deisNode0 sh[1782]: 14127: 10.0.0.4:6804/1 'deisNode0' mds.-1.0 up:standby seq 1
Oct 23 17:01:34 deisNode0 sh[1782]: 14142: 10.0.0.5:6804/1 'deisNode1' mds.-1.0 up:standby seq 1
Oct 23 17:01:34 deisNode0 sh[1782]: 14136: 10.0.0.6:6804/1 'deisNode2' mds.0.1 up:creating seq 1
Oct 23 17:01:34 deisNode0 sh[1782]: 2015-10-23 17:01:34.851793 7f33f2a55700 0 log_channel(cluster) log [INF] : mds.? 10.0.0.5:6804/1 up:boot
Oct 23 17:01:34 deisNode0 sh[1782]: 2015-10-23 17:01:34.852050 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e5: 1/1/1 up {0=deisNode2=up:creating}, 2 up:standby
Oct 23 17:01:39 deisNode0 sh[1782]: 2015-10-23 17:01:39.706945 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v11: 192 pgs: 192 creating; 0 bytes data, 5618 MB used, 20715 MB / 27787 MB avail
Oct 23 17:01:40 deisNode0 sh[1782]: 2015-10-23 17:01:40.831794 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v12: 192 pgs: 192 creating; 0 bytes data, 11276 MB used, 41391 MB / 55574 MB avail
Oct 23 17:01:57 deisNode0 sh[1782]: 2015-10-23 17:01:57.210185 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(8) update_stats avail 74% total 27787 MB, used 5618 MB, avail 20715 MB
Oct 23 17:01:58 deisNode0 sh[1782]: 2015-10-23 17:01:58.726354 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v13: 192 pgs: 192 creating+peering; 0 bytes data, 16896 MB used, 62105 MB / 83362 MB avail
Oct 23 17:02:01 deisNode0 sh[1782]: 2015-10-23 17:02:01.695123 7f33f1252700 0 mon.deisNode0@0(leader) e3 handle_command mon_command({"prefix": "auth add", "entity": "client.radosgw.gateway"} v 0) v1
Oct 23 17:02:01 deisNode0 sh[1782]: 2015-10-23 17:02:01.695208 7f33f1252700 0 log_channel(audit) log [INF] : from='client.14111 :/0' entity='client.admin' cmd=[{"prefix": "auth add", "entity": "client.radosgw.gateway"}]: dispatch
Oct 23 17:02:01 deisNode0 sh[1782]: 2015-10-23 17:02:01.818902 7f33f2a55700 0 log_channel(audit) log [INF] : from='client.14111 :/0' entity='client.admin' cmd='[{"prefix": "auth add", "entity": "client.radosgw.gateway"}]': finished
Oct 23 17:02:03 deisNode0 sh[1782]: 2015-10-23 17:02:03.318683 7f33f2a55700 1 mon.deisNode0@0(leader).osd e10 e10: 4 osds: 3 up, 3 in
Oct 23 17:02:03 deisNode0 sh[1782]: 2015-10-23 17:02:03.397305 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e10: 4 osds: 3 up, 3 in
Oct 23 17:02:03 deisNode0 sh[1782]: 2015-10-23 17:02:03.494265 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v14: 256 pgs: 64 creating, 192 creating+peering; 0 bytes data, 16896 MB used, 62105 MB / 83362 MB avail
Oct 23 17:02:04 deisNode0 sh[1782]: 2015-10-23 17:02:04.709121 7f33f2a55700 1 mon.deisNode0@0(leader).osd e11 e11: 4 osds: 3 up, 3 in
Oct 23 17:02:04 deisNode0 sh[1782]: 2015-10-23 17:02:04.787250 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e11: 4 osds: 3 up, 3 in
Oct 23 17:02:04 deisNode0 sh[1782]: 2015-10-23 17:02:04.928653 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v15: 256 pgs: 64 creating, 192 creating+peering; 0 bytes data, 16896 MB used, 62105 MB / 83362 MB avail
Oct 23 17:02:07 deisNode0 sh[1782]: 2015-10-23 17:02:07.553656 7f33f2a55700 1 mon.deisNode0@0(leader).osd e12 e12: 4 osds: 3 up, 3 in
Oct 23 17:02:07 deisNode0 sh[1782]: 2015-10-23 17:02:07.615337 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e12: 4 osds: 3 up, 3 in
Oct 23 17:02:07 deisNode0 sh[1782]: 2015-10-23 17:02:07.787835 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v16: 256 pgs: 64 creating, 192 creating+peering; 0 bytes data, 16896 MB used, 62105 MB / 83362 MB avail
Oct 23 17:02:08 deisNode0 sh[1782]: 2015-10-23 17:02:08.662491 7f33f2a55700 1 mon.deisNode0@0(leader).osd e13 e13: 4 osds: 3 up, 3 in
Oct 23 17:02:08 deisNode0 sh[1782]: 2015-10-23 17:02:08.740581 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e13: 4 osds: 3 up, 3 in
Oct 23 17:02:08 deisNode0 sh[1782]: 2015-10-23 17:02:08.836438 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v17: 256 pgs: 45 creating, 209 creating+peering, 2 creating+activating; 0 bytes data, 16903 MB used, 62098 MB / 83362 MB avail
Oct 23 17:02:10 deisNode0 sh[1782]: 2015-10-23 17:02:10.194866 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v18: 256 pgs: 22 creating, 231 creating+peering, 3 creating+activating; 0 bytes data, 16904 MB used, 62097 MB / 83362 MB avail
Oct 23 17:02:13 deisNode0 sh[1782]: 2015-10-23 17:02:13.664714 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v19: 256 pgs: 2 creating+activating+undersized+degraded, 251 creating+peering, 3 creating+activating; 0 bytes data, 16906 MB used, 62095 MB / 83362 MB avail
Oct 23 17:02:15 deisNode0 sh[1782]: 2015-10-23 17:02:15.119049 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v20: 256 pgs: 2 creating+activating+undersized+degraded, 232 creating+peering, 22 creating+activating; 0 bytes data, 16906 MB used, 62095 MB / 83362 MB avail
Oct 23 17:02:15 deisNode0 sh[1782]: 2015-10-23 17:02:15.820435 7f33f2a55700 1 mon.deisNode0@0(leader).osd e14 e14: 4 osds: 3 up, 3 in
Oct 23 17:02:15 deisNode0 sh[1782]: 2015-10-23 17:02:15.882700 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e14: 4 osds: 3 up, 3 in
Oct 23 17:02:16 deisNode0 sh[1782]: 2015-10-23 17:02:16.007826 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v21: 256 pgs: 2 creating+activating+undersized+degraded, 232 creating+peering, 22 creating+activating; 0 bytes data, 16906 MB used, 62095 MB / 83362 MB avail
Oct 23 17:02:17 deisNode0 sh[1782]: 2015-10-23 17:02:17.685608 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v22: 256 pgs: 2 creating+activating+undersized+degraded, 212 creating+peering, 42 creating+activating; 0 bytes data, 16907 MB used, 62094 MB / 83362 MB avail
Oct 23 17:02:19 deisNode0 sh[1782]: 2015-10-23 17:02:19.969850 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v23: 256 pgs: 2 creating+activating+undersized+degraded, 181 creating+peering, 73 creating+activating; 0 bytes data, 16908 MB used, 62093 MB / 83362 MB avail
Oct 23 17:02:21 deisNode0 sh[1782]: 2015-10-23 17:02:21.312930 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v24: 256 pgs: 2 creating+activating+undersized+degraded, 137 creating+peering, 117 creating+activating; 0 bytes data, 16909 MB used, 62092 MB / 83362 MB avail
Oct 23 17:02:21 deisNode0 sh[1782]: 2015-10-23 17:02:21.574237 7f33f1a53700 0 log_channel(cluster) log [INF] : HEALTH_WARN; 2 pgs degraded; 137 pgs peering; 256 pgs stuck inactive; 256 pgs stuck unclean; 2 pgs undersized; 37 requests are blocked > 32 sec
Oct 23 17:02:22 deisNode0 sh[1782]: 2015-10-23 17:02:22.703529 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v25: 256 pgs: 136 creating+peering, 120 creating+activating; 0 bytes data, 16910 MB used, 62091 MB / 83362 MB avail
Oct 23 17:02:25 deisNode0 sh[1782]: 2015-10-23 17:02:25.047047 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v26: 256 pgs: 127 creating+peering, 5 active+clean, 124 creating+activating; 0 bytes data, 16910 MB used, 62091 MB / 83362 MB avail
Oct 23 17:02:26 deisNode0 sh[1782]: 2015-10-23 17:02:26.156511 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v27: 256 pgs: 86 creating+peering, 11 active+clean, 159 creating+activating; 0 bytes data, 16911 MB used, 62090 MB / 83362 MB avail; 0 B/s wr, 0 op/s
Oct 23 17:02:28 deisNode0 sh[1782]: 2015-10-23 17:02:28.835625 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v28: 256 pgs: 80 creating+peering, 11 active+clean, 165 creating+activating; 0 bytes data, 16912 MB used, 62089 MB / 83362 MB avail; 0 B/s wr, 0 op/s
Oct 23 17:02:30 deisNode0 sh[1782]: 2015-10-23 17:02:30.429682 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v29: 256 pgs: 1 active, 80 creating+peering, 22 active+clean, 153 creating+activating; 22 bytes data, 16912 MB used, 62089 MB / 83362 MB avail; 256 B/s wr, 0 op/s
Oct 23 17:02:31 deisNode0 sh[1782]: 2015-10-23 17:02:31.648922 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v30: 256 pgs: 3 active, 60 creating+peering, 45 active+clean, 148 creating+activating; 22 bytes data, 16913 MB used, 62088 MB / 83362 MB avail; 338 B/s wr, 0 op/s; 2/12 objects degraded (16.667%)
Oct 23 17:02:33 deisNode0 sh[1782]: 2015-10-23 17:02:33.087048 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v31: 256 pgs: 3 active, 38 creating+peering, 45 active+clean, 170 creating+activating; 22 bytes data, 16913 MB used, 62088 MB / 83362 MB avail; 0 B/s wr, 0 op/s; 2/12 objects degraded (16.667%)
Oct 23 17:02:35 deisNode0 sh[1782]: 2015-10-23 17:02:35.914157 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v32: 256 pgs: 2 active, 38 creating+peering, 71 active+clean, 145 creating+activating; 594 bytes data, 16913 MB used, 62088 MB / 83362 MB avail; 541 B/s wr, 1 op/s; 2/21 objects degraded (9.524%)
Oct 23 17:02:37 deisNode0 sh[1782]: 2015-10-23 17:02:37.230564 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v33: 256 pgs: 2 active, 37 creating+peering, 88 active+clean, 129 creating+activating; 1412 bytes data, 16914 MB used, 62087 MB / 83362 MB avail; 1012 B/s wr, 2 op/s
Oct 23 17:02:39 deisNode0 sh[1782]: 2015-10-23 17:02:39.811374 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v34: 256 pgs: 2 active, 21 creating+peering, 89 active+clean, 144 creating+activating; 1412 bytes data, 16915 MB used, 62086 MB / 83362 MB avail; 491 B/s wr, 1 op/s
Oct 23 17:02:41 deisNode0 sh[1782]: 2015-10-23 17:02:41.233264 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v35: 256 pgs: 21 creating+peering, 122 active+clean, 113 creating+activating; 1412 bytes data, 16917 MB used, 62084 MB / 83362 MB avail; 0 B/s wr, 1 op/s
Oct 23 17:02:44 deisNode0 sh[1782]: 2015-10-23 17:02:44.920010 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v36: 256 pgs: 18 creating+peering, 138 active+clean, 100 creating+activating; 1412 bytes data, 16917 MB used, 62084 MB / 83362 MB avail; 0 B/s wr, 0 op/s
Oct 23 17:02:46 deisNode0 sh[1782]: 2015-10-23 17:02:46.045247 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v37: 256 pgs: 18 creating+peering, 165 active+clean, 73 creating+activating; 1480 bytes data, 16918 MB used, 62083 MB / 83362 MB avail; 416 B/s wr, 0 op/s
Oct 23 17:02:49 deisNode0 sh[1782]: 2015-10-23 17:02:49.718200 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v38: 256 pgs: 235 active+clean, 21 creating+activating; 2158 bytes data, 16918 MB used, 62083 MB / 83362 MB avail; 643 B/s wr, 1 op/s
Oct 23 17:02:49 deisNode0 sh[1782]: 2015-10-23 17:02:49.825803 7f33f2a55700 0 mon.deisNode0@0(leader).mds e6 print_map
Oct 23 17:02:49 deisNode0 sh[1782]: epoch 6
Oct 23 17:02:49 deisNode0 sh[1782]: flags 0
Oct 23 17:02:49 deisNode0 sh[1782]: created 2015-10-23 17:01:28.362135
Oct 23 17:02:49 deisNode0 sh[1782]: modified 2015-10-23 17:02:49.705277
Oct 23 17:02:49 deisNode0 sh[1782]: tableserver 0
Oct 23 17:02:49 deisNode0 sh[1782]: root 0
Oct 23 17:02:49 deisNode0 sh[1782]: session_timeout 60
Oct 23 17:02:49 deisNode0 sh[1782]: session_autoclose 300
Oct 23 17:02:49 deisNode0 sh[1782]: max_file_size 1099511627776
Oct 23 17:02:49 deisNode0 sh[1782]: last_failure 0
Oct 23 17:02:49 deisNode0 sh[1782]: last_failure_osd_epoch 0
Oct 23 17:02:49 deisNode0 sh[1782]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table}
Oct 23 17:02:49 deisNode0 sh[1782]: max_mds 1
Oct 23 17:02:49 deisNode0 sh[1782]: in 0
Oct 23 17:02:49 deisNode0 sh[1782]: up {0=14136}
Oct 23 17:02:49 deisNode0 sh[1782]: failed
Oct 23 17:02:49 deisNode0 sh[1782]: stopped
Oct 23 17:02:49 deisNode0 sh[1782]: data_pools 1
Oct 23 17:02:49 deisNode0 sh[1782]: metadata_pool 2
Oct 23 17:02:49 deisNode0 sh[1782]: inline_data disabled
Oct 23 17:02:49 deisNode0 sh[1782]: 14127: 10.0.0.4:6804/1 'deisNode0' mds.-1.0 up:standby seq 1
Oct 23 17:02:49 deisNode0 sh[1782]: 14142: 10.0.0.5:6804/1 'deisNode1' mds.-1.0 up:standby seq 1
Oct 23 17:02:49 deisNode0 sh[1782]: 14136: 10.0.0.6:6804/1 'deisNode2' mds.0.1 up:active seq 21
Oct 23 17:02:49 deisNode0 sh[1782]: 2015-10-23 17:02:49.826223 7f33f2a55700 0 log_channel(cluster) log [INF] : mds.0 10.0.0.6:6804/1 up:active
Oct 23 17:02:49 deisNode0 sh[1782]: 2015-10-23 17:02:49.826318 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e6: 1/1/1 up {0=deisNode2=up:active}, 2 up:standby
Oct 23 17:02:50 deisNode0 sh[1782]: 2015-10-23 17:02:50.920494 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v39: 256 pgs: 250 active+clean, 6 creating+activating; 2640 bytes data, 16919 MB used, 62082 MB / 83362 MB avail; 417 B/s wr, 1 op/s
Oct 23 17:02:51 deisNode0 sh[1782]: 2015-10-23 17:02:51.021168 7f33f2a55700 1 mon.deisNode0@0(leader).osd e15 e15: 4 osds: 3 up, 3 in
Oct 23 17:02:51 deisNode0 sh[1782]: 2015-10-23 17:02:51.075836 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e15: 4 osds: 3 up, 3 in
Oct 23 17:02:51 deisNode0 sh[1782]: 2015-10-23 17:02:51.156611 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v40: 320 pgs: 64 creating, 254 active+clean, 2 creating+activating; 2810 bytes data, 16919 MB used, 62082 MB / 83362 MB avail; 2099 B/s wr, 2 op/s
Oct 23 17:02:52 deisNode0 sh[1782]: 2015-10-23 17:02:52.294698 7f33f2a55700 1 mon.deisNode0@0(leader).osd e16 e16: 4 osds: 3 up, 3 in
Oct 23 17:02:52 deisNode0 sh[1782]: 2015-10-23 17:02:52.372812 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e16: 4 osds: 3 up, 3 in
Oct 23 17:02:52 deisNode0 sh[1782]: 2015-10-23 17:02:52.513564 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v41: 320 pgs: 64 creating, 254 active+clean, 2 creating+activating; 2810 bytes data, 16919 MB used, 62082 MB / 83362 MB avail; 1362 B/s wr, 1 op/s
Oct 23 17:02:56 deisNode0 sh[1782]: 2015-10-23 17:02:56.611372 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v42: 320 pgs: 19 creating, 45 creating+activating, 256 active+clean; 2810 bytes data, 16921 MB used, 62080 MB / 83362 MB avail; 0 B/s wr, 0 op/s
Oct 23 17:02:57 deisNode0 sh[1782]: 2015-10-23 17:02:57.210562 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(8) update_stats avail 74% total 27787 MB, used 5625 MB, avail 20708 MB
Oct 23 17:02:57 deisNode0 sh[1782]: 2015-10-23 17:02:57.970467 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v43: 320 pgs: 64 creating+activating, 256 active+clean; 2810 bytes data, 16922 MB used, 62079 MB / 83362 MB avail; 0 B/s wr, 0 op/s
Oct 23 17:03:01 deisNode0 sh[1782]: 2015-10-23 17:03:01.610584 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v44: 320 pgs: 52 creating+activating, 268 active+clean; 2810 bytes data, 16923 MB used, 62078 MB / 83362 MB avail
Oct 23 17:03:02 deisNode0 sh[1782]: 2015-10-23 17:03:02.986296 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v45: 320 pgs: 38 creating+activating, 282 active+clean; 2810 bytes data, 16923 MB used, 62078 MB / 83362 MB avail
Oct 23 17:03:06 deisNode0 sh[1782]: 2015-10-23 17:03:06.644537 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v46: 320 pgs: 19 creating+activating, 301 active+clean; 2810 bytes data, 16923 MB used, 62078 MB / 83362 MB avail
Oct 23 17:03:07 deisNode0 sh[1782]: 2015-10-23 17:03:07.894322 7f33f2a55700 1 mon.deisNode0@0(leader).osd e17 e17: 4 osds: 3 up, 3 in
Oct 23 17:03:07 deisNode0 sh[1782]: 2015-10-23 17:03:07.987925 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v47: 320 pgs: 320 active+clean; 2810 bytes data, 16924 MB used, 62077 MB / 83362 MB avail
Oct 23 17:03:07 deisNode0 sh[1782]: 2015-10-23 17:03:07.988987 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e17: 4 osds: 3 up, 3 in
Oct 23 17:03:08 deisNode0 sh[1782]: 2015-10-23 17:03:08.113895 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v48: 384 pgs: 64 creating, 320 active+clean; 2810 bytes data, 16924 MB used, 62077 MB / 83362 MB avail
Oct 23 17:03:09 deisNode0 sh[1782]: 2015-10-23 17:03:09.568645 7f33f2a55700 1 mon.deisNode0@0(leader).osd e18 e18: 4 osds: 3 up, 3 in
Oct 23 17:03:09 deisNode0 sh[1782]: 2015-10-23 17:03:09.642325 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e18: 4 osds: 3 up, 3 in
Oct 23 17:03:09 deisNode0 sh[1782]: 2015-10-23 17:03:09.805163 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v49: 448 pgs: 128 creating, 320 active+clean; 2810 bytes data, 16924 MB used, 62077 MB / 83362 MB avail
Oct 23 17:03:10 deisNode0 sh[1782]: 2015-10-23 17:03:10.911674 7f33f2a55700 1 mon.deisNode0@0(leader).osd e19 e19: 4 osds: 3 up, 3 in
Oct 23 17:03:10 deisNode0 sh[1782]: 2015-10-23 17:03:10.974059 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e19: 4 osds: 3 up, 3 in
Oct 23 17:03:11 deisNode0 sh[1782]: 2015-10-23 17:03:11.102051 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v50: 512 pgs: 192 creating, 320 active+clean; 2810 bytes data, 16924 MB used, 62077 MB / 83362 MB avail
Oct 23 17:03:13 deisNode0 sh[1782]: 2015-10-23 17:03:13.115162 7f33f2a55700 1 mon.deisNode0@0(leader).osd e20 e20: 4 osds: 3 up, 3 in
Oct 23 17:03:13 deisNode0 sh[1782]: 2015-10-23 17:03:13.192805 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e20: 4 osds: 3 up, 3 in
Oct 23 17:03:13 deisNode0 sh[1782]: 2015-10-23 17:03:13.286931 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v51: 512 pgs: 192 creating, 320 active+clean; 2810 bytes data, 16924 MB used, 62077 MB / 83362 MB avail
Oct 23 17:03:17 deisNode0 sh[1782]: 2015-10-23 17:03:17.397458 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v52: 512 pgs: 126 creating, 47 creating+peering, 19 creating+activating, 320 active+clean; 2810 bytes data, 16926 MB used, 62075 MB / 83362 MB avail
Oct 23 17:03:18 deisNode0 sh[1782]: 2015-10-23 17:03:18.663687 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v53: 512 pgs: 57 creating, 96 creating+peering, 39 creating+activating, 320 active+clean; 2810 bytes data, 16927 MB used, 62074 MB / 83362 MB avail
Oct 23 17:03:19 deisNode0 sh[1782]: 2015-10-23 17:03:19.866328 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v54: 512 pgs: 128 creating+peering, 64 creating+activating, 320 active+clean; 2810 bytes data, 16929 MB used, 62072 MB / 83362 MB avail
Oct 23 17:03:21 deisNode0 sh[1782]: 2015-10-23 17:03:21.575044 7f33f1a53700 0 log_channel(cluster) log [INF] : HEALTH_WARN; 128 pgs peering; 192 pgs stuck inactive; 192 pgs stuck unclean
Oct 23 17:03:22 deisNode0 sh[1782]: 2015-10-23 17:03:22.490365 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v55: 512 pgs: 127 creating+peering, 62 creating+activating, 323 active+clean; 2810 bytes data, 16930 MB used, 62072 MB / 83362 MB avail
Oct 23 17:03:23 deisNode0 sh[1782]: 2015-10-23 17:03:23.851326 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v56: 512 pgs: 82 creating+peering, 96 creating+activating, 334 active+clean; 2810 bytes data, 16930 MB used, 62071 MB / 83362 MB avail
Oct 23 17:03:27 deisNode0 sh[1782]: 2015-10-23 17:03:27.163092 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v57: 512 pgs: 57 creating+peering, 121 creating+activating, 334 active+clean; 2810 bytes data, 16931 MB used, 62070 MB / 83362 MB avail
Oct 23 17:03:28 deisNode0 sh[1782]: 2015-10-23 17:03:28.271677 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v58: 512 pgs: 27 creating+peering, 145 creating+activating, 340 active+clean; 2810 bytes data, 16932 MB used, 62069 MB / 83362 MB avail
Oct 23 17:03:30 deisNode0 sh[1782]: 2015-10-23 17:03:30.648423 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v59: 512 pgs: 23 creating+peering, 140 creating+activating, 349 active+clean; 2810 bytes data, 16932 MB used, 62069 MB / 83362 MB avail
Oct 23 17:03:32 deisNode0 sh[1782]: 2015-10-23 17:03:32.931555 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v60: 512 pgs: 6 active, 16 creating+peering, 135 creating+activating, 355 active+clean; 2810 bytes data, 16932 MB used, 62069 MB / 83362 MB avail
Oct 23 17:03:34 deisNode0 sh[1782]: 2015-10-23 17:03:34.210977 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v61: 512 pgs: 7 active, 147 creating+activating, 358 active+clean; 2810 bytes data, 16933 MB used, 62068 MB / 83362 MB avail
Oct 23 17:03:35 deisNode0 sh[1782]: 2015-10-23 17:03:35.929023 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v62: 512 pgs: 8 active, 141 creating+activating, 363 active+clean; 2810 bytes data, 16934 MB used, 62067 MB / 83362 MB avail
Oct 23 17:03:38 deisNode0 sh[1782]: 2015-10-23 17:03:38.273968 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v63: 512 pgs: 2 active, 128 creating+activating, 382 active+clean; 2810 bytes data, 16934 MB used, 62067 MB / 83362 MB avail
Oct 23 17:03:39 deisNode0 sh[1782]: 2015-10-23 17:03:39.367659 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v64: 512 pgs: 1 active, 101 creating+activating, 410 active+clean; 2810 bytes data, 16935 MB used, 62066 MB / 83362 MB avail
Oct 23 17:03:40 deisNode0 sh[1782]: 2015-10-23 17:03:40.742724 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v65: 512 pgs: 84 creating+activating, 428 active+clean; 2810 bytes data, 16935 MB used, 62066 MB / 83362 MB avail
Oct 23 17:03:43 deisNode0 sh[1782]: 2015-10-23 17:03:43.884586 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v66: 512 pgs: 72 creating+activating, 440 active+clean; 2810 bytes data, 16936 MB used, 62065 MB / 83362 MB avail
Oct 23 17:03:46 deisNode0 sh[1782]: 2015-10-23 17:03:46.227829 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v67: 512 pgs: 64 creating+activating, 448 active+clean; 2810 bytes data, 16936 MB used, 62065 MB / 83362 MB avail
Oct 23 17:03:49 deisNode0 sh[1782]: 2015-10-23 17:03:49.702508 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v68: 512 pgs: 53 creating+activating, 459 active+clean; 2810 bytes data, 16937 MB used, 62064 MB / 83362 MB avail
Oct 23 17:03:50 deisNode0 sh[1782]: 2015-10-23 17:03:50.951688 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v69: 512 pgs: 43 creating+activating, 469 active+clean; 2810 bytes data, 16937 MB used, 62064 MB / 83362 MB avail
Oct 23 17:03:53 deisNode0 sh[1782]: 2015-10-23 17:03:53.844271 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v70: 512 pgs: 39 creating+activating, 473 active+clean; 2810 bytes data, 16937 MB used, 62064 MB / 83362 MB avail
Oct 23 17:03:55 deisNode0 sh[1782]: 2015-10-23 17:03:55.438235 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v71: 512 pgs: 8 creating+activating, 504 active+clean; 2810 bytes data, 16938 MB used, 62063 MB / 83362 MB avail
Oct 23 17:03:55 deisNode0 sh[1782]: 2015-10-23 17:03:55.531008 7f33f2a55700 1 mon.deisNode0@0(leader).osd e21 e21: 4 osds: 3 up, 3 in
Oct 23 17:03:55 deisNode0 sh[1782]: 2015-10-23 17:03:55.655226 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e21: 4 osds: 3 up, 3 in
Oct 23 17:03:55 deisNode0 sh[1782]: 2015-10-23 17:03:55.784059 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v72: 576 pgs: 64 creating, 512 active+clean; 2810 bytes data, 16938 MB used, 62063 MB / 83362 MB avail
Oct 23 17:03:57 deisNode0 sh[1782]: 2015-10-23 17:03:57.210957 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(8) update_stats avail 74% total 27787 MB, used 5630 MB, avail 20703 MB
Oct 23 17:03:57 deisNode0 sh[1782]: 2015-10-23 17:03:57.287602 7f33f2a55700 1 mon.deisNode0@0(leader).osd e22 e22: 4 osds: 3 up, 3 in
Oct 23 17:03:57 deisNode0 sh[1782]: 2015-10-23 17:03:57.359175 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e22: 4 osds: 3 up, 3 in
Oct 23 17:03:57 deisNode0 sh[1782]: 2015-10-23 17:03:57.875372 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v73: 576 pgs: 64 creating, 512 active+clean; 2810 bytes data, 16938 MB used, 62063 MB / 83362 MB avail
Oct 23 17:04:01 deisNode0 sh[1782]: 2015-10-23 17:04:01.313643 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v74: 576 pgs: 19 creating, 3 creating+peering, 42 creating+activating, 512 active+clean; 2810 bytes data, 16940 MB used, 62061 MB / 83362 MB avail
Oct 23 17:04:02 deisNode0 sh[1782]: 2015-10-23 17:04:02.439347 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v75: 576 pgs: 3 creating+peering, 61 creating+activating, 512 active+clean; 2810 bytes data, 16940 MB used, 62061 MB / 83362 MB avail
Oct 23 17:04:06 deisNode0 sh[1782]: 2015-10-23 17:04:06.262156 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v76: 576 pgs: 3 creating+peering, 51 creating+activating, 522 active+clean; 2810 bytes data, 16941 MB used, 62060 MB / 83362 MB avail
Oct 23 17:04:07 deisNode0 sh[1782]: 2015-10-23 17:04:07.366958 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v77: 576 pgs: 2 active, 40 creating+activating, 534 active+clean; 2810 bytes data, 16942 MB used, 62059 MB / 83362 MB avail
Oct 23 17:04:11 deisNode0 sh[1782]: 2015-10-23 17:04:11.272941 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v78: 576 pgs: 2 active, 30 creating+activating, 544 active+clean; 2810 bytes data, 16942 MB used, 62059 MB / 83362 MB avail
Oct 23 17:04:12 deisNode0 sh[1782]: 2015-10-23 17:04:12.413378 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v79: 576 pgs: 15 active, 6 creating+activating, 555 active+clean; 3128 bytes data, 16943 MB used, 62058 MB / 83362 MB avail; 408 B/s wr, 0 op/s; 2/99 objects degraded (2.020%)
Oct 23 17:04:16 deisNode0 sh[1782]: 2015-10-23 17:04:16.276018 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v80: 576 pgs: 15 active, 3 creating+activating, 558 active+clean; 3128 bytes data, 16943 MB used, 62058 MB / 83362 MB avail; 409 B/s rd, 409 B/s wr, 1 op/s; 2/102 objects degraded (1.961%)
Oct 23 17:04:17 deisNode0 sh[1782]: 2015-10-23 17:04:17.385969 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v81: 576 pgs: 576 active+clean; 3128 bytes data, 16944 MB used, 62057 MB / 83362 MB avail; 2865 B/s rd, 0 B/s wr, 7 op/s
Oct 23 17:04:18 deisNode0 sh[1782]: 2015-10-23 17:04:18.595433 7f33ee33f700 0 -- 10.0.0.4:6789/0 >> 10.0.0.4:0/2002610178 pipe(0x5eb8000 sd=27 :6789 s=0 pgs=0 cs=0 l=0 c=0x542b020).accept peer addr is really 10.0.0.4:0/2002610178 (socket is 10.0.0.4:35757/0)
Oct 23 17:04:21 deisNode0 sh[1782]: 2015-10-23 17:04:21.260455 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v82: 576 pgs: 576 active+clean; 3128 bytes data, 16944 MB used, 62057 MB / 83362 MB avail; 7370 B/s rd, 0 B/s wr, 18 op/s
Oct 23 17:04:21 deisNode0 sh[1782]: 2015-10-23 17:04:21.575455 7f33f1a53700 0 log_channel(cluster) log [INF] : HEALTH_OK
Oct 23 17:04:22 deisNode0 sh[1782]: 2015-10-23 17:04:22.370209 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v83: 576 pgs: 576 active+clean; 5917 bytes data, 16945 MB used, 62056 MB / 83362 MB avail; 10271 B/s rd, 1232 B/s wr, 25 op/s
Oct 23 17:04:51 deisNode0 sh[1782]: 2015-10-23 17:04:51.622093 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v84: 576 pgs: 576 active+clean; 5917 bytes data, 16957 MB used, 62044 MB / 83362 MB avail; 35 B/s wr, 0 op/s
Oct 23 17:04:52 deisNode0 sh[1782]: 2015-10-23 17:04:52.840159 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v85: 576 pgs: 576 active+clean; 8127 bytes data, 16969 MB used, 62032 MB / 83362 MB avail; 134 B/s wr, 0 op/s
Oct 23 17:04:56 deisNode0 sh[1782]: 2015-10-23 17:04:56.824586 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v86: 576 pgs: 576 active+clean; 10326 bytes data, 16969 MB used, 62032 MB / 83362 MB avail; 1197 B/s wr, 0 op/s
Oct 23 17:04:57 deisNode0 sh[1782]: 2015-10-23 17:04:57.212290 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(8) update_stats avail 74% total 27787 MB, used 5655 MB, avail 20678 MB
Oct 23 17:05:44 deisNode0 sh[1782]: 2015-10-23 17:05:44.001225 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:05:44 deisNode0 sh[1782]: 2015-10-23 17:05:44.001315 7f33f1252700 1 mon.deisNode0@0(electing).elector(9) init, last seen epoch 9
Oct 23 17:05:49 deisNode0 sh[1782]: 2015-10-23 17:05:49.032596 7f33f1a53700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,1
Oct 23 17:05:49 deisNode0 sh[1782]: 2015-10-23 17:05:49.063929 7f33f1a53700 0 log_channel(cluster) log [INF] : HEALTH_WARN; 1 mons down, quorum 0,1 deisNode0,deisNode1
Oct 23 17:05:49 deisNode0 sh[1782]: 2015-10-23 17:05:49.256529 7f33f2a55700 0 log_channel(cluster) log [INF] : monmap e3: 3 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0,deisNode2=10.0.0.6:6789/0}
Oct 23 17:05:49 deisNode0 sh[1782]: 2015-10-23 17:05:49.256638 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v87: 576 pgs: 576 active+clean; 14013 bytes data, 17213 MB used, 61788 MB / 83362 MB avail; 102 B/s wr, 0 op/s
Oct 23 17:05:49 deisNode0 sh[1782]: 2015-10-23 17:05:49.256858 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e6: 1/1/1 up {0=deisNode2=up:active}, 2 up:standby
Oct 23 17:05:49 deisNode0 sh[1782]: 2015-10-23 17:05:49.256979 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e22: 4 osds: 3 up, 3 in
Oct 23 17:05:50 deisNode0 sh[1782]: 2015-10-23 17:05:50.517968 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v88: 576 pgs: 576 active+clean; 21669 bytes data, 17923 MB used, 61078 MB / 83362 MB avail; 305 B/s wr, 0 op/s
Oct 23 17:05:57 deisNode0 sh[1782]: 2015-10-23 17:05:57.213867 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(10) update_stats avail 73% total 27787 MB, used 5930 MB, avail 20403 MB
Oct 23 17:06:02 deisNode0 sh[1782]: 2015-10-23 17:06:02.829369 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:06:02 deisNode0 sh[1782]: 2015-10-23 17:06:02.829509 7f33f1252700 1 mon.deisNode0@0(electing).elector(10) init, last seen epoch 10
Oct 23 17:06:02 deisNode0 sh[1782]: 2015-10-23 17:06:02.835782 7f33ee13d700 0 -- 10.0.0.4:6789/0 >> 10.0.0.6:0/2544123017 pipe(0x5fd9000 sd=28 :6789 s=0 pgs=0 cs=0 l=0 c=0x5a1f700).accept peer addr is really 10.0.0.6:0/2544123017 (socket is 10.0.0.6:39782/0)
Oct 23 17:06:06 deisNode0 sh[1782]: 2015-10-23 17:06:06.760942 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,1,2
Oct 23 17:06:06 deisNode0 sh[1782]: 2015-10-23 17:06:06.791016 7f33f1252700 0 log_channel(cluster) log [INF] : HEALTH_OK
Oct 23 17:06:07 deisNode0 sh[1782]: 2015-10-23 17:06:07.307577 7f33f2a55700 0 log_channel(cluster) log [INF] : monmap e3: 3 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0,deisNode2=10.0.0.6:6789/0}
Oct 23 17:06:07 deisNode0 sh[1782]: 2015-10-23 17:06:07.307711 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v89: 576 pgs: 576 active+clean; 21669 bytes data, 18011 MB used, 60990 MB / 83362 MB avail; 535 B/s wr, 0 op/s
Oct 23 17:06:07 deisNode0 sh[1782]: 2015-10-23 17:06:07.307864 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e6: 1/1/1 up {0=deisNode2=up:active}, 2 up:standby
Oct 23 17:06:07 deisNode0 sh[1782]: 2015-10-23 17:06:07.308023 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e22: 4 osds: 3 up, 3 in
Oct 23 17:06:08 deisNode0 sh[1782]: 2015-10-23 17:06:08.690953 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v90: 576 pgs: 576 active+clean; 34767 bytes data, 18203 MB used, 60798 MB / 83362 MB avail; 1135 B/s wr, 0 op/s
Oct 23 17:06:17 deisNode0 sh[1782]: 2015-10-23 17:06:17.048028 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v91: 576 pgs: 576 active+clean; 58561 bytes data, 18290 MB used, 60711 MB / 83362 MB avail; 3118 B/s wr, 1 op/s
Oct 23 17:06:21 deisNode0 sh[1782]: 2015-10-23 17:06:21.391334 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v92: 576 pgs: 576 active+clean; 64770 bytes data, 18307 MB used, 60694 MB / 83362 MB avail; 2867 B/s wr, 0 op/s
Oct 23 17:06:23 deisNode0 sh[1782]: 2015-10-23 17:06:23.813370 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v93: 576 pgs: 576 active+clean; 66949 bytes data, 18376 MB used, 60625 MB / 83362 MB avail; 2259 B/s wr, 0 op/s
Oct 23 17:06:25 deisNode0 sh[1782]: 2015-10-23 17:06:25.390716 7f33f2a55700 1 mon.deisNode0@0(leader).osd e23 e23: 4 osds: 3 up, 3 in
Oct 23 17:06:25 deisNode0 sh[1782]: 2015-10-23 17:06:25.485141 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e23: 4 osds: 3 up, 3 in
Oct 23 17:06:25 deisNode0 sh[1782]: 2015-10-23 17:06:25.673986 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v94: 640 pgs: 64 creating, 576 active+clean; 66949 bytes data, 18376 MB used, 60625 MB / 83362 MB avail; 964 B/s wr, 0 op/s
Oct 23 17:06:26 deisNode0 sh[1782]: 2015-10-23 17:06:26.907344 7f33f2a55700 1 mon.deisNode0@0(leader).osd e24 e24: 4 osds: 3 up, 3 in
Oct 23 17:06:27 deisNode0 sh[1782]: 2015-10-23 17:06:27.000290 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e24: 4 osds: 3 up, 3 in
Oct 23 17:06:27 deisNode0 sh[1782]: 2015-10-23 17:06:27.157146 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v95: 640 pgs: 64 creating, 576 active+clean; 66949 bytes data, 18376 MB used, 60625 MB / 83362 MB avail
Oct 23 17:06:32 deisNode0 sh[1782]: 2015-10-23 17:06:32.271151 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v96: 640 pgs: 44 creating, 20 creating+activating, 576 active+clean; 66949 bytes data, 18395 MB used, 60606 MB / 83362 MB avail
Oct 23 17:06:33 deisNode0 sh[1782]: 2015-10-23 17:06:33.493650 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v97: 640 pgs: 28 creating, 36 creating+activating, 576 active+clean; 69127 bytes data, 18396 MB used, 60605 MB / 83362 MB avail; 486 B/s wr, 0 op/s
Oct 23 17:06:34 deisNode0 sh[1782]: 2015-10-23 17:06:34.587836 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v98: 640 pgs: 20 creating+peering, 44 creating+activating, 576 active+clean; 69921 bytes data, 18506 MB used, 60495 MB / 83362 MB avail; 1674 B/s wr, 0 op/s
Oct 23 17:06:37 deisNode0 sh[1782]: 2015-10-23 17:06:37.010014 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v99: 640 pgs: 20 creating+peering, 44 creating+activating, 576 active+clean; 73008 bytes data, 18506 MB used, 60495 MB / 83362 MB avail; 1760 B/s wr, 0 op/s
Oct 23 17:06:42 deisNode0 sh[1782]: 2015-10-23 17:06:42.545852 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v100: 640 pgs: 64 creating+activating, 576 active+clean; 73008 bytes data, 18568 MB used, 60433 MB / 83362 MB avail; 520 B/s wr, 0 op/s
Oct 23 17:06:46 deisNode0 sh[1782]: 2015-10-23 17:06:46.578613 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v101: 640 pgs: 49 creating+activating, 591 active+clean; 85840 bytes data, 18569 MB used, 60432 MB / 83362 MB avail; 1481 B/s wr, 0 op/s
Oct 23 17:06:48 deisNode0 sh[1782]: 2015-10-23 17:06:48.720340 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v102: 640 pgs: 10 active, 8 creating+activating, 622 active+clean; 95334 bytes data, 18640 MB used, 60361 MB / 83362 MB avail; 5295 B/s wr, 1 op/s
Oct 23 17:06:51 deisNode0 sh[1782]: 2015-10-23 17:06:51.577969 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v103: 640 pgs: 10 active, 3 creating+activating, 627 active+clean; 100411 bytes data, 18640 MB used, 60361 MB / 83362 MB avail; 4914 B/s wr, 0 op/s
Oct 23 17:06:52 deisNode0 sh[1782]: 2015-10-23 17:06:52.829806 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v104: 640 pgs: 640 active+clean; 106 kB data, 18740 MB used, 60261 MB / 83362 MB avail; 4477 B/s wr, 0 op/s
Oct 23 17:06:57 deisNode0 sh[1782]: 2015-10-23 17:06:57.214243 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 73% total 27787 MB, used 6007 MB, avail 20326 MB
Oct 23 17:07:00 deisNode0 sh[1782]: 2015-10-23 17:07:00.064079 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v105: 640 pgs: 640 active+clean; 106 kB data, 18740 MB used, 60261 MB / 83362 MB avail; 2252 B/s wr, 0 op/s
Oct 23 17:07:01 deisNode0 sh[1782]: 2015-10-23 17:07:01.298993 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v106: 640 pgs: 640 active+clean; 106 kB data, 18827 MB used, 60174 MB / 83362 MB avail
Oct 23 17:07:02 deisNode0 sh[1782]: 2015-10-23 17:07:02.423520 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v107: 640 pgs: 640 active+clean; 106 kB data, 18827 MB used, 60174 MB / 83362 MB avail; 6817 B/s rd, 0 B/s wr, 11 op/s
Oct 23 17:07:07 deisNode0 sh[1782]: 2015-10-23 17:07:07.501493 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v108: 640 pgs: 640 active+clean; 106 kB data, 18827 MB used, 60174 MB / 83362 MB avail; 12636 B/s rd, 0 B/s wr, 20 op/s
Oct 23 17:07:09 deisNode0 sh[1782]: 2015-10-23 17:07:09.129117 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v109: 640 pgs: 640 active+clean; 107 kB data, 18883 MB used, 60118 MB / 83362 MB avail; 9410 B/s rd, 330 B/s wr, 15 op/s
Oct 23 17:07:11 deisNode0 sh[1782]: 2015-10-23 17:07:11.970329 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v110: 640 pgs: 640 active+clean; 109 kB data, 18883 MB used, 60118 MB / 83362 MB avail; 5768 B/s rd, 961 B/s wr, 9 op/s
Oct 23 17:07:17 deisNode0 sh[1782]: 2015-10-23 17:07:17.728329 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v111: 640 pgs: 640 active+clean; 109 kB data, 19035 MB used, 59967 MB / 83362 MB avail; 116 B/s rd, 466 B/s wr, 0 op/s
Oct 23 17:07:21 deisNode0 sh[1782]: 2015-10-23 17:07:21.580904 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v112: 640 pgs: 640 active+clean; 122 kB data, 19035 MB used, 59966 MB / 83362 MB avail; 105 B/s rd, 2327 B/s wr, 1 op/s
Oct 23 17:07:22 deisNode0 sh[1782]: 2015-10-23 17:07:22.711913 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v113: 640 pgs: 640 active+clean; 129 kB data, 19035 MB used, 59966 MB / 83362 MB avail; 0 B/s rd, 5984 B/s wr, 1 op/s
Oct 23 17:07:27 deisNode0 sh[1782]: 2015-10-23 17:07:27.461456 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v114: 640 pgs: 640 active+clean; 142 kB data, 19035 MB used, 59966 MB / 83362 MB avail; 0 B/s rd, 5976 B/s wr, 1 op/s
Oct 23 17:07:28 deisNode0 sh[1782]: 2015-10-23 17:07:28.696370 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v115: 640 pgs: 640 active+clean; 142 kB data, 19213 MB used, 59788 MB / 83362 MB avail; 4156 B/s wr, 1 op/s
Oct 23 17:07:41 deisNode0 sh[1782]: 2015-10-23 17:07:41.593094 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v116: 640 pgs: 640 active+clean; 142 kB data, 19213 MB used, 59788 MB / 83362 MB avail; 428 B/s wr, 0 op/s
Oct 23 17:07:42 deisNode0 sh[1782]: 2015-10-23 17:07:42.889985 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v117: 640 pgs: 640 active+clean; 153 kB data, 19422 MB used, 59579 MB / 83362 MB avail; 1304 B/s wr, 0 op/s
Oct 23 17:07:47 deisNode0 sh[1782]: 2015-10-23 17:07:47.484211 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v118: 640 pgs: 640 active+clean; 156 kB data, 19422 MB used, 59579 MB / 83362 MB avail; 3652 B/s wr, 0 op/s
Oct 23 17:07:50 deisNode0 sh[1782]: 2015-10-23 17:07:50.498995 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v119: 640 pgs: 640 active+clean; 156 kB data, 19440 MB used, 59562 MB / 83362 MB avail; 1215 B/s wr, 0 op/s
Oct 23 17:07:54 deisNode0 sh[1782]: 2015-10-23 17:07:54.390342 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v120: 640 pgs: 640 active+clean; 162 kB data, 19440 MB used, 59561 MB / 83362 MB avail; 1724 B/s wr, 0 op/s
Oct 23 17:07:55 deisNode0 sh[1782]: 2015-10-23 17:07:55.499875 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v121: 640 pgs: 640 active+clean; 164 kB data, 19440 MB used, 59561 MB / 83362 MB avail; 1490 B/s wr, 0 op/s
Oct 23 17:07:57 deisNode0 sh[1782]: 2015-10-23 17:07:57.214731 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 73% total 27787 MB, used 6008 MB, avail 20325 MB
Oct 23 17:08:01 deisNode0 sh[1782]: 2015-10-23 17:08:01.737439 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v122: 640 pgs: 640 active+clean; 166 kB data, 19440 MB used, 59561 MB / 83362 MB avail; 825 B/s wr, 0 op/s
Oct 23 17:08:02 deisNode0 sh[1782]: 2015-10-23 17:08:02.987566 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v123: 640 pgs: 640 active+clean; 168 kB data, 19440 MB used, 59561 MB / 83362 MB avail; 1249 B/s wr, 0 op/s
Oct 23 17:08:07 deisNode0 sh[1782]: 2015-10-23 17:08:07.396206 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v124: 640 pgs: 640 active+clean; 174 kB data, 19440 MB used, 59561 MB / 83362 MB avail; 1922 B/s wr, 0 op/s
Oct 23 17:08:08 deisNode0 sh[1782]: 2015-10-23 17:08:08.726089 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v125: 640 pgs: 640 active+clean; 174 kB data, 19549 MB used, 59453 MB / 83362 MB avail; 1447 B/s wr, 0 op/s
Oct 23 17:08:11 deisNode0 sh[1782]: 2015-10-23 17:08:11.879573 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v126: 640 pgs: 640 active+clean; 175 kB data, 19549 MB used, 59452 MB / 83362 MB avail; 614 B/s wr, 0 op/s
Oct 23 17:08:16 deisNode0 sh[1782]: 2015-10-23 17:08:16.571722 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v127: 640 pgs: 640 active+clean; 176 kB data, 19549 MB used, 59452 MB / 83362 MB avail; 892 B/s wr, 0 op/s
Oct 23 17:08:17 deisNode0 sh[1782]: 2015-10-23 17:08:17.682023 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v128: 640 pgs: 640 active+clean; 181 kB data, 19550 MB used, 59451 MB / 83362 MB avail; 530 B/s rd, 2831 B/s wr, 2 op/s
Oct 23 17:08:21 deisNode0 sh[1782]: 2015-10-23 17:08:21.295108 7f33f2a55700 1 mon.deisNode0@0(leader).osd e25 e25: 4 osds: 3 up, 3 in
Oct 23 17:08:21 deisNode0 sh[1782]: 2015-10-23 17:08:21.388812 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e25: 4 osds: 3 up, 3 in
Oct 23 17:08:21 deisNode0 sh[1782]: 2015-10-23 17:08:21.609944 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v129: 704 pgs: 64 creating, 640 active+clean; 181 kB data, 19550 MB used, 59451 MB / 83362 MB avail; 631 B/s rd, 2103 B/s wr, 2 op/s
Oct 23 17:08:22 deisNode0 sh[1782]: 2015-10-23 17:08:22.792097 7f33f2a55700 1 mon.deisNode0@0(leader).osd e26 e26: 4 osds: 3 up, 3 in
Oct 23 17:08:22 deisNode0 sh[1782]: 2015-10-23 17:08:22.888735 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e26: 4 osds: 3 up, 3 in
Oct 23 17:08:23 deisNode0 sh[1782]: 2015-10-23 17:08:23.014250 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v130: 704 pgs: 64 creating, 640 active+clean; 181 kB data, 19550 MB used, 59451 MB / 83362 MB avail
Oct 23 17:08:27 deisNode0 sh[1782]: 2015-10-23 17:08:27.052172 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v131: 704 pgs: 41 creating, 23 creating+activating, 640 active+clean; 182 kB data, 19550 MB used, 59451 MB / 83362 MB avail; 184 B/s wr, 0 op/s
Oct 23 17:08:28 deisNode0 sh[1782]: 2015-10-23 17:08:28.287915 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v132: 704 pgs: 21 creating, 43 creating+activating, 640 active+clean; 182 kB data, 19576 MB used, 59425 MB / 83362 MB avail; 196 B/s wr, 0 op/s
Oct 23 17:08:29 deisNode0 sh[1782]: 2015-10-23 17:08:29.804293 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v133: 704 pgs: 5 creating+peering, 59 creating+activating, 640 active+clean; 182 kB data, 19603 MB used, 59399 MB / 83362 MB avail
Oct 23 17:08:32 deisNode0 sh[1782]: 2015-10-23 17:08:32.726952 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v134: 704 pgs: 5 creating+peering, 59 creating+activating, 640 active+clean; 182 kB data, 19604 MB used, 59397 MB / 83362 MB avail; 450 B/s wr, 0 op/s
Oct 23 17:08:34 deisNode0 sh[1782]: 2015-10-23 17:08:34.057171 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v135: 704 pgs: 5 creating+peering, 59 creating+activating, 640 active+clean; 188 kB data, 19604 MB used, 59397 MB / 83362 MB avail; 2453 B/s wr, 0 op/s
Oct 23 17:08:35 deisNode0 sh[1782]: 2015-10-23 17:08:35.234936 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v136: 704 pgs: 64 creating+activating, 640 active+clean; 188 kB data, 19613 MB used, 59388 MB / 83362 MB avail; 3749 B/s wr, 0 op/s
Oct 23 17:08:37 deisNode0 sh[1782]: 2015-10-23 17:08:37.695701 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v137: 704 pgs: 55 creating+activating, 649 active+clean; 197 kB data, 19614 MB used, 59387 MB / 83362 MB avail; 4095 B/s wr, 1 op/s
Oct 23 17:08:39 deisNode0 sh[1782]: 2015-10-23 17:08:39.197993 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v138: 704 pgs: 44 creating+activating, 660 active+clean; 2163 kB data, 19614 MB used, 59387 MB / 83362 MB avail; 537 kB/s wr, 2 op/s
Oct 23 17:08:41 deisNode0 sh[1782]: 2015-10-23 17:08:41.541177 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v139: 704 pgs: 3 active, 38 creating+activating, 663 active+clean; 2163 kB data, 19614 MB used, 59387 MB / 83362 MB avail; 1056 B/s rd, 507 kB/s wr, 2 op/s
Oct 23 17:08:42 deisNode0 sh[1782]: 2015-10-23 17:08:42.698292 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v140: 704 pgs: 3 active, 34 creating+activating, 667 active+clean; 4126 kB data, 19616 MB used, 59385 MB / 83362 MB avail; 1078 B/s rd, 517 kB/s wr, 4 op/s
Oct 23 17:08:44 deisNode0 sh[1782]: 2015-10-23 17:08:44.307989 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v141: 704 pgs: 3 active, 31 creating+activating, 670 active+clean; 4126 kB data, 19618 MB used, 59383 MB / 83362 MB avail; 848 kB/s wr, 5 op/s
Oct 23 17:08:45 deisNode0 sh[1782]: 2015-10-23 17:08:45.637149 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v142: 704 pgs: 3 active, 29 creating+activating, 672 active+clean; 4126 kB data, 19631 MB used, 59370 MB / 83362 MB avail
Oct 23 17:08:47 deisNode0 sh[1782]: 2015-10-23 17:08:47.028267 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v143: 704 pgs: 3 active, 25 creating+activating, 676 active+clean; 4126 kB data, 19633 MB used, 59368 MB / 83362 MB avail
Oct 23 17:08:48 deisNode0 sh[1782]: 2015-10-23 17:08:48.715501 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v144: 704 pgs: 3 active, 21 creating+activating, 680 active+clean; 4126 kB data, 19635 MB used, 59366 MB / 83362 MB avail
Oct 23 17:08:51 deisNode0 sh[1782]: 2015-10-23 17:08:51.649538 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v145: 704 pgs: 6 active, 17 creating+activating, 681 active+clean; 6087 kB data, 19656 MB used, 59345 MB / 83362 MB avail; 503 B/s rd, 482 kB/s wr, 1 op/s; 2/261 objects degraded (0.766%)
Oct 23 17:08:52 deisNode0 sh[1782]: 2015-10-23 17:08:52.809520 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v146: 704 pgs: 6 active, 9 creating+activating, 689 active+clean; 8050 kB data, 19659 MB used, 59342 MB / 83362 MB avail; 475 B/s rd, 911 kB/s wr, 3 op/s; 2/264 objects degraded (0.758%)
Oct 23 17:08:55 deisNode0 sh[1782]: 2015-10-23 17:08:55.746277 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v147: 704 pgs: 704 active+clean; 10012 kB data, 19674 MB used, 59327 MB / 83362 MB avail; 8964 B/s rd, 859 kB/s wr, 17 op/s
Oct 23 17:08:57 deisNode0 sh[1782]: 2015-10-23 17:08:57.215070 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 73% total 27787 MB, used 6048 MB, avail 20285 MB
Oct 23 17:08:58 deisNode0 sh[1782]: 2015-10-23 17:08:58.183603 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v148: 704 pgs: 704 active+clean; 14482 kB data, 19679 MB used, 59322 MB / 83362 MB avail; 10838 B/s rd, 1513 kB/s wr, 25 op/s
Oct 23 17:08:59 deisNode0 sh[1782]: 2015-10-23 17:08:59.339815 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v149: 704 pgs: 704 active+clean; 14994 kB data, 19681 MB used, 59320 MB / 83362 MB avail; 1385 B/s rd, 1348 kB/s wr, 10 op/s
Oct 23 17:09:01 deisNode0 sh[1782]: 2015-10-23 17:09:01.015087 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v150: 704 pgs: 704 active+clean; 14994 kB data, 19692 MB used, 59309 MB / 83362 MB avail; 3329 B/s rd, 139 kB/s wr, 8 op/s
Oct 23 17:09:02 deisNode0 sh[1782]: 2015-10-23 17:09:02.371439 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v151: 704 pgs: 704 active+clean; 14997 kB data, 19697 MB used, 59304 MB / 83362 MB avail; 4339 B/s rd, 1808 B/s wr, 11 op/s
Oct 23 17:09:04 deisNode0 sh[1782]: 2015-10-23 17:09:04.065001 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v152: 704 pgs: 704 active+clean; 15079 kB data, 19702 MB used, 59299 MB / 83362 MB avail; 31375 B/s wr, 7 op/s
Oct 23 17:09:07 deisNode0 sh[1782]: 2015-10-23 17:09:07.061550 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v153: 704 pgs: 704 active+clean; 15081 kB data, 19702 MB used, 59299 MB / 83362 MB avail; 19485 B/s wr, 3 op/s
Oct 23 17:09:12 deisNode0 sh[1782]: 2015-10-23 17:09:12.031376 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v154: 704 pgs: 704 active+clean; 15083 kB data, 19702 MB used, 59299 MB / 83362 MB avail; 1108 B/s wr, 0 op/s
Oct 23 17:09:13 deisNode0 sh[1782]: 2015-10-23 17:09:13.140072 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v155: 704 pgs: 704 active+clean; 15088 kB data, 19719 MB used, 59282 MB / 83362 MB avail; 1837 B/s wr, 0 op/s
Oct 23 17:09:17 deisNode0 sh[1782]: 2015-10-23 17:09:17.336544 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v156: 704 pgs: 704 active+clean; 15092 kB data, 19719 MB used, 59282 MB / 83362 MB avail; 2252 B/s wr, 0 op/s
Oct 23 17:09:20 deisNode0 sh[1782]: 2015-10-23 17:09:20.629115 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v157: 704 pgs: 704 active+clean; 15092 kB data, 19746 MB used, 59255 MB / 83362 MB avail; 686 B/s wr, 0 op/s
Oct 23 17:09:22 deisNode0 sh[1782]: 2015-10-23 17:09:22.831617 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v158: 704 pgs: 704 active+clean; 15093 kB data, 19746 MB used, 59255 MB / 83362 MB avail; 351 B/s wr, 0 op/s
Oct 23 17:09:27 deisNode0 sh[1782]: 2015-10-23 17:09:27.035689 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v159: 704 pgs: 704 active+clean; 15095 kB data, 19746 MB used, 59255 MB / 83362 MB avail; 638 B/s wr, 0 op/s
Oct 23 17:09:29 deisNode0 sh[1782]: 2015-10-23 17:09:29.004785 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v160: 704 pgs: 704 active+clean; 15099 kB data, 19747 MB used, 59254 MB / 83362 MB avail; 1922 B/s wr, 0 op/s
Oct 23 17:09:37 deisNode0 sh[1782]: 2015-10-23 17:09:37.026143 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v161: 704 pgs: 704 active+clean; 15101 kB data, 19748 MB used, 59253 MB / 83362 MB avail; 1126 B/s wr, 0 op/s
Oct 23 17:09:38 deisNode0 sh[1782]: 2015-10-23 17:09:38.189165 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v162: 704 pgs: 704 active+clean; 15104 kB data, 19748 MB used, 59253 MB / 83362 MB avail; 868 B/s wr, 0 op/s
Oct 23 17:09:41 deisNode0 sh[1782]: 2015-10-23 17:09:41.311493 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v163: 704 pgs: 704 active+clean; 15104 kB data, 19791 MB used, 59210 MB / 83362 MB avail; 1704 B/s wr, 0 op/s
Oct 23 17:09:43 deisNode0 sh[1782]: 2015-10-23 17:09:43.234569 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v164: 704 pgs: 704 active+clean; 15112 kB data, 19791 MB used, 59210 MB / 83362 MB avail; 2589 B/s wr, 0 op/s
Oct 23 17:09:47 deisNode0 sh[1782]: 2015-10-23 17:09:47.030516 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v165: 704 pgs: 704 active+clean; 15119 kB data, 19791 MB used, 59210 MB / 83362 MB avail; 3360 B/s wr, 0 op/s
Oct 23 17:09:48 deisNode0 sh[1782]: 2015-10-23 17:09:48.754367 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v166: 704 pgs: 704 active+clean; 15122 kB data, 19791 MB used, 59210 MB / 83362 MB avail; 2881 B/s wr, 0 op/s
Oct 23 17:09:52 deisNode0 sh[1782]: 2015-10-23 17:09:52.046605 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v167: 704 pgs: 704 active+clean; 15123 kB data, 19791 MB used, 59210 MB / 83362 MB avail; 1228 B/s wr, 0 op/s
Oct 23 17:09:57 deisNode0 sh[1782]: 2015-10-23 17:09:57.215725 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 72% total 27787 MB, used 6067 MB, avail 20266 MB
Oct 23 17:09:57 deisNode0 sh[1782]: 2015-10-23 17:09:57.827408 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v168: 704 pgs: 704 active+clean; 15123 kB data, 19792 MB used, 59210 MB / 83362 MB avail; 212 B/s wr, 0 op/s
Oct 23 17:10:00 deisNode0 sh[1782]: 2015-10-23 17:10:00.834272 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v169: 704 pgs: 704 active+clean; 15123 kB data, 19831 MB used, 59170 MB / 83362 MB avail; 477 B/s rd, 238 B/s wr, 1 op/s
Oct 23 17:10:02 deisNode0 sh[1782]: 2015-10-23 17:10:02.521604 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v170: 704 pgs: 704 active+clean; 15872 kB data, 19831 MB used, 59170 MB / 83362 MB avail; 876 B/s rd, 160 kB/s wr, 3 op/s
Oct 23 17:10:05 deisNode0 sh[1782]: 2015-10-23 17:10:05.615588 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v171: 704 pgs: 704 active+clean; 15872 kB data, 19840 MB used, 59161 MB / 83362 MB avail; 409 B/s rd, 149 kB/s wr, 2 op/s
Oct 23 17:10:22 deisNode0 sh[1782]: 2015-10-23 17:10:22.073227 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v172: 704 pgs: 704 active+clean; 15875 kB data, 19841 MB used, 59160 MB / 83362 MB avail; 104 B/s rd, 157 B/s wr, 0 op/s
Oct 23 17:10:24 deisNode0 sh[1782]: 2015-10-23 17:10:24.138535 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v173: 704 pgs: 704 active+clean; 15877 kB data, 19842 MB used, 59160 MB / 83362 MB avail; 406 B/s wr, 0 op/s
Oct 23 17:10:26 deisNode0 sh[1782]: 2015-10-23 17:10:26.514406 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v174: 704 pgs: 704 active+clean; 15877 kB data, 19890 MB used, 59111 MB / 83362 MB avail; 1158 B/s wr, 0 op/s
Oct 23 17:10:32 deisNode0 sh[1782]: 2015-10-23 17:10:32.031399 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v175: 704 pgs: 704 active+clean; 15879 kB data, 19890 MB used, 59111 MB / 83362 MB avail; 464 B/s wr, 0 op/s
Oct 23 17:10:33 deisNode0 sh[1782]: 2015-10-23 17:10:33.343658 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v176: 704 pgs: 704 active+clean; 15880 kB data, 19890 MB used, 59111 MB / 83362 MB avail; 1218 B/s wr, 0 op/s
Oct 23 17:10:57 deisNode0 sh[1782]: 2015-10-23 17:10:57.216145 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 72% total 27787 MB, used 6068 MB, avail 20265 MB
Oct 23 17:10:58 deisNode0 sh[1782]: 2015-10-23 17:10:58.063808 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v177: 704 pgs: 704 active+clean; 15882 kB data, 19890 MB used, 59111 MB / 83362 MB avail; 128 B/s wr, 0 op/s
Oct 23 17:10:59 deisNode0 sh[1782]: 2015-10-23 17:10:59.204371 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v178: 704 pgs: 704 active+clean; 15884 kB data, 19891 MB used, 59110 MB / 83362 MB avail; 315 B/s wr, 0 op/s
Oct 23 17:11:01 deisNode0 sh[1782]: 2015-10-23 17:11:01.157958 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v179: 704 pgs: 704 active+clean; 15884 kB data, 19946 MB used, 59055 MB / 83362 MB avail; 1580 B/s wr, 0 op/s
Oct 23 17:11:32 deisNode0 sh[1782]: 2015-10-23 17:11:32.042621 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v180: 704 pgs: 704 active+clean; 15886 kB data, 19946 MB used, 59055 MB / 83362 MB avail; 98 B/s wr, 0 op/s
Oct 23 17:11:33 deisNode0 sh[1782]: 2015-10-23 17:11:33.134920 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v181: 704 pgs: 704 active+clean; 15887 kB data, 19946 MB used, 59055 MB / 83362 MB avail; 158 B/s wr, 0 op/s
Oct 23 17:11:36 deisNode0 sh[1782]: 2015-10-23 17:11:36.279894 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v182: 704 pgs: 704 active+clean; 15887 kB data, 19992 MB used, 59009 MB / 83362 MB avail; 857 B/s wr, 0 op/s
Oct 23 17:11:57 deisNode0 sh[1782]: 2015-10-23 17:11:57.216499 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 72% total 27787 MB, used 6068 MB, avail 20265 MB
Oct 23 17:12:02 deisNode0 sh[1782]: 2015-10-23 17:12:02.396094 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v183: 704 pgs: 704 active+clean; 15893 kB data, 19992 MB used, 59009 MB / 83362 MB avail; 309 B/s wr, 0 op/s
Oct 23 17:12:05 deisNode0 sh[1782]: 2015-10-23 17:12:05.660492 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v184: 704 pgs: 704 active+clean; 15893 kB data, 20037 MB used, 58964 MB / 83362 MB avail; 307 B/s wr, 0 op/s
Oct 23 17:12:08 deisNode0 sh[1782]: 2015-10-23 17:12:08.239993 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v185: 704 pgs: 704 active+clean; 15897 kB data, 20037 MB used, 58964 MB / 83362 MB avail; 1221 B/s wr, 0 op/s
Oct 23 17:12:12 deisNode0 sh[1782]: 2015-10-23 17:12:12.051153 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v186: 704 pgs: 704 active+clean; 15898 kB data, 20038 MB used, 58964 MB / 83362 MB avail; 1912 B/s rd, 1275 B/s wr, 3 op/s
Oct 23 17:12:15 deisNode0 sh[1782]: 2015-10-23 17:12:15.144577 7f33f2a55700 1 mon.deisNode0@0(leader).osd e27 e27: 4 osds: 3 up, 3 in
Oct 23 17:12:15 deisNode0 sh[1782]: 2015-10-23 17:12:15.206965 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e27: 4 osds: 3 up, 3 in
Oct 23 17:12:15 deisNode0 sh[1782]: 2015-10-23 17:12:15.381075 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v187: 768 pgs: 64 creating, 704 active+clean; 15898 kB data, 20038 MB used, 58964 MB / 83362 MB avail; 1669 B/s rd, 278 B/s wr, 2 op/s
Oct 23 17:12:16 deisNode0 sh[1782]: 2015-10-23 17:12:16.488700 7f33f2a55700 1 mon.deisNode0@0(leader).osd e28 e28: 4 osds: 3 up, 3 in
Oct 23 17:12:16 deisNode0 sh[1782]: 2015-10-23 17:12:16.565811 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e28: 4 osds: 3 up, 3 in
Oct 23 17:12:16 deisNode0 sh[1782]: 2015-10-23 17:12:16.738267 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v188: 768 pgs: 64 creating, 704 active+clean; 15898 kB data, 20038 MB used, 58964 MB / 83362 MB avail
Oct 23 17:12:21 deisNode0 sh[1782]: 2015-10-23 17:12:21.151932 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v189: 768 pgs: 38 creating, 11 creating+peering, 15 creating+activating, 704 active+clean; 15899 kB data, 20131 MB used, 58870 MB / 83362 MB avail; 350 B/s wr, 1 op/s
Oct 23 17:12:22 deisNode0 sh[1782]: 2015-10-23 17:12:22.310339 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v190: 768 pgs: 11 creating+peering, 53 creating+activating, 704 active+clean; 15899 kB data, 20132 MB used, 58869 MB / 83362 MB avail; 10190 B/s rd, 727 B/s wr, 19 op/s
Oct 23 17:12:26 deisNode0 sh[1782]: 2015-10-23 17:12:26.520637 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v191: 768 pgs: 12 active, 50 creating+activating, 706 active+clean; 15899 kB data, 20133 MB used, 58868 MB / 83362 MB avail; 10611 B/s rd, 378 B/s wr, 18 op/s
Oct 23 17:12:31 deisNode0 sh[1782]: 2015-10-23 17:12:31.569330 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v192: 768 pgs: 12 active, 35 creating+activating, 721 active+clean; 15899 kB data, 20135 MB used, 58867 MB / 83362 MB avail
Oct 23 17:12:32 deisNode0 sh[1782]: 2015-10-23 17:12:32.897258 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v193: 768 pgs: 768 active+clean; 15900 kB data, 20136 MB used, 58865 MB / 83362 MB avail; 825 B/s rd, 165 B/s wr, 1 op/s
Oct 23 17:12:36 deisNode0 sh[1782]: 2015-10-23 17:12:36.255979 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v194: 768 pgs: 768 active+clean; 17948 kB data, 20142 MB used, 58859 MB / 83362 MB avail; 1023 B/s rd, 409 kB/s wr, 3 op/s
Oct 23 17:12:37 deisNode0 sh[1782]: 2015-10-23 17:12:37.584470 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v195: 768 pgs: 768 active+clean; 26142 kB data, 20153 MB used, 58848 MB / 83362 MB avail; 2187 B/s rd, 2188 kB/s wr, 11 op/s
Oct 23 17:12:41 deisNode0 sh[1782]: 2015-10-23 17:12:41.273103 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v196: 768 pgs: 768 active+clean; 26146 kB data, 20163 MB used, 58838 MB / 83362 MB avail; 2047 B/s rd, 1640 kB/s wr, 9 op/s
Oct 23 17:12:42 deisNode0 sh[1782]: 2015-10-23 17:12:42.621134 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v197: 768 pgs: 768 active+clean; 31276 kB data, 20178 MB used, 58823 MB / 83362 MB avail; 1023 B/s rd, 1026 kB/s wr, 5 op/s
Oct 23 17:12:46 deisNode0 sh[1782]: 2015-10-23 17:12:46.256654 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v198: 768 pgs: 768 active+clean; 36405 kB data, 20178 MB used, 58823 MB / 83362 MB avail; 1228 B/s rd, 2051 kB/s wr, 8 op/s
Oct 23 17:12:47 deisNode0 sh[1782]: 2015-10-23 17:12:47.382423 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v199: 768 pgs: 768 active+clean; 41528 kB data, 20189 MB used, 58812 MB / 83362 MB avail; 2067 B/s rd, 2070 kB/s wr, 10 op/s
Oct 23 17:12:51 deisNode0 sh[1782]: 2015-10-23 17:12:51.276153 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v200: 768 pgs: 768 active+clean; 42552 kB data, 20199 MB used, 58802 MB / 83362 MB avail; 1842 B/s rd, 1229 kB/s wr, 7 op/s
Oct 23 17:12:52 deisNode0 sh[1782]: 2015-10-23 17:12:52.667302 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v201: 768 pgs: 768 active+clean; 49977 kB data, 20219 MB used, 58783 MB / 83362 MB avail; 7913 B/s rd, 1632 kB/s wr, 17 op/s
Oct 23 17:12:56 deisNode0 sh[1782]: 2015-10-23 17:12:56.245200 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v202: 768 pgs: 768 active+clean; 49978 kB data, 20227 MB used, 58774 MB / 83362 MB avail; 8190 B/s rd, 1485 kB/s wr, 18 op/s
Oct 23 17:12:57 deisNode0 sh[1782]: 2015-10-23 17:12:57.216916 7f33f1a53700 0 mon.deisNode0@0(leader).data_health(12) update_stats avail 72% total 27787 MB, used 6105 MB, avail 20228 MB
Oct 23 17:12:58 deisNode0 sh[1782]: 2015-10-23 17:12:58.105460 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v203: 768 pgs: 768 active+clean; 49994 kB data, 20151 MB used, 58851 MB / 83362 MB avail; 14892 B/s rd, 4893 B/s wr, 37 op/s
Oct 23 17:13:01 deisNode0 sh[1782]: 2015-10-23 17:13:01.309149 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v204: 768 pgs: 768 active+clean; 49995 kB data, 20151 MB used, 58850 MB / 83362 MB avail; 14537 B/s rd, 4709 B/s wr, 36 op/s
Oct 23 17:13:02 deisNode0 sh[1782]: 2015-10-23 17:13:02.464814 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v205: 768 pgs: 768 active+clean; 50020 kB data, 20151 MB used, 58850 MB / 83362 MB avail; 15570 B/s rd, 6673 B/s wr, 35 op/s
Oct 23 17:13:06 deisNode0 sh[1782]: 2015-10-23 17:13:06.436058 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v206: 768 pgs: 768 active+clean; 50022 kB data, 20152 MB used, 58849 MB / 83362 MB avail; 15561 B/s rd, 6961 B/s wr, 36 op/s
Oct 23 17:13:12 deisNode0 sh[1782]: 2015-10-23 17:13:12.190650 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v207: 768 pgs: 768 active+clean; 50022 kB data, 20152 MB used, 58849 MB / 83362 MB avail; 399 B/s rd, 599 B/s wr, 3 op/s
Oct 23 17:13:30 deisNode0 sh[1782]: 2015-10-23 17:13:30.848116 7f33f1a53700 1 mon.deisNode0@0(leader).paxos(paxos updating c 1..456) accept timeout, calling fresh election
Oct 23 17:13:30 deisNode0 sh[1782]: 2015-10-23 17:13:30.850426 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:13:30 deisNode0 sh[1782]: 2015-10-23 17:13:30.850564 7f33f1252700 1 mon.deisNode0@0(electing).elector(12) init, last seen epoch 12
Oct 23 17:13:35 deisNode0 sh[1782]: 2015-10-23 17:13:35.930265 7f33f1a53700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,2
Oct 23 17:13:35 deisNode0 sh[1782]: 2015-10-23 17:13:35.962284 7f33f1a53700 0 log_channel(cluster) log [INF] : HEALTH_WARN; 1 mons down, quorum 0,2 deisNode0,deisNode2
Oct 23 17:13:36 deisNode0 sh[1782]: 2015-10-23 17:13:36.118540 7f33f2a55700 0 log_channel(cluster) log [INF] : monmap e3: 3 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0,deisNode2=10.0.0.6:6789/0}
Oct 23 17:13:36 deisNode0 sh[1782]: 2015-10-23 17:13:36.118696 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v208: 768 pgs: 768 active+clean; 50022 kB data, 20152 MB used, 58849 MB / 83362 MB avail; 290 B/s rd, 0 B/s wr, 0 op/s
Oct 23 17:13:36 deisNode0 sh[1782]: 2015-10-23 17:13:36.118849 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e6: 1/1/1 up {0=deisNode2=up:active}, 2 up:standby
Oct 23 17:13:36 deisNode0 sh[1782]: 2015-10-23 17:13:36.118937 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e28: 4 osds: 3 up, 3 in
Oct 23 17:13:37 deisNode0 sh[1782]: 2015-10-23 17:13:37.509539 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v209: 768 pgs: 768 active+clean; 50031 kB data, 20152 MB used, 58849 MB / 83362 MB avail; 582 B/s wr, 0 op/s
Oct 23 17:13:41 deisNode0 sh[1782]: 2015-10-23 17:13:41.665399 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v210: 768 pgs: 768 active+clean; 50035 kB data, 21309 MB used, 57692 MB / 83362 MB avail; 362 B/s rd, 1051 B/s wr, 0 op/s
Oct 23 17:13:46 deisNode0 sh[1782]: 2015-10-23 17:13:46.690314 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0 calling new monitor election
Oct 23 17:13:46 deisNode0 sh[1782]: 2015-10-23 17:13:46.690445 7f33f1252700 1 mon.deisNode0@0(electing).elector(14) init, last seen epoch 14
Oct 23 17:13:46 deisNode0 sh[1782]: 2015-10-23 17:13:46.964929 7f33f1252700 0 log_channel(cluster) log [INF] : mon.deisNode0@0 won leader election with quorum 0,1,2
Oct 23 17:13:47 deisNode0 sh[1782]: 2015-10-23 17:13:47.012380 7f33f1252700 0 log_channel(cluster) log [INF] : HEALTH_OK
Oct 23 17:13:47 deisNode0 sh[1782]: 2015-10-23 17:13:47.481016 7f33f2a55700 0 log_channel(cluster) log [INF] : monmap e3: 3 mons at {deisNode0=10.0.0.4:6789/0,deisNode1=10.0.0.5:6789/0,deisNode2=10.0.0.6:6789/0}
Oct 23 17:13:47 deisNode0 sh[1782]: 2015-10-23 17:13:47.481149 7f33f2a55700 0 log_channel(cluster) log [INF] : pgmap v211: 768 pgs: 768 active+clean; 55155 kB data, 21314 MB used, 57687 MB / 83362 MB avail; 2017 B/s rd, 561 kB/s wr, 5 op/s
Oct 23 17:13:47 deisNode0 sh[1782]: 2015-10-23 17:13:47.481277 7f33f2a55700 0 log_channel(cluster) log [INF] : mdsmap e6: 1/1/1 up {0=deisNode2=up:active}, 2 up:standby
Oct 23 17:13:47 deisNode0 sh[1782]: 2015-10-23 17:13:47.481398 7f33f2a55700 0 log_channel(cluster) log [INF] : osdmap e28: 4 osds: 3 up, 3 in
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment