DC-ip-10-35-147-209.log

Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: - <cib admin_epoch="0" epoch="409" num_updates="16" />
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: + <cib epoch="410" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.5" update-origin="ip-10-35-147-209" update-client="crmd" cib-last-written="Wed Jun 26 16:44:32 2013" have-quorum="1" dc-uuid="ip-10-35-147-209" >
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   <configuration >
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     <crm_config >
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" >
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="false" __crm_diff_marker__="added:top" />
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       </cluster_property_set>
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     </crm_config>
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   </configuration>
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib:diff: + </cib>
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crm_attribute/5, version=0.410.1): ok (rc=0)
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: abort_transition_graph: te_update_diff:124 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.410.1) : Non-status change
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke: Query 316: Requesting the current CIB: S_POLICY_ENGINE
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke_callback: Invoking the PE: query=316, ref=pe_calc-dc-1372265234-161, seq=28, quorate=1
Jun 26 16:47:14 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Startup probes: enabled
Jun 26 16:47:14 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Jun 26 16:47:14 ip-10-35-147-209 pengine: [3944]: info: unpack_domains: Unpacking domains
Jun 26 16:47:14 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-34-151-73 is online
Jun 26 16:47:14 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-35-147-209 is online
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: crmd_ais_dispatch: Setting expected votes to 2
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/319, version=0.410.2): ok (rc=0)
Jun 26 16:47:14 ip-10-35-147-209 pengine: [3944]: notice: process_pe_message: Transition 40: PEngine Input stored in: /var/lib/pengine/pe-input-398.bz2
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: unpack_graph: Unpacked transition 40: 0 actions in 0 synapses
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_te_invoke: Processing graph 40 (ref=pe_calc-dc-1372265234-161) derived from /var/lib/pengine/pe-input-398.bz2
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: run_graph: ====================================================
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: notice: run_graph: Transition 40 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-398.bz2): Complete
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: te_graph_trigger: Transition 40 is now complete
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: notify_crmd: Transition 40 status: done - <null>
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Jun 26 16:47:14 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: Starting PEngine Recheck Timer
Jun 26 16:47:14 ip-10-35-147-209 cib: [3941]: info: Managed write_cib_contents process 6669 exited with return code 0.
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: - <cib admin_epoch="0" epoch="410" num_updates="2" />
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: + <cib epoch="411" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.5" update-origin="ip-10-35-147-209" update-client="crm_attribute" cib-last-written="Wed Jun 26 16:47:14 2013" have-quorum="1" dc-uuid="ip-10-35-147-209" >
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   <configuration >
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     <crm_config >
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" >
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="ignore" __crm_diff_marker__="added:top" />
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       </cluster_property_set>
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     </crm_config>
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   </configuration>
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib:diff: + </cib>
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crm_attribute/4, version=0.411.1): ok (rc=0)
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: abort_transition_graph: te_update_diff:124 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.411.1) : Non-status change
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke: Query 320: Requesting the current CIB: S_POLICY_ENGINE
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke_callback: Invoking the PE: query=320, ref=pe_calc-dc-1372265239-162, seq=28, quorate=1
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Startup probes: enabled
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: notice: unpack_config: On loss of CCM Quorum: Ignore
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: info: unpack_domains: Unpacking domains
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-34-151-73 is online
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-35-147-209 is online
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: crmd_ais_dispatch: Setting expected votes to 2
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/323, version=0.411.2): ok (rc=0)
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: unpack_graph: Unpacked transition 41: 0 actions in 0 synapses
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_te_invoke: Processing graph 41 (ref=pe_calc-dc-1372265239-162) derived from /var/lib/pengine/pe-input-399.bz2
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: run_graph: ====================================================
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: notice: run_graph: Transition 41 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-399.bz2): Complete
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: te_graph_trigger: Transition 41 is now complete
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: notify_crmd: Transition 41 status: done - <null>
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Jun 26 16:47:19 ip-10-35-147-209 pengine: [3944]: notice: process_pe_message: Transition 41: PEngine Input stored in: /var/lib/pengine/pe-input-399.bz2
Jun 26 16:47:19 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: Starting PEngine Recheck Timer
Jun 26 16:47:19 ip-10-35-147-209 cib: [3941]: info: Managed write_cib_contents process 6671 exited with return code 0.
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: - <cib admin_epoch="0" epoch="411" num_updates="2" />
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: + <cib epoch="412" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.5" update-origin="ip-10-35-147-209" update-client="crm_attribute" cib-last-written="Wed Jun 26 16:47:19 2013" have-quorum="1" dc-uuid="ip-10-35-147-209" >
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   <configuration >
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     <crm_config >
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       <cluster_property_set id="cib-bootstrap-options" >
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         <nvpair id="cib-bootstrap-options-symmetric-cluster" name="symmetric-cluster" value="false" __crm_diff_marker__="added:top" />
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       </cluster_property_set>
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     </crm_config>
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   </configuration>
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib:diff: + </cib>
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crm_attribute/4, version=0.412.1): ok (rc=0)
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: abort_transition_graph: te_update_diff:124 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.412.1) : Non-status change
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke: Query 324: Requesting the current CIB: S_POLICY_ENGINE
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke_callback: Invoking the PE: query=324, ref=pe_calc-dc-1372265243-163, seq=28, quorate=1
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Startup probes: enabled
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: notice: unpack_config: On loss of CCM Quorum: Ignore
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: info: unpack_domains: Unpacking domains
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-34-151-73 is online
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-35-147-209 is online
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: unpack_graph: Unpacked transition 42: 0 actions in 0 synapses
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_te_invoke: Processing graph 42 (ref=pe_calc-dc-1372265243-163) derived from /var/lib/pengine/pe-input-400.bz2
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:47:23 ip-10-35-147-209 pengine: [3944]: notice: process_pe_message: Transition 42: PEngine Input stored in: /var/lib/pengine/pe-input-400.bz2
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: crmd_ais_dispatch: Setting expected votes to 2
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: run_graph: ====================================================
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: notice: run_graph: Transition 42 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-400.bz2): Complete
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: te_graph_trigger: Transition 42 is now complete
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: notify_crmd: Transition 42 status: done - <null>
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Jun 26 16:47:23 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: Starting PEngine Recheck Timer
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/327, version=0.412.2): ok (rc=0)
Jun 26 16:47:23 ip-10-35-147-209 cib: [3941]: info: Managed write_cib_contents process 6673 exited with return code 0.
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib_replace_notify: Replaced: 0.412.2 -> 0.413.1 from <null>
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: - <cib admin_epoch="0" epoch="412" num_updates="2" />
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: + <cib epoch="413" num_updates="1" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.5" update-origin="ip-10-35-147-209" update-client="crm_attribute" cib-last-written="Wed Jun 26 16:47:23 2013" have-quorum="1" dc-uuid="ip-10-35-147-209" >
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   <configuration >
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     <resources >
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       <primitive class="ocf" id="ping" provider="pacemaker" type="ping" __crm_diff_marker__="added:top" >
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         <instance_attributes id="ping-instance_attributes" >
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +           <nvpair id="ping-instance_attributes-host_list" name="host_list" value="10.34.151.73" />
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         </instance_attributes>
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         <operations >
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +           <op id="ping-monitor-15s" interval="15s" name="monitor" timeout="5s" />
Jun 26 16:48:01 ip-10-35-147-209 attrd: [3943]: info: do_cib_replaced: Sending full refresh
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +         </operations>
Jun 26 16:48:01 ip-10-35-147-209 attrd: [3943]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +       </primitive>
Jun 26 16:48:01 ip-10-35-147-209 crmd: [3945]: info: abort_transition_graph: te_update_diff:124 - Triggered transition abort (complete=1, tag=diff, id=(null), magic=NA, cib=0.413.1) : Non-status change
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +     </resources>
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: +   </configuration>
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib:diff: + </cib>
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_replace for section 'all' (origin=local/cibadmin/2, version=0.413.1): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE [ input=I_PE_CALC cause=C_FSA_INTERNAL origin=abort_transition_graph ]
Jun 26 16:48:01 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_sync_one for section 'all' (origin=ip-10-34-151-73/ip-10-34-151-73/(null), version=0.413.1): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke: Query 330: Requesting the current CIB: S_POLICY_ENGINE
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_ELECTION [ input=I_ELECTION cause=C_FSA_INTERNAL origin=do_cib_replaced ]
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: update_dc: Unset DC ip-10-35-147-209
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/328, version=0.413.4): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_ELECTION -> S_INTEGRATION [ input=I_ELECTION_DC cause=C_FSA_INTERNAL origin=do_election_check ]
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: Managed write_cib_contents process 6719 exited with return code 0.
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_dc_takeover: Taking over DC status for this partition
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_master for section 'all' (origin=local/crmd/331, version=0.413.8): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/332, version=0.413.9): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/334, version=0.413.10): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/336, version=0.413.11): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_dc_join_offer_all: join-10: Waiting on 2 outstanding join acks
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: crmd_ais_dispatch: Setting expected votes to 2
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/339, version=0.413.12): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: crmd_ais_dispatch: Setting expected votes to 2
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section crm_config (origin=local/crmd/341, version=0.413.13): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: update_dc: Set DC to ip-10-35-147-209 (3.0.5)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_INTEGRATION -> S_FINALIZE_JOIN [ input=I_INTEGRATED cause=C_FSA_INTERNAL origin=check_join_state ]
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: All 2 cluster nodes responded to the join offer.
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_dc_join_finalize: join-10: Syncing the CIB from ip-10-35-147-209 to the rest of the cluster
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_sync for section 'all' (origin=local/crmd/343, version=0.413.13): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/344, version=0.413.14): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/345, version=0.413.15): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_dc_join_ack: join-10: Updating node state to member for ip-10-34-151-73
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='ip-10-34-151-73']/lrm (origin=local/crmd/346, version=0.413.16): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_dc_join_ack: join-10: Updating node state to member for ip-10-35-147-209
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_delete for section //node_state[@uname='ip-10-35-147-209']/lrm (origin=local/crmd/348, version=0.413.18): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: erase_xpath_callback: Deletion of "//node_state[@uname='ip-10-34-151-73']/lrm": ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: erase_xpath_callback: Deletion of "//node_state[@uname='ip-10-35-147-209']/lrm": ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_FINALIZE_JOIN -> S_POLICY_ENGINE [ input=I_FINALIZED cause=C_FSA_INTERNAL origin=check_join_state ]
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section nodes (origin=local/crmd/350, version=0.413.22): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: All 2 cluster nodes are eligible to run resources.
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_dc_join_final: Ensuring DC, quorum and node attributes are up-to-date
Jun 26 16:48:02 ip-10-35-147-209 attrd: [3943]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: crm_update_quorum: Updating quorum status to true (call=352)
Jun 26 16:48:02 ip-10-35-147-209 cib: [3941]: info: cib_process_request: Operation complete: op cib_modify for section cib (origin=local/crmd/352, version=0.413.24): ok (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 attrd: [3943]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: abort_transition_graph: do_te_invoke:167 - Triggered transition abort (complete=1) : Peer Cancelled
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke: Query 353: Requesting the current CIB: S_POLICY_ENGINE
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_pe_invoke_callback: Invoking the PE: query=353, ref=pe_calc-dc-1372265282-171, seq=28, quorate=1
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Startup probes: enabled
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: notice: unpack_config: On loss of CCM Quorum: Ignore
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: unpack_config: Node scores: 'red' = -INFINITY, 'yellow' = 0, 'green' = 0
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: unpack_domains: Unpacking domains
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-34-151-73 is online
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: determine_online_status: Node ip-10-35-147-209 is online
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: native_print: ping (ocf::pacemaker:ping):	Stopped
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: info: native_color: Resource ping cannot run anywhere
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: notice: LogActions: Leave   ping	(Stopped)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_POLICY_ENGINE -> S_TRANSITION_ENGINE [ input=I_PE_SUCCESS cause=C_IPC_MESSAGE origin=handle_response ]
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: unpack_graph: Unpacked transition 43: 5 actions in 5 synapses
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_te_invoke: Processing graph 43 (ref=pe_calc-dc-1372265282-171) derived from /var/lib/pengine/pe-input-401.bz2
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: te_rsc_command: Initiating action 6: monitor ping_monitor_0 on ip-10-35-147-209 (local)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_lrm_rsc_op: Performing key=6:43:7:2b376a8b-9d80-4149-abec-7ba73cfb1aca op=ping_monitor_0 )
Jun 26 16:48:02 ip-10-35-147-209 lrmd: [3942]: info: rsc:ping probe[10] (pid 6722)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: te_rsc_command: Initiating action 4: monitor ping_monitor_0 on ip-10-34-151-73
Jun 26 16:48:02 ip-10-35-147-209 pengine: [3944]: notice: process_pe_message: Transition 43: PEngine Input stored in: /var/lib/pengine/pe-input-401.bz2
Jun 26 16:48:02 ip-10-35-147-209 lrmd: [3942]: WARN: Managed ping:monitor process 6722 exited with return code 7.
Jun 26 16:48:02 ip-10-35-147-209 lrmd: [3942]: info: operation monitor[10] on ping for client 3945: pid 6722 exited with return code 7
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: process_lrm_event: LRM operation ping_monitor_0 (call=10, rc=7, cib-update=354, confirmed=true) not running
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: match_graph_event: Action ping_monitor_0 (6) confirmed on ip-10-35-147-209 (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: te_rsc_command: Initiating action 5: probe_complete probe_complete on ip-10-35-147-209 (local) - no waiting
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: match_graph_event: Action ping_monitor_0 (4) confirmed on ip-10-34-151-73 (rc=0)
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: te_rsc_command: Initiating action 3: probe_complete probe_complete on ip-10-34-151-73 - no waiting
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: te_pseudo_action: Pseudo action 2 fired and confirmed
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: run_graph: ====================================================
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: notice: run_graph: Transition 43 (Complete=5, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pengine/pe-input-401.bz2): Complete
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: te_graph_trigger: Transition 43 is now complete
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: notify_crmd: Transition 43 status: done - <null>
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Jun 26 16:48:02 ip-10-35-147-209 crmd: [3945]: info: do_state_transition: Starting PEngine Recheck Timer

ip-10-34-151-73.log

Jun 26 16:47:14 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:47:14 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:47:14 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:47:14 ip-10-34-151-73 crmd: [3113]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:47:14 ip-10-34-151-73 cib: [3109]: info: Managed write_cib_contents process 3517 exited with return code 0.
Jun 26 16:47:19 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:47:19 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:47:19 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:47:19 ip-10-34-151-73 crmd: [3113]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:47:19 ip-10-34-151-73 cib: [3109]: info: Managed write_cib_contents process 3518 exited with return code 0.
Jun 26 16:47:23 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Shutdown escalation occurs after: 1200000ms
Jun 26 16:47:23 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Checking for expired actions every 900000ms
Jun 26 16:47:23 ip-10-34-151-73 crmd: [3113]: info: config_query_callback: Sending expected-votes=2 to corosync
Jun 26 16:47:23 ip-10-34-151-73 crmd: [3113]: info: ais_dispatch_message: Membership 28: quorum retained
Jun 26 16:47:23 ip-10-34-151-73 cib: [3109]: info: Managed write_cib_contents process 3519 exited with return code 0.
Jun 26 16:47:59 ip-10-34-151-73 cib: [3109]: info: cib_stats: Processed 79 operations (506.00us average, 0% utilization) in the last 10min
Jun 26 16:48:01 ip-10-34-151-73 cib: [3109]: info: apply_xml_diff: Digest mis-match: expected f73e8862886a127d45eb7739b4c01961, calculated 6b15d526a6b022916600cf4b9df841d8
Jun 26 16:48:01 ip-10-34-151-73 cib: [3109]: notice: cib_process_diff: Diff 0.412.2 -> 0.413.1 not applied to 0.412.2: Failed application of an update diff
Jun 26 16:48:01 ip-10-34-151-73 cib: [3109]: info: cib_server_process_diff: Requesting re-sync from peer
Jun 26 16:48:01 ip-10-34-151-73 cib: [3109]: info: cib_replace_notify: Replaced: -1.-1.-1 -> 0.413.1 from ip-10-35-147-209
Jun 26 16:48:01 ip-10-34-151-73 attrd: [3111]: info: do_cib_replaced: Sending full refresh
Jun 26 16:48:01 ip-10-34-151-73 attrd: [3111]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Jun 26 16:48:02 ip-10-34-151-73 cib: [3109]: info: Managed write_cib_contents process 3520 exited with return code 0.
Jun 26 16:48:02 ip-10-34-151-73 crmd: [3113]: info: update_dc: Unset DC ip-10-35-147-209
Jun 26 16:48:02 ip-10-34-151-73 crmd: [3113]: info: do_state_transition: State transition S_NOT_DC -> S_PENDING [ input=I_PENDING cause=C_FSA_INTERNAL origin=do_election_count_vote ]
Jun 26 16:48:02 ip-10-34-151-73 crmd: [3113]: info: update_dc: Set DC to ip-10-35-147-209 (3.0.5)
Jun 26 16:48:02 ip-10-34-151-73 crmd: [3113]: info: do_state_transition: State transition S_PENDING -> S_NOT_DC [ input=I_NOT_DC cause=C_HA_MESSAGE origin=do_cl_join_finalize_respond ]
Jun 26 16:48:02 ip-10-34-151-73 attrd: [3111]: notice: attrd_local_callback: Sending full refresh (origin=crmd)
Jun 26 16:48:02 ip-10-34-151-73 attrd: [3111]: notice: attrd_trigger_update: Sending flush op to all hosts for: probe_complete (true)
Jun 26 16:48:02 ip-10-34-151-73 cib: [3109]: info: Managed write_cib_contents process 3521 exited with return code 0.
Jun 26 16:48:02 ip-10-34-151-73 crmd: [3113]: info: do_lrm_rsc_op: Performing key=4:43:7:2b376a8b-9d80-4149-abec-7ba73cfb1aca op=ping_monitor_0 )
Jun 26 16:48:02 ip-10-34-151-73 lrmd: [3110]: info: rsc:ping probe[10] (pid 3522)
Jun 26 16:48:02 ip-10-34-151-73 lrmd: [3110]: WARN: Managed ping:monitor process 3522 exited with return code 7.
Jun 26 16:48:02 ip-10-34-151-73 lrmd: [3110]: info: operation monitor[10] on ping for client 3113: pid 3522 exited with return code 7
Jun 26 16:48:02 ip-10-34-151-73 crmd: [3113]: info: process_lrm_event: LRM operation ping_monitor_0 (call=10, rc=7, cib-update=69, confirmed=true) not running