Last active
August 29, 2015 14:17
-
-
Save trozet/89f53dc11c6dbc704f62 to your computer and use it in GitHub Desktop.
pacemaker miracles with foreman
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| At the end of this everything works, just missing neutron. | |
| 1. First I made changes to quickstack, clone my latest opendaylight branch por favor | |
| 2. Disable hiera in foreman (I guess its on by default?). Hiera will auto replace variable names not defined in smart class parameters for Foreman, and it automatically reads astapor defaults which ruins everything. To disable it: | |
| data_binding_terminus = none | |
| ^put that in your puppet.conf file under master on Foreman. | |
| 3. restart system services: foreman, foreman-proxy, httpd, puppet (also needed to pick up the library changes in quickstack) | |
| 4. Add all in one HA hostgroup type: | |
| {:name=>"HA All In One Controller", | |
| :class=>["quickstack::openstack_common", | |
| "quickstack::pacemaker::common", | |
| "quickstack::pacemaker::params", | |
| "quickstack::pacemaker::keystone", | |
| "quickstack::pacemaker::swift", | |
| "quickstack::pacemaker::load_balancer", | |
| "quickstack::pacemaker::memcached", | |
| "quickstack::pacemaker::qpid", | |
| "quickstack::pacemaker::rabbitmq", | |
| "quickstack::pacemaker::glance", | |
| "quickstack::pacemaker::nova", | |
| "quickstack::pacemaker::heat", | |
| "quickstack::pacemaker::cinder", | |
| "quickstack::pacemaker::horizon", | |
| "quickstack::pacemaker::galera", | |
| "quickstack::pacemaker::neutron", | |
| "quickstack::pacemaker::nosql", | |
| "quickstack::pacemaker::ceilometer", | |
| ]}, | |
| 5. Also add the trystack class to it so that it gets the RDO repo. Also add NTP or you get a clock out of sync error for Ceph. | |
| 6. set all the settings by overriding class parameters here: https://gist.github.com/trozet/4144b44b96ab03a8a086 | |
| 7. Copy these three files to /tmp/ and /etc/ceph on your Foreman Node. Foreman server will be your OSD host for now: | |
| https://gist.github.com/trozet/1a968e66602e56b369ef | |
| https://gist.github.com/trozet/c049f46a904823b05b4b | |
| https://gist.github.com/trozet/867b9172aacbff383c80 | |
| 8. Now we need to get ceph-deploy on foreman. It will deploy ceph OSD to foreman node and ceph mon to the three controllers: | |
| rpm -ivh http://cdn.stage.redhat.com/content/dist/rhel/server/7/7Server/x86_64/redhat-ceph-installer/1.2/os/Packages/ceph-deploy-1.5.22-0.1.rc1.el7cp.noarch.rpm | |
| 9. Now grab the bash script to deploy the nodes and put it on the foreman server: | |
| https://gist.github.com/trozet/7f4bf84157218639ee8e | |
| 10. Now rebuild your 3 controllers in foreman. When they complete kill puppet agent so you can intervene before everything gets configured. | |
| 11. Before we run puppet we want ceph up. Modify the 4 variables in the bash script to match your hostnames and ips. | |
| 12. Add your foreman ssh key to authorized hosts on your hosts so that you can ssh without a password to them. | |
| 13. Execute create_ceph.sh. If everything works then on your foreman node do: | |
| [root@foreman-server ~]# ceph -s | |
| cluster 904c8491-5c16-4dae-9cc3-6ce633a7f4cc | |
| health HEALTH_OK | |
| monmap e1: 3 mons at {oscompute11=10.4.9.4:6789/0,oscontroller1=10.4.9.2:6789/0,tempest1=10.4.8.51:6789/0}, election epoch 6, quorum 0,1,2 tempest1,oscontroller1,oscompute11 | |
| osdmap e25: 3 osds: 3 up, 3 in | |
| pgmap v427: 448 pgs, 5 pools, 16 bytes data, 3 objects | |
| 24238 MB used, 126 GB / 149 GB avail | |
| 448 active+clean | |
| 14. Now run puppet agent --test --debug on all 3 controllers. All three controllers should sync up for every resource and you can see it in the debug output. At the end you should get something like this (isn't it beautiful?): | |
| [root@oscompute11 ~]# pcs status | |
| Cluster name: openstack | |
| Last updated: Sat Mar 28 02:55:28 2015 | |
| Last change: Sat Mar 28 02:17:43 2015 via crmd on pcmk-oscontroller1 | |
| Stack: corosync | |
| Current DC: pcmk-tempest (3) - partition with quorum | |
| Version: 1.1.10-32.el7_0.1-368c726 | |
| 3 Nodes configured | |
| 59 Resources configured | |
| Online: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Full list of resources: | |
| ip-amqp-pub-10.4.9.81 (ocf::heartbeat:IPaddr2): Started pcmk-oscompute11 | |
| ip-loadbalancer-pub-10.4.9.89 (ocf::heartbeat:IPaddr2): Started pcmk-oscontroller1 | |
| Clone Set: memcached-clone [memcached] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: haproxy-clone [haproxy] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| ip-galera-pub-10.4.9.88 (ocf::heartbeat:IPaddr2): Started pcmk-tempest | |
| Master/Slave Set: galera-master [galera] | |
| Masters: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: rabbitmq-server-clone [rabbitmq-server] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| ip-keystone-pub-10.2.84.71 (ocf::heartbeat:IPaddr2): Started pcmk-oscompute11 | |
| ip-keystone-prv-10.4.9.71 (ocf::heartbeat:IPaddr2): Started pcmk-oscompute11 | |
| Clone Set: keystone-clone [keystone] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| ip-glance-pub-10.2.84.75 (ocf::heartbeat:IPaddr2): Started pcmk-oscontroller1 | |
| ip-glance-prv-10.4.9.75 (ocf::heartbeat:IPaddr2): Started pcmk-oscontroller1 | |
| Clone Set: glance-registry-clone [glance-registry] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: glance-api-clone [glance-api] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| ip-nova-pub-10.2.84.77 (ocf::heartbeat:IPaddr2): Started pcmk-tempest | |
| ip-nova-prv-10.4.9.77 (ocf::heartbeat:IPaddr2): Started pcmk-tempest | |
| Clone Set: openstack-nova-novncproxy-clone [openstack-nova-novncproxy] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: openstack-nova-consoleauth-clone [openstack-nova-consoleauth] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: openstack-nova-conductor-clone [openstack-nova-conductor] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: openstack-nova-api-clone [openstack-nova-api] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: openstack-nova-scheduler-clone [openstack-nova-scheduler] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| ip-cinder-pub-10.2.84.67 (ocf::heartbeat:IPaddr2): Started pcmk-oscompute11 | |
| ip-cinder-prv-10.4.9.67 (ocf::heartbeat:IPaddr2): Started pcmk-oscompute11 | |
| Clone Set: cinder-api-clone [cinder-api] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| Clone Set: cinder-scheduler-clone [cinder-scheduler] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| cinder-volume (systemd:openstack-cinder-volume): Started pcmk-oscontroller1 | |
| ip-horizon-pub-10.2.84.78 (ocf::heartbeat:IPaddr2): Started pcmk-oscontroller1 | |
| ip-horizon-prv-10.4.9.78 (ocf::heartbeat:IPaddr2): Started pcmk-oscontroller1 | |
| Clone Set: horizon-clone [horizon] | |
| Started: [ pcmk-oscompute11 pcmk-oscontroller1 pcmk-tempest ] | |
| PCSD Status: | |
| pcmk-oscontroller1: Online | |
| pcmk-oscompute11: Online | |
| pcmk-tempest: Online | |
| Daemon Status: | |
| corosync: active/enabled | |
| pacemaker: active/enabled | |
| pcsd: active/enabled | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment