We currently use devops (stackforge/fuel-devops) as a middle-ware to handle building and deploying tests. The test framework starts by deploying clusters in different modes (haubuntu). Then it uses snap-shots to revert between scenarios.
This process is time intensive and doesn't scale well if there are sufficient resources. Furthermore, it relies on a middle-ware that implements an API similar to the OpenStack nova interface, it should instead be using the OpenStack API. This will prevent duplicate development and allow for far more features than are implemented in the current fuel-devops middle-ware
Additionally, we are under-testing some areas, by opening the testing framework to a underlying infrastructure that can be much larger than a previous single node, we can run iterative tests in some areas that need to be tested more. For example, provisioning results (pmanager specifically) is under tested.
Lastly, moving towards running on OpenStack, (devstack too) would enable more people to work with fuel dev environments as they could stand up ENV's using methods that they are already familiar with.
Need to write a OpenStack driver for devops. This could end up being a throw-away work, but will be necessary to migrate to OpenStack. It may later be pulled into the system tests.
Need to test provisioning, and a empty role to help. With a empty role, we can run the provision task for each OS release type. These can then be validated and saved to an image. Further provisioning testing can be performed by meshing volumes and roles and nics to improve the testing coverage of combinations during provisioning. (This requires Neutron PXE support same as TrippleO)
Need to create a slightly tricky on-demand method to create instances. Eventually, fuel should be able to manage on-demand nodes, but until then, we can use the testing framework to stand up the cluster and the instances necessary from the empty role images created during provision testing.
Instances from images will need some tweaking from cloud-init. Images will to be able to:
- Grow rootfs
- create partitions / LVM / sgdisk attached volumes (maybe by having pmanager ask over an api)
- Use expected Admin IP
The testing framework will then need to be adapted, Firstly we will need to start consuming nodes in the method described here. Second, we will need to be able to run the tests in parallel.
We might be able to adapt all of the fuel-main/fuelweb_tests to tempest since most of the work is API based anyways. Tempest also already exclusively creates separate tenants to run tests.
Fuel should be able to manage on-demand nodes, in that it can ask an underlying system to yield nodes to it at assignment of a node that it could have no prior knowledge of. This is tricky, but could greatly help when running in development where not all nodes need to be pre-defined, or in cases like SoftLayer where scale out is the point, but not until you ask for more (because you are charged for them)
Ok, these changes are great, how is this useful. We will need to further link jobs from the ISO build script to then download the ISO and kick off all the fun stuff described here. We will also need to set up an environment that can be used this way. We will also want to make sure that this will work on top of devstack to help ensure that the community can participate more easily.
Additionally, Ceph support will be able to greatly reduce provisioning time, and given a cluster of any reasonable size should be highly encouraged.