The riak_test loaded_upgrade does some cool stuff in that it runs a set of client
processes as workers against specified versions of riak while upgrading nodes. We can take
this a step further if we also randomly generate configurations, where the configuration
includes general riak application settings as well as versions of riak and possibly the
worker mix as well. This might be a good candidate for testing the interoperation of new
features of riak 2.0 (and existing features too). It lets us keep the existing riak_tests
as they are without trying to figure out composable tests, which was also talked about
(and might still be a different good idea).
This could be accomplished by using EQC to generate the configurations. We'd need just one legitimate version of most settings and just enough riak versions to make a reasonable sized cluster. Cluster sizes would be randomized within a reasonable range.
Workers, are a good candidate for random EQC commands, though they would have to be pretty high level so that the constraints could be verified.
Reporting to Giddyup could require some new work since we'd have variable and random configurations that need to be included in the test results.
Some references from Joe B:
old, but useful hints on driving riak_test from EQC: https://github.com/basho/riak_test/blob/jdb-eqc-example/tests/verify_dynamic_eqc.erl
On generating configuration stuff (not commands): https://github.com/basho/riak_core/blob/develop/src/hashtree.erl#L1117-L1140