Organisation: Beam Community
Team: Riak
Title: Improving Riak testing suites using Common Test and Intercepts
Mentors: Bryan Hunt, Gordon Guthrie, Russell Brown
We proposed to take riak_test, the main integration and regression testing component of the Riak database and improve its reporting abilities. This involved breaking riak_test into its discrete components and using standard tools rather than a bespoke test runner facility that was infamous for being difficult to set up and use.
Here is a list of the open-source contributions I made or was involved in, not only for the Riak Test tool but also for other components required to make the project work, and other separate projects.
- New Riak Test tool
- Riak Test Library
- Erlang Code Intercept Libray
- Rebar3 compatible Riak dependencies
- Demos and Proofs of Concept
- Rebar3 bug fix
- Profanity filter library
This is the main contribution: a successor to the original riak_test utility. Significant time and attention was given into making this tool work with standard Erlang tools such as the Common Test framework and rebar3, widely regarded as the official Erlang build tool.
Since this is a standalone tool, it now lives under a basho repository:
Details about how it works and what it does are included in the readme document on the repository itself, but I wanted
to disclose that significant portions of code are not mine despite being committed by me to the repository. Instead they
were brought back from the old repository, such as the content of the intercepts folder.
When separating out the old repository into its components, it made sense to cut the code base that was solely responsible for setting up Riak tests. This is left on its own repository:
Since the code was cut there were minimal changes to this repository, so this contribution is mostly a conceptual separation. Nonetheless, it was important to correctly select the Erlang modules that fit under this category in order to keep a clean project architecture in the main repository.
The original Riak Test code contained a set of modules that were able to perform code intercepts, which is very useful for building robust test suites. This code is not tightly coupled to the Riak code itself, and we separated this code into a library because this will certainly be useful for the rest of the Erlang community can use. We plan to disseminate information about this library in an industry conference talk (Code BEAM Lite) later this year.
The code for the intercept library is therefore hosted on a separate repository:
Like in the Riak Test Library, most of this code was written before and there were minimal code changes.
Riak is still built as a rebar2 application, and incorporating rebar3 into the project required some changes to some
of its dependencies in order to get the project to compile successfully. To achieve this, the lucene_parser dependency
of Riak Search had to be separated into its own repository:
Then, a small yet important set of changes was made in order to allow riak_search, a dependency of Riak, to use the
modified version of lucene_parser:
In my research to design a rebar3 architecture for the new project I had to build several small demos and proofs
of concept to slowly progress into a single application that was able to perform everything we needed. One of the
proofs of concept was particularly challenging, which was the sharing of files between Common Test suites.
I received some suggestions from the Erlang Slack channel to create symlinks for each test suite data directory,
but that did not scale as we have plenty of test suites. I believe to have come up with a better design that
involves the priv directory, available by default on OTP applications, and I have left that demo in a specific
repository:
In the Erlang Slack community Fred Hebert made a request for contributions on a particular bug which was tagged as beginner friendly. I applied and made the contribution, and worked with Fred until the code changes were decent. This resulted in a merged PR:
The little free time I had this summer I wanted to build a URL shortener but generating random URL slugs has a non-zero probability of returning something that might offend some users. I made a simple profanity filter that works with multiple languages:
I also published it as a Hex package. NOTE: Repository contains profanity! (hopefully obvious)
In brief, consider the project goals to be the following set of tasks:
- Breakdown the old riak_test into its discrete components ✔️
- Design a new architecture of Riak Test that allows usage of rebar3 and Common Test ✔️
- Convert a significant subset of test suites to Common Test ✔️
- Write documentation to help migrate remaining test suites from old Riak Test Runner to Common Test ✔️
- Ensure that code intercepts are able to be used after splitting from the old repository ✔️
(✔️ - done, ❌ - not finished)
Unfortunately, due to time constraints it was not possible to migrate all existing test suites over to Common Test, but since there is a significant number of them, we purposely set a subset as the goal of the project. Over time I will be able to make small contributions to finish the remaining ones.
I want to thank Bryan, Gordon and Russell for putting up with me when they probably could be doing more important things, and also the people from the Erlang Slack community, in particular Tristan Sloughter, Fred Hebert, Bryan Paxton, Craig Everet and Brujo Benavides for getting me unstuck several times.
We plan to submit the result of this year's GSoC project as an industry talk at Code BEAM Lite later this year, as we see the contributions to have potential for impact in the Erlang community, and also prove that Riak is gaining momentum!