After spending a lot of time on the awesome exercism.io platform, I was able to learn a lot of new languages and received great advice. After having accumulated a lot of exercism, I tried to automate and industrialize it. I was able to discover advanced features proposed by gitlab-ci. Here is my feedback
Exercism is a community site, where people wishing to learn a new language can receive expert advice from volunteer mentors. To learn a language, it is advisable to follow a path that corresponds to a logical sequence of exercises to have a regular progression in the language.
Here is the list of all the tracks (learning paths) available on the platform.
Exercism offers a cli to download and upload these exercises. This system is good but does not replace a versioning with git. So I use git to version my exercises.
The exercism folder is composed of subfolders for each language, and each language contains a folder for each exercism.
Here is an example of the architecture of an exercism folder.
https://gist.github.com/0821484a8a58d155a4dec53f5ce36d32
Each exercise has tests to validate. The exercises use the TDD method. Once the tests have been validated, the solution can be submitted and the return of a mentor is requested.
After accumulating many solutions, I wanted to have an IC that would allow me to stabilize and keep my exercises up to date if new tests were added.
Moreover, the way of executing the tests is similar depending on the tracks/languages. For example the exercises in go will be tested with the command go test or for java exercises gradle test.
Wanting to follow the DRY principle, I would like to have a CI template that I could apply to all the exercises in the same language and add this CI to all the new exercises.
I compared the different continuous integration solutions available for open source projects (Jenkins, CircleCI, Travis CI, Codeship Gitlab CI). I chose gitlab-ci because it offers the most interesting specifications and features.
Gitlab offers a free CI, which can be easily integrated into a git project and even integrated into a Github project. Cool, my repo is on github
[Parent Child Pipelines] (https://docs.gitlab.com/ee/ci/parent_child_pipelines.html)
They allow to execute a CI from a CI. It is therefore possible to create a meta-CI. Despite some limitations that we will see later, Parent Child Pipelines allow me to spread my CI over 3 nested levels :
- At the root of the exercises folder which will execute a CI for each language. The simplest file, it just has to trigger the ci of the languages: https://gist.github.com/e09140093979c7b1435b0e1718a23d36
- At the root of each language that will execute a CI for each exercise: https://gist.github.com/22a90b64c77df59ea859fbd268413fcc
- At the root of each exercise to run the tests provided by the platform : https://gist.github.com/3668e36d42dbdd2491ce3dca43faefd3
Note the use of the rules field which allows each new push to execute the CI task only if there has been a modification in the given path.
To be DRY, I need to templating my CI, an example is proposed with jsonnet. Here is an example proposed by the gitlab team: https://gitlab.com/gitlab-org/project-templates/jsonnet.
A gitlab-ci.yml file can be described in jsonnet format.
For example : https://gist.github.com/03593f6f8e59f481445c7999214ce010
This template allows you to describe gitlab-ci.yml for C exercisms.
Morality, jsonnet templates to avoid repetition, it's perfect!
Parent Child Pipeline are awesome but have several flaws, 2 were notable to me:
- Only 2 levels for Nested child pipelines. In my case I need 3 levels. The limitation can be bypassed by using multi-project pipelines.
- Management of files modified by the last commit to avoid rebuilding all exercisms.
For the management of modified files, it is possible to recalculate the modified files using a pre-step.
https://gist.github.com/26b571452cf11563b7b9b62ebbcd8007
Once the exercisms has been modified, it can be exported to artifact to share it with the job that will be generated afterwards.
Because of the limitation of the number of nested and also a wish to separate the different tracks, I created a git repository for each track (manual action that could be automated with ansible).
The classic way to manage these nested projects would be to use the git submodule. However, keeping a consistent set of git submodule up to date can become quite restrictive.
There is an alternative ! The subtree : https://gist.github.com/f2d58ffd544106908d0eaba11d05e2d1
The advantage I found in subtrees is that I can mirror only the tracks and automate this one simply with the git hooks. And this without modifying my existing one.
To create a subtree nothing could be simpler : https://gist.github.com/12908bda485dab63869f70f433507c3b
Then to make sure the subtrees is up to date, just add a hook to the post-commit..
Here is my hook file .git/hooks/post-commit :
https://gist.github.com/d161963364054379703b65a63e92440d
At each commit, each subtree is updated in the background.
I hope this article has made you want to try exercism.io and, why not, join this community.
Industrialize, as quickly as possible. Exercism has a version management system, however, complementing it with git allows you to go further:
- For example, I can do my exercises on several machines
- The automated tasks save me a lot of time
- have a backup system
- Just manage the history
- share my work and make it visible on github.com.
Test yourself, the basis of exercise is the TDD. It is important to test yourself and your code all the time. Moreover, I want to keep my exercism compatible with the latest version of each language and in the future with new tests added by the community. Having up to date exercism allows you to always have up to date exercism that make a very good example, and it allows you to have a lot of stars ;)
DRY ! Do not repeat, Do not repeat (oops)! At first, factorized allows to gain clarity. Allows for sanctuarisation and transmission of knowledge. How many times you make a set of commands that are almost automatic, but after several months without doing them, you end up forgetting them. Taking the time to template, this time is quickly gained!
Thank you for reading :)