Chris Black (@infotroph), 2020-02-29
PEcAn is mostly developed by scientists who are skilled at programming but whose main job is not to write software, but rather to answer questions about the world. Therefore when someone has written code that helped them answer a question, we want it to be as easy as possible for them to contribute it to PEcAn without spending undue time worrying about configuring their development toolchain.
But at the same time, we need continuous integration for the same reasons all software does: PEcAn needs to continue working as intended after each contribution. When a proposed change would break something we want the tests to fail loudly and quickly, but overall we want contributors to think of our CI system as the helpful robots who make PRs easier rather than as the gatekeeper robots who demand you change your code. When there is tension between ease of contibution and ease of maintenance, we want to lean toward favoring ease of contribution.
Our existing Travis CI build gets this balance wrong in several important ways: It takes too long to be used for immediate feedback, tests the installation process more than the PEcAn code, gives errors that are hard to interpret (and often hard to even reproduce on your own machine), and in practice its build failures are more often caused by dependency issues or transient infrastructure failures than by problems with the changed code.
We think GitHub Actions has great potential to make the testing process both easier and more robust:
- Generous limits on simultaneous jobs -> ability to perform tests in parallel for faster results
- Generous limits on run time -> allows more complicated integration tests
- Integration with other GitHub features -> ability to streamline the review process by e.g. posting comments back to review threads or contributing automated build updates
- Good Docker support -> eases testing of Dockerized components, simplifies deployment
Here are a few ways of using GitHub Actions that the PEcAn team has discussed, listed in roughly decreasing order of urgency. See also issue #2412 for (a little) more context, and our existing prototype of a basic build & test pipeline.
- Refine existing prototype CI workflow to build/test/run on multiple R versions and system environments. (Currently done on Travis)
- Create a suite of longer-running integration tests that use several different ecosystem models (Currently not done. Has been on the wishlist for years!)
- Compile and automatically deploy documentation changes (including the PEcAn book, package function help, and vignettes) to the PEcAn website. (Book is currently deployed by Travis, others only deployed by hand; see also #2437)
- Build and automatically push Docker images (Currently done on @kooper's machine)
- Implement server-side rebuilding of package documentation on request, similar to this example. (Currently must be done by the user before commit, checked by Travis in a way that annoys everyone)
- Test Shiny apps (#2121)
- Automate various code-cleanup tasks: Linting, formatting, etc. (Currently not done)
- Anything else that makes life easier for PEcAn's users, contributors, developers, and/or maintainers
The wishlist above should give an idea what we had in mind, but a good GSOC proposal is one that makes the project yours. Feel free to propose something we didn't think of! Although we hope your proposal incorporates items from the wishlist, it need not implement everything. We expect that many of these will require care and thought to do correctly, and we'd rather you finish the summer with a few features working well than with many roughed out. A successful proposal will tell us:
- What specific features you plan to implement, with at least an approximate timeline
- How your existing skills will help you implement these features
- What new skills you will need to learn (make sure the time to learn them is included in your timeline!)
- How your work on this project will advance the scientific goals of the PEcAn project
- Which features are must-haves to consider the project successful, and which are stretch goals to be done as time allows
- What kind of contributions you expect to need from the rest of the PEcAn team; e.g. answering questions about R, help implementing tests of specific ecosystem models, code review, etc. If you'll need help from a specific person who's not your project mentor, have you discussed the proposal with them before submitting it?
See also Google's proposal guidelines
- R-specific Actions examples
- Wide-ranging Actions examples from a GitHub developer
- The GitHub Actions organization, with good examples and many tools we can use
- PEcAn issues labeled "testing" (Not all directly CI-related, but useful to think about anyway)
See also: ideas for some warmup exercises to get comfortable with Actions and start thinking about how it should work in the context of PEcAn.