Skip to content

Instantly share code, notes, and snippets.

@tallysmartins
Last active August 18, 2021 14:30
Show Gist options
  • Save tallysmartins/83bbe000cbc385e8b651e650248e8192 to your computer and use it in GitHub Desktop.
Save tallysmartins/83bbe000cbc385e8b651e650248e8192 to your computer and use it in GitHub Desktop.
ElixirBench final report - GSOC 2018

Google Summer of Code 2018 - ElixirBench

Project Long Running Elixir Benchmarks
Organization Beam Community
Project site https://elixirbench.org
Repositories https://github.com/elixir-bench
GSoC link https://summerofcode.withgoogle.com/organizations/6486585449644032
Mentors Tobias Pfeiffer, Michal Muskala
Student Tallys Martins

Project goal

ElixirBench aims to provide a service for performance monitoring similar to a continuous integration system and display results in meaningful graphs, for analysis and comparisons between code changes. The project goal was to put the service up and running, with basic features that would make it possible to monitor at least two Elixir projects. For this purpose we have developed new features, fixed bugs, setup of servers and we have written benchmarks for Ecto and Plug projects.

Work done

We did a lot of work during the project and we have reached most of our main goals. Many contributions were made to the project, from design, documentation, addition of new features to servers setup, putting many different skills into practice. Working on ElixirBench was a very pleasant experience with lots of learning and fun.

There were different kinds of contributions to the different repositories of the project. Here are the following Pull Requests, divided by repository and a small description for reference:

ElixirBench Web Repo

This is the frontend application, built with React. Here we have made several improvements like addition of new pages, menu links, bug fixes and features like displaying the status of a job in the job list page.

ElixirBench API Repo

This repository received most of the contributions. We have done several enhancements in the code base, improving the test suite and refactoring when needed. In this process we could detect and fix bugs and consequently raise the confidence for making other changes. The most important feature added was the integration with Github webhooks to automate the execution of jobs.

ElixirBench Runner

The runner application is responsible for running the benchmarks, collect the results and send it to the api server. Many changes were made to this repository in order to follow the modifications of the other applications and also small fixes that were detected occasionally. The biggest difficult here was the integration of changes with impact in the other services and ensure everything was working. Here are the contributions made so far:

ElixirBench Runner Container

The runner container is our docker files that ships with the Elixir dependencies for each different versions of the language that we support. There was only one contribution to this repository about adding support for synchronization between the containers that run the benchmarks and external dependencies, like databases and others. This feature had a big impact in terms of affected code, requiring changes also in the Api and Runner repositories.

Other Contributions

One of our goals was to monitor the performance of at least two Elixir projects using ElixirBench. For this, we have written benchmarks for Ecto and Plug, and we have also set a demo application with documentation purposes.

Future work

There is a lot of work to be done on ElixirBench I am looking forward to continue to contribute with it. The following list presents some issues that we would like to address the near future :

Features

  • Integrate with the Github App in the Market place
  • Enhancements on the graphics page (ddatetime, title, and information about hardware)
  • Add max retries to job claiming to avoid the runner to always claim a job that fails
  • Display project benchmarks per branch
  • Wrap ElixirBench standard configurations in a formatter
  • Be able to search repository
  • Link retried jobs to its parent (today is a completely new job and is not linked to retried one)
  • Be able to Save/Download docker-compose file from a job
  • Automatic detect and notify performance regressions in benchmark runs

DevOps

  • Automate deploy and define the release workflow
  • Build container releases for more elixir and erlang versions and push to docker hub

Documentation

  • Add about page
  • Add a contact page
  • Add help/docs page
  • Write about config.yml settings
  • Write about runners and infrastructure
  • Write about benchmarks and examples
  • Write about how to Contribute
  • Define the issues/development workflow

Acknowledgments

I would like to thanks my mentors, Tobias Pfeiffer and Michal Muskala for all the support and patience. They supervised my work with such a big wise and taught me many things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment