GSoC 2021 | Sarthak Bhardwaj
- Project Link: https://summerofcode.withgoogle.com/projects/#6129398324920320
- Project Idea: Develop a test framework for checking FreeType's rendering output.
- Proposal: https://docs.google.com/document/d/19YpZN3hqahHDhXuQKi7MrYmmGM9mhu3x
- Organization: FreeType
- Mentor: Werner Lemberg (wl)
- Student: Sarthak Bhardwaj (thesarthakbhardwaj)
- Clean Working Branch: https://gitlab.freedesktop.org/thesarthakbhardwaj/regressiontest/-/tree/Sarthak-GSOC-2021
FreeType is an open-source library used to render fonts and is packed with a lot of font-related features. This framework helps the developer to compare the rendering output of a ‘base’ version to a ‘test’ (developer) version of FreeType. Upon each commit, several builds are triggered for various operating systems and build configurations. Each build creates artifacts containing the newly-built FreeType libraries for developers and users to download.
The following goals were framed according to the requirements and the scope of the project:
- Modifying ftlint to output glyph md5 hashes for fast testing.
- Creating a script for regression testing, that can be run locally or in CI
- Adding regression test CI.
- Adding macOS builds for Autotools and meson.
- Adding image comparison functionality to the website.
- Improving the layout of generated web pages.
Modifying ftlint from ft-demos repo to output glyph md5 hash.
Writing a proof-of-concept script for detection of rendering differences in glyphs, using output from tools like ftlint and ftgrid.
Integrating the output with HTML pages, to provide an easy-to-use interface to see regressions. I ended up storing the directory structure inside a string variable in a .js file and loading it from HTML to show the data.
Linking the different web pages with each other, since it's not easy to pass data from one local web page to other, I solved this by appending the data to the URL of the web page, separated by "-" as a delimiter.
Adding a CI build for macOS
After the integration part was over, I focused on improving the layout of the generated web pages. Added image comparison functionality to the website, The diff was generated by XOR-ing the base and test images. Added translucent overlay of the diff/another image which appears over the glyph images by clicking it, with a short animation. This also made blinking comparison made easy by pressing and releasing a mouse button in quick succession. Added a selectable button so that, image to be used in the overlay can be selected from the select menu beneath both images.
- https://gitlab.freedesktop.org/freetype/freetype/-/merge_requests/47 (Added regression test CI)
- https://gitlab.freedesktop.org/freetype/freetype/-/merge_requests/49 (Added macOS build)
The script checks a "base" version of FreeType vs a "test" version.
Follow these steps to test FreeType
- Build and install
ft-demoslinked with the "base" version of FreeType.
- Build and install
ft-demoslinked with the "test" version of FreeType.
- If regressions are found,
the script generates a file
that contains web pages to browse the list of regressions.
You can download the artifacts from the test job on the jobs page here: https://gitlab.freedesktop.org/thesarthakbhardwaj/freetype/-/pipelines/358373/builds. The zip file will extract from the "report" folder, open the index.html file inside it to browse the regressions.
Follow this link to have the visual result of the semi-permanent web page, and see how it work: https://1sarthakbhardwaj.github.io/FreeType/
You can simply add/remove any fonts by simply putting the font file inside
the script will automatically read it.
And finally, I'd like to thank everyone, who has helped me in completing my project:
I would like to thank Werner Lemberg, my mentor for guiding me throughout the process and for helping me out to write well-structured and clean code. Our interaction has been very productive which led me to write code with better logic and quality, He was so cooperative and supported me everytime i needed help.
In addition, I would like to thanks Alexei Podtelezhnikov for helping me, Alexei's knowledge of the FreeType codebase is incredible and He helped me learn more about FreeType's internals and fonts.
Last but not the least, I want to thank Anurag Thakur, He was super supportive throughout the process and He helped me with many issues.
It was great working along with such amazing and helpful people. It is because of them that I was able to complete the project on time. I would also like to thank Google and The FreeType Project for this amazing opportunity. Looking forward to keeping the project growing.