Oppia is an open-source project whose aim is to empower learners across the globe by providing access to high-quality, engaging education. Oppia envision a society in which access to high-quality education is a human right rather than a privilege.
Oppia uses python as its backend language, and by default, python is a dynamically typed language. Dynamic typing means that the type of the variable is determined only during runtime. So, of course, it brings some performance costs and the potential risk of errors occurring at runtime.
Now, the aim of this project is to:
- Convert the whole backend codebase from dynamic typing to static typing. So that the type checks are enforced during compile time with exceptions and errors more likely to occur during compilation rather than runtime.
- Implement new pylint linters, to enforce contributors to use static typing rather than dynamic typing.
- Enable schema validations for all the HTTP handlers, so that the values coming from the front end will have a well-defined structure and malicious data being prevented from entering into the backend codebase.
For more information regarding the project. Please check out my accepted proposal
The following two milestones list the contributions made during and for the GSoC Project.
Fully type core/domain, core/tests, and core/jobs directories. Put measures in place to ensure that the files in these folders have full backend typing in perpetuity, to Oppia’s standards (i.e. not using "Any", casts, and objects, and only using a narrow, fully-documented subset of type-ignore pragmas).
Covered whole core/domain directory with strict typing.
| Merged PRs |
|---|
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.2 #15580 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.3 #15596 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.4 #15618 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.5 #15640 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.6 #15663 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.7 #15693 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.8 #15762 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.9 #15801 |
| Fix part of #14033: Added Mypy type annotations to some files of domain folder. -- M1.10 #15827 |
Covered core/jobs + core/tests directories with strict typing.
| Merged PRs |
|---|
Added Mypy type annotations to validation_decorators.py and job_result_transforms.py #15501 |
Added Mypy type annotations to the files of core/jobs/transforms/validation directory #15687 |
Added Mypy type annotations to the files of core/jobs/batch_jobs directory. #15711 |
| Added Mypy type annotations to core/test #15864 |
Implemented a pylint linter that will stop contributors from using generic(or loose) types into the codebase and ask contributors to write a code comment if an exception occurs in typing.
| Merged PR |
|---|
| Added linters for exceptional types in backend type annotations #15905 |
Fixed assertion issues in the codebase so that contributors will handle the ambiguous types gracefully rather than just arresting them. Established typing patterns for generic classes(like ChangeDomain). Fixed minor typing issues and unified typing patterns all over the codebase.
| Merged PRs |
|---|
| Fixing ChangeDomain typing issue in the backend type annotations #15958 |
| Fixes some assert issues. #15980 |
| Added Mypy type annotations ~ miscellaneous #15870 |
After completion of milestone 1, we are ended up with:
- Fully annotated(or strictly typed) half backend codebase with all sorts of checks passed.
- Total 220 files strictly typed along with errors fixed.
- 40,000+ lines of code written into the codebase.
- Robust pylint linter that will throw error if any contributor use loose types like,
Any,objectandcast.
Fully type the entire backend codebase, including schema validation for all handlers. Also, ensure that measures are in place to prevent backend typing coverage from regressing in the codebase.
Covered the whole core/controllers directory with strict typing enabling schema validation for all HTTP handlers.
| Merged PRs |
|---|
| Added Mypy type annotations to the core/controller directory's files #16342 |
| Added Mypy type annotations to the core/controller directory's files - 2 #16461 |
| Added Mypy type annotations to the core/controller directory's files - 3 #16473 |
| Added Mypy type annotations to the core/controller directory's files - 4 #16501 |
| Added Mypy type annotations to the core/controller directory's files - 5 #16532 |
Covered core/extensions + core/scripts directories with strict typing.
| Merged PRs |
|---|
| Added Mypy type annotations to the extensions directory. #16083 |
| Fix part of #13341: Added Mypy type annotations to the script directory's root level files. #16169 |
| Fix part of #13341: Added Mypy type annotations to the script directory's pending files #16226 |
Removed dead (or redundant) code related to voice-over application. Established typing patterns for generic functionalities so that they can also fall under the classification of strict typing. Implemented a new pylint linter that will restrict contributors from hiding typing errors in the codebase.
| Merged PRs |
|---|
| Removed voiceover application from codebase. #16343 |
| Fix #15995: Adding more strict types to customization args. #16551 |
| Added support to stop the usage of prohibited type ignores in the codebase #16104 |
After completion of milestone 2, we are ended up with:
- Fully annotated(or strictly typed) backend codebase with all sorts of checks passed.
- Total 247 files strictly typed along with errors fixed.
- 20,000+ lines of code written into the codebase.
- Pylint linter imposed on whole codebase so that it will throw error if any contributor use generic
type: ignorecomment or usetype: ignorecomment with restricted error code to hide the typing error in the codebase.
It was an awesome learning experience working with Oppia this summer and 1 month of winter as well (because this is a large project😬). Special thanks to Aasif Faizal, Vojtěch Jelínek, and sean lip who was always ready for helping me whenever I faced any problem in completing this project. I would also like to thank Oppia for giving me this opportunity.
About the project, I would say this project was way more complex than we thought. I have faced lots of road blockers while working on issues sometimes it took 3-4 days to come to a solid solution but in the end, everything is sorted. This project requires lots of debugging and tracing the code back and forth which was time taken and now I think why it is considered a large project.
At last but not the least, I would like to thank all the contributors of oppia community, who were always available whenever needed just a one email away and I would love to keep contributing to the organization in the future as well with LaCE quality team!