The Tasking Manager is the most used application in the OSM ecosystem for volunteers, but also professionals, to team up and coordinate mapping. In order to ease up the user experience of fulfilling the guidelines for organized editing, the goal of my project this summer is to make the process of reporting back data for OpenStreetMap automatically.
During the project, I used my workplan to direct which activities I should do. The list of activities to be done looked like this:
- Proof of concept for communication with OpenStreetMap, containing the following:
- Save and update data into a mediawiki instance, according to the Organised Editing Guidelines
- Save and update data in a human-readable in a git repository
- Move code from proof of concept to OEG Reporter
- Update Tasking Manager according to the Organised Editing Guidelines
- Integrate OEG Reporter in Tasking Manager
During the community bounding period was defined what data should be updated in the Tasking Manager to fulfill the guidelines for organized editing. Also, I started to implement the proof of concept service with the initial structure of what it would be necessary to report data from the Tasking Manager to a MediaWiki instance and a git repository. With everything properly defined, I commented on everything that was planned for the project in the project issue and also wrote a blog post talking about how my experience during the first month in GSoC was.
The core of my work was done in the OEG Reporter repository. I also worked in the integration of the work in the Tasking Manager.
The first thing I did in the coding period was to finalize the proof of concept implementation, with the data report working for a MediaWiki instance and a git repository. After that, we validated how should occur the communication between Tasking Manager and OpenStreetMap with the community in the state of the map 2020. Based on feedback from the community and people from OpenStreetMap itself, we decided to implement both solutions (report data to Mediawiki instance and git repository), instead of using just one of the solutions as previously thought. With the proof of concept already implemented, I started moving the code to the HOTOSM organization's repository (OEG Reporter) and during this process, I also added unit tests and some code improvements. The Pull Requests are listed below:
OEG Reporter Pull Requests
-
Repository initial structure (merged)
- The first thing I did was adding the initial structure of the repository with the organization of folders and a flask server with an endpoint, test configuration and swagger in order to test if the server was configured correctly. It is important to mention that I organized all folders and files according to the Tasking Manager repository with some slight differences, which would help in maintenance in the future.
-
- With a simple server configured, I added serializers for validating data from Tasking Manager using the Marshmallow library over schematics - the library currently in use by Tasking Manager). We choose this library because in the Tasking Manager some problems with API filters are caused by serialization.
-
- With the server and serializers already configured, I added the git report. The idea of reporting the data to a git repository was to parse the data into a human-readable format and store it in a git repository, so we choose the YAML file format.
Parsing data from the Tasking Manager into a YAML file was easily done with the pyYAML library.
To report data to a git repository we used the gitpython library because this would allow us to report data to any cloud hosting service for git repository (GitHub, Gitlab, Bitbucket, etc).
- With the server and serializers already configured, I added the git report. The idea of reporting the data to a git repository was to parse the data into a human-readable format and store it in a git repository, so we choose the YAML file format.
-
Add wiki report (waiting review)
- After finishing the git report the only missing part from OEG Reporter was to report data to a MediaWiki instance. Definitely, this was the trickiest task of the summer because of the syntax of the files used by a MediaWiki instance and for helping me with some abstractions for sections, tables, and other things from wikitext files I choose the wikitextparser library.
- After getting everything formatted according to the wikitext file syntax I should save it to a MediaWiki instance and this was a much easier task than formatting text, due to the very detailed documentation of the MediaWiki Action API.
-
- Before merging everything into the main branch it was important to choose a license and add it to the repository. The choosen one was the BSD-2 because it is a permissive license that avoids problems for potential HOT partners who use proprietary software.
During the process of moving the code from the proof of concept to the OEG Reporter I also added some fields that were not present in the Tasking Manager, but were suggested by the Organised Editing Guidelines, a list of this PRs is described with more details next:
Tasking Manager Pull Requests
-
Add project due date (waiting review)
- Added estimated date for finishing a project in the project create/edit page.
-
Add organisation description (merged)
- Added organisation description in the organisation create/edit page.
-
Add metrics and quality assurance to project (closed)
- This Pull Request was closed because after a conversation with my mentors we decided not to include these fields in the Tasking Manager.
-
Integrate OEG Reporter in Tasking Manager (waiting review)
- Finally, the time to integrate the OEG Reporter in the Tasking Manager came. And for this, it was necessary to add some new environment variables, a new controller, and also a serializer for formatting the data according to the one defined in the OEG Reporter. You can see this with more details in this Pull Request.
I have done all the tasks that I proposed in my initial workplan. So the automatized support for organised editing guidelines is already integrated in the Tasking Manager (when the PR get merged). An idea of future work is to add the data reported to a git repository into a simple web page with some filters that may be of interest of the community.
I would like to thank Google for allowing me to participate in such a cool project that helps so many people, I will take many lessons from this experience for the rest of my life.
I would also like to thank my mentors during the summer, Wille, and Felix for all their dedication and effort in guiding me through this project and always encouraging me to participate in community events.
Last but not least, I would also like to thank all the people who have supported me so far.
- Organised Editing Guidelines - All suggestions made by OSM for an organised editing
- State of the map 2020 self organised sessions - Wiki page in the OSM Wiki with the list of all self organised sessions from state of the map 2020
- Project issue - Issue with content related to the project
- Integration of OEG Reporter in the Tasking Manager - The most important PR that integrates all the automation process into the Tasking Manager
- Add git report - Pull request that add the report to a git repository
- Add wiki report - Pull request that add the report to a MediaWiki instance