This document contains a brief summary of all the work that I did as part of my Google Summer of Code 2020 (GSoC '20) internship.
Project Details
Student: Abhineet Tamrakar
Organization: moja global (under the umbrella organization DIAL)
Mentors: Guy, Kaushik, Max
Project: Interactive Visualization Tool for GCBM
Links
- Project Plan
- Weekly Stand-ups posted by me during GSoC.
- Weekly Blog Posts posted by me during GSoC.
- Code Repository
- Commits made by me during GSoC.
- Pull Requests created by me during GSoC.
- Issues created by me during GSoC.
- Documents written by me during GSoC.
My mentoring organization's primary project is the Full Lands Integration Tool (FLINT). Its a modular accounting system for the land sector to estimate greenhouse gas emissions and removals. The Generic Carbon Budget Model (GCBM) is a packaging of FLINT with some other modules developed by the Canadian Forest Service.
During my GSoC internship, I developed a tool that creates a browser-based interactive visualization from the output generated by GCBM. I've lovingly named this tool Taswira (which is Swahili for "visualization"). I built it on the work of another moja global member, Max Fellows, who has created a tool which is similar but produces static animated videos instead.
I was able to develop the tool almost entirely in Python. Thanks to a framework called Dash, even the client-side code (which is traditionally written in JavaScript) is in Python! Also, for the server side, I had originally planned on writing it from scratch but thankfully I came upon Terracotta. It took care of all the heavy raster conversion for me and so I was able to focus more on the front-end. Speaking of which, here are some of the features of the UI:
- View graphs generated from non-spatial data of GCBM output.
- View the spatial output from different ecosystem indicators overlaid on top of a map.
- Visually cycle through the spatial output by going through the different years of the output's time-series.
- Automatically do the above by using a "Play" button.
See the GIF attached to this gist for a sample of the final product in action.
In its current form, the tool has more or less fulfilled its original intention of showing how the visualisation of GCBM output is possible in an interactive and dynamic way.
Now, looking at the future of the tool, I have some — rather ambitious — plans that sprang from discussions with my mentors and other members of the organization.
The dataset that I worked with while developing Taswira had ZSTD-compressed raster files, whereas most GCBM implementations are by default configured to produce DEFLATE-compressed raster files. This does not mean that DEFLATE-compressed files are not supported. They work fine but the tool takes a lot of time to load them and process them. This can contribute to bad user experience.
A possible resolution for this is to convert the files into ZSTD-compressed format and save them in temporary storage. Terracotta already comes with support for this, so implementing this should be straightforward but before doing that, I will need to assess whether doing this on-the-fly would be beneficial or not.
Terracotta is a wrapper on a wrapper for GDAL, ie., Rasterio. This means that, theoretically we can do away with Terracotta and directly work with Rasterio. This would require understanding Rasterio and taking a deep-dive into the source code of Terracotta.
At the moment, Taswira supports GCBM produced output but there is scope for it to become a general purpose utility for visualising arbitrary combinations of spatial and non-spatial data. This would require considerable abstraction in the source code.
Specifying how the spatial and non-spatial data relate to each other will be a core problem in implementing this. My mentor, Kaushik has suggested that we can let the user provide custom SQL queries.
This is by far the most ambitious idea that I have for Taswira. I'll try my best to carry it out along with all the other plans that I mentioned above.
These past three months have been immensely challenging and fun for me. I've had the opportunity to work along with some insanely kind and supportive people. I'm wholeheartedly grateful to Guy and Michael for accepting my proposal and giving me this wonderful opportunity.
It would have been impossible for me complete this project without the patient and detailed guidance of Max and Kaushik. Thank you so much!
I look forward to continue collaborating with the awesome people of moja global!
