A Blog About GSoC 2021 With ML4SCI by Georgios Pipilis
Mentors: Patrick Peplowski, Sergei Gleyzer, Jason Terry
The following post serves as a synopsis of my work for Google Summer of Code 2021, under the Machine Learning for Science umbrella organisation. The project, titled "Machine Learning Model for the Albedo of Mercury", was part of ML4SCI's research on the planet Mercury, based on data retrieved from NASA's MESSENGER probe. This was my second time participating in GSoC (I worked with the Debian Foundation back in 2018), this time as a postgraduate student.
We can observe planets using all kind of electromagnetic wavelengths (infrared, ultraviolet, gamma-ray and so on). Each of these different approaches provides unique information about the chemistry and the history of a planet's surface. The objective of this project is to use existing data about the planet Mercury, as observed by the MESSENGER space probe, in order to figure out relationships between the various elements on the planet's surface.
These relationships will help us make predictions about the chemical composition of the planet in places where we have no data available. Essentially, we aim to use the data from areas of Mercury that were observed by MESSENGER in order to predict the chemical composition in areas that MESSENGER did not measure.
MESSENGER (Mercury Surface, Space Environment, Geochemistry, and Ranging) was a NASA spacecraft that orbited Mercury between 2011 and 2015. Among its accomplishments, it determined Mercury’s surface composition, revealed its geological history and discovered details about its internal magnetic field. The mission ended on April 30, 2015 when the spacecraft used the last of its propellant in order to deorbit and impact on the planet's surface.
The dataset we are using consists of several maps of Mercury. These maps can be divided in two categories: albedo maps and element composition maps.
Albedo maps: When physicists talk about the albedo of a surface, they refer to a measurement of how reflective said surface is. Planets reflect light from the Sun and other stars which we can measure in order to learn more information about the chemical composition of the planet's surface. In essence, the chemical composition of the ground determines how bright or dark it is. For example, the image below is the Lunar albedo. You can clearly see that, certain areas close to the center of the map, are darker than others. These areas are richer in Iron and other elements which make the rocks less reflective and, therefore, darker.
Element maps: These maps depict ratios between elements that are found on the planet's surface. For instance, the map shown below, depicts the ratio between Calcium (Ca) and Silicon (Si) on Mercury.
While the albedo maps of Mercury are of high quality, the element maps are not. The highly elliptical orbit of MESSENGER in combination with the low resolution instruments used have resulted in maps with missing data and spatial resolution that varies depending on the latitude. Of course, missing data refers to areas of the planet for which we have no measurements at all. But what does "spatial resolution that varies depending on the latitude" mean? Let's take it one step at a time.
Spatial resolution refers to the minimum size a planetary feature must have in order to be detected by an instrument. For example, if we're measuring a planet's surface at a spatial resolution of 250 kilometers, a crater that is 100 km in diameter would not appear on our measurement. This is because, with a resolution of 250 km, we are only able to extract features that are at least 250km.
Unfortunately, in the element maps we are using, the spatial resolution varies. Taking a look at the image below (Magnesium (Mg) Silicone (Si) map) we can clearly see that, as we approach the North pole of the planet (in other words, the top part of the image), the quality of the image improves but we also encounter a lot of areas with missing data (the areas in purple). Conversely, as we get closer to the South pole (the bottom part of the image) we see no missing data but the image quality deteriorates and the map starts looking blurry. Obviously, we're looking at a trade-off between high image resolution and the existence of data - in this dataset, we can not have both at the same time.
As mentioned, this trade-off is affected by how close we are to the top or the bottom of the image. In other words, it is affected by the latitude, the measurement of distance north or south of the equator of the planet.
We can now see how different in quality the albedo and the element maps are. Our goal is to predict the chemical composition in the solid purple areas (the areas we have no data for) using data from the albedo. However, before we do this, we need to deal with the problem of varying spatial resolution.
The idea is simple. Create an algorithm that re-creates the blurry, varying spatial resolution effect we see on Mercury's element maps and applies it to the albedo. In this way, it is as if the albedo, which was originally measured using a high-resolution camera, was viewed through the eyes of the low-resolution instruments that created the chemical composition maps.
Before we delve deeper into our latitude dependent smoothing algorithm, which is the core of this project, we should briefly discuss the basics of gaussian blurring.
Gaussian blurring (sometimes referred to as Gaussian smoothing) is a technique commonly used in computer vision and image processing in order to reduce the noise or detail of an image. Essentially, we "edit" each pixel of the image by changing its color in order to bring it closer to the colors of its neighbors. In the case of Gaussian smoothing, how much a neighboring pixel contributes to the color of the pixel we're editing depends only on the distance between the two pixels. The further away they are, the less they will affect each other's color. In other words, we're trying to make pixels that are close to each other look more similar in color, hence creating a blur effect and making some of the finer details of the image disappear.
Mathematically speaking, Gaussian blurring is simply the result of convolving an image with the Gaussian function. The formulas for one and two dimensions are as follows:
When we're dealing with data in the continuous domain, all we need in order to define the Gaussian blur is sigma (σ) - the standard deviation of the distribution. However, since image data is inherently discrete, we need two parameters so as to define a discrete approximation of the continuous Gaussian. Apart from sigma, we also need to define r - the radius of the kernel to be applied on the image. In practice, the radius is calculated after we select a value for sigma and, while this can be calculated in multiple ways, for this project are using the formula radius = 2 * sigma.
What is very interesting from a computational point of view is the fact that Gaussian blur is a separable filter. In practice, this means that, instead of calculating a 2D matrix and applying it to the image, we can apply a series of one dimensional matrices horizontally and, after that, repeat the process vertically thus significantly reducing the algorithmic complexity of our blurring function. The explanation behind this is better visualized in the following image.
Back to our goal of creating an algorithm that simulates the blurring effect seen on Mercury. Mercury is a sphere, a three dimensional object, yet its map is a two dimensional projection. This means that, the distances between two places on the surface Mercury (or any other planet really), are not represented accurately on a two dimensional map. Intuitively, this can be understood by looking at the following image of the Moon.
When we look at the Moon as a 3D object, its northernmost point is just a single dot on the very top of the sphere. However, as we "unwrap" the sphere and turn it into a map (in effect moving from 3D to 2D) the northernmost point of the Moon is now the topmost row of pixels in the image. This row has exactly the same amount of pixels as any other row on the image suggesting, for example, that the northernmost point on the Moon (which in reality is just a dot) has the same size as a line that circumnavigates its equator (which is represented by the row of pixels in the middle of the image) which is 10,921 kilometers in length!
Evidently, this assumption is not true. But, then, how can the North pole and the equator have the same "size" on a map when, in reality, they do not? The answer is that, the pixels of the map that are close to the poles, represent less physical distance in kilometers than the pixels close to the equator do. While a pixel close to the center of the image might represent hundreds of kilometers in distance, the pixels in the very top might even represent a few meters or less.
This fact prevents us from using a pre-made Gaussian smoothing algorithm on Mercury or the Moon, since it would apply smoothing based on a radius measured in pixels - a distance metric that is, as described above, inaccurate in our case (reminder: a pixel on the north pole doesn't represent the same area as a pixel on the equator). Consequently, our first goal is to create a Gaussian smoothing algorithm that takes as input the kernel radius in kilometers instead of pixels.
In order to achieve this, we first need to calculate the distance in kilometers of each pixel from the pixels right next to it. Starting from the pixel in [0, 0], we calculate its distance from the pixels in [1, 0] and [0, 1]. We can calculate this distance with the use of the haversine formula:
After these distances are calculated and stored into a matrix, we can use them in order to create a different Gaussian kernel for each pixel, based on its latitude and longitude on the planet. Fundamentally, the algorithm starts from the pixel located at [0, 0] and iterates through all pixels. For each of them it calculates the correct gaussian blur radius based on the kilometers parameter. For example, if we need to blur with a radius of 150 km, the algorithm will first calculate how many pixels p correspond to 150km depending on the latitude and longitude of the current pixel. Subsequently, it will apply a Gaussian convolution centered around said pixel with radius equal to p.
Our algorithm is almost ready! Before we start applying it, we need to deal with how the kernel functions at the edges of the map. In order to create a proper wrap-around effect we create a grid of maps, as shown below. We make all of our computations on the central map of the grid. The reasoning behind this grid is that, when we work with pixels close to the left and right edges of the central image, it's easy to wrap around to the other side. But, when doing so on the poles, we need to wrap around to the same side of the image (since, for example, the other side of the planet's North pole is still on the top of the image). That's why the maps on the first and third row are flipped and mirrored.
Below are the results after using our algorithm on the Moon, blurring with a kernel of 120 km in radius. We notice that the top and bottom of the image shift towards a unifrom color. Intuitively, this makes sense since these areas represent the poles which, as previously mentioned, are points on a 3D sphere and don't cover the same area as the equator does.
Plotting the average color per row of pixels before and after the smoothing, we can see the effects of the blur.
All we need in order to simulate the desired blurring effect found on Mercury, is one update to our algorithm. Instead of the user providing a single value for the kernel's radius (always in kilometers), the algorithm should be able to pick a different radius size for each degree of latitude. In this way, we move from simple adaptive blurring to latitude dependent blurring.
After this change, we are able to qualitatively simulate the effect found on Mercury. However, we also need to match it quantitatively. Unfortunately, this is another part of the project for which we have no discrete data. Thankfully, with the aid of online computer vision tools, we were able to export data from the following average spatial resolution curves. These maps depict the Mg/Si and Al/Si ratios for Mercury. The curves we are interested in are on the far right of the plot and show how the average spatial resolution changes as we move North or South on the planet.
Originally, continuous data like the aforementioned curve were smoothed using Gaussian kernels where the spatial resolution corresponded to the full width at half maximum of the Gaussian. Using the relationships FWHM = sigma * 2.335 and radius = 2 * sigma we are able to use the data of the curve, which represents the FWHM, in order to get the correct sigma and radius for our kernel.
Now, we have everything we need to simulate the variable smoothing effect found on Mercury. Below, you can see an example of that effect applied on the lunar albedo.
The blurring algorithm we wanted is ready! Now we can blur Mercury's albedo, apply machine learning to it and predict the elemental composition in areas we have no data for. You may be thinking "Since we're missing so much data on Mercury, how can we know if our algorithm is working and the predictions we make are accurate?". The answer to this is: the Moon. Or, rather.. "Mooncury".
In contrast to the datasets we have for Mercury, our Lunar datasets have no missing data and no spatial resolution anomalies. We can use this to our advantage by trying out our algorithms and assessing their performance on the Moon.
First, we make a map that shows the parts where data is missing on Mercury. Subsequently, we apply this map on the Moon. Essentially, on the lunar surface, we simulate the loss of data that appears on Mercury. The product is the image shown below. The yellow areas are the simulated "no data" parts of the albedo.
Now, if we apply our latitude dependent smoothing on this image, following the average resolution curves of the MESSENGER data, we'll get a lunar map that has varying spatial resolution similar to Mercury's. We dubbed this creation "Mooncury".
In the final week of GSoC, all of the problems that plagued our datasets were solved and we were ready to start experimenting. We tried predicting the chemical composition of missing data both on "Mooncury" and Mercury. The models of choice were gradient boosting regression (using XGBoost) and polynomial regression (using SKLearn) since they had the best performance in benchmarks we ran throughout the summer (on the lunar dataset).
But how was training and testing done? All element composition maps of Mercury and "Mooncury" have two distinct parts. One part where data is missing and one part where data is available. For each of the two planets (separately), we used the parts of the element maps where data was not missing as well as the corresponding parts of the albedos for training. Fundamentally, we taught the models how to predict the element maps by looking at parts of the albedo. Then, by using the rest of the albedo (the part of it that matches the areas where the element maps are missing data) as input, the models predict values for the data in the missing parts of the element maps.
Below you can see some of the preliminary results. Unfortunately, there was not enough time for hyperparameter tuning or working with different models and algorithms but that is one of the project goals for the future. The image on the left is the result of our models "filling in" the missing data. The image on the right is the original map, with missing data.
"Mooncury" results:
Mercury results:
The results on "Mooncury" are evidently better than the results on Mercury. However, this was only the first round of applying ML to the data we prepared. These results serve as a great indicator of what we should aim for in the future.
As with all research and open source projects, the work does not stop with the end of GSoC. There is a lot more to be done in the future. We've dealt with all of the problems in the MESSENGER dataset, we've prepared a test bench for our experiments ("Mooncury") and now we're ready to try more machine learning and computer vision techniques.
Some of the ideas for the future include:
- Creating models that are trained using multiple maps and not only the albedo
- Experimenting with GANs (generative adversarial networks) and CNNs (convolutional neural networks)
- Applying different image segmentation and computer vision methodologies on Mercury
This has been a really eye opening and productive experience. I would like to thank my mentors Patrick Peplowski, Sergei Gleyzer and Jason Terry for their feedback and support throughout the summer.
The code produced during GSoC '21 can be found here.