| title | description | type | slideOptions | ||
|---|---|---|---|---|---|
Teaching machines about our planet: Viewing, Learning, Imagining |
EARTHSC 8898 seminar talk |
slide |
|
EARTHSC 8898 seminar talk, Friday 8 Apr 2022, 17:45-19:00 (UTC)
by Wei Ji Leong
P.S. Slides are at https://hackmd.io/@weiji14/2022earthsc8898
To see how our planet is changing, and understand patterns over time
Iceberg A-74 (1270 sq km, or 2x size of Chicago) calving off Brunt Ice Shelf, Antarctica
https://earthobservatory.nasa.gov/images/148009/breakup-at-brunt
Exponential increase in amount of satellite data
Source: https://www.radiant.earth/infographic
Higher spatial resolution calls for new image analysis techniques
Source: https://www.radiant.earth/infographic
Do you look at the leaves, trees, or the forest?
- Leaf - the colour of an individual pixel
- Tree - a group of pixels forming an object
- Forest - an entire optical satellite/aerial image
Supervised / Unsupervised / Reinforcement
Recognition / Detection / Segmentation
*Note, sometimes Recognition is called Classification
Pan-sharpening
Cloud removal
Cloudy image Attention map Model output Ground-truth
Art (Style Transfer)
More at https://github.com/robmarkcole/satellite-image-deep-learning
- 🏁 Pixel-based image classification (2014)
- Classying proglacial lakes (Classic ML)
- 🔍 Object-based image analysis (2017-present)
- Iceberg or ship? (ConvNet)
- Building outlines/Ship detection (Unet)
- Crevasse detection & Navigation (A3C)
- Super-resolution bed topography (ESRGAN)
- Point cloud anomaly detection (DBSCAN)
- 🔭 Beyond pixels and objects ... (2022-future?)
in Aoraki/Mount Cook National Park, NZ using Landsat 5 and 7 from 2000-2011
an undergraduate project done in 2014
*back when we didn't call it machine learning
Tracking proglacial lake growth tells us how glaciers are retreating
Specify how many 'classes' you want to separate image
Algorithm (e.g. K-means, ISODATA) groups pixels with similar 'colour' into many classes
Telling the algorithm what a lake's colour looks like by drawing regions of interest (ROI)
Algorithm (e.g. Parallelepiped, Minimum distance, Mahalanobis distance) finds pixels with similar 'colour' to those ROI examples
Lake (red), ice (blue), rock/debris-covered glacier (green)
 |
 |
As a proglacial lake grows in size, that means the glacier behind it is retreating
- Pixel-based classification
- The algorithm calculates statistics based on each band/channel (i.e. colour)
- Individual pixels separated into classes (water, rock, ice) based on some statistical 'similarity' measure
- All the time spent
- Finding cloud-free imagery to download (and dealing with missing Landsat 7 stripes)
- Days of trial and error to find a good algorithm and suitable parameters
| Year | What came out | What I did |
|---|---|---|
| 2015 | Unet paper on image segmentation | More remote sensing BSc (Hons) |
| 2016 | AlphaGo beat Lee Sedol 4-1 | Worked in geospatial industry |
| 2017 | Andrew Ng's popular Deep Learning Coursera course | Started PhD and took that course! 👈 |
Kernels/Filters are learned to detect 'Features' like edges, corners, etc.
|
|
Subsequent convolutional layers build on these basic representations to learn complex high level features like texture, patterns and shapes.
To detect potential hazards out at sea using radar imagery
Image Classification using a ConvNet - Kaggle competition in 2018 Jan Code at https://www.kaggle.com/weiji14/keras-tinyyolo-based-model/notebook
Allowing arbitrary sized image input and same scale output
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. https://arxiv.org/abs/1505.04597
Developing a solution for maritime monitoring on Earth Observation data
 |
 |
Image Segmentation using Unet -- 2-day hackathon in 2018 Apr Code at https://github.com/weiji14/actinspacenz
Automating a tedious task of manually digitizing shapes in images
Image Segmentation using Unet -- side-project from 2018 Feb-Mar Code at https://github.com/weiji14/nz_convnet
To avoid falling into big holes when traversing the Ross ice shelf
Crevasse Classifier module | Route Navigator module
Image Segmentation using Unet + Reinforcement Learning using A3C -- Yet another challenge project from 2018 Mar-May
Satellite image --> Crevasse map --> Actions ⬅️ ⬆️ ⬇️ ➡️
 |
 |
Code at https://github.com/weiji14/nz_space_challenge
An async (parallel) version of an actor-critic (agent vs reward) model, whereby the model learns to do good actions that maximizes rewards
Good infographic at https://hackernoon.com/intuitive-rl-intro-to-advantage-actor-critic-a2c-4ff545978752
Going from BEDMAP2 (1km) to a 4x higher resolution (250m) DeepBedMap DEM
Using an Enhanced Super-Resolution Generative Adversarial Network (ESRGAN) -- First PhD project from 2018 Jun - 2019 Sep
Code at https://github.com/weiji14/deepbedmap
ESRGAN model learns to produce finer topography textures after many iterations (example over Thwaites Glacier, ice flow from right to left)
Epoch 1
|
Training
|
Epoch 100
|
Two competing neural networks working to improve the image's finer details
Generator (artist) learns to produce better image to convince Discriminator, Discriminator (teacher) points out where the image is incorrect
Detected from areas of rapid ice surface elevation change in ICESat-2 point clouds
|
|
Anomaly detection using unsupervised density-based clustering (DBSCAN) -- Final PhD project from 2019 Oct - 2021 Oct
Code at https://github.com/weiji14/deepicedrain
Code at https://github.com/weiji14/agu2021
Used to find dense clusters of where a pattern (e.g. rapid elevation change) is occuring, without specifying the number of clusters beforehand
Set a distance (epsilon) to cluster points, and a minimum points (minPts) threshold required to form a cluster
https://www.naftaliharris.com/blog/visualizing-dbscan-clustering!
Up and coming stuff to keep an eye out for!
Sentinel-2 image (10m @ 5-day repeat) -> Super-Resolution river masks (2m) over Alaska/Canada
Multi-task learning with Vision Transformers -- 2022 (ongoing) with Ziwei Li & Joachim Moortgat
Vision Transformer - An image worth 16x16 words
Able to capture long-range dependencies (e.g. how a river is connected) in an image, overcoming limited field of view in standard Convolutional Neural Networks
Taking in the whole scene at once, but paying 'attention' to what matters A model architecture from Natural Language Processing adapted for Computer Vision
Locating oval features in optical imagery that may potentially yield hydrogen gas
Object Detection project using YOLOX -- 2022 (ongoing) with team at SES/Byrd From Australia to the whole world? - Every bit of efficiency counts!
(Clean) labels are still very important!
Model-centric - How can I tune the model architecture to improve performance? Data-centric - How can I modify my data (new examples, more labels, etc) to improve performance?
Watch Andrew Ng's video - https://youtu.be/06-AZXmwHjo
Are you aware of how much electricity goes into training a neural network?
|
|
Hint: Use https://github.com/mlco2/codecarbon
- As a geographer 🌏, we need to ask:
- Whose problems are being solved? Who defines these problems? :thinking_face:
- Who is involved in the generation of solutions 💡? For whom are these solutions valid and good?
- How do alternative approaches or worldviews fit in? 👫🏽
Put another way, how can our map 🗺️ affect people?
...
Viewing forests, not trees
Learning smarter, from each other
Imagining a future, a blue planet still to be
...