Program of AstroVič team for Astro Pi 2019/20 challenge

README.md

AstroVič

Introduction

The objectives of the experiment are to develop a system for land categorization and classification system of satellite photos from Raspberry Pi NoIR camera. Such technologies (but on a bigger scale) would also allow analysing unknown land and easy constructions of rudimentary maps on less discovered places on Earth or analysing land of newly discovered planets, as well as navigation systems based on land images.

Our algorithm uses OpenCV’s Python library and Numpy to process images. For every photo, it uses colour-based classification functions to generate grayscale masks for clouds, oceans, and land, as well as edge detection. We decided to use OpenCV’s threshold with a value between 185 and 255 for cloud detection, ocean mask was detected with checking if red and green components of each pixel were smaller than blue one, and land mask additionally checked that all three components are similar enough.

Program takes all photos in directory, process each of them in separate process, and saves grayscale masks to new directories. We also made additional program which uses Matplotlib to display all masks as coloured overlays on original photo.

Usage

Program is written in Python 3 and requires numpy, cv2 and matplotlib modules.

All raw photos need to be stored in raw directory relative to main.py. It is recommended that directory only contains both Python scripts, raw photos and virtual environment if needed. All other directories will be created on run and will be overwritten.

To analyse photos, you need to run python main.py. The program uses multiple processes and it may use 100% of your CPU so it is recommended to close all other programs and to not use your computer while the program is running. After program is finishes, various other directories will be created and will contain masks of original photos.

To view analysed photo run python display.py [filename], where [filename] is filename of photo you want to view, without any directory path. This will open Matplotlib window with original photo, overlayed with clouds, ocean and land masks.

Other masks are currently unused, but it may be used in the future.

display.py

import matplotlib.pyplot as plt
import numpy as np
import cv2

import sys


def show_raw(ax, img):
    return ax.imshow(img)

def show_alpha(ax, img, color):
    rgba = np.zeros((*(img.shape)[:2], 4))

    rgba[:,:,:3] = color
    rgba[:,:,3] = img

    return ax.imshow(rgba.astype(np.uint8), alpha=0.5)


def onpick(event):
    line = event.artist
    overlay = line.overlay

    visible = overlay.get_visible()
    overlay.set_visible(not visible)

    if visible:
        overlay.set_alpha(0)
        line.set_alpha(0.25)
    else:
        overlay.set_alpha(0.5)
        line.set_alpha(1)

    event.canvas.draw()

def main():
    if len(sys.argv) != 2:
        print('usage: python display.py [filename]')
        sys.exit(1)

    filename = sys.argv[1]
    fig, ax = plt.subplots()

    raw = cv2.cvtColor(cv2.imread(f'raw/{filename}'), cv2.COLOR_BGR2RGB)
    show_raw(ax, raw)

    overlays = []

    ocean = cv2.imread(f'ocean/{filename}', 0)
    ocean_label, = ax.plot(1, 1, linewidth=10, color='blue', label='Ocean')
    overlays.append(show_alpha(ax, ocean, (0, 0, 255)))

    land = cv2.imread(f'land/{filename}', 0)
    land_label, = ax.plot(1, 1, linewidth=10, color='green', label='Land')
    overlays.append(show_alpha(ax, land, (0, 255, 0)))

    clouds = cv2.imread(f'clouds/{filename}', 0)
    clouds_label, = ax.plot(1, 1, linewidth=10, color='red', label='Clouds')
    overlays.append(show_alpha(ax, clouds, (255, 0, 0)))

    leg = ax.legend(loc='upper left', borderpad=0.75, fancybox=True, shadow=True)
    leg.get_frame().set_alpha(0.4)

    for overlay, line in zip(overlays, leg.get_lines()):
        line.overlay = overlay
        line.set_picker(5)

    fig.canvas.mpl_connect('pick_event', onpick)
    plt.show()


if __name__ == '__main__': 
    main()

main.py

import numpy as np
import cv2

from multiprocessing import Process
import glob
import pathlib


def mask_edges(img):
    edges = cv2.Canny(img, 150, 210)
    return edges


def mask_terrain(img):
    terrain = cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, cv2.THRESH_BINARY, 11, 2)
    return terrain

def mask_clouds(img):
    _, clouds = cv2.threshold(img, 185, 255, cv2.THRESH_TOZERO)
    return clouds    

def mask_ocean(img):
    ocean = np.zeros_like(img, np.uint8)
    
    for row in range(img.shape[0]):
        for col in range(img.shape[1]):
            pixel = img[row][col]
            
            if pixel[2] > pixel[1] and pixel[2] > pixel[0]:
                ocean[row][col] = 255

    return ocean

def mask_land(img):
    land = np.zeros_like(img, np.uint8)

    for row in range(img.shape[0]):
        for col in range(img.shape[1]):
            pixel = list(map(int, img[row][col]))
            
            if (
                abs(pixel[0] - pixel[1]) <= 60 and abs(pixel[1] - pixel[2]) <= 60 and abs(pixel[0] - pixel[2]) <= 60 and
                pixel[2] < pixel[1] and pixel[2] < pixel[0]
            ):
                land[row][col] = 255

    return land


def process_image(filename):
    print(f'Processing {filename} started')
    original = cv2.imread(filename)
    
    gray = cv2.cvtColor(original, cv2.COLOR_BGR2GRAY)
    rgb = cv2.cvtColor(original, cv2.COLOR_BGR2RGB)

    edges = mask_edges(gray)
    cv2.imwrite(filename.replace('raw', 'edges'), edges)

    terrain = mask_terrain(gray)
    cv2.imwrite(filename.replace('raw', 'terrain'), terrain)

    clouds = mask_clouds(gray)
    cv2.imwrite(filename.replace('raw', 'clouds'), clouds)

    ocean = mask_ocean(rgb)
    cv2.imwrite(filename.replace('raw', 'ocean'), ocean)
    
    land = mask_land(rgb)
    cv2.imwrite(filename.replace('raw', 'land'), land)

    print(f'Processing {filename} done')


def main():
    pathlib.Path('edges').mkdir(exist_ok=True)
    pathlib.Path('terrain').mkdir(exist_ok=True)
    pathlib.Path('clouds').mkdir(exist_ok=True)
    pathlib.Path('ocean').mkdir(exist_ok=True)
    pathlib.Path('land').mkdir(exist_ok=True)

    for filename in glob.glob('raw/*.jpg'):
        process = Process(target=process_image, args=(filename, ))
        process.start()


if __name__ == '__main__': 
    main()