celoyd

from skimage import io, filters
import numpy as np
import sys

# ./hi8-deband.py input.png output.png
# Cherrypicked sample before/after: http://basecase.org/2016/1/hi8corr

'''
Himawari-8 has slight noise along scanlines. It seems basically uncorrelated
between rows, and varies smoothly on the scale of about 100 columns.

brunosan

This is a list inspired by some of our current or potential lines of work at the World Bank Innovation Labs. The “Innovations in Big Data Analytics” program helps to strengthen the World Bank capabilities to effectively use big data in its operational and strategic work.

We are always looking for great Data Scientists. If you can solve any of these [using open software], you'll be heads down helping us from day one. Email us to brunosanchez@worldbank.org

(This list is updated frequently).

1. Nightlights from Satellite

We are building an open stack to process nightly data from satellite and query light output from all known villages. Currently we are doing 20 years of nightly data for 600,000 villages in India.

celoyd

Himawari-8 animation tutorial

Here’s how to make animations like this one. It requires intermediate Unix command-line knowledge, to install some tools and to debug if they don’t work. You’ll need these utilities:

• curl (or you can translate to wget)
• convert and montage, part of ImageMagick
• ffmpeg, plus whatever codecs
• parallel, for iteration that’s nicer than shell for loops or xargs
• run everything in zsh for leading 0s in numerical ranges to work

kgjenkins

albert-decatur

#!/bin/bash

# see Development Seed landsat-api scene extent on geojson.io
# example use: $0 LC81660362014196LGN00

scene=$1
# get scene metadata from devseed landsat-api
curl -s "https://api.developmentseed.org/landsat?search=${scene}"  |\
# get corner coords
jq '.results[]|[.lowerLeftCornerLongitude,.lowerLeftCornerLatitude,"",.lowerRightCornerLongitude,.lowerRightCornerLatitude,"",.upperRightCornerLongitude,.upperRightCornerLatitude,"",.upperLeftCornerLongitude,.upperLeftCornerLatitude,"",.lowerLeftCornerLongitude,.lowerLeftCornerLatitude]|@csv' |\

celoyd

#!/usr/bin/env python
# Very simple rasterio NDVI (or general normalized difference ratio) demo.
# python ndvi.py $landsat_scene/*B{4,5}.TIF ndvi.tif

import rasterio as rio
from sys import argv
import numpy as np

math_type = np.float32 # type for internal calculations

briantjacobs

# This is a shell function to quickly grab the data for a given Landsat 8 tile ID from Google's servers
# For example:
# l8get LC81690352014169LGN00

# The echo at the end is to remind myself of the syntax for extracting bands 8, 4, 3, and 2. (Pansharp, Red, Green, Blue)

# On OSX this would go into your ~/.bash_profile file.
# Requires gsutil from https://developers.google.com/storage/docs/gsutil_install
# Most useful in conjunction with USGS' Earth Explorer: http://earthexplorer.usgs.gov/

dwtkns

# These are shell functions to quickly create pansharpened RGB images ready for color correction 
# from a set of raw Landsat 8 TIF files in a given directory.

# For example:
#  l8_pansharp LC81690352014169LGN00
# Creates:
#  LC81690352014169LGN00_rgb_pansharp.tif

# l8_rush_pansharp downloads a L8 tile from Google Storage (if they have it), extracts the relevant bands, then creates the pansharpened output.

bsweger

Useful Pandas Snippets

A personal diary of DataFrame munging over the years.

Data Types and Conversion

Convert Series datatype to numeric (will error if column has non-numeric values)
(h/t @makmanalp)

dwtkns

# This is a shell function to quickly grab the data for a given Landsat 8 tile ID from Google's servers
# For example:
# l8get LC81690352014169LGN00

# The echo at the end is to remind myself of the syntax for extracting bands 8, 4, 3, and 2. (Pansharp, Red, Green, Blue)

# On OSX this would go into your ~/.bash_profile file.
# Requires gsutil from https://developers.google.com/storage/docs/gsutil_install
# Most useful in conjunction with USGS' Earth Explorer: http://earthexplorer.usgs.gov/