sharkinsspatial

Holy grail

Before diving too deeply into the various friction points when working with archives of earth observation data in xarray, let's look at a more optimal case from the earth systems world. In the notebook here we demonstrate how using zarr's consolidated metadata option to access the dimensional and chunk reference information, a massive dataset's dimensions and variables can be loaded extremely quickly. With this consolidated metadata available to reference chunks on disk, we can leverage xarray's dask integration to use normal xarray operations to lazily load chunks in parallel and perform our calculations using dask's blocked algorithm implementations. Gravy.

Challenges

But the earth observation story is more complicated... Not everything lives in standardized file containers and more importantly our grid coordinate systems are "all over the map" :] Here are some of the current challenges.

1. Consolida

ocefpaf

phobson

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
cycler=0.10.0=py35_0
decorator=4.0.9=py35_0
ipykernel=4.3.1=py35_0
ipython=4.1.2=py35_0
ipython_genutils=0.1.0=py35_0
ipywidgets=4.1.1=py35_0
jinja2=2.8=py35_0

m3t

On Mar 28, 2013 Dan Allen explained how to deploy to github-pages automatically. Many other GitHub users like Steve Klabnik and Domenic Denicola followed with the same approach:

Use an OAuth token with public_repo or repo permission to access the Github account over HTTPS inside a virtual build environment.

The same applies to Travis CI's built-in abilities, and there are more restrictions as deploying GitHub Releases works only for tags, not for branches.

Security concerns

The token grants write access for all of your (public) repositories and

lpinner

import numpy as np
import rasterio as rio
import dask
import dask.array as da

class RioDataset():
    def __init__(self, filepath, band=1):
        self.band = band
        self.dataset = rio.open(filepath)
        self.dtype = self.dataset.dtypes[band-1]

arthur-e

USERNAME=heyyouguys

# NOTE: Could not determine which jpeg library should be installed; it's probably installed by default
sudo apt-get install libtiff5 libtiff5-dev libgeotiff2 libgeotiff-dev libxml2 libxml2-dev

# http://www.hdfgroup.org/release4/obtainsrc.html
sudo mkdir /usr/local/hdf4 && sudo chown $USERNAME /usr/local/hdf4 && cd /usr/local/hdf4
wget http://www.hdfgroup.org/ftp/HDF/HDF_Current/src/hdf-4.2.11.tar.gz
tar -xzvf hdf-4.2.11.tar.gz
cd hdf-4.2.11

domenic

Auto-deploying built products to gh-pages with GitHub Actions

This is a set up for projects which want to check in only their source files, but have their gh-pages branch automatically updated with some compiled output every time they push.

A file below this one contains the steps for doing this with Travis CI. However, these days I recommend GitHub Actions, for the following reasons:

• It is much easier and requires less steps, because you are already authenticated with GitHub, so you don't need to share secret keys across services like you do when coordinate Travis CI and GitHub.
• It is free, with no quotas.
• Anecdotally, builds are much faster with GitHub Actions than with Travis CI, especially in terms of time spent waiting for a builder.

Thrilleratplay

Installing Arch Linux on an LUKS Encrypted root and booting from UEFI.

---

Sources

• Gist mattiaslundberg / arch-linux-install
• HOWTO: Disk encryption with dm-crypt / LUKS and Debian

Before you begin

• For Gummiboot, EFI must be enabled in the bios setup BEFORE YOU START the installation process. It will yell at you if this is not enabled.
• If you do not have UEFI, it may be easier to use rEFInd or Grub2

briantjacobs

Storytelling from Space: Tools/Resources

This list of resources is all about acquring and processing aerial imagery. It's generally broken up in three ways: how to go about this in Photoshop/GIMP, using command-line tools, or in GIS software, depending what's most comfortable to you. Often these tools can be used in conjunction with each other.

Acquiring Landsat & MODIS

Web Interface

• USGS Earth Explorer - Browser and data access (create a login)
  http://earthexplorer.usgs.gov/

• Landsat archive

rashkur

import ConfigParser, mmap

config_file = "/usr/share/applications/google-chrome.desktop"
add_string_to_each_section = ["StartupWMClass", "Google-chrome-stable"]
option = add_string_to_each_section[0]
value = add_string_to_each_section[1]


class Fixer:
    def check(self, cf, option, value):