lossyrob

The Gang Mistakes a Cat

A week ago, I was woken up by my partner Annie, who in panick delivered the news that Cleopatra was missing. Cleo is her 15-turning-4 year old cat (seriously this old cat is spry like you wouldn't believe), who had apparently slipped out the door as we left the house the night before. I say "her" cat, because she's spent all 15 years of Cleo’s life with her, but in the years I've known her and the bit we’ve been roommates, I've grown quite fond of this cat. I’ve even written her a song that I would be embarrassed if you heard (the true sign of Rob’s love). So she's a bit "our" cat now. We spent all day and night looking for her, but no dice. The week that followed was morose: Long stretches sitting on our porch shaking dry cat food at an empty night, a very depressed Sadie (the football-shaped granddaughter of Cleo), a have-a-heart trap on our patio that was only catching unbemused street cats that were clearly not Cleo. Not happy times in t

lossyrob

# For ISPRS Potsdam data, Download the validation prediction probability rasters,
# give them the proper extent and bounding box, and upload them back to S3 for visualization.

import os, re, json, shutil
from subprocess import call
import rasterio

ul = r"""Upper Left  \(  ([\d.]+), ([\d.]+)"""
ll = r"""Lower Left  \(  ([\d.]+), ([\d.]+)"""
ur = r"""Upper Right  \(  ([\d.]+), ([\d.]+)"""

lossyrob

I definitely understand the head spinning, code that relies heavily on implicits can be a tricky stack of code to sort through. Once you get the knack of them, hopefully you are converted into a fan as I was :)

First thing to read through would be this notebook, specifically sections "Organization" and "Type constraints"

Let's first take a look at why the compiling code compiles.

We are calling "tileToLayout" on an "RDD[(ProjectedExtent, Tile)]", with a specific cellType and layout definition. The reason we can call this is because of this implicit, as you pointed out:

https://github.com/locationtech/geotrellis/blob/bc09ab8c49fc0c34a98d53c626a0fdd7e82f858f/spark/src/main/scala/geotrellis/spark/tiling/TilerMethods.scala#L55-L57

lossyrob

;; This buffer is for text that is not saved, and for Lisp evaluation. ;; To create a file, visit it with C-x C-f and enter text in its buffer.

RasterRDD[K] is renamed to TileLayerRDD[SpatialKey]

The layer reader API changed a bit. So this:

https://github.com/azavea/usace-flood-geoprocessing/blob/cpc/docker/server/src/main/scala/com/azavea/usaceflood/server/ElevationData.scala#L29-L33

  def apply(multiPolygon: MultiPolygon)(implicit sc: SparkContext): RasterRDD[SpatialKey] =

lossyrob

  def time[T](msg: String)(f: => T) = {
    val start = System.currentTimeMillis
    val v = f
    val end = System.currentTimeMillis
    println(s"[TIMING] $msg: ${java.text.NumberFormat.getIntegerInstance.format(end - start)} ms")
    v
  }

lossyrob

import os, csv, sys, traceback
import mechanicalsoup
from subprocess import call

browser = mechanicalsoup.Browser()

def pull_table_links(s):
    tables = s.select("table")
    if tables and len(tables[0].select('tr')) > 1:
        return map(lambda x: x.select("a")[0].attrs['href'], page.soup.select("tr")[0].select('td'))

lossyrob

#!/bin/sh

#to change L&F if desired.  Blank is default
JAVA_LOOKANDFEEL="-Dswing.defaultlaf=javax.swing.plaf.metal.MetalLookAndFeel"
#JAVA_LOOKANDFEEL=""

JAVA_OPTS="-Xms256M -Xmx1024M"

APP_OPTS=""

lossyrob

Major Features

• GeoTools support
  • Add Support for GeoTools SimpleFeature #1495
  • Conversions between GeoTools GridCoverage2D and GeoTrellis Raster types #1502
• Streaming GeoTiff reading #1559
• Windowed GeoTiff ingests into GeoTrellis layers, allowing users to ingest large GeoTiffs #1763
  • Reading TiffTags via MappedByteBuffer #1541
  • Cropped Windowed GeoTiff Reading #1559
• Added documentation to the GeoTiff* files #1560

lossyrob

import cv2
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt


def buffer_coastline(img, threshold=135):
    arr = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY)
    mask = np.ndarray(arr.shape + (3,)).astype(np.uint8)
    below = arr < threshold

lossyrob

/**
* The reason this doesn't work is that you have to pass each value through the target cell type conversions. This will have
* unique problems for each cell type. For instance, if your inner tile is a DoubleConstantNoDataCellType and has a Double.NaN
* value, and your target cell type is a ByteCellType, you have to flatten the NaN out to 0 before passing it back from
* a getDouble in order to match what a convert(ByteCellType).toArrayTile.get(0, 0) would be.
* Like wise, think of pulling out a (Byte.MinValue.toInt - 5).toInt value on a IntConstantNoDataCellType lazily converted
* to a ByteCellType vs pulled out after a toArray call.
*/

package geotrellis.raster