drwelby

Maxar Open Data GeoParquet STAC Catalog

GeoParquet is an experimental standard for storing geospatial data in the Parquet format. Because Parquet's columnar architecture allow for efficient reads over HTTP it's considered to be an initial attempt at a "Cloud-optimized" vector format.

While it lacks the ability to do optimized spatial reads, since it can filter on a small fraction of a features fields it fits will with storing and querying STAC Items. A STAC Item inherits from GeoJSON so has a spatial component, but also can store a large number of metadata field, many of which may be redundant and rarely useful to query. GeoParquet lets us query for features with simple filter requirements like "all images with a low cloud cover percentage".

The first attempt at converting the full Open Data Catalog to GeoParquet is at:

s3://maxar-opendata/events/maxar-opendata.parquet

drwelby

--extra-index-url https://packages.ard.maxar.com/
maxar-canvas-grid
click
pyjwt
s3fs
boto3
requests
requests_oauthlib
pydantic
backports.cached-property

drwelby

Acini

Grid

Global grid of uniquely identified tiles, 256px x 0.3125m

specs: PDA (default), PD, PA

acini <catid> <file.geojson> generate all Acini tiles intersecting geojson features from cat id

drwelby

Roadmap - what we want to do

???


Sketch

shaping

Pitch - how we're going to build a piece of it

drwelby

{
	"baseLayer": {
		"tilejson": "2.2.0",
		"name": "base",
		"version": "1.0.0",
		"scheme": "xyz",
		"tiles": [
			"https://a.tile.openstreetmap.org/{z}/{x}/{y}.png"
		]
	},

drwelby

canvas.town

Mastodon works like Twitter, but instead of a central server users are on independent instances that can talk to each other.

Let's say every Canvas grid tile is an account on a server called canvas.town. It can post activities, like:

Z40-0df769@canvas.town: WV04 acquired imagery over me, ID 1001212350

It can also take commands if someone sends it a message.

drwelby

class Point(object):
  def __init__(self, x, y):
    self.x = float(x)
    self.y = float(y)

def square(x):
  return x * x

def distance_squared(v, w):
  return square(v.x - w.x) + square(v.y - w.y)

drwelby

Soundcloud home page:

https://soundcloud.com/farm-to-taber-podcast

The actual RSS url that's hidden:

http://feeds.soundcloud.com/users/soundcloud:users:432549147/sounds.rss

drwelby

Stop the Geocortex service (task manager / services tab)

Delete (not rename!) the following indexes (default directories listed here, yours may be different):

C:\Program Files\Latitude Geographics\Geocortex Core\Data\DocumentStore\DocumentStore\[YourClusterName]\nodes\0\indices\geocortex.essentials.default.instantsearch
C:\Program Files\Latitude Geographics\Geocortex Core\Data\DocumentStore\DocumentStore\[YourClusterName]\nodes\0\indices\geocortex.search
C:\Program Files\Latitude Geographics\Geocortex Core\Data\DocumentStore\DocumentStore\[YourClusterName]\nodes\0\indices\geocortex.search.messagequeue

(optional if it's getting out of hand: rename or delete the Logs folder (E:/Geocortex Essentials/Logs))

drwelby

import urllib2
import tempfile
from zipfile import Zipfile
import os
import shutil

def web_import(url, target):
    ''' Takes a url, downloads the file, and copies it to a feature class
        For zipped shapefiles at the moment!!
        Expects the target to be consistent since it truncate/appends'''