manzt

My Minimal TypeScript Monorepo Setup for ESM Libraries

After a deep dive looking up best practices for TypeScript monorepos recently, I couldn't find anything that suited my needs:

1. Publish individual (typed) packages to NPM with minimal config.
2. Supports fast unit testing that spans the entire project (e.g., via Vitest)
3. Ability to have an interactive playground to experiment with the API in a real-time (e.g., via Vite)

Most solutions point to TypeScript project references,

sharkinsspatial

Holy grail

Before diving too deeply into the various friction points when working with archives of earth observation data in xarray, let's look at a more optimal case from the earth systems world. In the notebook here we demonstrate how using zarr's consolidated metadata option to access the dimensional and chunk reference information, a massive dataset's dimensions and variables can be loaded extremely quickly. With this consolidated metadata available to reference chunks on disk, we can leverage xarray's dask integration to use normal xarray operations to lazily load chunks in parallel and perform our calculations using dask's blocked algorithm implementations. Gravy.

Challenges

But the earth observation story is more complicated... Not everything lives in standardized file containers and more importantly our grid coordinate systems are "all over the map" :] Here are some of the current challenges.

1. Consolida

vincentsarago

Important

DO NOT USE VSCODE terminal and make sure uname -m returns arm64

1. First Install Xcode command line tool

xcode-select --install

2. Create and change /opt directories owner

ajfriend

jorisvandenbossche

neharkarvishal

/**
 * zipObject.js
 * tags: { JavaScript, Array, Object }
 * Given an array of valid property identifiers and an array of values,
 * return an object associating the properties to the values.
 * Since an object can have undefined values but not undefined property 
 * pointers, the array of properties is used to decide the structure of
 * the resulting object using Array.prototype.reduce().
 */

pbugnion

vincentsarago

import mercantile
from shapely.geometry import box, shape
from supermercado import burntiles

minzoom = 7
maxzoom = 12
maximum_items_per_tile = 20

def stac_to_mosaicJSON(data):
    dataset = data["features"]

vincentsarago

Let's see how much it will cost to translate all the Sentinel-2 JPEG2000 to COGs using AWS Lambda https://github.com/developmentseed/sentinel-2-cog (Note at this scale AWS Lambda might not be the cheapest option).

How many S2 scenes do we have on AWS?

$ curl https://sat-api.developmentseed.org/collections/sentinel-2-l1c/items | jq -r '.meta.found'
5 633 438

Lambda config

madchap

# get api token with proper perms, ensure full "repos" for private repo access.

$ export auth="Authorization: token 123345465hfghfghfghgfhgfhfg"
$ export org="your_org"

# get the last page in the Link header. github API limits per_page to 100. Anything over this will require pagination.
$ curl -I -H "$auth" https://api.github.com/orgs/$org/repos

# go over all pages. Put this in a script, and save it.
#!/bin/bash