Skip to content

Instantly share code, notes, and snippets.

View omad's full-sized avatar

Damien Ayers omad

41 followers · 9 following
View GitHub Profile
@omad
omad / click_experiment.py
Created November 16, 2023 01:29
Experiments with Click arguments nargs
import click
@click.command()
@click.argument("uri", type=str, nargs=-1)
@click.argument("product", nargs=1, required=False)
def cli(uri, product):
print(f"{repr(uri)}")
print(f"{repr(product)}")
@omad
omad / README.md
Last active November 18, 2022 05:39
Open Data Cube PostgreSQL Database Schema

Run SchemaSpy to generate database table diagrams

Using Docker

docker run -it --rm -v "$PWD:/output" --network="host" schemaspy/schemaspy -u dra547 -host localhost -port 15432 -db datacube -t pgsql -schemas agdc,cubedash -norows -noviews -pfp -imageformat svg

If SchemaSpy is downloaded Locally

java -jar schemaspy-6.1.0.jar -o . -u dra547 -host localhost -port 15432 -db datacube -t pgsql -dp postgresql-42.5.0.jar -s agdc -norows -noviews
@omad
omad / environment.yaml
Created October 14, 2022 05:42
Libs for DEA Python Environment
name: dea-env
channels:
- conda-forge
dependencies:
## Python
- python=3.8
## Necessary CLI Tools
## DEA Critical Python Libraries and Tools
@omad
omad / find-missing-tiffs.py
Created September 24, 2021 02:38
Scan for missing Data files in S3
#!/usr/bin/env python3
#
# Generate an inventory dump with:
# print(entry)
# s3-inventory-dump --prefix baseline | tqdm > baseline-inventory.txt
import dawg
from tqdm import tqdm
from pathlib import Path
import sys
@omad
omad / pod-config.yaml
Last active March 30, 2021 23:59
Kubernetes: Prototype for killing hung pods in a compute Job
apiVersion: v1
kind: Pod
metadata:
labels:
test: liveness
name: liveness-exec
spec:
# Don't recreate the pod, if it was hung, we want it gone.
restartPolicy: Never
containers:
@omad
omad / odc_to_postgis.sql
Created February 6, 2020 22:55
Experiments in converting ODC Datasets to PostGIS Geometries
-- Experiments with PostgreSQL, connecting to a remote server.
CREATE EXTENSION postgres_fdw;
DROP SERVER IF EXISTS collection2;
CREATE SERVER collection2 FOREIGN DATA WRAPPER postgres_fdw OPTIONS (dbname 'datacube', host 'localhost');
DROP USER MAPPING IF EXISTS FOR CURRENT_USER SERVER collection2;
CREATE USER MAPPING FOR dra547 SERVER collection2 OPTIONS (user 'dra547', password 'xxxxx');
@omad
omad / axis.csv
Last active November 19, 2019 03:15
Axis table from Proj.6
EPSG 1039 Easting M east EPSG 1024 1 EPSG 9001
EPSG 1040 Northing P north EPSG 1024 2 EPSG 9001
EPSG 1062 Easting X North along 130°W EPSG 1025 1 EPSG 9001
EPSG 1063 Northing Y North along 140°E EPSG 1025 2 EPSG 9001
EPSG 1065 Easting E South along 90°E EPSG 1026 1 EPSG 9001
EPSG 1066 Northing N South along 180°E EPSG 1026 2 EPSG 9001
EPSG 1056 Easting E North along 90°E EPSG 1027 1 EPSG 9001
EPSG 1058 Northing N North along 0°E EPSG 1027 2 EPSG 9001
EPSG 1073 Easting E east EPSG 1028 1 EPSG 9037
EPSG 1074 Northing N north EPSG 1028 2 EPSG 9037
@omad
omad / delete_odc_product.sql
Last active March 17, 2021 17:22
Delete an Open Data Cube Product using RAW SQL
--------------------------------------
-- SQL to Delete a Data Cube Product
--------------------------------------
--
-- Use with psql from the command line:
--
-- psql -v product_name=<product-to-delete> -f delete_odc_product.sql -h <database-hostname> <dbname>
--
--
@omad
omad / ODC Dataset Metadata Updates.md
Created November 8, 2018 00:04
ODC Dataset Metadata Updates

ODC Dataset Metadata Updates

Problems

  • Too many definitions of spatial extents
    • Including
  • Insufficient spatial metadata
    • Not enough information to define the pixel grid
    • Requires CRS + Affine transform + Shape (width, height)
  • Shape and Transform can be different per band
@omad
omad / README.md
Created September 14, 2018 01:22
MODIS Ocean Colour Available Through DEA Datacube

MODIS Ocean Colour Available Through DEA Datacube