Skip to content

Instantly share code, notes, and snippets.

@omad

omad/ODC Dataset Metadata Updates.md

Created November 8, 2018 00:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save omad/614d33880897668a91a4e710c93e163d to your computer and use it in GitHub Desktop.
Save omad/614d33880897668a91a4e710c93e163d to your computer and use it in GitHub Desktop.
Download ZIP
ODC Dataset Metadata Updates
Raw
ODC Dataset Metadata Updates.md

ODC Dataset Metadata Updates

Problems

  • Too many definitions of spatial extents
    • Including
  • Insufficient spatial metadata
    • Not enough information to define the pixel grid
    • Requires CRS + Affine transform + Shape (width, height)
    • Shape and Transform can be different per band
  • Storing a Valid Region Polygon as an optional extra

Why

  • The eo metadata does not have a schema or any other strict definition
  • Requires file access to determine the pixel grid
  • OR
  • Requires out of band information like pixel size
  1. Can not perform efficient native resolution data load()
  2. Inefficient and complex database queries

Impacts

  • Currently indexed data
  • All applications generating data
    • Ingest
    • WOfS
    • Stats
    • Fractional Cover
  • All dataset prepare scripts
  • Datasets embedded in NetCDF files
  • ODC Indexing Code
  • ODC Querying Code
  • ODC Load Code
  • ODC Database Structure
  • Likely implies a change of model - No longer store and index inside a verbatim dataset document

Dependencies

It would be very useful to have in place first:

  • Database versioning
  • Database migrations

References

  1. Kirill - ODC Metadata Change Proposal/Discussion
  2. Issue 536: Propose to add new field to eo metadata
  3. Issue 537: Inaccurate computation of Lon/Lat bounds in geobox_info
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment