Skip to content

Instantly share code, notes, and snippets.

@aaronspring
Created March 17, 2020 11:54
Show Gist options
  • Save aaronspring/7d743cf6d96a34479ec8da6b3aee3181 to your computer and use it in GitHub Desktop.
Save aaronspring/7d743cf6d96a34479ec8da6b3aee3181 to your computer and use it in GitHub Desktop.
intake-esm_require_all_on_usage
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@andersy005
Copy link

By the way, I thought I would let you know that intake-esm now supports relative paths for the csv files. This will make it easy for other users to explore the catalog even when they don't have access to the data storage.

As example

{
  "esmcat_version": "0.1.0",
  "id": "mistral-cmip6",
  "description": "This is an ESM collection for CMIP6 data accessible on the DKRZ's MISTRAL disk storage system in /work/ik1017/CMIP6/data/CMIP6",
  "catalog_file": "/home/mpim/m300524/intake-esm-datastore/catalogs/mistral-cmip6.csv.gz",
...
}

becomes

{
  "esmcat_version": "0.1.0",
  "id": "mistral-cmip6",
  "description": "This is an ESM collection for CMIP6 data accessible on the DKRZ's MISTRAL disk storage system in /work/ik1017/CMIP6/data/CMIP6",
  "catalog_file": "mistral-cmip6.csv.gz",
...
}

So, If the json and the csv are in a GitHub repo, I can easily open both the json and csv without needing access to mistral. In earlier versions of intake-esm, this would have failed since I cannot access /home/mpim/m300524/intake-esm-datastore/catalogs/mistral-cmip6.csv.gz without access to the system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment