Skip to content

Instantly share code, notes, and snippets.

View nocollier's full-sized avatar

Nathan Collier nocollier

  • Oak Ridge National Laboratory
  • Tennessee
View GitHub Profile
@nocollier
nocollier / concept.py
Last active January 26, 2024 20:08
A proof of concept REST API for ESGF based on FastAPI
"""A REST API for Globus-based searches that mimics esg-search.
This API is so that community tools that are based on the esg-search RESTful API need
not change to be compatible with the new Globus indices. If you are designing a new
project, you should look to use the globus-sdk directly and this is only a small wrapper
around the `post_search` functionality.
The standalone script does not need installed. You will need FastAPI on which it is
based and the Globus sdk.
@nocollier
nocollier / intake_globus_prototype.py
Created January 20, 2024 02:11
intake-esgf globus integration prototype
import globus_sdk as sdk
# I created a client_uuid for intake-esgf
CLIENT_ID = "81a13009-8326-456e-a487-2d1557d8eb11"
client = sdk.NativeAppAuthClient(CLIENT_ID)
# Authorize our interactions with Globus.
client.oauth2_start_flow()
authorize_url = client.oauth2_get_authorize_url()
print(
@nocollier
nocollier / ztest.py
Created January 10, 2024 19:14
Possible CMIP test
"""ESGF could consider maintaining a database of mean and standard deivations for each
variable in each experiment/table. There should be a public API to this so tools could
be created to test model data that is to be submitted. For example, a z-statistic could
be used:
z = | (submission_mean - esgf_mean) / sqrt(submission_std**2 + esgf_std**2) |
If z < 2, then the distributions are to be considered similar. Perhaps there are better
statistical tests that could be performed if ESGF does the work of providing these
important scalars.
@nocollier
nocollier / esgf_globus_query.py
Last active September 6, 2023 14:26
ESGF Globus query
import re
import pandas as pd
from globus_sdk import SearchClient
from globus_sdk.response import GlobusHTTPResponse
# the Globus uuid of the test ESGF2 Argonne index
INDEX_ID = "d927e2d9-ccdb-48e4-b05d-adbc3d97bbc5"
@nocollier
nocollier / SunHan.yaml
Created September 6, 2022 18:53
Intake YAML Catalog for Photosynthesis data
description: "The Physiological Basis for Estimating Photosynthesis from
Chlorophyll a Fluorescence. Please consult
[here](https://ecommons.cornell.edu/handle/1813/110978) for more information
regarding this data. Columns in each source are species, Tleaf (leaf temperature
in °C), LeafReplicateNumber (replicate number for individual leaf), PARi
(photosynthetically active radiation in µmol m-2 s-1), CO2R (reference CO2
concentration in µmol mol-1), Ci (intercellular CO2 concentration in µmol
mol-1), Anet (net photosynthesis in µmol m-2 s-1), Fm' (maximum chlorophyll a
fluorescence under light in mV), Fs (steady-state chlorophyll a fluorescence
under light in mV), VpdL (leaf-to-air vapour pressure deficit in kPa), Press
@nocollier
nocollier / data.csv
Created February 17, 2020 14:55
Sample data for violin plots
model param1 param2 param3 gpp score
model00 1.3347 0.4306 1.8052 2.2859 0.1868
model01 1.3733 0.4412 2.5093 2.8943 0.3432
model02 1.2145 0.4409 1.0973 1.6951 0.1034
model03 0.9550 0.4576 2.0603 2.3166 0.1926
model04 0.2911 0.4852 2.5538 2.6158 0.2597
model05 1.1281 0.4175 2.6145 2.8779 0.3376
model06 1.2911 0.4828 1.5981 2.1104 0.1567
model07 1.3396 0.4747 1.6167 2.1526 0.1634
model08 1.7115 0.3911 1.3351 2.2056 0.1723
@nocollier
nocollier / GCPtoNetCDF4.py
Last active January 14, 2020 15:00
Converts GCP carbon emission data to netCDF4 for use in ILAMB
import argparse
import glob
from netCDF4 import Dataset
import numpy as np
import xlrd,os,time
remote_source = "doi:10.18160/GCP-2019"
gist_source = "https://gist.github.com/nocollier/6f848e5b643d6f83537ac5a9bab0f9e0"
local_source = "Global_Carbon_Budget_2019v1.0.xlsx"
stamp = time.strftime('%Y-%m-%d', time.localtime(os.path.getmtime(local_source)))
@nocollier
nocollier / my-petsc-config.py
Created October 22, 2019 16:20
My commonly used options for configuring PETSc
import sys
import os
sys.path.insert(0, os.path.abspath('config'))
import configure
mode = 'debug'
if len(sys.argv) == 2: mode = sys.argv[1]
configure_options = [
'--download-exodusii',
@nocollier
nocollier / ilamb_variable.py
Created August 9, 2019 17:06
an xarray accessor implementing functions we need in ILAMB
from cf_units import Unit
import xarray as xr
import numpy as np
@xr.register_dataarray_accessor('ilamb')
class ilamb_variable:
def __init__(self, xarray_obj):
self._obj = xarray_obj
self.measure = None
self.bounds = None
@nocollier
nocollier / AMFtoNetCDF4.py
Created February 6, 2019 03:13
A tool to convert Ameriflux CSV files into a CF-compliant netCDF4 format
"""
Converts csv files downloaded from the Ameriflux archive into a CF-compliant netCDF4 file.
"""
import argparse
import glob
import re
import xlrd
import numpy as np
import cftime as cf
from netCDF4 import Dataset