Skip to content

Instantly share code, notes, and snippets.

Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Multiple Writer/Multiple Reader (MWMR) Example\n",
"\n",
"The Goddard Satellite-based Surface Turbulent Fluxes (GSSTF) Version 3 Dataset is a product of the NASA MEaSURES project. The dataset consists of 7,850 files in the HDF-EOS5 format. Each file contains one-day worth of data in the Equidistant Cylindrical projection on the global 0.25×0.25 degree grid cell. The dataset's temporal coverage is from July 1, 1987 to January 1, 2009. More information is available at https://disc.gsfc.nasa.gov/datasets/GSSTF_NCEP_V3/summary.\n",
"\n",
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jreadey
jreadey / ca30_s3fs_smoke_test.ipynb
Created December 24, 2019 17:56
ca30_s3fs_smoke_test
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jreadey
jreadey / ca30_h5pyd_smoke_test.ipynb
Created December 24, 2019 18:00
ca30_h5pyd_smoke_test
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
# install docker (from instrucitons on: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-18-04)
echo "install docker"
apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"
apt update
apt-cache policy docker-ce
apt install docker-ce
systemctl --no-page status docker
# install docker-compose (from: https://www.digitalocean.com/community/tutorials/how-to-install-docker-compose-on-ubuntu-18-04)
version: '2'
services:
head:
image: hdfgroup.azurecr.io/hsds:${IMAGE_TAG}
container_name: hsds_head
restart: ${RESTART_POLICY}
mem_limit: ${HEAD_RAM}
environment:
- TARGET_SN_COUNT=${SN_CORES}
- TARGET_DN_COUNT=${DN_CORES}
@jreadey
jreadey / aad_open.py
Created November 11, 2021 06:59
Open HSDS folder without a .hscfg file
import h5pyd
api_key = {}
api_key["AD_APP_ID"] = "47b78af2-4d62-41a2-849f-38b0615196bc"
api_key["AD_TENANT_ID"] = "670728e4-f05c-4505-9642-0d1e5754ea9e"
api_key["AD_RESOURCE_ID"] = "7c56e32a-edad-4e64-be49-74e4fedc0ce9"
api_key["openid_provider"] = "azure"
endpoint = "http://20.83.239.208"
@jreadey
jreadey / nsrdb_test.py
Created December 21, 2021 17:08
Fancy Indexing
import random
import sys
import h5pyd
import numpy as np
#
# main
#
if len(sys.argv) <= 1 or sys.argv[1] in ("-h", "--help"):