charlesbluca

name: dask-sql
channels:
- conda-forge
- defaults
dependencies:
- adagio>=0.2.3
- antlr4-python3-runtime>=4.9.2
- black=19.10b0
- ciso8601>=2.2.0
- dask-ml>=1.7.0

charlesbluca

Keybase proof

I hereby claim:

• I am charlesbluca on github.
• I am charlesbluca (https://keybase.io/charlesbluca) on keybase.
• I have a public key ASAajBblO28S1jupn7M_6Fw2VUW2FR9z7ULGWoBALcfRtgo

To claim this, I am signing this object:

charlesbluca

BCC_CSM2_MR.historical.r1i1p1f1.pr:
  description: CMIP6 3-hourly precipitation for BCC-CSM2-MR historical r1i1p1f1
  metadata:
    tags:
      - atmosphere
      - model
  driver: zarr
  args:
    storage_options:
      token: anon

charlesbluca

import os
import json
import zarr
import xarray as xr

paths = glob("/lsdf/kit/imk-tro/projects/MOD/Gruppe_Voigt/TRACMIP_ESGFCOPY/*/*/*/*/*/*/*/*/*/*.nc")

for path in paths:

  # get GCS path

charlesbluca

import xarray as xr
from glob import glob

paths = glob("/lsdf/kit/imk-tro/projects/MOD/Gruppe_Voigt/TRACMIP_ESGFCOPY/*/%s/%s*/%s/*/*/*/*/*/*" 
             % ("ECHAM63", "aquaControl", "mon"))
                         
for version in set([i.split("/")[-2] for i in paths]):

  ver_paths = [i for i in paths if version in i]

charlesbluca

from dask.distributed import Client

client = Client("tcp://127.0.0.1:38395")
client

charlesbluca

ssh -L %port%:localhost:%port% hz8227@os-login.lsdf.kit.edu

charlesbluca

jupyter lab --port=%port%

charlesbluca

gcloud auth login

charlesbluca

echo "export CLOUDSDK_PYTHON=/usr/bin/python2.7" >> .bashrc