Skip to content

Instantly share code, notes, and snippets.

@rsignell-usgs

rsignell-usgs/NWM-Zarr-rechunk_v3.ipynb

Created December 10, 2020 18:36
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save rsignell-usgs/c0b87ed1fa5fc694e665fb789e8381bb to your computer and use it in GitHub Desktop.
Save rsignell-usgs/c0b87ed1fa5fc694e665fb789e8381bb to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
NWM-Zarr-rechunk_v3.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Convert lots of small NetCDFs to one big Zarr\n",
"The National Water Model writes a new NetCDF file for each hour, resulting in 8760 files for a year, and 227904 files for the entire 26 year reanalysis (1993-01-01 00:00 - 2018-12-31 23:00). \n",
"\n",
"For small datasets, rechunking the data to Zarr would be as simple as:\n",
"\n",
"```\n",
"import xarray as xr\n",
"ds = xr.open_mfdataset('*.nc')\n",
"ds = ds.chunk({'time':672, 'feature_id':30000})\n",
"ds.to_zarr('all_nc.zarr', consolidated=True)\n",
"```\n",
"For large datasets, this approach is slow and uses too much memory. Here we process the data in batches of 672 time files at a time (one time chunk). \n",
"\n",
"For each batch, we create an xarray dataset with open_mfdataset, then use [rechunker](https://github.com/pangeo-data/rechunker), which creates a rechunked Zarr dataset for that batch. We then append each batch (each time chunk) along the time dimension, building up our overall dataset. \n",
"\n",
"The nice part of this approach is that if something goes wrong with the batch, we can fix the problem and just carry on appending."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import xarray as xr\n",
"import pandas as pd\n",
"import numcodecs\n",
"from dask.distributed import Client, progress, LocalCluster, performance_report\n",
"import zarr\n",
"import time\n",
"import shutil"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.chdir('/usgs/gamone/data2/rsignell/data/NWM2')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Build a list of filenames for open_mfdataset"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"dates = pd.date_range(start='1993-01-01 00:00',end='2018-12-31 23:00', freq='1h')\n",
"\n",
"files = ['./nc/{}/{}.CHRTOUT_DOMAIN1.comp'.format(date.strftime('%Y'),date.strftime('%Y%m%d%H%M')) for date in dates]"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"227904"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(files)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"dset = xr.open_dataset(files[0])"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div><svg style=\"position: absolute; width: 0; height: 0; overflow: hidden\">\n",
"<defs>\n",
"<symbol id=\"icon-database\" viewBox=\"0 0 32 32\">\n",
"<path d=\"M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z\"></path>\n",
"<path d=\"M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
"<path d=\"M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
"</symbol>\n",
"<symbol id=\"icon-file-text2\" viewBox=\"0 0 32 32\">\n",
"<path d=\"M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z\"></path>\n",
"<path d=\"M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
"<path d=\"M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
"<path d=\"M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
"</symbol>\n",
"</defs>\n",
"</svg>\n",
"<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.\n",
" *\n",
" */\n",
"\n",
":root {\n",
" --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));\n",
" --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));\n",
" --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));\n",
" --xr-border-color: var(--jp-border-color2, #e0e0e0);\n",
" --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);\n",
" --xr-background-color: var(--jp-layout-color0, white);\n",
" --xr-background-color-row-even: var(--jp-layout-color1, white);\n",
" --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);\n",
"}\n",
"\n",
"html[theme=dark],\n",
"body.vscode-dark {\n",
" --xr-font-color0: rgba(255, 255, 255, 1);\n",
" --xr-font-color2: rgba(255, 255, 255, 0.54);\n",
" --xr-font-color3: rgba(255, 255, 255, 0.38);\n",
" --xr-border-color: #1F1F1F;\n",
" --xr-disabled-color: #515151;\n",
" --xr-background-color: #111111;\n",
" --xr-background-color-row-even: #111111;\n",
" --xr-background-color-row-odd: #313131;\n",
"}\n",
"\n",
".xr-wrap {\n",
" display: block;\n",
" min-width: 300px;\n",
" max-width: 700px;\n",
"}\n",
"\n",
".xr-text-repr-fallback {\n",
" /* fallback to plain text repr when CSS is not injected (untrusted notebook) */\n",
" display: none;\n",
"}\n",
"\n",
".xr-header {\n",
" padding-top: 6px;\n",
" padding-bottom: 6px;\n",
" margin-bottom: 4px;\n",
" border-bottom: solid 1px var(--xr-border-color);\n",
"}\n",
"\n",
".xr-header > div,\n",
".xr-header > ul {\n",
" display: inline;\n",
" margin-top: 0;\n",
" margin-bottom: 0;\n",
"}\n",
"\n",
".xr-obj-type,\n",
".xr-array-name {\n",
" margin-left: 2px;\n",
" margin-right: 10px;\n",
"}\n",
"\n",
".xr-obj-type {\n",
" color: var(--xr-font-color2);\n",
"}\n",
"\n",
".xr-sections {\n",
" padding-left: 0 !important;\n",
" display: grid;\n",
" grid-template-columns: 150px auto auto 1fr 20px 20px;\n",
"}\n",
"\n",
".xr-section-item {\n",
" display: contents;\n",
"}\n",
"\n",
".xr-section-item input {\n",
" display: none;\n",
"}\n",
"\n",
".xr-section-item input + label {\n",
" color: var(--xr-disabled-color);\n",
"}\n",
"\n",
".xr-section-item input:enabled + label {\n",
" cursor: pointer;\n",
" color: var(--xr-font-color2);\n",
"}\n",
"\n",
".xr-section-item input:enabled + label:hover {\n",
" color: var(--xr-font-color0);\n",
"}\n",
"\n",
".xr-section-summary {\n",
" grid-column: 1;\n",
" color: var(--xr-font-color2);\n",
" font-weight: 500;\n",
"}\n",
"\n",
".xr-section-summary > span {\n",
" display: inline-block;\n",
" padding-left: 0.5em;\n",
"}\n",
"\n",
".xr-section-summary-in:disabled + label {\n",
" color: var(--xr-font-color2);\n",
"}\n",
"\n",
".xr-section-summary-in + label:before {\n",
" display: inline-block;\n",
" content: '►';\n",
" font-size: 11px;\n",
" width: 15px;\n",
" text-align: center;\n",
"}\n",
"\n",
".xr-section-summary-in:disabled + label:before {\n",
" color: var(--xr-disabled-color);\n",
"}\n",
"\n",
".xr-section-summary-in:checked + label:before {\n",
" content: '▼';\n",
"}\n",
"\n",
".xr-section-summary-in:checked + label > span {\n",
" display: none;\n",
"}\n",
"\n",
".xr-section-summary,\n",
".xr-section-inline-details {\n",
" padding-top: 4px;\n",
" padding-bottom: 4px;\n",
"}\n",
"\n",
".xr-section-inline-details {\n",
" grid-column: 2 / -1;\n",
"}\n",
"\n",
".xr-section-details {\n",
" display: none;\n",
" grid-column: 1 / -1;\n",
" margin-bottom: 5px;\n",
"}\n",
"\n",
".xr-section-summary-in:checked ~ .xr-section-details {\n",
" display: contents;\n",
"}\n",
"\n",
".xr-array-wrap {\n",
" grid-column: 1 / -1;\n",
" display: grid;\n",
" grid-template-columns: 20px auto;\n",
"}\n",
"\n",
".xr-array-wrap > label {\n",
" grid-column: 1;\n",
" vertical-align: top;\n",
"}\n",
"\n",
".xr-preview {\n",
" color: var(--xr-font-color3);\n",
"}\n",
"\n",
".xr-array-preview,\n",
".xr-array-data {\n",
" padding: 0 5px !important;\n",
" grid-column: 2;\n",
"}\n",
"\n",
".xr-array-data,\n",
".xr-array-in:checked ~ .xr-array-preview {\n",
" display: none;\n",
"}\n",
"\n",
".xr-array-in:checked ~ .xr-array-data,\n",
".xr-array-preview {\n",
" display: inline-block;\n",
"}\n",
"\n",
".xr-dim-list {\n",
" display: inline-block !important;\n",
" list-style: none;\n",
" padding: 0 !important;\n",
" margin: 0;\n",
"}\n",
"\n",
".xr-dim-list li {\n",
" display: inline-block;\n",
" padding: 0;\n",
" margin: 0;\n",
"}\n",
"\n",
".xr-dim-list:before {\n",
" content: '(';\n",
"}\n",
"\n",
".xr-dim-list:after {\n",
" content: ')';\n",
"}\n",
"\n",
".xr-dim-list li:not(:last-child):after {\n",
" content: ',';\n",
" padding-right: 5px;\n",
"}\n",
"\n",
".xr-has-index {\n",
" font-weight: bold;\n",
"}\n",
"\n",
".xr-var-list,\n",
".xr-var-item {\n",
" display: contents;\n",
"}\n",
"\n",
".xr-var-item > div,\n",
".xr-var-item label,\n",
".xr-var-item > .xr-var-name span {\n",
" background-color: var(--xr-background-color-row-even);\n",
" margin-bottom: 0;\n",
"}\n",
"\n",
".xr-var-item > .xr-var-name:hover span {\n",
" padding-right: 5px;\n",
"}\n",
"\n",
".xr-var-list > li:nth-child(odd) > div,\n",
".xr-var-list > li:nth-child(odd) > label,\n",
".xr-var-list > li:nth-child(odd) > .xr-var-name span {\n",
" background-color: var(--xr-background-color-row-odd);\n",
"}\n",
"\n",
".xr-var-name {\n",
" grid-column: 1;\n",
"}\n",
"\n",
".xr-var-dims {\n",
" grid-column: 2;\n",
"}\n",
"\n",
".xr-var-dtype {\n",
" grid-column: 3;\n",
" text-align: right;\n",
" color: var(--xr-font-color2);\n",
"}\n",
"\n",
".xr-var-preview {\n",
" grid-column: 4;\n",
"}\n",
"\n",
".xr-var-name,\n",
".xr-var-dims,\n",
".xr-var-dtype,\n",
".xr-preview,\n",
".xr-attrs dt {\n",
" white-space: nowrap;\n",
" overflow: hidden;\n",
" text-overflow: ellipsis;\n",
" padding-right: 10px;\n",
"}\n",
"\n",
".xr-var-name:hover,\n",
".xr-var-dims:hover,\n",
".xr-var-dtype:hover,\n",
".xr-attrs dt:hover {\n",
" overflow: visible;\n",
" width: auto;\n",
" z-index: 1;\n",
"}\n",
"\n",
".xr-var-attrs,\n",
".xr-var-data {\n",
" display: none;\n",
" background-color: var(--xr-background-color) !important;\n",
" padding-bottom: 5px !important;\n",
"}\n",
"\n",
".xr-var-attrs-in:checked ~ .xr-var-attrs,\n",
".xr-var-data-in:checked ~ .xr-var-data {\n",
" display: block;\n",
"}\n",
"\n",
".xr-var-data > table {\n",
" float: right;\n",
"}\n",
"\n",
".xr-var-name span,\n",
".xr-var-data,\n",
".xr-attrs {\n",
" padding-left: 25px !important;\n",
"}\n",
"\n",
".xr-attrs,\n",
".xr-var-attrs,\n",
".xr-var-data {\n",
" grid-column: 1 / -1;\n",
"}\n",
"\n",
"dl.xr-attrs {\n",
" padding: 0;\n",
" margin: 0;\n",
" display: grid;\n",
" grid-template-columns: 125px auto;\n",
"}\n",
"\n",
".xr-attrs dt, dd {\n",
" padding: 0;\n",
" margin: 0;\n",
" float: left;\n",
" padding-right: 10px;\n",
" width: auto;\n",
"}\n",
"\n",
".xr-attrs dt {\n",
" font-weight: normal;\n",
" grid-column: 1;\n",
"}\n",
"\n",
".xr-attrs dt:hover span {\n",
" display: inline-block;\n",
" background: var(--xr-background-color);\n",
" padding-right: 10px;\n",
"}\n",
"\n",
".xr-attrs dd {\n",
" grid-column: 2;\n",
" white-space: pre-wrap;\n",
" word-break: break-all;\n",
"}\n",
"\n",
".xr-icon-database,\n",
".xr-icon-file-text2 {\n",
" display: inline-block;\n",
" vertical-align: middle;\n",
" width: 1em;\n",
" height: 1.5em !important;\n",
" stroke-width: 0;\n",
" stroke: currentColor;\n",
" fill: currentColor;\n",
"}\n",
"</style><pre class='xr-text-repr-fallback'>&lt;xarray.Dataset&gt;\n",
"Dimensions: (feature_id: 2729077, reference_time: 1, time: 1)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 1993-01-01\n",
" * reference_time (reference_time) datetime64[ns] 1992-11-01\n",
" * feature_id (feature_id) int32 101 179 181 ... 1180001803 1180001804\n",
" latitude (feature_id) float32 ...\n",
" longitude (feature_id) float32 ...\n",
"Data variables:\n",
" crs |S1 b&#x27;&#x27;\n",
" order (feature_id) int32 ...\n",
" elevation (feature_id) float32 ...\n",
" streamflow (feature_id) float64 ...\n",
" q_lateral (feature_id) float64 ...\n",
" velocity (feature_id) float64 ...\n",
" qSfcLatRunoff (feature_id) float64 ...\n",
" qBucket (feature_id) float64 ...\n",
" qBtmVertRunoff (feature_id) float64 ...\n",
"Attributes:\n",
" featureType: timeSeries\n",
" proj4: +proj=lcc +units=m +a=6370000.0 +b=6370000.0 ...\n",
" model_initialization_time: 1992-11-01_00:00:00\n",
" station_dimension: feature_id\n",
" model_output_valid_time: 1993-01-01_00:00:00\n",
" model_total_valid_times: 1464\n",
" stream_order_output: 1\n",
" cdm_datatype: Station\n",
" Conventions: CF-1.6\n",
" code_version: v5.1.0-alpha11\n",
" model_output_type: channel_rt\n",
" model_configuration: retrospective\n",
" dev_OVRTSWCRT: 1\n",
" dev_NOAH_TIMESTEP: 3600\n",
" dev_channel_only: 0\n",
" dev_channelBucket_only: 0\n",
" dev: dev_ prefix indicates development/internal me...</pre><div class='xr-wrap' hidden><div class='xr-header'><div class='xr-obj-type'>xarray.Dataset</div></div><ul class='xr-sections'><li class='xr-section-item'><input id='section-c4051d78-494f-42a8-bc26-074ec1a92956' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-c4051d78-494f-42a8-bc26-074ec1a92956' class='xr-section-summary' title='Expand/collapse section'>Dimensions:</label><div class='xr-section-inline-details'><ul class='xr-dim-list'><li><span class='xr-has-index'>feature_id</span>: 2729077</li><li><span class='xr-has-index'>reference_time</span>: 1</li><li><span class='xr-has-index'>time</span>: 1</li></ul></div><div class='xr-section-details'></div></li><li class='xr-section-item'><input id='section-ff249faa-e5e2-40f7-bea4-b6929dd5614e' class='xr-section-summary-in' type='checkbox' checked><label for='section-ff249faa-e5e2-40f7-bea4-b6929dd5614e' class='xr-section-summary' >Coordinates: <span>(5)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>time</span></div><div class='xr-var-dims'>(time)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>1993-01-01</div><input id='attrs-93ba34b5-dd70-4646-9235-31fc5650bdc8' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-93ba34b5-dd70-4646-9235-31fc5650bdc8' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-65d6a718-05d0-4203-a1e3-3023dd76d554' class='xr-var-data-in' type='checkbox'><label for='data-65d6a718-05d0-4203-a1e3-3023dd76d554' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>valid output time</dd><dt><span>standard_name :</span></dt><dd>time</dd><dt><span>valid_min :</span></dt><dd>12009660</dd><dt><span>valid_max :</span></dt><dd>12097440</dd></dl></div><div class='xr-var-data'><pre>array([&#x27;1993-01-01T00:00:00.000000000&#x27;], dtype=&#x27;datetime64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>reference_time</span></div><div class='xr-var-dims'>(reference_time)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>1992-11-01</div><input id='attrs-6ddc08fb-b9b0-4640-8d42-1c75fde356bb' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-6ddc08fb-b9b0-4640-8d42-1c75fde356bb' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-dd011c49-6eb2-4690-9eca-85fd39cc19e6' class='xr-var-data-in' type='checkbox'><label for='data-dd011c49-6eb2-4690-9eca-85fd39cc19e6' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>model initialization time</dd><dt><span>standard_name :</span></dt><dd>forecast_reference_time</dd></dl></div><div class='xr-var-data'><pre>array([&#x27;1992-11-01T00:00:00.000000000&#x27;], dtype=&#x27;datetime64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>feature_id</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>int32</div><div class='xr-var-preview xr-preview'>101 179 ... 1180001803 1180001804</div><input id='attrs-bc5eec98-6e02-43ee-a679-0bbfa9ece9b0' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-bc5eec98-6e02-43ee-a679-0bbfa9ece9b0' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-b7c3e993-3716-48af-b97e-e6bdc718db69' class='xr-var-data-in' type='checkbox'><label for='data-b7c3e993-3716-48af-b97e-e6bdc718db69' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Reach ID</dd><dt><span>comment :</span></dt><dd>Gage Points Specified by User in Routelink file</dd><dt><span>cf_role :</span></dt><dd>timeseries_id</dd></dl></div><div class='xr-var-data'><pre>array([ 101, 179, 181, ..., 1180001802, 1180001803,\n",
" 1180001804], dtype=int32)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>latitude</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-cf6921d2-3c2b-43b1-bbe5-d035253d82d0' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-cf6921d2-3c2b-43b1-bbe5-d035253d82d0' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-b3ec1c9d-3b74-43bd-b3ec-e7bd0b5e7782' class='xr-var-data-in' type='checkbox'><label for='data-b3ec1c9d-3b74-43bd-b3ec-e7bd0b5e7782' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Feature latitude</dd><dt><span>standard_name :</span></dt><dd>latitude</dd><dt><span>units :</span></dt><dd>degrees_north</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float32]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>longitude</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-16e6b7fa-90f0-4324-85d1-6336cafc2934' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-16e6b7fa-90f0-4324-85d1-6336cafc2934' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-78f19b04-9fec-4b75-a42b-e674aaa13e8e' class='xr-var-data-in' type='checkbox'><label for='data-78f19b04-9fec-4b75-a42b-e674aaa13e8e' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Feature longitude</dd><dt><span>standard_name :</span></dt><dd>longitude</dd><dt><span>units :</span></dt><dd>degrees_east</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float32]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-e1a1d8ee-a1bd-4780-85a3-dfef2cea7a66' class='xr-section-summary-in' type='checkbox' checked><label for='section-e1a1d8ee-a1bd-4780-85a3-dfef2cea7a66' class='xr-section-summary' >Data variables: <span>(9)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span>crs</span></div><div class='xr-var-dims'>()</div><div class='xr-var-dtype'>|S1</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-1aa14f3f-a40b-496d-a5cc-bc151dd630f8' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-1aa14f3f-a40b-496d-a5cc-bc151dd630f8' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-bcd611d9-9931-4faf-8b29-f728680628b6' class='xr-var-data-in' type='checkbox'><label for='data-bcd611d9-9931-4faf-8b29-f728680628b6' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>transform_name :</span></dt><dd>latitude longitude</dd><dt><span>grid_mapping_name :</span></dt><dd>latitude longitude</dd><dt><span>esri_pe_string :</span></dt><dd>GEOGCS[&quot;GCS_WGS_1984&quot;,DATUM[&quot;D_WGS_1984&quot;,SPHEROID[&quot;WGS_1984&quot;,6378137.0,298.257223563]],PRIMEM[&quot;Greenwich&quot;,0.0],UNIT[&quot;Degree&quot;,0.0174532925199433]];-400 -400 1000000000;-100000 10000;-100000 10000;8.98315284119521E-09;0.001;0.001;IsHighPrecision</dd><dt><span>spatial_ref :</span></dt><dd>GEOGCS[&quot;GCS_WGS_1984&quot;,DATUM[&quot;D_WGS_1984&quot;,SPHEROID[&quot;WGS_1984&quot;,6378137.0,298.257223563]],PRIMEM[&quot;Greenwich&quot;,0.0],UNIT[&quot;Degree&quot;,0.0174532925199433]];-400 -400 1000000000;-100000 10000;-100000 10000;8.98315284119521E-09;0.001;0.001;IsHighPrecision</dd><dt><span>long_name :</span></dt><dd>CRS definition</dd><dt><span>longitude_of_prime_meridian :</span></dt><dd>0.0</dd><dt><span>_CoordinateAxes :</span></dt><dd>latitude longitude</dd><dt><span>semi_major_axis :</span></dt><dd>6378137.0</dd><dt><span>semi_minor_axis :</span></dt><dd>6356752.5</dd><dt><span>inverse_flattening :</span></dt><dd>298.25723</dd></dl></div><div class='xr-var-data'><pre>array(b&#x27;&#x27;, dtype=&#x27;|S1&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>order</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>int32</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-e7a2698f-066a-4af7-b6c1-cba49b44268c' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-e7a2698f-066a-4af7-b6c1-cba49b44268c' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-6b726e4c-1c76-4935-bf4b-fba8294b550a' class='xr-var-data-in' type='checkbox'><label for='data-6b726e4c-1c76-4935-bf4b-fba8294b550a' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Streamflow Order</dd><dt><span>standard_name :</span></dt><dd>order</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=int32]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>elevation</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-e1be5495-4872-4ee6-8f5b-7c057ad8a8b5' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-e1be5495-4872-4ee6-8f5b-7c057ad8a8b5' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-e8dd908b-ddc7-4bbd-8f5e-6063af0204e8' class='xr-var-data-in' type='checkbox'><label for='data-e8dd908b-ddc7-4bbd-8f5e-6063af0204e8' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Feature Elevation</dd><dt><span>standard_name :</span></dt><dd>Elevation</dd><dt><span>units :</span></dt><dd>meters</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float32]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>streamflow</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-d660c258-2f94-4e7d-819c-802f203df22e' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-d660c258-2f94-4e7d-819c-802f203df22e' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-0df8935c-1cbd-440b-bb1f-03d4a6f4cb00' class='xr-var-data-in' type='checkbox'><label for='data-0df8935c-1cbd-440b-bb1f-03d4a6f4cb00' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>River Flow</dd><dt><span>units :</span></dt><dd>m3 s-1</dd><dt><span>grid_mapping :</span></dt><dd>crs</dd><dt><span>valid_range :</span></dt><dd>[ 0 50000000]</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float64]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>q_lateral</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-843671c6-74b3-4000-8950-3ccf03a7e083' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-843671c6-74b3-4000-8950-3ccf03a7e083' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-04f0c0b5-3477-44c1-a3a7-ede346dbc523' class='xr-var-data-in' type='checkbox'><label for='data-04f0c0b5-3477-44c1-a3a7-ede346dbc523' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Runoff into channel reach</dd><dt><span>units :</span></dt><dd>m3 s-1</dd><dt><span>grid_mapping :</span></dt><dd>crs</dd><dt><span>valid_range :</span></dt><dd>[ 0 5000000]</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float64]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>velocity</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-b06a6d9a-037e-4923-a770-9210d6b05203' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-b06a6d9a-037e-4923-a770-9210d6b05203' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-560f4ef1-d5f7-4160-a695-1be7325230c9' class='xr-var-data-in' type='checkbox'><label for='data-560f4ef1-d5f7-4160-a695-1be7325230c9' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>River Velocity</dd><dt><span>units :</span></dt><dd>ms-1</dd><dt><span>grid_mapping :</span></dt><dd>crs</dd><dt><span>valid_range :</span></dt><dd>[ 0 50000000]</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float64]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>qSfcLatRunoff</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-8cf45295-faf7-4a0c-baf4-a32efbcd7d8c' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-8cf45295-faf7-4a0c-baf4-a32efbcd7d8c' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-70fce9bc-6d47-47d1-ba4f-e8910a612618' class='xr-var-data-in' type='checkbox'><label for='data-70fce9bc-6d47-47d1-ba4f-e8910a612618' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Runoff from terrain routing</dd><dt><span>units :</span></dt><dd>m3 s-1</dd><dt><span>grid_mapping :</span></dt><dd>crs</dd><dt><span>valid_range :</span></dt><dd>[ 0 -2147483648]</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float64]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>qBucket</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-d6cca3c2-5b99-4a36-b220-655773e49742' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-d6cca3c2-5b99-4a36-b220-655773e49742' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-9f53ab7a-4585-4b99-85f3-441521760ad6' class='xr-var-data-in' type='checkbox'><label for='data-9f53ab7a-4585-4b99-85f3-441521760ad6' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Flux from gw bucket</dd><dt><span>units :</span></dt><dd>m3 s-1</dd><dt><span>grid_mapping :</span></dt><dd>crs</dd><dt><span>valid_range :</span></dt><dd>[ 0 -2147483648]</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float64]</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>qBtmVertRunoff</span></div><div class='xr-var-dims'>(feature_id)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-89eab4ea-b91c-45ac-831f-f58a50d24e01' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-89eab4ea-b91c-45ac-831f-f58a50d24e01' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-40baa16d-6af5-4c05-9a32-4080179d0442' class='xr-var-data-in' type='checkbox'><label for='data-40baa16d-6af5-4c05-9a32-4080179d0442' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Runoff from bottom of soil to bucket</dd><dt><span>units :</span></dt><dd>m3</dd><dt><span>grid_mapping :</span></dt><dd>crs</dd><dt><span>valid_range :</span></dt><dd>[ 0 499999968]</dd></dl></div><div class='xr-var-data'><pre>[2729077 values with dtype=float64]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-0767161d-9423-4887-afa0-f4c890b87e1f' class='xr-section-summary-in' type='checkbox' ><label for='section-0767161d-9423-4887-afa0-f4c890b87e1f' class='xr-section-summary' >Attributes: <span>(17)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>featureType :</span></dt><dd>timeSeries</dd><dt><span>proj4 :</span></dt><dd>+proj=lcc +units=m +a=6370000.0 +b=6370000.0 +lat_1=30.0 +lat_2=60.0 +lat_0=40.0 +lon_0=-97.0 +x_0=0 +y_0=0 +k_0=1.0 +nadgrids=@</dd><dt><span>model_initialization_time :</span></dt><dd>1992-11-01_00:00:00</dd><dt><span>station_dimension :</span></dt><dd>feature_id</dd><dt><span>model_output_valid_time :</span></dt><dd>1993-01-01_00:00:00</dd><dt><span>model_total_valid_times :</span></dt><dd>1464</dd><dt><span>stream_order_output :</span></dt><dd>1</dd><dt><span>cdm_datatype :</span></dt><dd>Station</dd><dt><span>Conventions :</span></dt><dd>CF-1.6</dd><dt><span>code_version :</span></dt><dd>v5.1.0-alpha11</dd><dt><span>model_output_type :</span></dt><dd>channel_rt</dd><dt><span>model_configuration :</span></dt><dd>retrospective</dd><dt><span>dev_OVRTSWCRT :</span></dt><dd>1</dd><dt><span>dev_NOAH_TIMESTEP :</span></dt><dd>3600</dd><dt><span>dev_channel_only :</span></dt><dd>0</dd><dt><span>dev_channelBucket_only :</span></dt><dd>0</dd><dt><span>dev :</span></dt><dd>dev_ prefix indicates development/internal meta data</dd></dl></div></li></ul></div></div>"
],
"text/plain": [
"<xarray.Dataset>\n",
"Dimensions: (feature_id: 2729077, reference_time: 1, time: 1)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 1993-01-01\n",
" * reference_time (reference_time) datetime64[ns] 1992-11-01\n",
" * feature_id (feature_id) int32 101 179 181 ... 1180001803 1180001804\n",
" latitude (feature_id) float32 ...\n",
" longitude (feature_id) float32 ...\n",
"Data variables:\n",
" crs |S1 ...\n",
" order (feature_id) int32 ...\n",
" elevation (feature_id) float32 ...\n",
" streamflow (feature_id) float64 ...\n",
" q_lateral (feature_id) float64 ...\n",
" velocity (feature_id) float64 ...\n",
" qSfcLatRunoff (feature_id) float64 ...\n",
" qBucket (feature_id) float64 ...\n",
" qBtmVertRunoff (feature_id) float64 ...\n",
"Attributes:\n",
" featureType: timeSeries\n",
" proj4: +proj=lcc +units=m +a=6370000.0 +b=6370000.0 ...\n",
" model_initialization_time: 1992-11-01_00:00:00\n",
" station_dimension: feature_id\n",
" model_output_valid_time: 1993-01-01_00:00:00\n",
" model_total_valid_times: 1464\n",
" stream_order_output: 1\n",
" cdm_datatype: Station\n",
" Conventions: CF-1.6\n",
" code_version: v5.1.0-alpha11\n",
" model_output_type: channel_rt\n",
" model_configuration: retrospective\n",
" dev_OVRTSWCRT: 1\n",
" dev_NOAH_TIMESTEP: 3600\n",
" dev_channel_only: 0\n",
" dev_channelBucket_only: 0\n",
" dev: dev_ prefix indicates development/internal me..."
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dset"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A nice chunk size for object storage is on the order of 100Mb. "
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"time_chunk_size = 672 \n",
"feature_chunk_size = 30000"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"121.0"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"81312/672"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"90.96923333333334"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"nh_chunks = len(dset.feature_id)/feature_chunk_size\n",
"nh_chunks"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"340"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"nt_chunks = int(np.ceil(len(files)/time_chunk_size))\n",
"nt_chunks"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"161.28"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(time_chunk_size * feature_chunk_size )*8 / 1e6"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"... Close enough to 100Mb"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a function to drop stuff that messes up `open_mfdataset`"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"def drop_coords(ds):\n",
" ds = ds.drop(['reference_time','feature_id', 'crs'])\n",
" return ds.reset_coords(drop=True)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"#cluster.close(); client.close()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a local dask cluster"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "b98aabe52ffd41eca9786a151266d5ec",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"VBox(children=(HTML(value='<h2>LocalCluster</h2>'), HBox(children=(HTML(value='\\n<div>\\n <style scoped>\\n …"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#cluster = LocalCluster() # for this first step, having 32 workers is faster\n",
"cluster = LocalCluster(n_workers=8, threads_per_worker=1) \n",
"cluster"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"client = Client(cluster)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Tell blosc not to use threads since we are using dask to parallelize"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"numcodecs.blosc.use_threads = False"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"zarr_step = 'zarr/step'\n",
"zarr_chunked = 'zarr/nwm'\n",
"\n",
"zarr_temp = 'zarr/tmp'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Step our way through the dataset, reading one chunk along the time dimension at a time, to avoid dask reading too many chunks before writing and blowing out memory. First time chunk is written to zarr, then others are appended. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Set up Rechunker"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"from rechunker import rechunk"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"13.50GB\n"
]
}
],
"source": [
"chunk_mem_factor = 0.8 #fraction of worker memory for each chunk (seems to be the max possible)\n",
"worker_mem = cluster.worker_spec[0]['options']['memory_limit']/1e9\n",
"max_mem = '{:.2f}GB'.format(chunk_mem_factor * worker_mem)\n",
"print(max_mem)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Do the big loop"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"319\n",
"320\n",
"321\n",
"322\n",
"323\n",
"324\n",
"325\n",
"326\n",
"327\n",
"328\n",
"329\n",
"330\n",
"331\n",
"332\n",
"333\n",
"334\n",
"335\n",
"336\n",
"337\n",
"338\n",
"339\n"
]
},
{
"ename": "ValueError",
"evalue": "Invalid chunk_limits (96, 2729077).",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<timed exec>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n",
"\u001b[0;32m~/miniconda3/envs/pangeo/lib/python3.7/site-packages/rechunker/api.py\u001b[0m in \u001b[0;36mrechunk\u001b[0;34m(source, target_chunks, max_mem, target_store, target_options, temp_store, temp_options, executor)\u001b[0m\n\u001b[1;32m 301\u001b[0m \u001b[0mtarget_options\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtarget_options\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 302\u001b[0m \u001b[0mtemp_store\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtemp_store\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 303\u001b[0;31m \u001b[0mtemp_options\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtemp_options\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 304\u001b[0m )\n\u001b[1;32m 305\u001b[0m \u001b[0mplan\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mexecutor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mprepare_plan\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcopy_spec\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/miniconda3/envs/pangeo/lib/python3.7/site-packages/rechunker/api.py\u001b[0m in \u001b[0;36m_setup_rechunk\u001b[0;34m(source, target_chunks, max_mem, target_store, target_options, temp_store, temp_options)\u001b[0m\n\u001b[1;32m 381\u001b[0m \u001b[0mtemp_store_or_group\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtemp_group\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 382\u001b[0m \u001b[0mtemp_options\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 383\u001b[0;31m \u001b[0mname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 384\u001b[0m )\n\u001b[1;32m 385\u001b[0m \u001b[0mcopy_spec\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwrite\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mattrs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mupdate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvariable_attrs\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# type: ignore\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/miniconda3/envs/pangeo/lib/python3.7/site-packages/rechunker/api.py\u001b[0m in \u001b[0;36m_setup_array_rechunk\u001b[0;34m(source_array, target_chunks, max_mem, target_store_or_group, target_options, temp_store_or_group, temp_options, name)\u001b[0m\n\u001b[1;32m 484\u001b[0m \u001b[0mitemsize\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 485\u001b[0m \u001b[0mmax_mem\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 486\u001b[0;31m \u001b[0mconsolidate_reads\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconsolidate_reads\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 487\u001b[0m )\n\u001b[1;32m 488\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/miniconda3/envs/pangeo/lib/python3.7/site-packages/rechunker/algorithm.py\u001b[0m in \u001b[0;36mrechunking_plan\u001b[0;34m(shape, source_chunks, target_chunks, itemsize, max_mem, consolidate_reads, consolidate_writes)\u001b[0m\n\u001b[1;32m 127\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 128\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mconsolidate_writes\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 129\u001b[0;31m \u001b[0mwrite_chunks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconsolidate_chunks\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtarget_chunks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mitemsize\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_mem\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 130\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 131\u001b[0m \u001b[0mwrite_chunks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtuple\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget_chunks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m~/miniconda3/envs/pangeo/lib/python3.7/site-packages/rechunker/algorithm.py\u001b[0m in \u001b[0;36mconsolidate_chunks\u001b[0;34m(shape, chunks, itemsize, max_mem, chunk_limits)\u001b[0m\n\u001b[1;32m 51\u001b[0m \u001b[0mchunk_limit_per_axis\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mn_ax\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mn_ax\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 52\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 53\u001b[0;31m \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mf\"Invalid chunk_limits {chunk_limits}.\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 54\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 55\u001b[0m \u001b[0mchunk_mem\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mitemsize\u001b[0m \u001b[0;34m*\u001b[0m \u001b[0mprod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchunks\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mValueError\u001b[0m: Invalid chunk_limits (96, 2729077)."
]
}
],
"source": [
"%%time\n",
"for i in range(nt_chunks):\n",
" print(i)\n",
" istart = i * time_chunk_size\n",
" istop = int(np.min([(i+1) * time_chunk_size, len(files)]))\n",
" \n",
" ds = xr.open_mfdataset(files[istart:istop], parallel=True, \n",
" preprocess=drop_coords, combine='by_coords', \n",
" concat_dim='time', join='override')\n",
" \n",
" # add back in the 'feature_id' coordinate removed by preprocessing \n",
" ds.coords['feature_id'] = dset.coords['feature_id']\n",
" \n",
" # chunk this step to zarr using rechunker\n",
"\n",
" # remote the temp and step zarr datasets\n",
" try:\n",
" shutil.rmtree(zarr_temp)\n",
" while os.path.exists(zarr_temp): # check if it still exists\n",
" pass\n",
" except:\n",
" pass\n",
"\n",
" try:\n",
" shutil.rmtree(zarr_step)\n",
" while os.path.exists(zarr_step): # check if it still exists\n",
" pass\n",
" except:\n",
" pass\n",
" \n",
" chunk_plan={}\n",
" for var in ds.data_vars:\n",
" if len(ds[var].dims)==2:\n",
" var_chunk = (time_chunk_size, feature_chunk_size)\n",
" chunk_plan[var] = var_chunk\n",
"\n",
" array_plan = rechunk(ds, chunk_plan, max_mem, zarr_step, \n",
" temp_store=zarr_temp)\n",
" \n",
" with performance_report(filename=\"dask-report.html\"):\n",
" result = array_plan.execute(retries=10)\n",
"\n",
" # read back in the zarr chunk rechunker wrote\n",
" ds = xr.open_zarr(zarr_step)\n",
"\n",
" if i==0:\n",
" ds.to_zarr(zarr_chunked, consolidated=True, mode='w')\n",
" else:\n",
" ds.to_zarr(zarr_chunked, consolidated=True, append_dim='time')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The workflow threw an error on the last partial chunk because Rechunker doesn't think the chunk_plan is valid. But it is valid to have a partial last chunk. Here we just rechunk the last partial chunk without rechunker and append it to the overall Zarr dataset. "
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"ds1 = ds.chunk({'feature_id':feature_chunk_size, 'time':time_chunk_size})"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<xarray.backends.zarr.ZarrStore at 0x7f53d6518530>"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds1.to_zarr('./zarr/last_step', consolidated=True, mode='w')"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"ds2 = xr.open_zarr('/usgs/gamone/data2/rsignell/data/NWM2/zarr/last_step', consolidated=True)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<xarray.backends.zarr.ZarrStore at 0x7f544c8b4410>"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ds2.to_zarr(zarr_chunked, consolidated=True, append_dim='time')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check the resulting chunked dataset for correct start time, stop time and for any gaps. If there are no gaps we should get just a single unique value of 3600s for the difference between the hourly time steps. "
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1993-01-01T00:00:00.000000000\n",
"2018-12-31T23:00:00.000000000\n"
]
}
],
"source": [
"ds1 = xr.open_zarr('/usgs/gamone/data2/rsignell/data/NWM2/zarr/nwm', consolidated=True)\n",
"print(ds1.time[0].values)\n",
"print(ds1.time[-1].values)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"d1 = ds1.time.diff(dim='time').values/1e9 # convert datetime64 nanoseconds to seconds"
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([3600], dtype='timedelta64[ns]')"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.unique(d1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#cluster.close(); client.close()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Matplotlib is building the font cache; this may take a moment.\n"
]
},
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x7f53f34c1110>]"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEiCAYAAADjxEWuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABL90lEQVR4nO2dd7wU1fXAv4eqCFawgCLYO6gENfaOJRqNRo2aWBI0atRfiqKxRYmiiTUakWjsEXul2QVEpUlvUqU/ens8eOX8/pjZx7x9M7uzZXZ2H+f7+ex7OzN37pyduXPPveeee66oKoZhGIaRTKO4BTAMwzCKE1MQhmEYhi+mIAzDMAxfTEEYhmEYvpiCMAzDMHwxBWEYhmH4YgoiABHZV0S+F5E1InJj3PJkg4gcKyJTUxx/QUR6FlKmuBCRK0RkaA7nDxCR3+RRni9F5Lf5ys/IHRGZLSKnFOA6a0Vkj6ivkw9MQQRzC/ClqrZS1SdyySiuykBVh6jqvoW8pohcLCJTRWSViJSJyIsisrXn+P4i8rl7fLqInJd0/i9FZLKrmCeJyM89xwa4L1fis1FExkfwG+4RkVe8+1T1DFV90T2ek7KJGxG5W0TUWxm6v7ky6f4GVmIi0kJE/i0iS91nOTjp+GEiMtjNZ7GI3OQ59oWILBGR1SIyVkTO9Rw7UUTGi8hKEVkmIu+KSDvP8e1F5HX3uktF5NVE+XIbRGuTPioiv8jXvcsHqtpSVWdC8TfSTEEEszswMW4hAESkSdwyZMDXwNGqug2wB9AE6Am1v+N94CNge6A78IqI7OMebwe8AvwR2Br4C/A/EdkRaivplokPMAx4s5A/rtQRkT2BC4CFPodf997fRCUWQB+cZ7i/+///PNdoDQwEngF2APYCPvacexOwi6puzaYysIt7bBJwuqpuC7QFfgCe9pzbE9gOp2ztCewE3AO1DSJv+TgbWOvKYmSDqton6QN8DlQDFTgFbB+gOfBP4EdgMdAb2NJNvx1OpbcEWOF+39U99vekvJ4EOgAKNPFc80vgt+73K3Aq2keB5TgvReD1U/yOE4B5nu1DgdHAGuB1oC/QM8L72BJ4Cejvbh/k3gPxpPkYuM/9fgRQlpTHEuAon7w7uPe1Y0hZrgCGerYfB+YCq4FRwLHu/m7ARqDSlXWs9/ngVIgV7rXXAiuTn1/A9U4FpgCr3DLwVVL6q4DJbvkZBOwe0TMZAJwJzAZO8ey/B3glZB77uvdt64Dj9wMvh8yrq3s/u/ocaw48AExKkv86z/b1wKCAvJ8Hns/g3tTeE/fajwEL3M9jQHPvewX8CSjDUbZXevLZAfjQvUcjcN5fb1lQHKXZ3S1nG92y9GEUzzyXj/UgfFDVk4AhwA3qtEamAQ/iKIrOOA+3HXCXe0ojnMK4O9AeWI9TCaCqf03K64aQYhwBzAR2xFEyqa6fFhFpBrwHvIzT4nsTCOx6i8gxbjc/6HNMmnNX4SiiX+C8XADilxxHcQCMBCaLyDki0tg1L20Axvmc92tgiKrOCv7VKRmBcy+3B/4HvCkiW6jqQJwKLtGa7uQ9SVUnA9cC37jHt013IbdF/TZwB9AamAEc7Tn+c+B24HygDU55eS1FfqmeS48U510IbFTV/gFJfiYiy0Vkooj8PsVPOgKYA/zNNfOMTzLjHAksF5FhrpnxQxFpnyTLRyJSAXyHo1xHeo61F5GVOO/Rn4GHPKc+BZwtItuJyHY45WuAz29tgdNTejHF70jFX93f0RnohKPI7vAc3xnYBuc9vBp4ypUnIeM6N81v3E89VLUP8CrwkFuWfpalrNERt4Yq1g91W/SC88D39Bw/CpgVcG5nYIVfXu52B9L3IH70HMvo+p40J+D2IIDjcFpC3tb7MKLtQbTDaZnu4243xVF6t7jfT8NpPQ3ynHM1TmuqCigHzgrIezpwRQayXIGnFedzfAXQyf1+D0mtaZ/nMzToeHIaHGX2bdLznOfJbwBwted4I/e3757HZ9ESx1zT0d2eTd0exAE4Jp3GwE9xWsWXBOR1u1t+7wGaAce7z2x/9/g0YCXwE2AL4Anga598mgJnAP8XcJ3tgVuBIz372gKfAjXu5xOgmc+5lwOzvOU9xD2qvSc4SvxMz7HTgdme92o9dd/fMhyF0hinV7Cv55hvD8L9/gIRvoO5fqwHEY42QAtgVKKlhmPXbAO1A3bPiMgcEVkNDAa2FZHGOVxzbtjrh6QtMF/dUukyJwf50qKq83Hk7OtuVwI/B84CFuF00d/AqSxxB00fwnkBExXPsyLS2Zuv23vZGXgrW9lE5E/iDIavcu/nNjit+yhoi+d5us/A+3x3Bx73PNvlOEqkHfnjbzhmH98el6pOUtUFqlqtqsNwTHAXBOS1HqcS7KmqG1X1K+ALHIWfOP6uqo5Q1Qr32j8VkW2SrlmpqgOA00XkHB+ZluP0AN73jMO9iaOAWuGMU83AGbdK5jfAS0nlPRPaUvf9mOPuS7BMVas82+U4SrgNzrib9/l6v5cUpiDCsRSn0B+oqtu6n23UGQgDp6LbFzhCnYG349z9CZNKciFd5/5v4dm3c1Ia7znprh+GhUA7EfGaedoHJQ7wCPF+jg153SY4g4nOj1Idp6rHq+oOqno6zmDjcPdwZ2Cwqo5U1RpVHYFjgkh2PfwN8I6qrg0pQ73fhtMy/SWwnTpmolUEP69k/I6vI/h5LgR281xfvNs4Fcg1nme7rapu6VbUfvKnei63B8h8MnCjiCwSkUXu9d8QkVtT/EY/kyD4m/ySj3vvUeJ7UH51yojPsR1xlAE45p5nVHWd+/x744yp1CIiu+E0Ml5KI2cqFuAo7gTt3X3pWILT+93Vs2+3gLSQvqzFiimIEKhqDfAf4NGER42ItBOR090krXAq8JUisj1wd1IWi3EqwkR+S4D5wGWurf0qgl+QMNcPwzc4BfdGEWkiIufj2FWDrlnHI8TnM8TvPBG51LUhi4jsjjN+8pnn+CEisoXb6/ozsAtONxuccYFjEz0GETkUOBZPhSQiWwIXes7xXvtLEbknxL1o5d6LJUATEbmLTRUQOM+rg4gEvR+LgV3dcZ0EY4Dz3d+1F46pLEE/4EAROd9tCd9IXQXSG7hNRA50f8c27niBL2mey/0Bp52MM9bT2f0sAK7BsZcjIue6dn0Rka6ujO8H5DUYx1niNrcsHY1TIQ9yjz8PnCcinUWkKXAnjollpYjsJyJniMiWItJURC7DaVB95cpxvjhzkBqJSBvgEeB7tzcBThn5rXv+ljgDvWOT5LscGKaqM7w7ReQEEQlbIb8G3CEibcQZQ7oL/55KHVS1GngHuMctC/vhmBiDqFM3FBumIMJzK47d+1vXjPQpTq8BnEHYLXFa+t9S363uceACEVkhIok5Fb/DceNcBhyIMx6Q7fXToqobcQZBr8Cxt1+EU5DzzQE4v2UtjifWVJzfmuBynBZ1GU6ldaqqbnBl/ArHrv2WiKzBGdi9X1W9LpI/x2ntf+Fz7d3ca6ZjEI7dfxqO6aCCumaAhOvsMhEZ7XP+5zgu0ItEZKm771Gc8ZTFOGaRVxOJVXUpjlLrhfO89/bKqarv4jgh9HWf7QQc23zeUNVlqroo8cHxwlrh6YVdjFO+1uC0vB9Ud94HgDgD15e6eVUC5+K03FfhNF5+rapT3OOf44xT9MN5znsBv0pkhfOMy3AU9E3ARaqauM/tcN6fNcB4nHEG71yZq3DG8ObhNLL2wCnTXn6N/+D0bjgNpTD0xBk4H+fKMdrdF4YbcEyWi3CcQl7Dcbbw4zngANe8+B7Uzvep7Ql6e+yJnn1IOXJGsjfRGUbxICK7Am+q6lFxy2IUJyLyLE4ZGZQ2cX6v+yCws6r6ejMVM6YgDMMw8ohrVmqG0/P4CdAfx2PtvTjlygYzMZU4InJ7wGBlPd9wwzAKQisc8+06HC+9hwkezylqIutBuJ4EL+EMxtUAfVT1cXcQ93UcO+Js4JequsLn/G44tvvGwLOq2isSQQ3DMAxfolQQu+DEWxktIq1wwhn8HGdAabmq9hJn1ud2qnpr0rmNcQYQT8UZjBqBM2lnUiTCGoZhGPWIzMSkqgsTngmqugYnzkw7HO+HhIfBizhKI5muwHRVnel63/R1zzMMwzAKREGihIpIB5xAcd8BO6nqQnCUSMKvP4l21HU7nIcT/yUlrVu31g4dOuQsr2EYxubCqFGjlqqqb1SGyBWEiLTE8We/WVVXiwRNpqx7ms8+X1uYiHTHmSxD+/btGTlypF8ywzAMwwcRCQy5E6kXkzuL8m3gVVVNTMpa7I5PJMYpynxOnUfd6em7EjDNXVX7qGoXVe3Spk0moYkMwzCMVESmINx4M88Bk1X1Ec+hD9gU/vY3+Lt/jQD2FpGObjiDi93zDMMwjAIRZQ/iaJywCieJyBj3cyZOuIFTReQHHC+lXgAi0lZE+gO4URJvwAmJMBl4Q1WLYnU3wzCMzYXIxiBUdSjB0RtP9km/AE9URnUWNQla2MQwDMOIGJtJbRiGYfhiCsIwDMPwxRSEYRiG4YspCMMoQdZtqOLQez9m6A9L0yc2jCwxBWEYJci0xWtYUV7JPz6eGrcoRgPGFIRhGIbhiykIwzAMwxdTEIZhGIYvpiAMwzAMX0xBGIZhGL6YgjAMwzB8MQVhGKVMREsGGwaYgjCMkiTkwluGkROmIAzDMAxfTEEYhmEYvpiCMAzDMHwxBWEYhmH4EtmKciLyX+BsoExVD3L3vQ7s6ybZFlipqp19zp0NrAGqgSpV7RKVnIZhGIY/kSkI4AXgSeClxA5VvSjxXUQeBlalOP9EVbVYxoZhGDER5ZrUg0Wkg98xcXz0fgmcFNX1DWNzwGZBGFES1xjEscBiVf0h4LgCH4vIKBHpniojEekuIiNFZOSSJUvyLqhhFCM2C8IoBHEpiEuA11IcP1pVDwPOAK4XkeOCEqpqH1Xtoqpd2rRpk285DcMwNlsKriBEpAlwPvB6UBpVXeD+LwPeBboWRjrDMAwjQRw9iFOAKao6z++giGwlIq0S34HTgAkFlM8wDMMgQgUhIq8B3wD7isg8EbnaPXQxSeYlEWkrIv3dzZ2AoSIyFhgO9FPVgVHJaRiGYfgTpRfTJQH7r/DZtwA40/0+E+gUlVyGYRhGOGwmtWEYhuGLKQjDKGFsOQgjSkxBGEYJYstBGIXAFIRhbCZMmL+Kt0b5Og8ahi9RxmIyDKOIOPtfQwG44PBdY5bEKBWsB2EYhmH4YgrCMAzD8MUUhGEYhuGLKQjDMAzDF1MQhlHCqK0IYUSIKQjDKEHEVoQwCoApCMMwDMMXUxCGYRiGL6YgDMMwDF9MQRiGYRi+mIIwDMMwfDEFYRiGYfgS5ZKj/xWRMhGZ4Nl3j4jMF5Ex7ufMgHO7ichUEZkuIj2iktEwDMMIJsoexAtAN5/9j6pqZ/fTP/mgiDQGngLOAA4ALhGRAyKU0zAMw/AhMgWhqoOB5Vmc2hWYrqozVXUj0Bc4N6/CGYZhGGmJYwziBhEZ55qgtvM53g6Y69me5+7zRUS6i8hIERm5ZMmSfMtqGIax2VJoBfE0sCfQGVgIPOyTxi+GQGDAGVXto6pdVLVLmzZt8iKkYRiGUWAFoaqLVbVaVWuA/+CYk5KZB+zm2d4VWFAI+QzDMIxNFFRBiMguns3zgAk+yUYAe4tIRxFpBlwMfFAI+QzDMIxNRLYmtYi8BpwAtBaRecDdwAki0hnHZDQbuMZN2xZ4VlXPVNUqEbkBGAQ0Bv6rqhOjktMwDMPwJzIFoaqX+Ox+LiDtAuBMz3Z/oJ4LrGEYdVFbDsKIEJtJbRgliNhyEEYBMAVhGIZh+GIKwjAMw/Al5RiEiJwfIo8Kv5AZhmEYRmmTbpD6P8D7+E9eS3AcNqBsGIbR4EinIAao6lWpEojIK3mUxzAMwygSUo5BqOpl6TIIk8YwDMMoPbIepBaRU/MpiGEYmWPzIIwoycWLyXfSm2EYhtEwSOfFFBQDSYAd8i+OYRiGUSykG6Q+FrgMWJu0X/CPxGoYhmE0ENIpiG+BclX9KvmAiEyNRiTDMAyjGEipIFT1jBTHjsu/OIZhGEaxkPEgtYicHYUghmEYRnGRjRfTvXmXwmhQrK6o5LcvjqRsTUXcohiGkQPZKAgLNGyk5M2R8/h08mKe/nJG3KI0eGwahBEl2SiIa8IkEpH/ikiZiEzw7PuHiEwRkXEi8q6IbBtw7mwRGS8iY0RkZBYyGkaDxtaDMApBKAUhIheKSCt383QReUdEDktz2gtAt6R9nwAHqeohwDTgthTnn6iqnVW1SxgZDcMwjPwStgdxp6quEZFjgFOBF4GnU52gqoOB5Un7PlbVKnfzW2DXDOU1DMPImHdGz+OtUfPiFqPkCKsgqt3/ZwG9VfV9oFmO174KGBBwTIGPRWSUiHTP8TqGYWzm/PGNsfz5zbFxi1FypJsol2C+iDwDnAI8KCLNyS3Q31+BKuDVgCRHq+oCEdkR+EREprg9Er+8ugPdAdq3b5+tSIZhGEYSYSv5XwKDgG6quhLYHvhLNhcUkd8AZwOXqvrHolTVBe7/MuBdUoT1UNU+qtpFVbu0adMmG5EMwzAMH0L1IFS1HHjHs70QWJjpxUSkG3ArcLybp1+arYBG7pjHVsBp2NwLwzCMgpNLuO+UiMhrwDfAviIyT0SuBp4EWuGYjcaISG83bVsRSSxbuhMwVETGAsOBfqo6MCo5DcMwDH/CjkFkjKpe4rPbdw0J16R0pvt9JtApKrkMoyERYKU1jLwQWQ/CMIzoEAtoYBSAlApCRPYTkQEi0k9E9hSRF0RkpYgMF5H9CyWkYRiGUXjS9SD6AP8GXgE+BwYC2wH34YwnGEY9zOxhGA2DdAqilap+qKqvAZWq2lcdPsRRFIYRSLGYQfqNW8iq8sq4xTCMkiOdgmjs+f5I0rFcZ1IbRuTMXV7O9f8bzQ2vjY5bFMMoOdIpiKdEpCWAqv47sVNE9gI+jVIwI1oqKqtZsW5j3GJEzoYqJ0rMgpXrY5Ykv6gF+jYKQEoFoarPqOpan/3TVfXmyKQyIueiPt9y6H2fxC2GkSNicb+NCAk1D0JEOgJ/ADp4z1HVc6IRy4iasXNXxi2CkQfMIcCIkrAT5d7DmeT2IVATmTSGYYSiWBwAjIZNWAVRoapPRCqJYRiGUVSEVRCPi8jdwMfAhsROVTXXEMMwjAZKWAVxMHA5cBKbTEzqbhuGYRgNkLAK4jxgD1Vt+H6RhmEYBhA+WN9YYNsI5TAMwzCKjLA9iJ2AKSIygrpjEObmahiG0UAJqyDujlQKwzCMImfUnBWsWLeRUw7YKW5RCkbYJUe/iloQo+FRKpN8126ooqKymtYtm8ctSmiK8d6Om7eSls2bsEeblnGLEgm/eHoYALN7nRWzJIUj3XoQH6XLICiNiPxXRMpEZIJn3/Yi8omI/OD+940IKyLdRGSqiEwXkR7pZDCMXDjpn1/SpaeFFsuVc578mpMetrZkQyJdD+IYEfkgxXEBDgg49gLOmhEvefb1AD5T1V5uxd8DuLVOhiKNgaeAU4F5wAgR+UBVJ6WR1TCyomzNhvSJDGMzJJ2CODdEHr6ur6o6WEQ6+OR3gvv9ReBLkhQE0BWY7q5NjYj0dc8zBWGUHL99cQSttmjKu9/P55vbTmKXbbaMWyTDCE1KBRHB2MNOqrrQzXuhiOzok6YdMNezPQ84Is9yGEbkrFpfyaeTy2q3h89azrmd28UokWFkRth5EIXEb/gtMGSliHQXkZEiMnLJkiURimUYGVJEgVYnLlgVtwhGCVJoBbFYRHYBcP+X+aSZB+zm2d4VWBCUoar2UdUuqtqlTZs2eRXWKH0sGrbDoAmL4hbBKEFCKQgROUlEWuTheh8Av3G//wZ43yfNCGBvEekoIs2Ai93zDCNrGurCOlErwPKNVfz8qa+ZvHB1tBcyipKwPYgrgDEi8o2IPCQiPwtyUU0gIq8B3wD7isg8Ebka6AWcKiI/4Hgp9XLTthWR/gCqWgXcAAwCJgNvqOrELH6bYcRLhDqpUPpu+KzljJm7kgcGTCnMBY2iIuxEuV+DU5EDF+C4obZNdb6qXhJw6GSftAuAMz3b/YH+YWQzjFKhofZijIZL2CVHLwOOxQn7vRRnfsOQCOUyjAaHLQ9qlBphYzE9BswAegNfqOrsqAQyjIaCdRiMUifUGISqtgauArYA/i4iw0Xk5UglM0oWaygbRsMgrBfT1kB7YHegA7ANm1aWM4w6qDsBwBrQdbExCKPUCGtiGur5PKmq86ITyWgoFHt9uKq8kg3V1XGLYRhFS1gvpkMARKQVRTU/tGFSUVlNs8aNaNSoyGvYEuewnp9QXRNdcS7E01N7HY0ICWtiOkhEvgcmAJNEZJSIHBStaJsv+905kB7vjItbjAZPlMohaiRD9VO6v9SIk7AT5foAf1TV3VW1PfAnd58REW+MNCueUTyYi25dlq3dwPNfz2rw9yXsGMRWqvpFYkNVvxSRrSKSyTDyThwvckMYlG4IvyEKbn59DEN+WErXjttzYNtt4hYnMsIqiJkicieQcG29DJgVjUiGkT+sfjOiYNX6SgCqqht2DyKsiekqoA3wDvCu+/3KqIQyDMMw4iesF9MK4MaIZTEMwzCKiJQKQkQ+JIUDhKqek3eJDCMPlK2pYMdWW8QqQ7J1y6xdRqmRrgfxz4JIYRh55IspZVz5wgiev/In7LZdw14DuoE70RQ9v3l+OANvOo6dt4m3MRIV6cYgZqnqV0GfgkhoADD6xxW8MWJu+oRFQNyV1vdzVwIw1v0fJWs3VDFlUbjFdPJ5W2zwvThYWV7JU19Mj1uMyEinIN5LfBGRt6MVxUjF+f8exi1vl9bkuWJwkYxaWV39wgi6PTbE1422CH6+YeREOgXhLeJ7RCmIYURJVMrqu1nLw8sQiQThiLtX19DYXHR/OgWhAd+zRkT2FZExns9qEbk5Kc0JIrLKk+aufFy7FGjoMzMNwygd0g1SdxKR1TgKc0v3O+62qurWmV5QVacCnQFEpDEwH2duRTJDVPXsTPM3jDhQrW9SyjReUqbXi5INVdXMXV6+2bSUDX9S9iBUtbGqbq2qrVS1ifs9sZ2xcvDhZGCGqs7JQ16GUXDiHmcIc/2Fq9YzZPrSjPLt8fZ4TnlkMCvdGcPG5knYmdRRcTHwWsCxo0RkrIgMEJEDCylUnJiFycg3Zzw+JGOPrm9nLgNg/caqCCRqWDTkkOuxKQgRaQacA7zpc3g0sLuqdgL+hcebyief7iIyUkRGLlmyJBJZDSMdcVURqRoUy9dtpKq6hpXlpdkLUFUbk4uZOHsQZwCjVXVx8gFVXa2qa93v/YGmItLaLxNV7aOqXVS1S5s2baKVuADY61BapLLwRGl+Spf3xqoaDrvvE25/d3x0QkTMda+OpuNt/eMWY7MmrYIQkcYi8mkE176EAPOSiOwsrl+iiHTFkXNZBDIYDZhSa3xWVFbnrcVcWe0sGf/RuIV5yS8OBkxYFLcIsbFs7QbWb4x/Ody0CkJVq4FyEclb0HMRaQGcihMdNrHvWhG51t28AJggImOBJ4CL1fqaRkiiHjfu8fY4fv/KqDr7ci2eC1etZ787B/LAgCk55WMUiIi9Ew7v+SkX9B4W6TXCEHY9iApgvIh8AqxL7FTVrCK8qmo5sEPSvt6e708CT2aTd6ljerD46esJeSIiobsqqeqUH5eVA9Bn8ExuP3P/nOQzGgYTF4QL4RIlYccg+gF3AoOBUZ6PYRiEGzsaPK10nSiSdeCTn/9Ahx792FAVvxnEiI6w60G8KCJbAu3diW6GEcjm1AfKxNAwYPwiHrqgU2SyhGVNRSWttmiaUx7/GeIsKLl+YzXNmzTOh1glS0Pu9IfqQYjIz4AxwEB3u7OIfBChXJstDams2SzcuqR6tlHFivKrvD6fUpb2vFJ/dqsrStO1t9gIa2K6B+gKrARQ1TFAx0gk2kyZtGA1pz36FWsrbGJSKRJXKzLosvnSN6XYOh794woOuedjBm4mXlCV1TVU10TzoMIqiCpVXZW0rwSLTvHy8MdTmbZ4Ld/NMm/eUiKuUBv5vGxldQ13vjeBJWs2BF+vhLoU49xZ49/MyCy8SKmy918HcNULIyLJO6yCmCAivwIai8jeIvIvIH4frAZIKbbYjNyJswL+ZNJiXv52Dvd8MDE+IYyc+CoiB4iwCuIPwIHABuB/wCrg5kgk2sxJ6IdSarEVK4XUtWHi8RSLC3OyGIntxG8oW1PBglUVWeVVTGQi2pdTy1i7Ibx5d3N5PcMqiH1V9a+q+hP3c4eqhitBRigSCqGYX7hSoZDKNSik99oNVQzPYDGhYuK3L45Mm6aYGzCZDvjPXV7OFc+P4M9vjM3qeg35lQ07Ue4REdkFJ7BeX1W1vmhENOTIkHEQ19288bXv63kLpaq48lXfLl5dwbTFazh89+0A//KU2Pfe9/OZs6ycvXZsWS+PhkDYxtY6N2LtrKXr0qTc/AjVg1DVE4ETgCVAHxEZLyJ3RCnY5kcRN8kaAFHd3Y1uzKPkymiSzyzYQpiYznlyKJc/N7y2Z1NRWROY9ubXx/Dop9NS5leKzZVsezfWOKtP6GiuqrpIVZ8ArsWZE7HZLANaSMzE1DBY5NMKTz0PIj/XXbw62BMpLH5ms1Isl2Er/ChX/it1wk6U219E7hGRCTgxkoYBu0Yq2WZG7RhEYjs2SYxSIpteSSanpCuH+dQbs5eu45qXR1JRmVv4jmzfnVJUglETtgfxPLACOE1Vj1fVp1U1/XRMI2OKxdOlodHQ7mouPY5EKPBc8o6iAXPn+xMYNHEx3xV4cL+YB9zjJuwYxJGq+riqLohaoM0VK6PRUDr3tXCSPjSwYYVTm710HR169GNY0sS4KNtam4tSSakgROQN9/94ERnn+YwXkXGFEXHzIlGoo4rNAzBgfLSLyFgnqLhZtm5j3CLklcT62e9/77ZfC1x7N+Tyns7N9Sb3/9lRC7K5s2kMIvrS9kPZ2sivAZRS8z1nwlQSxV6RJOSLKq5PeDkyu35Q6kx/RZE/nlhI2YNQ1YXu/zneDzAPOKYQAm6ubEZ1a3QUe43sId+N3kzyS05b5onJlO4O5nPMLNdec+L0THPJ9dZ/N3MZfQbPyDGX4iSdiWlrEblNRJ4UkdPE4Q/ATOCX2V5URGa7ZqoxIlJv2qZ7nSdEZLpr0jos22uVGoWo06JWPnH7k5vbYvREaQItNWYuXcf9/RvmUrHpTEwv43gvfQP8FvgL0Aw41w35nQsnqmpQuMUzgL3dzxHA0+7/BsugiYuB4mz0XvD0MEbOWcGwHifRdtstQ5+3OVXUoWIxmREjFBmbhgJOyPRdMg/C+qTzYtpDVa9Q1WeAS4AuwNl5UA7pOBd4SR2+BbZ1Q300eAoRrC/TvEfOWQFAj3fGRyCNATn47udVivqMdUNnF0K5Bd2DDj36sX5j+rkRtSamDG+mdYaCSacgapdlUtVqYJaqrsnDdRX4WERGiUh3n+PtgLme7XnuPmMzp6q6hrnLy0Onz3e1luskrvxRmFqt74i56RP5sH5jdcaD3SNmB89/WLYueIZ4sPKKRqm99M3sSPItRtIpiE4istr9rAEOSXwXkfrBZsJztKoehmNKul5Ejks67lf6fZ+2iHQXkZEiMnLJktJdFD5BIbq5pWw/fnDgFI596AsWrlqf0Xn5+sW/e8k/0qlZJ+qy/10D+dMbYzI6pzzRS/C5l6nu76Zj4v51/v+4vJwJ85PXOcudu96fWDDj6ZRFuVSzuZPOi6mxqm7tflqpahPP962zvWhiwp07G/tdnOVMvcwDdvNs7wr4TtJT1T6q2kVVu7Rp0yZbkYqGTaE2Mi+CVdU1XPXCCEb/uCJv8ngLaDHYaL+e7vi8L1sbjy//kB82j1XKwpIoEX5l470x+ZtX+/GkxWllSG73fD19GWf/a2joayxYWXxRbB/5OHUwxagJHawvX4jIViLSKvEdOA2YkJTsA+DXrjfTkcCqhMttgyeHOnjuivV8PqWMP74+Ji+iDJywkG6PDclLXkb4XkZNzPMQwpDcfIk6PMZ9H01Kmyb7Vr1z5voMzIfF/4TyQ9j1IPLJTsC7rpmjCfA/VR0oItcCqGpvoD9wJjAdKAeujEHOWMjHYGC6HMJamKYtzmxC3RdTytht+xYZndNQCPPUUkdzlVDp4uSLqWUctccObNG0cb1j6eI7ZYJ5exUPBVcQqjoT6OSzv7fnuwLXF1KuoiOL5lDilDnLMhjEVQ0ck0jem64FfKW7cPot3fYNff2GwrNDZnLTyXvnZVEgp/gXfpwo1fOdvHANVz4/gou67MaDFxxSOKHCkKPps1SH5J4dMpOe/SZHeo2Cm5iKndMfHcwh9wzyPfbppMV06NGP6WX5cOTypzAT5Ta9EZmE3QjbsiuCoQqgsC3xxz79gYk+iwR5CVsRFfr2hRFrTYXj0Pj6yOy8mgpBGDdXVeW5obNY564/XVVdw4YUiyoVM49/+kPk1zAFkcTUxWtYXeG/eHn/Cc4wyCOfRDdwlMt6ENm0hE57dHDO+XXo0Y+b+36fF3nCMmVRsJKOq0WYzq0zleL0E3nqojUM+SGNZ16BtEmQ00QUjYGMJ7hlkPazyWXc99Gk2pb3hc98w5lPFO84W9xtLVMQWdB//KLI8s7lhQvr+RS2Ak02l6SSLZ8eK6lIiPTnN7NbYL4USNzn0x8bzOXPDfdNk+4ZRq0kk/MvhlnzYWRIDESvXu/0iL7/cWWUIkVLAW65KYgMKMRLUMwDdMViOmroZFsGVlfUzmvN/7OKv/4PJPm3phI1X4qzULdj/LxVdOjRj5lLChSBOQlTEBlQCNPFpvUgMj/3tRE/hkpXxO96UXLVCyPoH2INjVzq5HyUrZEpZiJHTZwNm8QcjEzuYTE3xLwk1jb/cmp9U2Mh3mNTEAVmRYSLtXySYjJRNiS/cKXyUuWbz6eUcd2ro3POJ+zdy7b1XwxmnjhIHrdLpSga0j0qREQEUxAZkOvj+HjiIg6975PaFbD8KKYquCG9TIWgkHfLT4lEGSeqFErCi9/MYWjIme6lZi6NS1xTEBmQq8Ie7s42HTdvZXCiRHe5CF/JTF+qRaviD11QyIogJxNTts/bc9oLw2bnIEF2ZNKrLFtTwag56cPA5PLMLnvuu5THa1dujFFBfDB2AW/kwV24ECZvUxAFJEzhrHJdJasjLMHhvZjqbmcq0bvfz8/wjPRk+1KU0mSobB99tkuFLl6djSIP9nALigV2xmND+MXTw7K4VjBPfTGdv31YNwxHKmUbthis21DFhqpoemQ3vvY9t7w1LqNzvp4eTwwwUxAZkGurPmEzTPUa/+87Z6B5Y1X8k3fS/dq5y8t5NMI5IbnirbTKQ6wnUEh+XFYeqjWdljy0I+5xK9h5K4Nn4GeiYFeW+4+zLYtg/O0fg6ZmdV66ns+Bdw/inH99nVXeUfD5lDIGT6s7UF2INk8csZg2WxIPNFULcUXAy5VfObKcL5Ekd/eXRzF5YbzhiP3wip2YATxvRWbhwSPBvX81Ncpx//gCgNm9zkp72qrySrZp0bTOPr8KO9deUlV1cMH0lpnqGmXp2uD1GXIlZ52XapA6AxPT1MWpJmMWvku6ZE109zwI60FkgF+Z6NCjH70GhFyPNkSZKkiojSzLdnKra0PRLJ4TzIYC9sTS3dbE/eszZGaodAk63ftxyOt7Av7luRx5y8xbo+rbz+/+YGJk124o1NQo1748Km/5mRdTkRH0PHp/NSOjfFJ1bzN5t9ZvrI50jYa0PY0SsOsX1LMoZDq/mE15edc9edREWC5Wr/eEonEvM2vpusiul8ycZev4YGxhZu7nk2XrNjJwYv6iMNg8iAZGosJNvTpWuBd71fpK9r9rII9FGLCr3iB1yBmrcS8sVKwN2ErXhJPu/mQ/D2ITd7yXvMRKPGRTFv7w2mjeGT0v8Hi3x4Zw42v1Y38lSF1xpq9W4y6/QcThaGEKooCEecDpimbCpr7cHfB7b0z+PYWCKIbXJhNHARHy2syqSlrzYFmSHf7eDycShnT1Tz7ucxQeZGHxlvPBWazAV1FZwx/fCI61lcnCPkGkusdRrdeU7wre3FyLjhy9mNz/qVoo6SqPxGCr34B32JZPvmyXueTT86NJ3PNBuArVy7qN/pF2g8jnfJIHksaarnLXv0gwOmTgt6hmpBeqhZnuOt5iuD4G77GUa3KEGKTO1l04aqwH0cDJxySdXGI1ZUr9aK51Bc9FhGeHzspqYtfMJZnZujO9T+Ubq7jlrbGsKq+sd2x40rKac7P0jKrxGTf3VkrZmjiinFyZbc7FNv9kWoow8QmyHb/xm8nu7XXm+1YsLcC67HGsSb2biHwhIpNFZKKI3OST5gQRWSUiY9zPXYWW049cC3vtGESKNGnt03lofYb9GenSFdvL70frls0zSv+/737kjZHz+Nfn9cd28tXy98vnwYFTPMczyWsTuT6PlOtVpMh8zrJgpT1l4ZpY3DO9eN+ph2vn7QT/WK+CeH/MfKYsCufKvWBl3QbDtzOXsddfBzDCDaKY3MAoBeLoQVQBf1LV/YEjgetF5ACfdENUtbP7ubewIvqT/IoEzRgNPD9EDyJd5ZB8bpQB9HKdSV0MtGzuTPXZsVU4RZG4v36/td69z7Kl6WfBGDN3ZUZ5FItuVqjn5u29LY9+Oo1THvkqdR6qeTVF1Zu+43O/w5qYbuo7hm6P1V9QyG8ia3KWidnP38xwYq/NXZF+KeBV6ysDFVIc4XfiWJN6IbDQ/b5GRCYD7YBJKU8sQjI2d7j/U1bqIescP4+osNVV1vMg6nkxFUs15Y/qpt8a1qxciFg9foql7lhSdvmuCVgJMSypymWqJ12/0VKXVevrm+u8vDhsdu1s7mLAzwSYzPj5q+rtC3putWbhEO/LL3t/k3KCXqGJdQxCRDoAhwJ+EbaOEpGxIjJARA4srGThGDghQ5/mEDVz2B5EIqt5K9anbX2Vb6yiS89P0y9fmUQhTUyDJi7ivSw8b655eSQn/fPLOvu8YiXu19K1G9JGO527PHULL19KI20+WV7Hr9LKGymedXLLONNi0T/ke3TovR9z0TPfZJh7QG8wRfrs55CkHqML874Uk3KAGBWEiLQE3gZuVtXkPtVoYHdV7QT8C3gvRT7dRWSkiIxcsiSzCjAVfl3I5Af86eT8rr8A4WfjehmcpuKfUbaOpWs31JoCsq3Xw7422bxf17w8iptfH5PxeYMmLmZmikla38zc5Gb5016fp8zLG5wtjOku22okyklsuZCtWMkT/6L6dSvKK/muAHb8bANlBvYg8nRHNhsvJhFpiqMcXlXVd5KPq+pqVV3rfu8PNBWR1n55qWofVe2iql3atGmTNxlTrdmQK6nK35oNqc0EqeypNQF2lLyZTbLIIKqImGHxeiMtTxMszjFJ1X0LvRO28lXxpe9AFJ8C8d6VdO7NmY7NRF3vZSpP0HuU9jrJOzz3ad6KciYlKdLvIqxj8kUcXkwCPAdMVtVHAtLs7KZDRLriyBnZ3Xx71Dw+S+oN+BWRnKO5psg7LIlzve/o4tUVVFbXMHtZGhNJhtdKVxEEHfeujZzpOE0uTJi/imVrN9T5ndl6BAHMWLK2zoStfM2w9dY/HXr0442Rc+s8z0J0MPxCY/xQtpayNZmF/s6HrFFPIPN3OAgWvCpbBZHitGMe/IJ3kkyoF/X5lnkhBq7jJI5orkcDlwPjRWSMu+92oD2AqvYGLgB+LyJVwHrgYo1w/vuf3nQqgbeuPap2XxRmgNqCm0Hec5eX88+PN4U0VlVWrNvIWk9P4+4PJvKLw3dNe92MW3ZJL1ryixP0XntnuhbSnHL2v4ayyzZbcEnX9oAjfyaTnryyqsKGytSjldn+tOTn8NDAKRnnddLDqT2D0nFi0rhNguMf8t+fz8BwI2Yv5ycdts9bfvkm24lyQWU91bMdNWcFu27XIlT+6zYUvjcehxfTUNL0KlX1SeDJwki0Ca95x68y/SbHLmFizCJs8Vu7oYrrXh1dZ/BRgUPv+6Re2jUVwZ4iyT2fsC97cqpkW3NQNq98+2Pt90Kb2xcmrWIXdH2/gX3Vur+5UUT962SZlq7dyBZNN13sne/n02X37ULmld8bHCaMRa6q4sLe39QJc55vb7h0McQg9Tu4sTq7CMBhY5XVSZOB4q0O416VZ2wmtYepnlmWfoVqetnajPL7aNwCOvToV7uAyoT5qwPz9uOguwfV80wJOjeVi2NyGaz/AkVXi0e5vkWYll6yPb9Dj36sWl/Ja8N/rJ9W696bqNx4040x3PfRJM59qngWq4G69+Xej1K7pIYpTR169Kv1qot68DXTMZ10Pcd015letoYrnx9eG2o+1dUbZfDb4xiZMgXhoaKOaST3/J4bOguAGUl2+NwGITM/d9NcAP9zAyf8pR2DSH/ty58bnjbN1BDhD/z433dz0qbxe47f/7ii3qxXqPtcVLXe70teWyLbis2vIVikjk21BP3UVLOovfjF3br8ueHs89cBOUgVjkwnymVrFk2cdtf7E/li6hJGh1gxsFEGhSiOMmIKwoP3YeXDdh4UnC8fsZiCruV/rO6kuuS0QT3qtPMg8tTCXpvGcyuI5ev8zWqPeJZB9XuOihMLqt5+j4lJqf/7kwd2g55FukrTr4GQbmGjIM+aKNcuD8P9Potl+YkUFHdrY3VN3nsQkxemb3AE3bVlazfkf9wsRX4Z9SBieNamIDxU5RAwzS99QuFo0vFcHvPw2f5+4KnyTPcCNg4opelCA8Qdi8krtl9wPVX/d/Opz6eHyj991FL/uz5jSbApcnVFJd/OzNyXP0gRzF2+viDrlwfZysdmGCLEN+8sGxpB0YDD9GqCWveH9/w067Wz6z2iED8rkzEIMzHFzBOfbQrQlqmr23+/nl1vX0JB1NRoHVNHckEaNj18zPyHBvov0h6m1VObIqlQ+imI8o1VPPNV6qUx88WHntXBRsxezsJV4aKken/G9f8b7ZvGz5Q0MqByqFFNemFTv7xBc1ZSVXiH3BNu+dBkUo23DJiwMOteWFjKMwyzXgiCeiXJUU793o1U9+vLKWVZyRNkOk5VlZiJqUTJ1NVt/LyV9fYlWvs1mvoF+8fH/pV+JiQXnheHzWbJmg0M/WEpX7gFPqjFe+tb4+rtCzNQl68OhHcC0YW9v+GEf3wZ6rx3Rm/yKw/q7WQSkjs5HlIm6x7UIYKeVaoGwE19x/CnN8bk/6IeKjIauI3X7DUqqQEQdOuCJk5mO/5Y7zx3O9WSxN6iki5qbBx3NY55ECVBOgWRvCZuoxTGRFXl7vc3dYfHJSmT6Ysz847yv0bd7bs/mEjfEXOZvHBToUskSZZ06uI1vPf9fH5+aDtUlXe/n89x+/jPSp+5ZC1L126ka8ft82ZjSm55pbPH18riGRMIDpQW/rVSNrl5Lly1nr9lEUBOVSPxfUpXHgu5JnS+SeWinQ+C7lyvAZMD0tc/I8wa2Ne9MooWzZvUKp5EAzGVNcLrSv1juomuqqgqw2Ys46d77pBWnnxgPYgA5qR5WMlr4jZOUVkqdUN3DJuxrN7xXPFrYf64LNygKlAbB+m9MfP54xtj6TPY37x00sNf8csMA6bNX7ne19STIB8eY8nLgSY4YJetQ+dRWV1TG6/q08llDJ6WeWyvN0cFr6WcC+kURDEtgpapKWTsvAiDDKYg6J75yT8yYOzPy4JVFUwvW5s2pIuXq14YWfs9jEt4v/ELufTZ73jlu/pu2lFgCiKAxz+rv2BMKpo0TtWDgI3VdUvdZ5MX17Zuo5ptvC4gymuqhv9Kd7B3aYhFXsK2lI/u9XmdQHlvjJxb53hllhOTvCxYVT9EhEhm9/bfX4QbvE7FvOXleZ11nCBdkLpiDQBYDAT1IoPumZ/iyGSsIFtufXt82jSJhlZy4y8qTEHkiVQFaPaydfXGIK5+cWRtRZmPdztMcMGEDKlkzeQ1yPaduSVpzGNchC3ITHTPF1Nzjwb8xOfTeTgPY0rJPDekvluul2LSD0UkCpCidxWw/5NJ9aM0F5sJr1DP2xREnng1RZfvjvcmUO7Tml+w0mn15iN6Z89+/vZUL4tXO72C5k2CH3smrd/iXi7IGcuIumXtd7uiUHjDZy9PGRm32CqwYmJmgNtxULj+pWvr956/ysLcmG969pvM/f0dE2ihlLApiBR87XE/veblkSlS1qWisjrU7ODEQy60/bh5k8a++zdUVddWeMmRJ/PBm0mmpVSoKuUbq3IawOwzeGbkCqKQLfd7PiieVddKiaD4UqtzXIFvc8AURAouffa7Ws+CQRPDLw70q/98y+mPDU6brja0R4EVRLNUPYgM8pmSYYiMv/i40wZxy1vj6Pr3zzg4y3kDCRqSbT6dG2SxELVXUqb8J8DholjI5rmaialIyGYC0ugfV4ZKl/AUKnQltlUz/x7E1EVrMjIx+ZnN0nF0mlXdErw5al7Ok79O3LdNRmMQxU6zxqXxuq7wmdUeJ/kYW4qSdEsG+1GoRaVsHkQRkO0CJVkToAMmLVgdefiM+SncXfPNyvWV9UKUp2K7Fk2LrnLzkg9vr0IwINO12jdzirmPWxpNkpgJq+G9QeKKmTEB8XNeHzk3shDXQUQZgGx62VrKQrjrJijmRWwgfM80bvIRn2lzIpt3oKq6MGrFFEQahs1YGtpt8YkM507ERVA8p+9/XBm6B7HMx9MjG6LsUaRaI8OPj33cG3PlsYs65zW/mhqtE5Y+U+44a/88SuNPqZjCioW/fTiJ8Rl6vr38bfpQ9/kglicpIt1EZKqITBeRHj7HRUSecI+PE5HDopJldZoBtee/ns3K9eHNDj8szmzgts/g4DgtUXD+v1MvRHPbO+kn6wAc/WC4sYR0nPTP9EtnfjdzGU9/Gd19ev6Kn0SWd5cO4VaGC8set/dnvzsHZnVuI4Gf7tk6r/J4md3rLFpt0aR2Rbb9M5jFHjcXpFiy1w+vo8cp+++U07XHzVvFz54cmlMeUVFwBSEijYGngDOAA4BLROSApGRnAHu7n+7A01HJs1Wz1MMw81eu5y1P+ATvUol+nPpoeu8lLwm/5mSu+GkH3r3up1x9TMfAc08/cCeeuOTQOvtat2yW8npeM8Wt3fYLJWPTxlLvOsnB20becQoPnH9w7fYdZ+1P65bN0+YdZnnHi/p8y4MDp3DnexNq9w246djA9P/5dRf2aLNV2nwBfntMR07cb0du6bZvqPQJbjhxr1Dpdt2uBdcctwe9LwvXxpl8b7eUXmbJNE0xgz+ZqT3PYL+dW9XZ99VfTghM/8rVR9TZ/uxPx6e9RptWm575zafsHZhurx1bMune09Pml+D5K6NT4gD/vLATfz0zXO+q343HMPrOU+l92eE8fGEnnrr00PQnebjjrP0ZeHNw+d1+q03v8Iz7z2Tyvd1S5rf9Vs148leZyRCWOHoQXYHpqjpTVTcCfYFzk9KcC7ykDt8C24rILlEI07iRMLvXWQzrcVLtvt8ctXvKc6b27Ebn3bYF4KM/HMPfzjkwMO2Yu06l+3F71G5PuS/4YXtbIn84aS8Obb8dd559AL84zL9188zlXTinU1teuqpr7b7P/nhCStm9XHv8HukTASP+egrndGqbMk3rls25pGt7/nK6U9GeesBOjLzjlMD0r3c/kv/8uktoWaFutzpV6/TUA3bi8z+dUKcyfOvao7jsyPb10k5zl5HtfuwetNrCaSwctUdwILRrjtuD2b3O4s+n78vvT9gTgL+fdxBfe8pPgp+59+y2M/en20G78P71R9Np121S/US2bNaYaT3P4Do3b0hdMff8+UHM7nUWb157FHectT8z7j/TN90TlxxK08aNaNRI6tz33XcIVqTH7N2aaT3P4P7zDmbcPaexZ5uWgWkT78Pr3Y+q3bexqsa3DPzw9zP49I/H06JZkzrvQ4cdWvjmPejm4zhx3x257QynQfOzTm0DG2rH79OGczu3Zfw9p/HudT+l343H1B5L1wj43XF70MvTyNlnJ//fe2DbbWjZvAndDtqZXxy+K82bNObzFM/okq67MbvXWfS+7DB+uucOXHj4buy3c3D5vbXbvjxxyaF83eMkGjcStgzwOkww+s5TOfuQ1O9ntkihVykSkQuAbqr6W3f7cuAIVb3Bk+YjoJeqDnW3PwNuVdWUs9W6dOmiI0eGn9CWTNnqCgZNWszlR+7OolUVHPnAZ3WOt9t2S9+KAJw1dv0IKsh+6cfefRrbbNk0UL6PJy6i+8ujAJh5/5n1IshWVFazRdPGXNh7GCNmbwp5fPReO/DSVUcgwO3vjqfviLlce/ye9DhjUw8iWZ5bu+1XWwEm+GHxmsAeUtDvfH/MfG7qOyYw/ZdTy7ji+REAvP37o/jF0+ECASbOv/blUQycuMlr5tD22/LudUcDjr3+8c9+YO6Kch6+sBMiUu93Tut5Rm2LvbpGGTF7OT/psD2NGwm/eHpYbejovXdsySd/TN2C7vH2OPqO2DQZcNYDZ6Z0G15VXsnL387mV0fszkMDp/DA+QfXSf/c0Fkcu3dr9tmpFXvc1s93QuWM+8/0Xc9j3YYqytZsYNfttqSpz5jA0rUb2LJpY7Zq3gRVpbJaadakUe3zuu2M/bjm+D3rnQfOIHTLLZpw8sObzIPe57983UZe/XYO156wJ00bN2JjVQ373DHAN20yqkrH2/rX2ZdcVp781aGcfUhbNlRVM/SHpajCb18aGXi/q6prmLdiPR1ab1KG3nJwTqe2dXrIZasrqKisof0OLaipUfa43V+eZM56Yoiv19x3t5/MTltvUW//sOlL+dWz39Xb71cPVFbXsHfA8qzprBrpEJFRqurbWotDQVwInJ6kILqq6h88afoBDyQpiFtUdZRPft1xzFC0b9/+8Dlz8jd4s2LdRh75ZBp7ttmKrZo34cIuu6VM/+mkxXw0bgEHtduGIzruwAFttw5crW1jVQ23vzsewQlbff95B7NvUvffj7I1FWzfohlNQgwEDp+1nA47tGBHn8KZTGV1De9+Px8UmjdtxLmd2wXK3ePtcWy9ZVN23W5LVldU8X+n7J12/sSKdRt5fthszju0HR1bB7daKyqr+fV/h3P8Pm0YM3clTRoJHVtvxe+OdXo7LZo39p0J/taoeTQSOD+gt1UoVpZvZMmaDWy3VbNQJrZMmLJoNX98fSxnHbILLZs34YyDd2bHVumfbVSUra7gxW9mc/Mp+/gqIS8VldWsWl/pW1H6sW5DFT8uL6dj663Youmm5/3jsnLaB/Q0MqXSVRypymOCmhpl+OzlHNFx+5RlfcTs5fT+cgZ77diSsw9py947tawjfzKLV1fw5zfHsvsOLfhJh+05dLftUv6+b2cuo9eAKey7Uyu6dtye8w9rl3NwyGJTEEcB96jq6e72bQCq+oAnzTPAl6r6mrs9FThBVRemyjvXHoRhGMbmRioFEccYxAhgbxHpKCLNgIuBD5LSfAD82vVmOhJYlU45GIZhGPml4DOpVbVKRG4ABgGNgf+q6kQRudY93hvoD5wJTAfKgSsLLadhGMbmTiyhNlS1P44S8O7r7fmuwPWFlsswDMPYhE15NAzDMHwxBWEYhmH4YgrCMAzD8MUUhGEYhuFLwedBRImILAESM+VaA0tTJC9mTPZ4KFXZS1VuMNnjwiv77qraxi9Rg1IQXkRkZNDkj2LHZI+HUpW9VOUGkz0uwspuJibDMAzDF1MQhmEYhi8NWUH0iVuAHDDZ46FUZS9VucFkj4tQsjfYMQjDMAwjNxpyD8IwDMPIAVMQRkZIrsHnjc0OKzPxkI/7XvIKolQLn4jEEigxDwQveVfkiEhr93/qNRyLEBHZxvO91Mp8ydYzItJFRHaMW44syfldLckHJyL7uwsPJSK/lgwicpSI/AeIdhX2POPK/SbwTxE5oFQqWXdNkRYi8hrwPoCqVscsVmhE5AgReR94VkSuEpHmpVLmRaSriLwCPCAiB4tIydQ3InKgiAwD7ga2jVmcjMjnu1oyDwycVpRbufYF7hORv4vIXnHLFRYR+R2O98Bo4PsSqmR3BJ7ECdG+FLgJuMo9VtStWXUodzdbi8jvAUqhshKRQ4CngLeAN4GTgKIv7yLSSETuBp4FBuAsK3A90ClWwTLjJuBdVf2Zqk6D4i/rkP93tehfkiT+guN51Qm4BtgB6BCrRJnRHvirqj6tqhUl1JLtBExT1eeBh4F3gHNFZB9V1WJ+cUSkiYjsAiwGrgZ+LyLbqmpNCSiJrsB0VX0Z+ATYAvgxcbBY77uq1uCEvLlCVV8F/g7sjrNAWFEjIo1FZHtAcSpaROQ8EdkV2NLdLsr77pLXd7XYXxDcpUm3dDf/A9wFoKozcLp+B8ckWlpc2Zu737cHDgKGi8hJIjJIRG4XkfPd40VT6ETkEhH5m4ic4+76HugiInuq6jqcZWNH4ijpojLzeWT/GTgrGLrL1XYEZgNfAT3c31ITo6j18Mh+rrvrQ+A8Efk7MB7YFXhCRG6Forvvx4vIEZ5dfYExrklsGbAG2CUe6VLjld1ttJUDxwEnuSaya4CewGNummK67z9365Gz3F1jyOO7WrQKQkQ6iMgAnG7qKyKyr6rOUdUF4qxlDbAemBGflP4kyf4/EdlfVZcDy4BXgZ8D/wYWAneJSKdiKHSuvf5a4BacyvQfIvJbYC3wEk53FWAl8CnQwm2dx46P7P8UkStFZCsR2R2YparzcFri1wFvikhzEYl90N1H9odEpLuqLgb2wxlsvF1VjwReAI4RdwwubkSklYi8A7wLXOM2hAA2qGqNqm5w7/GuwNTYBPXBR/btAFS1Angex7w3SFW7AX8FDhKRM2IT2IOItBGR94A/AsuB50XkAlVdArwN/MFNupIc3tWiUhBJreg/A9+p6snAFzhjDge6xxKmmXbAXPfcWH9LCtk/B3qKSEecAa+DgQWq+r7bDewPnFsvwxhwldRRQC9XtuuBE4CTceTcS0ROcVvey3Du/6qYxK1DgOynAMcCK4COIvIh8A+cXsQcVd2gqpVxyZwgQPbjReQMVZ2FM+4wz00+CigDNsQibH024pTxy4AFwAVQr6W6P7BYVae5lXLXwovpS7LsF3qO/RvHpNQGQFXnA0OBYul17gl8rarHucs1/wn4P/fYa8B+InJyru9qUSkIHBur1wV0IoCqPoljj/2ViOyoqtXiDE4vV9XvxRl4vFNEto1DaJcg2Z8CDge6A0twehUXeM7bERhWODHrIiK/drvYiZbfZKCdiDRR1U+BCcCROLL/D3jMvfcnAwI088u3EISQfRxwDLAPMB+YCRyuqj8DdhORw2MRnNCynyDOoOMg4G63EXIxcCDOSx8LHtm3VdUNOGX6U2AajnljHzdd4l3YHigXkStwyvrBcZlUw8quqmtxWuG/EZHObh1zCk4PLxZc2U8QkRY4DYWX3P2NgUnuBxxzZF/g8Vzf1aLwxReRU3G611NFZLCqviEiy4FDRWSam2wCzkDXDjgtqD2An4jIF0AFcLOqrixS2SfiDFC3V9XbRWQ/EemF0zpf4B4vpMwC7IxT4dfgmOm2cl+CuTi9nL2AKTgF7VFgB1V9RUR2A3rgmD5+V+h7nqHsb+AM1L2JUz42erI6WVUL2vvJUPbXce57W1V9RkSOZ5NH0FWqOsfnEoWWvbuI3KSqS9003+Ao418CPVW1yj39dOASnF7Ppao6rthlB3DfZXH3HQhcrqoFNZMFyP474CZVXSwijd0G8/7ANq7cNcALbuMit3dVVWP94LwQ3+GYWQ7F6R5dB7QC7gQ+wunadXFv0o3ueZfi2N5OKSHZ/889b2v3oZ0Wg8yN3f/7AK+435vgdKlfxLF3/xe4HNjGPf4C8HdPHs1iut/ZyP4icK/7XYBGJSb7fe73psDORSb7v4C3k9Ke5/6mvYAW7r6fAheVmOxbAU0T5aYIZX8nKc1LwC/d7zt78sjpXY2lB5EYL1BH0x0BjFLV991jn+K2+lT1PhHZQ1Vnuse+ZpPtta86LnSlKPsaVZ2C01IslNxNgHuBxiLSH0dJVbu/pUpEbsAZND8AR5n9HGdg8QGclkutGUzrtsSLXfZqHCWOOm9MQZ0B8iD7t27aSmBRkcl+I7BARI5X1a/c/e+6rdmBQEsROVFVC25CzYfswInAZLfcFLXsOI4ks0TkXuB8EemmqvNyfVcLPgYhIlfiDLjd5+4aD1wiIh3c7SY43ahH3e1Z7nndcfzYR0M8s2HzKHuhC9zxODbL7YDpOPJXAicmBgxdhXcv8KA6NvA+ON4y37nnfVlImROY7EUtu+LIfo/nvAtxPH6+AA5R1cmFlXzzk90dg7gKZ0Ll1sCJ6njs5U6Bu0wtgfdw3CVHA/u5+x/DMc98DbyCY4vtB+zkHr8Zx5/3J4WUtwHJfiyO/TSx/W/g98AVOD0gcBoLO+PY6zu4+7YF2sUlt8leMrK/AXT0nHesyV4w2XfH8Wh6DDgs7/LEcAPau/97Aa+73xvjeDoc427vhmP3bu5ut4jzoZW67EALoDmb7JWXAg+438cAf3C/dwFei1tekz3+j8leErL3jVqegpuYVDURKuAxHN/009UxF61S1aHusWtxZjNWueeU18soBkpVdlUtV8fnP2GWOxXHbRXgSmB/EfkIpyc0Og4ZgzDZ48Fkj4cMZR8FEUdhiFlbXgN85dnuihNxsz8xeWw0ZNlxejuNcNwl93L37YVj0jiGmM0aJnvxfUz2zVv22JYcFZFG6gRMewvHi2MDzoSVH9SJs1S0lKrsbkujGc7koHdxBraW4XRbV8cpWzpM9ngw2eOhaGSPWUu2AAbjhKW9MW6tvTnIjjMrugZnfsbVcctjshf/x2TffGWPrQcBICJ/xvH5vlWdae8lQ6nKLk7Y4suBR0pJbjDZ48Jkj4dikD1uBdFIiyzkclhKWXbDMIwwxKogDMMwjOKl2KK5GoZhGEWCKQjDMAzDF1MQhmEYhi+mIAzDMAxfTEEYmxUistb939ad6OiX5ksR6ZKHa50gIj/NMY/bwx4TkdhWJjQaJqYgjM0SVV2gqhekT5kTJ+AslpMLgQoi+Ziq5notw6iDKQijZBGRB0XkOs/2PSLyJxFpKSKfichoERkvIuf6nNtBRCa437cUkb4iMk5EXsdZrN7veieLyPdunv8Vkebu/tki0tr93sXtgXTACdz4fyIyRkSOFZEXRKS3iAwRkWkicrZ7zhUi8qTnOh+5vY9ewJbu+a8myVLvmKd3dIKIfCUib7jX6SUil4rIcFf2Pd10bUTkbREZ4X6OzvZZGA0TUxBGKdMXuMiz/UucdRUqgPNU9TCcVcEeThPx8vdAuaoeAvwdODw5gYhsgRPG/SJVPRhncajfB2WoqrOB3sCjqtpZVYe4hzoAxwNnAb3dfIPy6AGsd8+/NOwxl044a5ccjDMbdx9V7YoT2+cPbprHXfl+AvzCPWYYtcSy5Khh5ANV/V5EdhSRtkAbYIWq/igiTYH7ReQ4nFg27YCdCF6y8zjgCTfPcSIyzifNvsAsVZ3mbr8IXI8T+j0T3nBn4P8gIjNx1iaPghGquhBARGYAH7v7x+MoTYBTgAM8unNrEWmlqmsikskoMUxBGKXOW8AFOCts9XX3XYqjMA5X1UoRmQ0EttRd0oUUSNUDqWJTbzzT62jS+WHyCIM3dk+NZ7uGTe99I+AoVV2fh+sZDRAzMRmlTl/gYhwlkfBK2gYoc5XDiTjLMqZiMI5SQUQOAg7xSTMF6CAie7nblwOJBeNns8ks9QvPOWuAVkn5XCgijdxxgD2Aqe75nd39u+GsLZKg0u0R+ZHqWBg+Bm5IbIhI5xzyMhogpiCMkkZVJ+JUwvMTJhXgVaCLiIzEqfinpMnmaaCla1q6BRjuc50KnBW93hSR8Tgt8d7u4b8Bj4vIEKDac9qHwHmJQWp331QcxTIAuNbN92tgFo7555/UXeWsDzAueZA6xLEw3Ihzn8aJyCScQXXDqMWC9RlGgRCRF4CPVNV3/oVhFBvWgzAMwzB8sR6EYRiG4Yv1IAzDMAxfTEEYhmEYvpiCMAzDMHwxBWEYhmH4YgrCMAzD8MUUhGEYhuHL/wOcS9UELVvXngAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import hvplot.xarray\n",
"ds1.streamflow[:,1000].plot()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.8"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {
"09f60392b4f441f392f7aa3fa8d685ca": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"0bd9e97ffe06417ca2f9ecbd58b8c7ce": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"0bde0522f4944f9497cc25fd281f4f34": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"0cbfede68ed74f2ebec869ddaac41bd2": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_166f3bbbf8f24df89172b53a48fa5816",
"IPY_MODEL_35b998ec9ddf432ea878503e4cb57009",
"IPY_MODEL_bd9e1db6ebae464d81372683366da6f4"
],
"layout": "IPY_MODEL_7fa59a02422f404ca36d887d3d34f527"
}
},
"10444f48e4fb406d8e74d70cc61992db": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"min_width": "150px"
}
},
"116b525d7c1a45dea10afdf32575fc2a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_5c270e206e2e475eb0fc2b54aa8185b8",
"style": "IPY_MODEL_fad38e1d0d474d4e9a3d3ee879f20572",
"value": "<h2>LocalCluster</h2>"
}
},
"14edd9679e994692848cf3fa73e4e0c2": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"width": "150px"
}
},
"166f3bbbf8f24df89172b53a48fa5816": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Minimum",
"layout": "IPY_MODEL_4e98f0fa3b474197b3fe7a3578f84694",
"step": 1,
"style": "IPY_MODEL_5d47dac17314417ea9650f660884a5de"
}
},
"18aaea04573e46f48af8e85af387e6f3": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_7ca4114c01f749b6a0bcbb04f36e8816",
"style": "IPY_MODEL_96b4210ee9c74accb8859fd04b1a2bc8",
"value": "<h2>LocalCluster</h2>"
}
},
"1a514a329bed4cb6815fce201ec81a25": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"1b70ea94a58348e281d3c8f59aa11e21": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonModel",
"state": {
"description": "Adapt",
"layout": "IPY_MODEL_7ada416dd4654703a9de7d85695b5046",
"style": "IPY_MODEL_9a71a8911eb34a2ca4ef363006191dd2"
}
},
"1d2a9582bbbe4b9fb7708e804500aa88": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_939a097c3dfd48c1995b927e17535930",
"IPY_MODEL_4aa6653eaed845679ba2e1e841e0cc76",
"IPY_MODEL_1b70ea94a58348e281d3c8f59aa11e21"
],
"layout": "IPY_MODEL_a45f633ac50b4805852214e4b26f0c70"
}
},
"2807c7610de342a48b1f9764cc10bac3": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_3fece436655c4e8b884947da0d1077c8",
"style": "IPY_MODEL_f73a46a0ab0b43dea33108da0a005707",
"value": "\n<div>\n <style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n </style>\n <table style=\"text-align: right;\">\n <tr> <th>Workers</th> <td>8</td></tr>\n <tr> <th>Cores</th> <td>8</td></tr>\n <tr> <th>Memory</th> <td>540.91 GB</td></tr>\n </table>\n</div>\n"
}
},
"2c51f71ae92f41ffb08f5e1fa42b7942": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Maximum",
"layout": "IPY_MODEL_14edd9679e994692848cf3fa73e4e0c2",
"step": 1,
"style": "IPY_MODEL_341582d6265c4722961af219b4b73c0d"
}
},
"341582d6265c4722961af219b4b73c0d": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"34781e3f3e374d1bae0bf8515c788526": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"35515d765a8d4512a0b21c0523c96a9b": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"35b998ec9ddf432ea878503e4cb57009": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Maximum",
"layout": "IPY_MODEL_4e98f0fa3b474197b3fe7a3578f84694",
"step": 1,
"style": "IPY_MODEL_45012b509e82447a882168c89cc3294a"
}
},
"384da4a05e45440db06de4e61c267994": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"399a1411fbfe47fe82f6bf804b434c09": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonModel",
"state": {
"description": "Scale",
"layout": "IPY_MODEL_14edd9679e994692848cf3fa73e4e0c2",
"style": "IPY_MODEL_ef4283a199be4280a7473f78921ebb7a"
}
},
"3a4a70223ad04577afe21c0a2caeb8f5": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_a580d96051814c57bde0f41678f00504",
"style": "IPY_MODEL_0bde0522f4944f9497cc25fd281f4f34",
"value": "<p><b>Dashboard: </b><a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a></p>\n"
}
},
"3b193b9eb576419fa6d90ad278476d44": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"3b59684f3fec4aa2ae0067c92b30f114": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"3fabe6bdaa1f4c1d86f5a775271c3e5a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"3fece436655c4e8b884947da0d1077c8": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"min_width": "150px"
}
},
"41d8470e436544949947590cf2d81b0b": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"4326bbd2e1f04b96ba1651f0457d7ab6": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"45012b509e82447a882168c89cc3294a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"46632b2539ad4841ac37a98cb241ce90": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonStyleModel",
"state": {}
},
"4a16b2b89dd74c449df5db89fb352b59": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"4aa6653eaed845679ba2e1e841e0cc76": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Maximum",
"layout": "IPY_MODEL_7ada416dd4654703a9de7d85695b5046",
"step": 1,
"style": "IPY_MODEL_e9def914fe924407bab482718bf010b1"
}
},
"4e554b6f74464e34b9e4d90e8594d0e3": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Workers",
"layout": "IPY_MODEL_7ada416dd4654703a9de7d85695b5046",
"step": 1,
"style": "IPY_MODEL_3fabe6bdaa1f4c1d86f5a775271c3e5a"
}
},
"4e98f0fa3b474197b3fe7a3578f84694": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"width": "150px"
}
},
"513369a2a1fc4edc9622cebfd45bdbf2": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"51b64c28ab574dc99c19181bb345a935": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "AccordionModel",
"state": {
"_titles": {
"0": "Manual Scaling",
"1": "Adaptive Scaling"
},
"children": [
"IPY_MODEL_6a8cdb8f59ec45b384902120d2a09f0a",
"IPY_MODEL_83585005cc2e4b27a170fc53685e103a"
],
"layout": "IPY_MODEL_c7c22c4e2c344b81a6a2e35b0b1bb94d",
"selected_index": null
}
},
"561afce9032e409bb5b3b032225d5541": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonModel",
"state": {
"description": "Scale",
"layout": "IPY_MODEL_4e98f0fa3b474197b3fe7a3578f84694",
"style": "IPY_MODEL_e1fc1a716fb94daebdb5cf168522ffce"
}
},
"56965f7145e043a2bbb2166cb9862555": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"5bd0da2940754727a532aefab927e1a4": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonStyleModel",
"state": {}
},
"5c270e206e2e475eb0fc2b54aa8185b8": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"5d47dac17314417ea9650f660884a5de": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"68c8449613ae4f2e885702d870fb8ec8": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"6a8cdb8f59ec45b384902120d2a09f0a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_ac287d6940104586860c1c76f58531e4",
"IPY_MODEL_399a1411fbfe47fe82f6bf804b434c09"
],
"layout": "IPY_MODEL_41d8470e436544949947590cf2d81b0b"
}
},
"6a9542952e694341b1c3c31735295f76": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_10444f48e4fb406d8e74d70cc61992db",
"style": "IPY_MODEL_97d85cf4e0da42859bcca341c35ee548",
"value": "\n<div>\n <style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n </style>\n <table style=\"text-align: right;\">\n <tr> <th>Workers</th> <td>8</td></tr>\n <tr> <th>Cores</th> <td>40</td></tr>\n <tr> <th>Memory</th> <td>540.91 GB</td></tr>\n </table>\n</div>\n"
}
},
"6adf9acec6544b7c8320442ed5ea7d4d": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"6b3bb80a29044d7a98771511718cb797": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_b790110b58254695af23818ec6d06f47",
"style": "IPY_MODEL_9091aaee251349eba0672cc81c3d8056",
"value": "\n<div>\n <style scoped>\n .dataframe tbody tr th:only-of-type {\n vertical-align: middle;\n }\n\n .dataframe tbody tr th {\n vertical-align: top;\n }\n\n .dataframe thead th {\n text-align: right;\n }\n </style>\n <table style=\"text-align: right;\">\n <tr> <th>Workers</th> <td>8</td></tr>\n <tr> <th>Cores</th> <td>40</td></tr>\n <tr> <th>Memory</th> <td>540.91 GB</td></tr>\n </table>\n</div>\n"
}
},
"6bbe1fbd905741fa8c0d7a2d0742f519": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"6fa3dd431b92447989bc63e59c1253e5": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"79c74a13946a43c3a7d55300bf085866": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "VBoxModel",
"state": {
"children": [
"IPY_MODEL_116b525d7c1a45dea10afdf32575fc2a",
"IPY_MODEL_ec90aea084414c71a8338c5c2d68e307",
"IPY_MODEL_ce9aaee26ddd48ab9f44bf616774d7fd"
],
"layout": "IPY_MODEL_3b59684f3fec4aa2ae0067c92b30f114"
}
},
"7ada416dd4654703a9de7d85695b5046": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"width": "150px"
}
},
"7ca4114c01f749b6a0bcbb04f36e8816": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"7f42b0885b844ceda8a213c4ab634df7": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonModel",
"state": {
"description": "Adapt",
"layout": "IPY_MODEL_14edd9679e994692848cf3fa73e4e0c2",
"style": "IPY_MODEL_46632b2539ad4841ac37a98cb241ce90"
}
},
"7fa59a02422f404ca36d887d3d34f527": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"83585005cc2e4b27a170fc53685e103a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_dfe452c763cf4ea4bb95bb1ab560ed2c",
"IPY_MODEL_2c51f71ae92f41ffb08f5e1fa42b7942",
"IPY_MODEL_7f42b0885b844ceda8a213c4ab634df7"
],
"layout": "IPY_MODEL_513369a2a1fc4edc9622cebfd45bdbf2"
}
},
"85f9adb9e20c4abea53873cbba30755c": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_4a16b2b89dd74c449df5db89fb352b59",
"style": "IPY_MODEL_4326bbd2e1f04b96ba1651f0457d7ab6",
"value": "<h2>LocalCluster</h2>"
}
},
"867cfdb1bcd2476987cb789f6c92f550": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_4e554b6f74464e34b9e4d90e8594d0e3",
"IPY_MODEL_e0330ca1a7c9475fad76b5912a8b9d3b"
],
"layout": "IPY_MODEL_68c8449613ae4f2e885702d870fb8ec8"
}
},
"8857abc3b0a44f7aab2e8295cfad8d1b": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "AccordionModel",
"state": {
"_titles": {
"0": "Manual Scaling",
"1": "Adaptive Scaling"
},
"children": [
"IPY_MODEL_91c9b6f075284222ac23934df67d5563",
"IPY_MODEL_0cbfede68ed74f2ebec869ddaac41bd2"
],
"layout": "IPY_MODEL_f9a36bc7fb1342578d03cd9acb56e119",
"selected_index": null
}
},
"9091aaee251349eba0672cc81c3d8056": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"91c9b6f075284222ac23934df67d5563": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_fb72fa920a65490c98934909cfaa56a8",
"IPY_MODEL_561afce9032e409bb5b3b032225d5541"
],
"layout": "IPY_MODEL_6adf9acec6544b7c8320442ed5ea7d4d"
}
},
"939a097c3dfd48c1995b927e17535930": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Minimum",
"layout": "IPY_MODEL_7ada416dd4654703a9de7d85695b5046",
"step": 1,
"style": "IPY_MODEL_dc0543b7b997483fb7f6142be03800a0"
}
},
"96b4210ee9c74accb8859fd04b1a2bc8": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"97d85cf4e0da42859bcca341c35ee548": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"9a71a8911eb34a2ca4ef363006191dd2": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonStyleModel",
"state": {}
},
"a45f633ac50b4805852214e4b26f0c70": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"a580d96051814c57bde0f41678f00504": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {}
},
"a8db9564850144d2aeeba623174f5ae2": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonStyleModel",
"state": {}
},
"ac287d6940104586860c1c76f58531e4": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Workers",
"layout": "IPY_MODEL_14edd9679e994692848cf3fa73e4e0c2",
"step": 1,
"style": "IPY_MODEL_e62ceffe4a9b40d196329b63c7cb1560"
}
},
"b0b6a9fdb5d74c41811b9c87a1889e7d": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "VBoxModel",
"state": {
"children": [
"IPY_MODEL_85f9adb9e20c4abea53873cbba30755c",
"IPY_MODEL_deac1bb711064eff846235fc15b63852",
"IPY_MODEL_3a4a70223ad04577afe21c0a2caeb8f5"
],
"layout": "IPY_MODEL_35515d765a8d4512a0b21c0523c96a9b"
}
},
"b74ee393e2564af9bd280cb095d1e590": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"min_width": "500px"
}
},
"b790110b58254695af23818ec6d06f47": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"min_width": "150px"
}
},
"bd9e1db6ebae464d81372683366da6f4": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonModel",
"state": {
"description": "Adapt",
"layout": "IPY_MODEL_4e98f0fa3b474197b3fe7a3578f84694",
"style": "IPY_MODEL_5bd0da2940754727a532aefab927e1a4"
}
},
"c7c22c4e2c344b81a6a2e35b0b1bb94d": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"min_width": "500px"
}
},
"cae579065b134bfb89083f984a0e8de5": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"cd0958c1e2a049269e5a8fb9337ea95e": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "AccordionModel",
"state": {
"_titles": {
"0": "Manual Scaling",
"1": "Adaptive Scaling"
},
"children": [
"IPY_MODEL_867cfdb1bcd2476987cb789f6c92f550",
"IPY_MODEL_1d2a9582bbbe4b9fb7708e804500aa88"
],
"layout": "IPY_MODEL_b74ee393e2564af9bd280cb095d1e590",
"selected_index": null
}
},
"ce9aaee26ddd48ab9f44bf616774d7fd": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_09f60392b4f441f392f7aa3fa8d685ca",
"style": "IPY_MODEL_6fa3dd431b92447989bc63e59c1253e5",
"value": "<p><b>Dashboard: </b><a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a></p>\n"
}
},
"d2a69396a4a44cb7be4f30a0afeeaa2d": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HTMLModel",
"state": {
"layout": "IPY_MODEL_6bbe1fbd905741fa8c0d7a2d0742f519",
"style": "IPY_MODEL_34781e3f3e374d1bae0bf8515c788526",
"value": "<p><b>Dashboard: </b><a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a></p>\n"
}
},
"d48d41d4e6b9440bb9f19db20a06b31b": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "VBoxModel",
"state": {
"children": [
"IPY_MODEL_18aaea04573e46f48af8e85af387e6f3",
"IPY_MODEL_d97648f5761e4602a3bf16be017947f2",
"IPY_MODEL_d2a69396a4a44cb7be4f30a0afeeaa2d"
],
"layout": "IPY_MODEL_56965f7145e043a2bbb2166cb9862555"
}
},
"d97648f5761e4602a3bf16be017947f2": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_6b3bb80a29044d7a98771511718cb797",
"IPY_MODEL_8857abc3b0a44f7aab2e8295cfad8d1b"
],
"layout": "IPY_MODEL_0bd9e97ffe06417ca2f9ecbd58b8c7ce"
}
},
"dc0543b7b997483fb7f6142be03800a0": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"deac1bb711064eff846235fc15b63852": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_6a9542952e694341b1c3c31735295f76",
"IPY_MODEL_cd0958c1e2a049269e5a8fb9337ea95e"
],
"layout": "IPY_MODEL_384da4a05e45440db06de4e61c267994"
}
},
"dfe452c763cf4ea4bb95bb1ab560ed2c": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Minimum",
"layout": "IPY_MODEL_14edd9679e994692848cf3fa73e4e0c2",
"step": 1,
"style": "IPY_MODEL_cae579065b134bfb89083f984a0e8de5"
}
},
"e0330ca1a7c9475fad76b5912a8b9d3b": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonModel",
"state": {
"description": "Scale",
"layout": "IPY_MODEL_7ada416dd4654703a9de7d85695b5046",
"style": "IPY_MODEL_a8db9564850144d2aeeba623174f5ae2"
}
},
"e1fc1a716fb94daebdb5cf168522ffce": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonStyleModel",
"state": {}
},
"e62ceffe4a9b40d196329b63c7cb1560": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"e9def914fe924407bab482718bf010b1": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"ec90aea084414c71a8338c5c2d68e307": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "HBoxModel",
"state": {
"children": [
"IPY_MODEL_2807c7610de342a48b1f9764cc10bac3",
"IPY_MODEL_51b64c28ab574dc99c19181bb345a935"
],
"layout": "IPY_MODEL_3b193b9eb576419fa6d90ad278476d44"
}
},
"ef4283a199be4280a7473f78921ebb7a": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "ButtonStyleModel",
"state": {}
},
"f73a46a0ab0b43dea33108da0a005707": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"f9a36bc7fb1342578d03cd9acb56e119": {
"model_module": "@jupyter-widgets/base",
"model_module_version": "1.2.0",
"model_name": "LayoutModel",
"state": {
"min_width": "500px"
}
},
"fad38e1d0d474d4e9a3d3ee879f20572": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "DescriptionStyleModel",
"state": {
"description_width": ""
}
},
"fb72fa920a65490c98934909cfaa56a8": {
"model_module": "@jupyter-widgets/controls",
"model_module_version": "1.5.0",
"model_name": "IntTextModel",
"state": {
"description": "Workers",
"layout": "IPY_MODEL_4e98f0fa3b474197b3fe7a3578f84694",
"step": 1,
"style": "IPY_MODEL_1a514a329bed4cb6815fce201ec81a25"
}
}
},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment