Skip to content

Instantly share code, notes, and snippets.

@rsignell-usgs

rsignell-usgs/Object_Storage_Costs.ipynb

Created October 27, 2023 14:58
Show Gist options
  • Save rsignell-usgs/02ed6f16be37b82d5d60ebe510714e6f to your computer and use it in GitHub Desktop.
Save rsignell-usgs/02ed6f16be37b82d5d60ebe510714e6f to your computer and use it in GitHub Desktop.
Download ZIP
Object_Storage_Costs.ipynb
Display the source blob
Display the rendered blob
Raw
Object_Storage_Costs.ipynb
{
"cells": [
{
"cell_type": "markdown",
"id": "17d83f3a-8c18-4fa3-b097-25b8414c99e9",
"metadata": {},
"source": [
"# Storage cost of 1PB for 1 year on AWS, Cloudflare R2 and OSN. \n",
"Object storage cost only, no egress, no puts, no gets (which would only make AWS more expensive). "
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "a7d9088b-c7e1-4ffd-a06d-0f34f5b7870a",
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import hvplot.pandas"
]
},
{
"cell_type": "markdown",
"id": "156a61c9-fd2f-4fc8-bf7f-c5c6461eb38b",
"metadata": {},
"source": [
"Cost of AWS and Cloudflare R2 for 1PB were calculated usign the [Cloudflare R2 Cost Calculator Tool](https://r2-calculator.cloudflare.com):"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "3ddc7bf4-1470-43ca-9c46-b639aeaeee1c",
"metadata": {},
"outputs": [],
"source": [
"cloudflare_r2_cost = 179.998 # $K/yr"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "b7b53933-c0b2-4798-87d6-bf8cdc083fd8",
"metadata": {},
"outputs": [],
"source": [
"aws_cost = 241.450 # $K/yr"
]
},
{
"cell_type": "markdown",
"id": "657dfe5d-47d4-4d79-b6ba-d53348d74aba",
"metadata": {},
"source": [
"Cost of Backblaze B2, GCS and Azure from the [Backblaze B2 Pricing page]( https://www.backblaze.com/cloud-storage/pricing), which prices 250TB for a year for Annual Capacity Bundles (so we multiply by 4 below)"
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "194703b7-9a2b-4ee3-9fbc-a6e040541e5b",
"metadata": {},
"outputs": [],
"source": [
"backblaze_cost = 19.500*4\n",
"aws_cost2 = 75.600*4\n",
"azure_cost = 52.392*4\n",
"gcs_cost = 69.000*4"
]
},
{
"cell_type": "markdown",
"id": "0d9003aa-8b0e-463a-890c-dff41b99c14b",
"metadata": {},
"source": [
"From the [AWS Calculator](calculator.aws):"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "8aed3ddd-238a-46bc-ae60-77171d736bc7",
"metadata": {},
"outputs": [],
"source": [
"aws_cost3 = 22.06720*12"
]
},
{
"cell_type": "markdown",
"id": "2858cd41-de45-4157-9828-2b85871640a0",
"metadata": {},
"source": [
"Cost of OSN: Each pods cost 100K for 1PB storage. If they last 5 years that's 20K/year. OSN is asking for a 15K/year admin fee to pay for salaried OSN folks. They also reserve 20% of each pod to allocate to the community. The estimated power consumption cost of a pod is 1K. So the total cost of 1PB for a year should be..."
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "08770f9c-29b8-49c5-a07a-1d38d3eab2f3",
"metadata": {},
"outputs": [],
"source": [
"osn_cost = ((100/5 * 1000/800) + 15 + 1)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "7ae6c303-9c52-4bf1-8f51-21e331b422c6",
"metadata": {},
"outputs": [],
"source": [
"data={'Provider':['AWS','GCS','Azure','Cloudflare R2', 'Backblaze B2', 'OSN'], 'Cost ($K)':[aws_cost3, gcs_cost, azure_cost, cloudflare_r2_cost, backblaze_cost, osn_cost]}\n",
"\n",
"df=pd.DataFrame(data)"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "d5b86845-8fdb-43dd-a386-f357c71a135b",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Provider</th>\n",
" <th>Cost ($K)</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>AWS</td>\n",
" <td>264.8064</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>GCS</td>\n",
" <td>276.0000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Azure</td>\n",
" <td>209.5680</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Cloudflare R2</td>\n",
" <td>179.9980</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Backblaze B2</td>\n",
" <td>78.0000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>OSN</td>\n",
" <td>41.0000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Provider Cost ($K)\n",
"0 AWS 264.8064\n",
"1 GCS 276.0000\n",
"2 Azure 209.5680\n",
"3 Cloudflare R2 179.9980\n",
"4 Backblaze B2 78.0000\n",
"5 OSN 41.0000"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "eee5cc6d-40ac-41de-99c9-987b3cdeab17",
"metadata": {},
"outputs": [
{
"data": {},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.holoviews_exec.v0+json": "",
"text/html": [
"<div id='p1181'>\n",
" <div id=\"db89bfeb-7a6f-4bd8-a573-9486f0699c25\" data-root-id=\"p1181\" style=\"display: contents;\"></div>\n",
"</div>\n",
"<script type=\"application/javascript\">(function(root) {\n",
" var docs_json = {\"5013b2a2-bd2f-4e89-ba14-f87f7f28dea3\":{\"version\":\"3.2.2\",\"title\":\"Bokeh Application\",\"roots\":[{\"type\":\"object\",\"name\":\"Row\",\"id\":\"p1181\",\"attributes\":{\"name\":\"Row01411\",\"tags\":[\"embedded\"],\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n background-size: auto calc(min(50%, 400px));\\n}\",{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"p1184\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.2.3/dist/css/loading.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"p1236\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.2.3/dist/css/listpanel.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"p1182\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.2.3/dist/bundled/theme/default.css\"}},{\"type\":\"object\",\"name\":\"ImportedStyleSheet\",\"id\":\"p1183\",\"attributes\":{\"url\":\"https://cdn.holoviz.org/panel/1.2.3/dist/bundled/theme/native.css\"}}],\"min_width\":700,\"margin\":0,\"sizing_mode\":\"stretch_width\",\"align\":\"start\",\"children\":[{\"type\":\"object\",\"name\":\"Spacer\",\"id\":\"p1185\",\"attributes\":{\"name\":\"HSpacer01422\",\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"p1184\"},{\"id\":\"p1182\"},{\"id\":\"p1183\"}],\"margin\":0,\"sizing_mode\":\"stretch_width\",\"align\":\"start\"}},{\"type\":\"object\",\"name\":\"Figure\",\"id\":\"p1193\",\"attributes\":{\"width\":700,\"height\":300,\"margin\":[5,10],\"sizing_mode\":\"fixed\",\"align\":\"start\",\"x_range\":{\"type\":\"object\",\"name\":\"FactorRange\",\"id\":\"p1186\",\"attributes\":{\"tags\":[[[\"Provider\",\"Provider\",null]],[]],\"factors\":[\"AWS\",\"GCS\",\"Azure\",\"Cloudflare R2\",\"Backblaze B2\",\"OSN\"]}},\"y_range\":{\"type\":\"object\",\"name\":\"Range1d\",\"id\":\"p1187\",\"attributes\":{\"tags\":[[[\"Cost ($K)\",\"Cost ($K)\",null]],{\"type\":\"map\",\"entries\":[[\"invert_yaxis\",false],[\"autorange\",false]]}],\"end\":299.5,\"reset_start\":0.0,\"reset_end\":299.5}},\"x_scale\":{\"type\":\"object\",\"name\":\"CategoricalScale\",\"id\":\"p1203\"},\"y_scale\":{\"type\":\"object\",\"name\":\"LinearScale\",\"id\":\"p1204\"},\"title\":{\"type\":\"object\",\"name\":\"Title\",\"id\":\"p1196\",\"attributes\":{\"text_color\":\"black\",\"text_font_size\":\"12pt\"}},\"renderers\":[{\"type\":\"object\",\"name\":\"GlyphRenderer\",\"id\":\"p1229\",\"attributes\":{\"data_source\":{\"type\":\"object\",\"name\":\"ColumnDataSource\",\"id\":\"p1220\",\"attributes\":{\"selected\":{\"type\":\"object\",\"name\":\"Selection\",\"id\":\"p1221\",\"attributes\":{\"indices\":[],\"line_indices\":[]}},\"selection_policy\":{\"type\":\"object\",\"name\":\"UnionRenderers\",\"id\":\"p1222\"},\"data\":{\"type\":\"map\",\"entries\":[[\"Provider\",[\"AWS\",\"GCS\",\"Azure\",\"Cloudflare R2\",\"Backblaze B2\",\"OSN\"]],[\"Cost_left_parenthesis_K_right_parenthesis\",{\"type\":\"ndarray\",\"array\":{\"type\":\"bytes\",\"data\":\"6bevA+eMcEAAAAAAAEBxQBkEVg4tMmpADi2yne9/ZkAAAAAAAIBTQAAAAAAAgERA\"},\"shape\":[6],\"dtype\":\"float64\",\"order\":\"little\"}]]}}},\"view\":{\"type\":\"object\",\"name\":\"CDSView\",\"id\":\"p1230\",\"attributes\":{\"filter\":{\"type\":\"object\",\"name\":\"AllIndices\",\"id\":\"p1231\"}}},\"glyph\":{\"type\":\"object\",\"name\":\"VBar\",\"id\":\"p1226\",\"attributes\":{\"tags\":[\"apply_ranges\"],\"x\":{\"type\":\"field\",\"field\":\"Provider\"},\"width\":{\"type\":\"value\",\"value\":0.8},\"top\":{\"type\":\"field\",\"field\":\"Cost_left_parenthesis_K_right_parenthesis\"},\"fill_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"hatch_color\":{\"type\":\"value\",\"value\":\"#30a2da\"}}},\"selection_glyph\":{\"type\":\"object\",\"name\":\"VBar\",\"id\":\"p1232\",\"attributes\":{\"tags\":[\"apply_ranges\"],\"x\":{\"type\":\"field\",\"field\":\"Provider\"},\"width\":{\"type\":\"value\",\"value\":0.8},\"bottom\":{\"type\":\"value\",\"value\":0},\"top\":{\"type\":\"field\",\"field\":\"Cost_left_parenthesis_K_right_parenthesis\"},\"line_color\":{\"type\":\"value\",\"value\":\"black\"},\"line_alpha\":{\"type\":\"value\",\"value\":1.0},\"line_width\":{\"type\":\"value\",\"value\":1},\"line_join\":{\"type\":\"value\",\"value\":\"bevel\"},\"line_cap\":{\"type\":\"value\",\"value\":\"butt\"},\"line_dash\":{\"type\":\"value\",\"value\":[]},\"line_dash_offset\":{\"type\":\"value\",\"value\":0},\"fill_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"fill_alpha\":{\"type\":\"value\",\"value\":1.0},\"hatch_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"hatch_alpha\":{\"type\":\"value\",\"value\":1.0},\"hatch_scale\":{\"type\":\"value\",\"value\":12.0},\"hatch_pattern\":{\"type\":\"value\",\"value\":null},\"hatch_weight\":{\"type\":\"value\",\"value\":1.0}}},\"nonselection_glyph\":{\"type\":\"object\",\"name\":\"VBar\",\"id\":\"p1227\",\"attributes\":{\"tags\":[\"apply_ranges\"],\"x\":{\"type\":\"field\",\"field\":\"Provider\"},\"width\":{\"type\":\"value\",\"value\":0.8},\"top\":{\"type\":\"field\",\"field\":\"Cost_left_parenthesis_K_right_parenthesis\"},\"line_alpha\":{\"type\":\"value\",\"value\":0.1},\"fill_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"fill_alpha\":{\"type\":\"value\",\"value\":0.1},\"hatch_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"hatch_alpha\":{\"type\":\"value\",\"value\":0.1}}},\"muted_glyph\":{\"type\":\"object\",\"name\":\"VBar\",\"id\":\"p1228\",\"attributes\":{\"tags\":[\"apply_ranges\"],\"x\":{\"type\":\"field\",\"field\":\"Provider\"},\"width\":{\"type\":\"value\",\"value\":0.8},\"top\":{\"type\":\"field\",\"field\":\"Cost_left_parenthesis_K_right_parenthesis\"},\"line_alpha\":{\"type\":\"value\",\"value\":0.2},\"fill_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"fill_alpha\":{\"type\":\"value\",\"value\":0.2},\"hatch_color\":{\"type\":\"value\",\"value\":\"#30a2da\"},\"hatch_alpha\":{\"type\":\"value\",\"value\":0.2}}}}}],\"toolbar\":{\"type\":\"object\",\"name\":\"Toolbar\",\"id\":\"p1202\",\"attributes\":{\"tools\":[{\"type\":\"object\",\"name\":\"WheelZoomTool\",\"id\":\"p1191\",\"attributes\":{\"tags\":[\"hv_created\"],\"zoom_together\":\"none\"}},{\"type\":\"object\",\"name\":\"HoverTool\",\"id\":\"p1192\",\"attributes\":{\"tags\":[\"hv_created\"],\"renderers\":[{\"id\":\"p1229\"}],\"tooltips\":[[\"Provider\",\"@{Provider}\"],[\"Cost ($K)\",\"@{Cost_left_parenthesis_K_right_parenthesis}\"]]}},{\"type\":\"object\",\"name\":\"SaveTool\",\"id\":\"p1215\"},{\"type\":\"object\",\"name\":\"PanTool\",\"id\":\"p1216\"},{\"type\":\"object\",\"name\":\"BoxZoomTool\",\"id\":\"p1217\",\"attributes\":{\"overlay\":{\"type\":\"object\",\"name\":\"BoxAnnotation\",\"id\":\"p1218\",\"attributes\":{\"syncable\":false,\"level\":\"overlay\",\"visible\":false,\"left_units\":\"canvas\",\"right_units\":\"canvas\",\"bottom_units\":\"canvas\",\"top_units\":\"canvas\",\"line_color\":\"black\",\"line_alpha\":1.0,\"line_width\":2,\"line_dash\":[4,4],\"fill_color\":\"lightgrey\",\"fill_alpha\":0.5}}}},{\"type\":\"object\",\"name\":\"ResetTool\",\"id\":\"p1219\"}],\"active_drag\":{\"id\":\"p1216\"},\"active_scroll\":{\"id\":\"p1191\"}}},\"left\":[{\"type\":\"object\",\"name\":\"LinearAxis\",\"id\":\"p1210\",\"attributes\":{\"ticker\":{\"type\":\"object\",\"name\":\"BasicTicker\",\"id\":\"p1211\",\"attributes\":{\"mantissas\":[1,2,5]}},\"formatter\":{\"type\":\"object\",\"name\":\"BasicTickFormatter\",\"id\":\"p1212\"},\"axis_label\":\"Cost ($K)\",\"major_label_policy\":{\"type\":\"object\",\"name\":\"AllLabels\",\"id\":\"p1213\"}}}],\"below\":[{\"type\":\"object\",\"name\":\"CategoricalAxis\",\"id\":\"p1205\",\"attributes\":{\"ticker\":{\"type\":\"object\",\"name\":\"CategoricalTicker\",\"id\":\"p1206\"},\"formatter\":{\"type\":\"object\",\"name\":\"CategoricalTickFormatter\",\"id\":\"p1207\"},\"axis_label\":\"Provider\",\"major_label_policy\":{\"type\":\"object\",\"name\":\"AllLabels\",\"id\":\"p1208\"}}}],\"center\":[{\"type\":\"object\",\"name\":\"Grid\",\"id\":\"p1209\",\"attributes\":{\"axis\":{\"id\":\"p1205\"},\"grid_line_color\":null}},{\"type\":\"object\",\"name\":\"Grid\",\"id\":\"p1214\",\"attributes\":{\"dimension\":1,\"axis\":{\"id\":\"p1210\"},\"grid_line_color\":null}}],\"min_border_top\":10,\"min_border_bottom\":10,\"min_border_left\":10,\"min_border_right\":10,\"output_backend\":\"webgl\"}},{\"type\":\"object\",\"name\":\"Spacer\",\"id\":\"p1234\",\"attributes\":{\"name\":\"HSpacer01425\",\"stylesheets\":[\"\\n:host(.pn-loading.pn-arc):before, .pn-loading.pn-arc:before {\\n background-image: url(\\\"data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHN0eWxlPSJtYXJnaW46IGF1dG87IGJhY2tncm91bmQ6IG5vbmU7IGRpc3BsYXk6IGJsb2NrOyBzaGFwZS1yZW5kZXJpbmc6IGF1dG87IiB2aWV3Qm94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVBc3BlY3RSYXRpbz0ieE1pZFlNaWQiPiAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSIjYzNjM2MzIiBzdHJva2Utd2lkdGg9IjEwIiByPSIzNSIgc3Ryb2tlLWRhc2hhcnJheT0iMTY0LjkzMzYxNDMxMzQ2NDE1IDU2Ljk3Nzg3MTQzNzgyMTM4Ij4gICAgPGFuaW1hdGVUcmFuc2Zvcm0gYXR0cmlidXRlTmFtZT0idHJhbnNmb3JtIiB0eXBlPSJyb3RhdGUiIHJlcGVhdENvdW50PSJpbmRlZmluaXRlIiBkdXI9IjFzIiB2YWx1ZXM9IjAgNTAgNTA7MzYwIDUwIDUwIiBrZXlUaW1lcz0iMDsxIj48L2FuaW1hdGVUcmFuc2Zvcm0+ICA8L2NpcmNsZT48L3N2Zz4=\\\");\\n background-size: auto calc(min(50%, 400px));\\n}\",{\"id\":\"p1184\"},{\"id\":\"p1182\"},{\"id\":\"p1183\"}],\"margin\":0,\"sizing_mode\":\"stretch_width\",\"align\":\"start\"}}]}}],\"defs\":[{\"type\":\"model\",\"name\":\"ReactiveHTML1\"},{\"type\":\"model\",\"name\":\"FlexBox1\",\"properties\":[{\"name\":\"align_content\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"align_items\",\"kind\":\"Any\",\"default\":\"flex-start\"},{\"name\":\"flex_direction\",\"kind\":\"Any\",\"default\":\"row\"},{\"name\":\"flex_wrap\",\"kind\":\"Any\",\"default\":\"wrap\"},{\"name\":\"justify_content\",\"kind\":\"Any\",\"default\":\"flex-start\"}]},{\"type\":\"model\",\"name\":\"FloatPanel1\",\"properties\":[{\"name\":\"config\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"contained\",\"kind\":\"Any\",\"default\":true},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"right-top\"},{\"name\":\"offsetx\",\"kind\":\"Any\",\"default\":null},{\"name\":\"offsety\",\"kind\":\"Any\",\"default\":null},{\"name\":\"theme\",\"kind\":\"Any\",\"default\":\"primary\"},{\"name\":\"status\",\"kind\":\"Any\",\"default\":\"normalized\"}]},{\"type\":\"model\",\"name\":\"GridStack1\",\"properties\":[{\"name\":\"mode\",\"kind\":\"Any\",\"default\":\"warn\"},{\"name\":\"ncols\",\"kind\":\"Any\",\"default\":null},{\"name\":\"nrows\",\"kind\":\"Any\",\"default\":null},{\"name\":\"allow_resize\",\"kind\":\"Any\",\"default\":true},{\"name\":\"allow_drag\",\"kind\":\"Any\",\"default\":true},{\"name\":\"state\",\"kind\":\"Any\",\"default\":[]}]},{\"type\":\"model\",\"name\":\"drag1\",\"properties\":[{\"name\":\"slider_width\",\"kind\":\"Any\",\"default\":5},{\"name\":\"slider_color\",\"kind\":\"Any\",\"default\":\"black\"},{\"name\":\"value\",\"kind\":\"Any\",\"default\":50}]},{\"type\":\"model\",\"name\":\"click1\",\"properties\":[{\"name\":\"terminal_output\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"debug_name\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"clears\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"FastWrapper1\",\"properties\":[{\"name\":\"object\",\"kind\":\"Any\",\"default\":null},{\"name\":\"style\",\"kind\":\"Any\",\"default\":null}]},{\"type\":\"model\",\"name\":\"NotificationAreaBase1\",\"properties\":[{\"name\":\"js_events\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"NotificationArea1\",\"properties\":[{\"name\":\"js_events\",\"kind\":\"Any\",\"default\":{\"type\":\"map\"}},{\"name\":\"notifications\",\"kind\":\"Any\",\"default\":[]},{\"name\":\"position\",\"kind\":\"Any\",\"default\":\"bottom-right\"},{\"name\":\"_clear\",\"kind\":\"Any\",\"default\":0},{\"name\":\"types\",\"kind\":\"Any\",\"default\":[{\"type\":\"map\",\"entries\":[[\"type\",\"warning\"],[\"background\",\"#ffc107\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-exclamation-triangle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]},{\"type\":\"map\",\"entries\":[[\"type\",\"info\"],[\"background\",\"#007bff\"],[\"icon\",{\"type\":\"map\",\"entries\":[[\"className\",\"fas fa-info-circle\"],[\"tagName\",\"i\"],[\"color\",\"white\"]]}]]}]}]},{\"type\":\"model\",\"name\":\"Notification\",\"properties\":[{\"name\":\"background\",\"kind\":\"Any\",\"default\":null},{\"name\":\"duration\",\"kind\":\"Any\",\"default\":3000},{\"name\":\"icon\",\"kind\":\"Any\",\"default\":null},{\"name\":\"message\",\"kind\":\"Any\",\"default\":\"\"},{\"name\":\"notification_type\",\"kind\":\"Any\",\"default\":null},{\"name\":\"_destroyed\",\"kind\":\"Any\",\"default\":false}]},{\"type\":\"model\",\"name\":\"TemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"BootstrapTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]},{\"type\":\"model\",\"name\":\"MaterialTemplateActions1\",\"properties\":[{\"name\":\"open_modal\",\"kind\":\"Any\",\"default\":0},{\"name\":\"close_modal\",\"kind\":\"Any\",\"default\":0}]}]}};\n",
" var render_items = [{\"docid\":\"5013b2a2-bd2f-4e89-ba14-f87f7f28dea3\",\"roots\":{\"p1181\":\"db89bfeb-7a6f-4bd8-a573-9486f0699c25\"},\"root_ids\":[\"p1181\"]}];\n",
" var docs = Object.values(docs_json)\n",
" if (!docs) {\n",
" return\n",
" }\n",
" const py_version = docs[0].version.replace('rc', '-rc.').replace('.dev', '-dev.')\n",
" const is_dev = py_version.indexOf(\"+\") !== -1 || py_version.indexOf(\"-\") !== -1\n",
" function embed_document(root) {\n",
" var Bokeh = get_bokeh(root)\n",
" Bokeh.embed.embed_items_notebook(docs_json, render_items);\n",
" for (const render_item of render_items) {\n",
" for (const root_id of render_item.root_ids) {\n",
"\tconst id_el = document.getElementById(root_id)\n",
"\tif (id_el.children.length && (id_el.children[0].className === 'bk-root')) {\n",
"\t const root_el = id_el.children[0]\n",
"\t root_el.id = root_el.id + '-rendered'\n",
"\t}\n",
" }\n",
" }\n",
" }\n",
" function get_bokeh(root) {\n",
" if (root.Bokeh === undefined) {\n",
" return null\n",
" } else if (root.Bokeh.version !== py_version && !is_dev) {\n",
" if (root.Bokeh.versions === undefined || !root.Bokeh.versions.has(py_version)) {\n",
"\treturn null\n",
" }\n",
" return root.Bokeh.versions.get(py_version);\n",
" } else if (root.Bokeh.version === py_version) {\n",
" return root.Bokeh\n",
" }\n",
" return null\n",
" }\n",
" function is_loaded(root) {\n",
" var Bokeh = get_bokeh(root)\n",
" return (Bokeh != null && Bokeh.Panel !== undefined)\n",
" }\n",
" if (is_loaded(root)) {\n",
" embed_document(root);\n",
" } else {\n",
" var attempts = 0;\n",
" var timer = setInterval(function(root) {\n",
" if (is_loaded(root)) {\n",
" clearInterval(timer);\n",
" embed_document(root);\n",
" } else if (document.readyState == \"complete\") {\n",
" attempts++;\n",
" if (attempts > 200) {\n",
" clearInterval(timer);\n",
"\t var Bokeh = get_bokeh(root)\n",
"\t if (Bokeh == null || Bokeh.Panel == null) {\n",
" console.warn(\"Panel: ERROR: Unable to run Panel code because Bokeh or Panel library is missing\");\n",
"\t } else {\n",
"\t console.warn(\"Panel: WARNING: Attempting to render but not all required libraries could be resolved.\")\n",
"\t embed_document(root)\n",
"\t }\n",
" }\n",
" }\n",
" }, 25, root)\n",
" }\n",
"})(window);</script>"
],
"text/plain": [
":Bars [Provider] (Cost ($K))"
]
},
"execution_count": 37,
"metadata": {
"application/vnd.holoviews_exec.v0+json": {
"id": "p1181"
}
},
"output_type": "execute_result"
}
],
"source": [
"df.hvplot.bar(x='Provider')"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "28bddd6f-6392-429a-a2be-c6cd8c9419d7",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"4.390195121951219"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(cloudflare_r2_cost/osn_cost) "
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "8aaff908-bab1-4c9d-99ae-9a6f49cfd1a2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"182.2242469360771"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"800 / (cloudflare_r2_cost/osn_cost)"
]
},
{
"cell_type": "markdown",
"id": "afd4f301-397a-4f0d-a05d-45a2f190f24d",
"metadata": {},
"source": [
"## Comments\n",
"* Certainly there is a cost to maintaining the infrastructure where the OSN pods are located, but until these facilities go away, OSN is the cheapest egress-fee-free solution for researchers.\n",
"\n",
"* OSN Pods are almost certainly less reliable and scalable than the big providers, so would not be a great solution for, say, the National Hurricane Center. And you need to have a home for the pod you buy, usually at a government funded HPC facility. So it won't work for everyone. For the USGS-led HyTEST project though, it seems a sweet spot of performance for price. "
]
},
{
"cell_type": "markdown",
"id": "0450a878-0888-4003-bd7c-9f9f0e35c5e5",
"metadata": {},
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "users-users-pangeo",
"language": "python",
"name": "conda-env-users-users-pangeo-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.12"
},
"widgets": {
"application/vnd.jupyter.widget-state+json": {
"state": {},
"version_major": 2,
"version_minor": 0
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment