Skip to content

Instantly share code, notes, and snippets.

@romer8
Last active November 13, 2024 22:11
Show Gist options
  • Save romer8/89c851014afb276b0f20cb61c9c731f6 to your computer and use it in GitHub Desktop.
Save romer8/89c851014afb276b0f20cb61c9c731f6 to your computer and use it in GitHub Desktop.
pywaterml_training.ipynb
Display the source blob
Display the rendered blob
Raw
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "pywaterml_training.ipynb",
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/romer8/89c851014afb276b0f20cb61c9c731f6/pywaterml_template.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PgIgUWTOwCqL"
},
"source": [
"# PywaterML Training"
]
},
{
"cell_type": "markdown",
"source": [
"## Requisites\n",
"\n",
"You need a Token:\n",
"[Get one](https://community.wmo.int/en/whos-web-services-and-supported-tools)"
],
"metadata": {
"id": "DnHG7wMPj_PJ"
}
},
{
"cell_type": "code",
"source": [
"# paste your peersonal token\n",
"Your_Personal_Token_Identifier = 'whos-60f68787-6a92-4d4e-95e2-08aee08cb239'"
],
"metadata": {
"id": "GclQF-5yY2X6"
},
"execution_count": 1,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "L1jSFY-jfnqs"
},
"source": [
"### Contents\n",
"1. [Variables for the Analysis](#intro)\n",
"2. [Install Pywaterml and Additional Dependencies](#install)\n",
"3. [Add WOF Web Service](#addService)\n",
"4. [Sites](#GetSites)\n",
"6. [Variables](#GetVariables)\n",
"8. [Site Information](#GetSiteInfo)\n",
"9. [Time Series Values for Variables](#GetValues)\n",
"\n",
"10. [Sites by Bounding Box](#GetSitesByBoxObject)\n",
"11. [Sites Filtered by Variables](#GetSitesByVariable)\n",
"12. [Interpolation](#GetInterpolation)\n",
"13. [Monthly Averages for Variables](#GetMonthlyAverage)\n",
"14. [Time Series Clusters from Monthly Averages](#GetClustersMonthlyAvg)\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iCZsmAefqm-D"
},
"source": [
"<a name=\"intro\"></a>\n",
"## Variables for the Analysis"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c3n1n3MVPUTs"
},
"source": [
"The following are variables to use this notebook, so just change the following, and you should be able to run the notebook and see the output.\n",
"\n",
"*Note that the VARIABLE and SITE variables are the index from the responses of GetVariables and GetSites respectively. For example, changing VARIABLE = 1 will give the second variable in from the GetVariables methods. Similarly,the SITES variable functions in the same way.*"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ioWosThiPbvg"
},
"source": [
"view = 'gs-view-and(whos,gs-view-source(russia-rihmi-wdc))'\n",
"WOF_URL = f\"https://whos.geodab.eu/gs-service/services/essi/token/{Your_Personal_Token_Identifier}/view/{view}/cuahsi_1_1.asmx?WSDL\"\n",
"\n",
"VARIABLE = 0\n",
"SITE = 0\n"
],
"execution_count": 8,
"outputs": []
},
{
"cell_type": "code",
"source": [
"WOF_URL"
],
"metadata": {
"id": "0SFxhgVZfRd_",
"outputId": "b98a826c-f3d0-47c2-e47b-750d5d8f5c10",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
}
},
"execution_count": 9,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"'https://whos.geodab.eu/gs-service/services/essi/token/whos-60f68787-6a92-4d4e-95e2-08aee08cb239/view/gs-view-and(whos,gs-view-source(russia-rihmi-wdc))/cuahsi_1_1.asmx?WSDL'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 9
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GdsIhCI1kW42"
},
"source": [
"<a name=\"install\"></a>\n",
"### Install Pywaterml and Additional Dependencies"
]
},
{
"cell_type": "code",
"metadata": {
"id": "MtWfYP3TgfR_",
"outputId": "0d8b0df7-a040-48a1-bab1-b43e3999391a",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"!pip install pywaterml\n",
"!pip install folium\n",
"!pip install plotly\n",
"!pip install pyproj"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Requirement already satisfied: pywaterml in /usr/local/lib/python3.10/dist-packages (1.2.26)\n",
"Requirement already satisfied: xmltodict in /usr/local/lib/python3.10/dist-packages (from pywaterml) (0.14.2)\n",
"Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from pywaterml) (2.2.2)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from pywaterml) (1.26.4)\n",
"Requirement already satisfied: owslib in /usr/local/lib/python3.10/dist-packages (from pywaterml) (0.32.0)\n",
"Requirement already satisfied: suds-community in /usr/local/lib/python3.10/dist-packages (from pywaterml) (1.2.0)\n",
"Requirement already satisfied: pyproj in /usr/local/lib/python3.10/dist-packages (from pywaterml) (3.7.0)\n",
"Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from owslib->pywaterml) (5.3.0)\n",
"Requirement already satisfied: python-dateutil in /usr/local/lib/python3.10/dist-packages (from owslib->pywaterml) (2.8.2)\n",
"Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from owslib->pywaterml) (6.0.2)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from owslib->pywaterml) (2.32.3)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->pywaterml) (2024.2)\n",
"Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->pywaterml) (2024.2)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from pyproj->pywaterml) (2024.8.30)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil->owslib->pywaterml) (1.16.0)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->owslib->pywaterml) (3.4.0)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->owslib->pywaterml) (3.10)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->owslib->pywaterml) (2.2.3)\n",
"Requirement already satisfied: folium in /usr/local/lib/python3.10/dist-packages (0.18.0)\n",
"Requirement already satisfied: branca>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from folium) (0.8.0)\n",
"Requirement already satisfied: jinja2>=2.9 in /usr/local/lib/python3.10/dist-packages (from folium) (3.1.4)\n",
"Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from folium) (1.26.4)\n",
"Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from folium) (2.32.3)\n",
"Requirement already satisfied: xyzservices in /usr/local/lib/python3.10/dist-packages (from folium) (2024.9.0)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2>=2.9->folium) (3.0.2)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->folium) (3.4.0)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->folium) (3.10)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->folium) (2.2.3)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->folium) (2024.8.30)\n",
"Requirement already satisfied: plotly in /usr/local/lib/python3.10/dist-packages (5.24.1)\n",
"Requirement already satisfied: tenacity>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from plotly) (9.0.0)\n",
"Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from plotly) (24.2)\n",
"Requirement already satisfied: pyproj in /usr/local/lib/python3.10/dist-packages (3.7.0)\n",
"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from pyproj) (2024.8.30)\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "axKiaVU1pjQe"
},
"source": [
"import pywaterml.waterML as pwml\n",
"import folium\n",
"import plotly.graph_objects as go\n",
"import plotly.express as px\n",
"import pandas as pd\n",
"import warnings\n",
"warnings.filterwarnings('ignore')\n"
],
"execution_count": 5,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "0YN3fO0dsmFC"
},
"source": [
"<a name=\"addService\"></a>\n",
"### WOF Web Service"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rqGI2HnKslup"
},
"source": [
"Any WOF web service can be added to the wrapper class WaterMLOperations from the pywaterml package. In the link below, please change the <em>hs_url</em> variable below.\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "hfZV_E02lEfs"
},
"source": [
"try:\n",
" water = pwml.WaterMLOperations(url = WOF_URL)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": 10,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "z8skY4N5tq7G"
},
"source": [
"<a name=\"GetSites\"></a>\n",
"\n",
"### Sites"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pG7eT4WqzIpb"
},
"source": [
"The GetSites method retrieves meta data for each site in the WOF web service. Data can be retrieved in three different formats: JSON, CSV, and WaterML"
]
},
{
"cell_type": "code",
"metadata": {
"id": "SZNMaxpwrc2D"
},
"source": [
"try:\n",
" sites = water.GetSites()\n",
" df = pd.DataFrame.from_dict(sites)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": 11,
"outputs": []
},
{
"cell_type": "code",
"source": [
"sites"
],
"metadata": {
"id": "6Gd3SmNpkWEf",
"outputId": "70a50ed5-b2b1-4d45-925f-efc2f75dc354",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"execution_count": 12,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[{'sitename': 'Chulym At Baturino',\n",
" 'latitude': '57.79684',\n",
" 'longitude': '85.11002',\n",
" 'sitecode': '13AD4325E252F6FE87425487DF9FE47E980B1C74',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:13AD4325E252F6FE87425487DF9FE47E980B1C74',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Vilyuy At Ust' Ambardakh\",\n",
" 'latitude': '62.15',\n",
" 'longitude': '117.65',\n",
" 'sitecode': '1ABC255238F7BA1CEE8C83435CE4270D133BCA4E',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:1ABC255238F7BA1CEE8C83435CE4270D133BCA4E',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Lena at Krestovskoe',\n",
" 'latitude': '59.76987',\n",
" 'longitude': '113.28421',\n",
" 'sitecode': '2106B6899E109E046C4EE5058700FBE6C6A510AF',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:2106B6899E109E046C4EE5058700FBE6C6A510AF',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Kolyma At Ust'-Srednekan\",\n",
" 'latitude': '62.41542',\n",
" 'longitude': '152.14512',\n",
" 'sitecode': '2B7567B01E128A058D004E056D8F374CCFF1A120',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:2B7567B01E128A058D004E056D8F374CCFF1A120',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Yana At Verkhoyansk',\n",
" 'latitude': '67.56',\n",
" 'longitude': '133.4',\n",
" 'sitecode': '3562900AAA46A40F398848553704E3D270EE6DA0',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:3562900AAA46A40F398848553704E3D270EE6DA0',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Ob At Kolpashevo',\n",
" 'latitude': '58.30033',\n",
" 'longitude': '82.90321',\n",
" 'sitecode': '39AE0651B8C54A3D8189D7567D386061D149F5F0',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:39AE0651B8C54A3D8189D7567D386061D149F5F0',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Podkamennaya Tunguska At Kuz'movka\",\n",
" 'latitude': '62.31',\n",
" 'longitude': '92.11',\n",
" 'sitecode': '3B8574020848AABC2DAAEA2D9B3CE6AC43DBBCDD',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:3B8574020848AABC2DAAEA2D9B3CE6AC43DBBCDD',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Vasyugan At Sredny Vasyugan',\n",
" 'latitude': '59.22037',\n",
" 'longitude': '78.21877',\n",
" 'sitecode': '3DE4BC25E98C5B9F39AD9B4BBFE619DDA27C1815',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:3DE4BC25E98C5B9F39AD9B4BBFE619DDA27C1815',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Indigirka At Indigirskiy',\n",
" 'latitude': '63.53',\n",
" 'longitude': '143.11',\n",
" 'sitecode': '3F4B76E377A1640B54BAC30CE28E8500824220B5',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:3F4B76E377A1640B54BAC30CE28E8500824220B5',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Mezen' At Malonisogorskaya\",\n",
" 'latitude': '65.0112',\n",
" 'longitude': '45.66589',\n",
" 'sitecode': '3FEE2ECB41030B81F3747906AC5DD549C2784C06',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:3FEE2ECB41030B81F3747906AC5DD549C2784C06',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Lena At Tabaga',\n",
" 'latitude': '61.85457',\n",
" 'longitude': '129.69776',\n",
" 'sitecode': '411A8632304C6330A58EC61AE3D0DEFEA0AF8AD0',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:411A8632304C6330A58EC61AE3D0DEFEA0AF8AD0',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Severnaya Dvina At Abramkovo',\n",
" 'latitude': '62.09741',\n",
" 'longitude': '45.10964',\n",
" 'sitecode': '43760A8D5B8BFAE475E0716BA76C26C7BD4E3F1F',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:43760A8D5B8BFAE475E0716BA76C26C7BD4E3F1F',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Severnaya Sosva At Igrim',\n",
" 'latitude': '63.19252',\n",
" 'longitude': '64.38967',\n",
" 'sitecode': '45955735E253BED9DC496F34B72A4487D1DACE6E',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:45955735E253BED9DC496F34B72A4487D1DACE6E',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Tuloma At Verkhne-Tulomskaya',\n",
" 'latitude': '68.607608',\n",
" 'longitude': '31.760284',\n",
" 'sitecode': '46DA28E5317BB5A422C4AC4782F7F8B074B6B566',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:46DA28E5317BB5A422C4AC4782F7F8B074B6B566',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Tobol At Yalotorovsk',\n",
" 'latitude': '56.66138',\n",
" 'longitude': '66.34654',\n",
" 'sitecode': '4D5A5F017C7E85C23136932605A4CF300773C9E6',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:4D5A5F017C7E85C23136932605A4CF300773C9E6',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Aldan At Ust'-Mil'\",\n",
" 'latitude': '59.63',\n",
" 'longitude': '133.03',\n",
" 'sitecode': '5205EFAEA89E4D84324ACB6C08D30936A7412A1A',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:5205EFAEA89E4D84324ACB6C08D30936A7412A1A',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Charysh At Charishskiy Zernovoy Sovkhoz',\n",
" 'latitude': '52.15006',\n",
" 'longitude': '83.30805',\n",
" 'sitecode': '53098E1E5CBAB95286569A28EDC49ADB7CC2B929',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:53098E1E5CBAB95286569A28EDC49ADB7CC2B929',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Sukhona At Kalikino',\n",
" 'latitude': '60.67169',\n",
" 'longitude': '45.86517',\n",
" 'sitecode': '5B144106ADD56CF8FC1EB0A2FEEC0105F7A6D616',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:5B144106ADD56CF8FC1EB0A2FEEC0105F7A6D616',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Alazeya At Argakhtakh',\n",
" 'latitude': '68.5',\n",
" 'longitude': '153.41',\n",
" 'sitecode': '5F17560DEAE32167CB5FF8B15353A983BF066F2D',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:5F17560DEAE32167CB5FF8B15353A983BF066F2D',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Tom' At Tomsk\",\n",
" 'latitude': '56.447785',\n",
" 'longitude': '84.960621',\n",
" 'sitecode': '630E5CB3E3E82957588BF1A2A4D99FFA7C3B1420',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:630E5CB3E3E82957588BF1A2A4D99FFA7C3B1420',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Nizhnyaya Tunguska At Yerbogachen',\n",
" 'latitude': '61.281153',\n",
" 'longitude': '108.008255',\n",
" 'sitecode': '6E1D17B8A259B7F06B9370AD79C2B7197C5EB36D',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:6E1D17B8A259B7F06B9370AD79C2B7197C5EB36D',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Ob At Salekhard',\n",
" 'latitude': '66.58773',\n",
" 'longitude': '66.4988',\n",
" 'sitecode': '6F04F9A3C4EA46C0E59F99555289D206E5B0F292',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:6F04F9A3C4EA46C0E59F99555289D206E5B0F292',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Selenga At Raz'ezd Mostovoy\",\n",
" 'latitude': '52.0556',\n",
" 'longitude': '107.49525',\n",
" 'sitecode': '705192B2AF8AB90A33B3171109C496339704455B',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:705192B2AF8AB90A33B3171109C496339704455B',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Nizhnyaya Tunguska At Bol'shoy Porog\",\n",
" 'latitude': '65.63',\n",
" 'longitude': '90.01',\n",
" 'sitecode': '73B19830FFE577EADE9718D21E22A1948A7468A1',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:73B19830FFE577EADE9718D21E22A1948A7468A1',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Bol'shoy Anuy At Konstantinovo\",\n",
" 'latitude': '68.1509',\n",
" 'longitude': '161.17462',\n",
" 'sitecode': '7A3E20DFE6C902B3917BCFEFC7A43FBDAC405C31',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:7A3E20DFE6C902B3917BCFEFC7A43FBDAC405C31',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Ishim At Vikulovo',\n",
" 'latitude': '56.82443',\n",
" 'longitude': '70.62875',\n",
" 'sitecode': '8040E91A0CEA9364BC56CDBE215B72A0606B1CBD',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:8040E91A0CEA9364BC56CDBE215B72A0606B1CBD',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Yenisey At Yeniseysk',\n",
" 'latitude': '58.45',\n",
" 'longitude': '92.15',\n",
" 'sitecode': '822F510BCB1CC7EFDABC3865097441927EC97C87',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:822F510BCB1CC7EFDABC3865097441927EC97C87',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Ket' (Bol'shaya Ket') At Rodionovka\",\n",
" 'latitude': '58.44233',\n",
" 'longitude': '83.51774',\n",
" 'sitecode': '8624731434C4D8E617CC6B07DE2EEF7CE14A0031',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:8624731434C4D8E617CC6B07DE2EEF7CE14A0031',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Biya At Biysk',\n",
" 'latitude': '52.54271',\n",
" 'longitude': '85.23564',\n",
" 'sitecode': '909E35ACC6804B54FD9F0647F0A64844A160A4B5',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:909E35ACC6804B54FD9F0647F0A64844A160A4B5',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Onega At Porog',\n",
" 'latitude': '63.82912',\n",
" 'longitude': '38.46988',\n",
" 'sitecode': '92CCD479B1AD5261A0C9BC08F56518739564949A',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:92CCD479B1AD5261A0C9BC08F56518739564949A',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Iset' At Isetskoye\",\n",
" 'latitude': '56.46551',\n",
" 'longitude': '65.34332',\n",
" 'sitecode': '93A8C17376A760C53C220B10BBD8A71E451F48FE',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:93A8C17376A760C53C220B10BBD8A71E451F48FE',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Vuchegda At Malaya Kuzhba',\n",
" 'latitude': '61.68868',\n",
" 'longitude': '53.64998',\n",
" 'sitecode': '96D63F579A4C0455F4ABC44F1B0032953AE6A27C',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:96D63F579A4C0455F4ABC44F1B0032953AE6A27C',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Angara At Irkutskaya GES',\n",
" 'latitude': '52.241602',\n",
" 'longitude': '104.311854',\n",
" 'sitecode': '9BA65A89073504D2FC7B2C2731CC886435BE7D06',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:9BA65A89073504D2FC7B2C2731CC886435BE7D06',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Yenisey At Igarka',\n",
" 'latitude': '67.43',\n",
" 'longitude': '86.48',\n",
" 'sitecode': '9F72240BBD022A60BBDAEEF26DBA7B3FC75B0338',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:9F72240BBD022A60BBDAEEF26DBA7B3FC75B0338',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Pur At Urengoy',\n",
" 'latitude': '65.9648',\n",
" 'longitude': '78.34363',\n",
" 'sitecode': 'A4F78995AEF41FAE17D7484BB9258A5D5648D9D1',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:A4F78995AEF41FAE17D7484BB9258A5D5648D9D1',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Konda At Bol'chary\",\n",
" 'latitude': '59.81385',\n",
" 'longitude': '68.79066',\n",
" 'sitecode': 'A5BEBCFAE322F7AAB087F01D969D13F4E31F3F6F',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:A5BEBCFAE322F7AAB087F01D969D13F4E31F3F6F',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Uchur At Chulbu',\n",
" 'latitude': '57.76',\n",
" 'longitude': '130.9',\n",
" 'sitecode': 'AD1A6162B89C61B22FB96B94E0F18AC4EF35931C',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:AD1A6162B89C61B22FB96B94E0F18AC4EF35931C',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Yenisey at Kyizyil',\n",
" 'latitude': '51.7262',\n",
" 'longitude': '94.44083',\n",
" 'sitecode': 'AE6A2E159118F782FBAC0AC2D5DC73C055DEDD9B',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:AE6A2E159118F782FBAC0AC2D5DC73C055DEDD9B',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Olenek At 7.5 km down of Buur's mouth\",\n",
" 'latitude': '71.85',\n",
" 'longitude': '123.65',\n",
" 'sitecode': 'AF4EF7DCA94F4CF38099A916129F3389EC03D56E',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:AF4EF7DCA94F4CF38099A916129F3389EC03D56E',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Vilyuy At Vilyuy',\n",
" 'latitude': '64.08',\n",
" 'longitude': '109.58',\n",
" 'sitecode': 'B1822A712B96AA466F4187A74B447F5FB931B899',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:B1822A712B96AA466F4187A74B447F5FB931B899',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Irtish At Tobol'sk\",\n",
" 'latitude': '58.20209',\n",
" 'longitude': '68.23542',\n",
" 'sitecode': 'BAD40F47C309FA0840B72037730BB6577DFE0421',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:BAD40F47C309FA0840B72037730BB6577DFE0421',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Voron'ya At Serebryanskaya\",\n",
" 'latitude': '68.840504',\n",
" 'longitude': '35.573592',\n",
" 'sitecode': 'BBBF5EB0057B38F127C3C17289883EAB9AD5D69A',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:BBBF5EB0057B38F127C3C17289883EAB9AD5D69A',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Abakan At Raikov',\n",
" 'latitude': '53.52249',\n",
" 'longitude': '91.18612',\n",
" 'sitecode': 'BCA0163170F4B845C636EAFC4DB077FED4B24921',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:BCA0163170F4B845C636EAFC4DB077FED4B24921',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Ob At Barnaul',\n",
" 'latitude': '53.36953',\n",
" 'longitude': '83.8111',\n",
" 'sitecode': 'BF073EE1ACB85D8B014CBB8BDE6738A1DDEAB301',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:BF073EE1ACB85D8B014CBB8BDE6738A1DDEAB301',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Kolyma At Srednekolymsk',\n",
" 'latitude': '67.45',\n",
" 'longitude': '153.7167',\n",
" 'sitecode': 'C0D204A27D56236A2CBF0C22C2E6FF26F3704566',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:C0D204A27D56236A2CBF0C22C2E6FF26F3704566',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Irtish At Omsk',\n",
" 'latitude': '55.00222',\n",
" 'longitude': '73.32482',\n",
" 'sitecode': 'C306ED5C4CCE4E0558A08AE594F9E13A2FDEECF6',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:C306ED5C4CCE4E0558A08AE594F9E13A2FDEECF6',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Severnaya Dvina At Ust'-Pinega\",\n",
" 'latitude': '64.1466',\n",
" 'longitude': '41.90859',\n",
" 'sitecode': 'C4EE9FB0F47E8B247EE17F597A2F1105A0006399',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:C4EE9FB0F47E8B247EE17F597A2F1105A0006399',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Anabar At Saskulakh',\n",
" 'latitude': '71.96',\n",
" 'longitude': '114.08',\n",
" 'sitecode': 'CAE5CB22C5098228E08823B59F519F2D170A2D42',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:CAE5CB22C5098228E08823B59F519F2D170A2D42',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Vilyuy At Khatyrik-Khomo',\n",
" 'latitude': '63.95',\n",
" 'longitude': '124.83',\n",
" 'sitecode': 'D77555185FCEDDAFC5A85E48B82E327C10B36D45',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:D77555185FCEDDAFC5A85E48B82E327C10B36D45',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Aldan At Verkhoyanskiy Perevoz',\n",
" 'latitude': '63.31',\n",
" 'longitude': '132.01',\n",
" 'sitecode': 'DA809CD33D947EA3E6CE599803AD81EA595DEDCA',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:DA809CD33D947EA3E6CE599803AD81EA595DEDCA',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Ob At Belogor'e\",\n",
" 'latitude': '61.06851',\n",
" 'longitude': '68.62401',\n",
" 'sitecode': 'E20AB0F9741F7F78C1AA9BDE19D047845FCC3DD9',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:E20AB0F9741F7F78C1AA9BDE19D047845FCC3DD9',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Adycha At Ust'-Charky\",\n",
" 'latitude': '66.8',\n",
" 'longitude': '136.68',\n",
" 'sitecode': 'F250F0DA53000F8AFE77183BBFDFAD735F662457',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:F250F0DA53000F8AFE77183BBFDFAD735F662457',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Yenisey At Bazaikha',\n",
" 'latitude': '55.98',\n",
" 'longitude': '92.8',\n",
" 'sitecode': 'F298591FECAE9FED553309CAA2342514C8BF3A5E',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:F298591FECAE9FED553309CAA2342514C8BF3A5E',\n",
" 'siteID': '1'},\n",
" {'sitename': \"Olekma At Kudu-Kel'\",\n",
" 'latitude': '59.36',\n",
" 'longitude': '121.31',\n",
" 'sitecode': 'F3C6391C503F691D3B46776BF23AFA8627CFE3C2',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:F3C6391C503F691D3B46776BF23AFA8627CFE3C2',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Olenek At Sukhona',\n",
" 'latitude': '68.61',\n",
" 'longitude': '118.33',\n",
" 'sitecode': 'F495F303EA92D02534E7556E05CCD0EDE9C39144',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:F495F303EA92D02534E7556E05CCD0EDE9C39144',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Nadym At Nadym',\n",
" 'latitude': '65.55411',\n",
" 'longitude': '72.70936',\n",
" 'sitecode': 'F8A8645D4A27C1558F1D1E7E44523CB2CE0371D7',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:F8A8645D4A27C1558F1D1E7E44523CB2CE0371D7',\n",
" 'siteID': '1'},\n",
" {'sitename': 'Ponoy At Kanevka',\n",
" 'latitude': '67.12523',\n",
" 'longitude': '39.67343',\n",
" 'sitecode': 'FDBB08309ADBA64CF0EB315BB85014A472E77D96',\n",
" 'network': 'russia-rihmi-wdc',\n",
" 'fullSiteCode': 'russia-rihmi-wdc:FDBB08309ADBA64CF0EB315BB85014A472E77D96',\n",
" 'siteID': '1'}]"
]
},
"metadata": {},
"execution_count": 12
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wGdgtYByuvb0"
},
"source": [
"The following code is a helpful code snipped to plot the different stations in a follium map"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ld6QSYA2j_Wk",
"outputId": "857a8f69-a1e2-4e86-d208-43712e3aa464",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"try:\n",
" m = folium.Map(location=[40.0150, -105.2705],\n",
" tiles = 'Stamen Terrain')\n",
" sitesLocations = []\n",
" for site in sites:\n",
" sitesLocations.append([float(site['latitude']) , float(site['longitude'])])\n",
" folium.Marker(\n",
" location= [float(site['latitude']) , float(site['longitude'])],\n",
" icon=folium.Icon()\n",
" ).add_to(m)\n",
" m.fit_bounds(sitesLocations)\n",
" m\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Custom tiles must have an attribution.\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0UKt9Ty7xQfP"
},
"source": [
"<a name=\"GetVariables\"></a>\n",
"### Variables"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Z-Ntb2bWxVR-"
},
"source": [
"The GetVariables method retrieves metadata from the different variables in the WOF web service."
]
},
{
"cell_type": "code",
"metadata": {
"id": "v3DOPKYXsJ7i",
"outputId": "92e17464-d400-4501-bbd8-2bbd5a6e9f68",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"pd.set_option('display.max_columns', None)\n",
"try:\n",
" variables = water.GetVariables()\n",
" df = pd.DataFrame.from_dict(variables['variables'])\n",
"\n",
" print(df.head())\n",
"except Exception as e:\n",
" print(e)\n"
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"ERROR:root:Not using the suds library directly, using a plugin\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/http.py\", line 89, in send\n",
" fp = self.u2open(u2request, timeout=request.timeout)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/http.py\", line 140, in u2open\n",
" return url.open(u2request, timeout=tm)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 525, in open\n",
" response = meth(req, response)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 634, in http_response\n",
" response = self.parent.error(\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 563, in error\n",
" return self._call_chain(*args)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 496, in _call_chain\n",
" result = func(*args)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 643, in http_error_default\n",
" raise HTTPError(req.full_url, code, msg, hdrs, fp)\n",
"urllib.error.HTTPError: HTTP Error 403: \n",
"\n",
"During handling of the above exception, another exception occurred:\n",
"\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 771, in send\n",
" reply = self.options.transport.send(request)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/https.py\", line 66, in send\n",
" return HttpTransport.send(self, request)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/http.py\", line 106, in send\n",
" raise TransportError(e.msg, e.code, e.fp)\n",
"suds.transport.TransportError\n",
"\n",
"During handling of the above exception, another exception occurred:\n",
"\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/pywaterml/waterML.py\", line 309, in GetVariables\n",
" variables = self.client.service.GetVariables('[:]')\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 586, in __call__\n",
" return client.invoke(args, kwargs)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 728, in invoke\n",
" result = self.send(soapenv, timeout=timeout)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 776, in send\n",
" return self.process_reply(content, e.httpcode, tostr(e))\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 845, in process_reply\n",
" raise Exception((status, description))\n",
"Exception: (403, '')\n",
"ERROR:root:Returning 0 variables\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/http.py\", line 89, in send\n",
" fp = self.u2open(u2request, timeout=request.timeout)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/http.py\", line 140, in u2open\n",
" return url.open(u2request, timeout=tm)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 525, in open\n",
" response = meth(req, response)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 634, in http_response\n",
" response = self.parent.error(\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 563, in error\n",
" return self._call_chain(*args)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 496, in _call_chain\n",
" result = func(*args)\n",
" File \"/usr/lib/python3.10/urllib/request.py\", line 643, in http_error_default\n",
" raise HTTPError(req.full_url, code, msg, hdrs, fp)\n",
"urllib.error.HTTPError: HTTP Error 403: \n",
"\n",
"During handling of the above exception, another exception occurred:\n",
"\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 771, in send\n",
" reply = self.options.transport.send(request)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/https.py\", line 66, in send\n",
" return HttpTransport.send(self, request)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/transport/http.py\", line 106, in send\n",
" raise TransportError(e.msg, e.code, e.fp)\n",
"suds.transport.TransportError\n",
"\n",
"During handling of the above exception, another exception occurred:\n",
"\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/pywaterml/waterML.py\", line 309, in GetVariables\n",
" variables = self.client.service.GetVariables('[:]')\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 586, in __call__\n",
" return client.invoke(args, kwargs)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 728, in invoke\n",
" result = self.send(soapenv, timeout=timeout)\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 776, in send\n",
" return self.process_reply(content, e.httpcode, tostr(e))\n",
" File \"/usr/local/lib/python3.10/dist-packages/suds/client.py\", line 845, in process_reply\n",
" raise Exception((status, description))\n",
"Exception: (403, '')\n",
"\n",
"During handling of the above exception, another exception occurred:\n",
"\n",
"Traceback (most recent call last):\n",
" File \"/usr/local/lib/python3.10/dist-packages/pywaterml/waterML.py\", line 314, in GetVariables\n",
" soap_namespace = new_raws[1].split(\"xmlns\")[1].split(\"=\")[1]\n",
"IndexError: list index out of range\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"list indices must be integers or slices, not str\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MoeHbfokzByc"
},
"source": [
"<a name=\"GetSiteInfo\"></a>\n",
"### Site Information"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JeVHdm8gzTx1"
},
"source": [
"Retrieves the information of a given site. GetSiteInfo() function is similar to the GetSiteInfo() WaterML function.\n",
"\n",
"<em>Please note that the site used in this example is configured to use the SITE variable at the beginning. If you want to change the site, you need to change the SITE variable at the beginning of the code</em>"
]
},
{
"cell_type": "code",
"metadata": {
"id": "v4J-ZB9esVlp",
"outputId": "6b45c195-6e8f-407b-cd66-c468796108cd",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"try:\n",
" site_full_code = sites[SITE]['fullSiteCode']\n",
" siteInfo = water.GetSiteInfo(site_full_code)\n",
" df = pd.DataFrame.from_dict(siteInfo['siteInfo'])\n",
" print(df)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"0\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UyYmqDND2pDj"
},
"source": [
"<a name=\"GetValues\"></a>\n",
"### Time Series Values for Variables"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "l3I2khI92hFu"
},
"source": [
"Retrieves the values for an specific variable in a site. GetValues() function is similar to the GetValues() WaterML function.\n",
"\n",
"<em>Please note that the variable used in this example is configured to use the VARIABLE variable at the beginning. If you want to change the variable, you need to change the VARIABLE variable at the beginning of the code</em>"
]
},
{
"cell_type": "code",
"metadata": {
"id": "I4mlk7O1mn_s",
"outputId": "734aa610-ddac-49ed-84f7-c2cf56a00b4c",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"source": [
"try:\n",
" variable_full_code = siteInfo['siteInfo'][VARIABLE]['fullVariableCode']\n",
" start_date = siteInfo['siteInfo'][VARIABLE]['beginDateTime'].split('T')[0]\n",
" end_date = siteInfo['siteInfo'][VARIABLE]['endDateTime'].split('T')[0]\n",
" variableResponse= water.GetValues(site_full_code, variable_full_code, start_date, end_date)\n",
" df = pd.DataFrame.from_dict(variableResponse['values'])\n",
" print(df)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"name 'siteInfo' is not defined\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yETx0fg33IoJ"
},
"source": [
"The following is a code snipped for plotting the results of the GetValues function using the Plotly library.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "diZKW5aB5qQv"
},
"source": [
"try:\n",
" ##PLOT THE TIME SERIES\n",
" timeStamps=[]\n",
" valuesTimeSeries = []\n",
" for index in variableResponse['values']:\n",
" timeStamps.append(index['dateTimeUTC'])\n",
" valuesTimeSeries.append(index['dataValue'])\n",
"\n",
" fig = go.Figure(data=go.Scatter(x=timeStamps, y=valuesTimeSeries))\n",
" # Edit the layout\n",
" fig.update_layout(title = variableResponse['values'][VARIABLE]['variableName'],\n",
" xaxis_title =variableResponse['values'][VARIABLE]['timeUnitAbbreviation'],\n",
" yaxis_title = variableResponse['values'][VARIABLE]['unitAbbreviation'])\n",
" fig.show()\n",
"\n",
" df = pd.DataFrame(dict(\n",
" data=valuesTimeSeries\n",
" ))\n",
" fig = px.box(df,y=\"data\", points=\"all\")\n",
" fig.show()\n",
"\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "7_TeQpmbMEgx"
},
"source": [
"## Extra Functionality Methods"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0vjseuQkMOWe"
},
"source": [
"The following are extra functionality from the package, and it includes the following methods:\n",
"\n",
"1. GetSitesByBoxObject : Get all the sites from a bounding box from a endpoint that complies to the SOAP protocol. The GetSitesByBoxObject() function is similar to the GetSitesByBoxObject() WaterML function.\n",
"\n",
"2. GetSitesByVariable: Get the specific sites according to a variable search array from a endpoint that complies to the SOAP protocol. The GetSitesByVariable() is an addition to the WaterML functions because it allows the user to retrieve sites that contains the epecific site/s.\n",
"\n",
"3. GetInterpolation : Interpolates the data given by the GetValues function in order to fix datasets with missing values. Three ooptions for interpolation are offered: mean, backward, forward. The default is the mean interpolation.\n",
"\n",
"4. GetMonthlyAverage : Gets the monthly averages for a given variable, or from the response given by the GetValues function for a given site.\n",
"\n",
"5. GetClustersMonthlyAvg : Gets “n” number of clusters using dtw time series interpolation for a given variable.\n",
"\n",
"Please note that some functions such as GetSitesByVariable, GetMonthlyAverage, and GetClustersMonthlyAvg takes time depending in the number of sites that are in the WOF service\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HZ4vWfkLwonY"
},
"source": [
"<a name=\"GetSitesByBoxObject\"></a>\n",
"\n",
"### Sites By Bounding box"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6rORLbDkwvj7"
},
"source": [
"The GetSitesByBoxObject method retrieves meta data for each site from a given lat/lon bounding box in the WOF web service. Data can be retrieved in three different formats: JSON, CSV, and WaterML"
]
},
{
"cell_type": "code",
"metadata": {
"id": "bgdyHVK7m_wY"
},
"source": [
"try:\n",
" Bounds = m.get_bounds()\n",
" BoundsRearranged = [Bounds[0][1],Bounds[0][0],Bounds[1][1],Bounds[1][0]]\n",
" SitesByBoundingBox = water.GetSitesByBoxObject(BoundsRearranged,'epsg:4326')\n",
" df = pd.DataFrame.from_dict(SitesByBoundingBox)\n",
" print(df)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "jYrhXF_5P9Fn"
},
"source": [
"<a name=\"GetSitesByVariable\"></a>\n",
"### Sites Filtered by Variables"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Psyx8nz3P9Fp"
},
"source": [
"The GetSitesByVariable methods retrieves metadata from the sites that contains certain variables in the WOF web service\n",
"\n",
"<em>Please note that the variable used in this example is configured to use the VARIABLE variable at the beginning. If you want to change the variable, you need to change the VARIABLE variable at the beginning of the code</em>\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Kvi1i7o6P9Ft"
},
"source": [
"try:\n",
" variablesTest = [variables['variables'][0]['variableCode']]\n",
"\n",
" sitesFiltered = water.GetSitesByVariable(variablesTest,sites)\n",
" df = pd.DataFrame.from_dict(sitesFiltered['sites'])\n",
" print(df)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "7tayhz1L4ShT"
},
"source": [
"<a name=\"GetInterpolation\"></a>\n",
"### Interpolations"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "V0xGv07qmeAP"
},
"source": [
"Interpolates the data given by the GetValues function in order to fix datasets with missing values. Three ooptions for interpolation are offered: mean, backward, forward. The default is the mean interpolation"
]
},
{
"cell_type": "code",
"metadata": {
"id": "jW9hBijwoqUn"
},
"source": [
"try:\n",
" mean_interpolation = water.GetInterpolation(variableResponse,'mean')\n",
" backward_interpolation = water.GetInterpolation(variableResponse,'backward')\n",
" forward_interpolation = water.GetInterpolation(variableResponse,'forward')\n",
" df_mean = pd.DataFrame.from_dict(mean_interpolation)\n",
" df_back = pd.DataFrame.from_dict(backward_interpolation)\n",
" df_ford = pd.DataFrame.from_dict(forward_interpolation)\n",
"\n",
" print(\"INTERPOLATIONS\")\n",
" # print(df_back)\n",
" print(df_mean)\n",
" # print(df_ford)\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "cXUBcAuomsNP"
},
"source": [
"The following is a code snipped for plotting the results of the GetInterpolation method using the Plotly library."
]
},
{
"cell_type": "code",
"metadata": {
"id": "DOxyWtwyo0sl"
},
"source": [
"try:\n",
" ##PLOT THE TIME SERIES FOR INTERPOLATIONS\n",
" timeStamps=[]\n",
" valuesTimeSeries = []\n",
" timeStampsBack=[]\n",
" valuesTimeSeriesBackward = []\n",
" timeStampsFor=[]\n",
" valuesTimeSeriesForward = []\n",
"\n",
" for index in mean_interpolation:\n",
" timeStamps.append(index[0])\n",
" valuesTimeSeries.append(index[1])\n",
"\n",
" for index in backward_interpolation:\n",
" timeStampsBack.append(index[0])\n",
" valuesTimeSeriesBackward.append(index[1])\n",
"\n",
" for index in forward_interpolation:\n",
" timeStampsFor.append(index[0])\n",
" valuesTimeSeriesForward.append(index[1])\n",
"\n",
" fig = go.Figure(data=go.Scatter(x=timeStamps, y=valuesTimeSeries, mode='lines',\n",
" name='Mean Interpolation'))\n",
" fig.add_trace(go.Scatter(x=timeStampsBack, y=valuesTimeSeriesBackward,\n",
" mode='lines',\n",
" name='Backward Interpolation'))\n",
" fig.add_trace(go.Scatter(x=timeStampsFor, y=valuesTimeSeriesForward,\n",
" mode='lines',\n",
" name='Forward Interpolation'))\n",
"\n",
" # Edit the layout\n",
" fig.update_layout(title = variableResponse['values'][0]['variableName'],\n",
" xaxis_title =variableResponse['values'][0]['timeUnitAbbreviation'],\n",
" yaxis_title = variableResponse['values'][0]['unitAbbreviation'])\n",
" fig.show()\n",
" ##ONLY MEAN INTERPOLATION WHISKER AND BOX PLOT\n",
"\n",
" df = pd.DataFrame(dict(\n",
" data=valuesTimeSeries\n",
" ))\n",
" fig = px.box(df,y=\"data\", points=\"all\")\n",
" fig.show()\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "VP69FShLm4wE"
},
"source": [
"<a name=\"GetMonthlyAverage\"></a>\n",
"### Monthly Averages for Variables"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BkGKWVn9m4ik"
},
"source": [
"Gets the monthly averages for a given variable, or from the response given by the GetValues function for a given site."
]
},
{
"cell_type": "code",
"metadata": {
"id": "3hu3bUo-uGtO"
},
"source": [
"try:\n",
" ##CALCULATING THE MONTHLY AVERAGES\n",
" m_avg = water.GetMonthlyAverage(None, site_full_code, variable_full_code, start_date, end_date)\n",
"\n",
" data = {'Months': ['Jan', 'Feb', 'Mar', 'Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'],\n",
" 'Monthly Average': m_avg,\n",
" }\n",
" df = pd.DataFrame (data, columns = ['Months','Monthly Average'])\n",
"\n",
" print (df)\n",
"except Exception as e:\n",
" print(e)\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "a6iV1Zl5oE45"
},
"source": [
"<a name=\"GetClustersMonthlyAvg\"></a>\n",
"\n",
"### Time Series Clusters from Monthly Averages"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nlBxm9wroHy9"
},
"source": [
"Gets “n” number of clusters using dtw time series interpolation for a given variable."
]
},
{
"cell_type": "code",
"metadata": {
"id": "KVT75Uk1_OfC"
},
"source": [
"try:\n",
" ## Calculating Clustering\n",
" Clusters = water.GetClustersMonthlyAvg(sites,siteInfo['siteInfo'][VARIABLE]['variableCode'])\n",
" print(Clusters)\n",
"\n",
"except Exception as e:\n",
" print(e)\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "zc_NJkEzoNgi"
},
"source": [
"The following is extra code to create a list of colors and plot the different clusters in a follium map."
]
},
{
"cell_type": "code",
"metadata": {
"id": "GbDmMvBPrRD7"
},
"source": [
"try:\n",
" ## Function to select the color for the cluster.\n",
" def colorToCluster(cluster):\n",
" colors = [\n",
" 'red',\n",
" 'blue',\n",
" 'green',\n",
" 'pink',\n",
" 'purple',\n",
" 'orange',\n",
" ]\n",
" return colors[cluster]\n",
"\n",
" ## Looking in the Map the different clusters\n",
" m = folium.Map(location=[40.0150, -105.2705],\n",
" tiles = 'Stamen Terrain')\n",
" sitesLocations = []\n",
" for site,singlecluster in zip(sitesFiltered['sites'],Clusters):\n",
" sitesLocations.append([float(site['latitude']) , float(site['longitude'])])\n",
" folium.Marker(\n",
" location= [float(site['latitude']) , float(site['longitude'])],\n",
" # popup= sites[\"name\"], # pop-up label for the marker\n",
" icon=folium.Icon(color=colorToCluster(singlecluster[1]))\n",
" ).add_to(m)\n",
" # Display m\n",
" m.fit_bounds(sitesLocations)\n",
" m\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "kFTcLsWcoZXR"
},
"source": [
"The following is extra code to plot the time series for the different clusters in the WOF web service"
]
},
{
"cell_type": "code",
"metadata": {
"id": "inFVIaDnNipE"
},
"source": [
" try:\n",
" m_arr = [\"Jan\", \"Feb\", \"Mar\", \"Apr\", \"May\", \"Jun\", \"Jul\",\"Aug\", \"Sep\", \"Oct\", \"Nov\", \"Dec\"]\n",
" fig = go.Figure(data=go.Scatter(x = m_arr, y = Clusters[0][0], mode='lines',\n",
" name='1'))\n",
" for index in range (1, len(Clusters)):\n",
" if Clusters[index][1] == 0:\n",
" fig.add_trace(go.Scatter(x=m_arr, y=Clusters[index][0],\n",
" mode='lines',\n",
" name=str(index)))\n",
"\n",
"\n",
" # Edit the layout\n",
" fig.update_layout(title = \"Graph for First Cluster\",\n",
" xaxis_title ='Months',\n",
" yaxis_title = variableResponse['values'][VARIABLE]['unitAbbreviation'])\n",
" fig.show()\n",
" fig = go.Figure(data=go.Scatter(x = m_arr, y = Clusters[0][0], mode='lines',\n",
" name='1'))\n",
" for index in range (1, len(Clusters)):\n",
" if Clusters[index][1] == 1:\n",
" fig.add_trace(go.Scatter(x=m_arr, y=Clusters[index][0],\n",
" mode='lines',\n",
" name=str(index)))\n",
"\n",
"\n",
" # Edit the layout\n",
" fig.update_layout(title = \"Graph for Second Cluster\",\n",
" xaxis_title ='Months',\n",
" yaxis_title =variableResponse['values'][VARIABLE]['unitAbbreviation'])\n",
" fig.show()\n",
" fig = go.Figure(data=go.Scatter(x = m_arr, y = Clusters[0][0], mode='lines',\n",
" name='1'))\n",
" for index in range (1, len(Clusters)):\n",
" if Clusters[index][1] == 2:\n",
" fig.add_trace(go.Scatter(x=m_arr, y=Clusters[index][0],\n",
" mode='lines',\n",
" name=str(index)))\n",
"\n",
"\n",
" # Edit the layout\n",
" fig.update_layout(title = \"Graph for Third Cluster\",\n",
" xaxis_title ='Months',\n",
" yaxis_title = variableResponse['values'][VARIABLE]['unitAbbreviation'])\n",
" fig.show()\n",
"except Exception as e:\n",
" print(e)"
],
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment