geoglows_package_tutorial.ipynb

geoglows_package_tutorial.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "geoglows_package_tutorial.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/rileyhales/873896e426a5bd1c4e68120b286bc029/geoglows_package_tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gLu_P8v5M2kb"
      },
      "source": [
        "# GEOGloWS ECMWF Streamflow\n",
        "The GEOGloWS ECMWF Streamflow model is a global hydrologic model driven by the HTESSEL land surface model. It produces new 15 day streamflow forecasts each day at midnight (UTC +0). It also has a 40+ year (and growing) historical simulation of streamflow.\n",
        "\n",
        "The GEOGloWS ECMWF Streamflow Data Service (REST API) and this python package client were developed at Brigham Young University in the Civil and Environmental Engineering Department by Riley Hales, Rohit Khatar, Chris Edwards, Kyler Ashby, Gio Romero, and others. This project is an axpansion and enhancement of the original work by Dr Jim Nelson and Dr Michael Suffront with funding from GEOGloWS, ECMWF, NASA, The World Bank, Microsoft Azure, BYU, and others.\n",
        "\n",
        "You can interact with the streamflow model using the geoglows python package. This notebook will take you through some of the functions available. For more information, please refer to https://geoglows.readthedocs.io."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uSd2bp8N4Vt-"
      },
      "source": [
        "# Start by installing the package and importing it to your code. Run this cell to do that.\n",
        "!pip install geoglows -q\n",
        "import geoglows\n",
        "from IPython.core.display import display, HTML\n",
        "from google.colab import files"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FAfqQDX14VuA"
      },
      "source": [
        "# Access the Streamflow Data Service\n",
        "\n",
        "Data in the GEOGloWS Model is organized by the numeric ID assigned to each stream segment in the model. This is refered to as the \"reach_id\" in the following code. If you need help finding this number, use the graphical tool on the home page of https://geoglows.ecmwf.int or view the tutorial on identifying reach ID's."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lqjPKeL6a8IM"
      },
      "source": [
        "# Pick the ID of a river. This ID is on the Mississippi River\n",
        "reach_id = 13081955"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OetLg2v7cSKj"
      },
      "source": [
        "## Forecasted streamflow products\n",
        "There are 3 Forecasted Streamflow products\n",
        "\n",
        "- **Forecast Ensembles** returns a csv containing 52, 15-day time series of discharge.\n",
        "- **Forecast Stats** returns a csv with a 15-day time series for the Max, Min, 25 & 75 percentile, and average of the ensemble members.\n",
        "- **Forecast Records** is a growing record of the previous forecast predictions. Each day the flow predictions for the first 24 hours are appended to the forecast record of each stream.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fgb13_rRa6q4"
      },
      "source": [
        "# 3 forecasted streamflow products:\n",
        "ensembles = geoglows.streamflow.forecast_ensembles(reach_id)\n",
        "stats = geoglows.streamflow.forecast_stats(reach_id)\n",
        "records = geoglows.streamflow.forecast_records(reach_id)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bfolP2PecyYz"
      },
      "source": [
        "## Historical simulation Products\n",
        "There are 4 historical streamflow products\n",
        "- **Historic Simulation** provides a timeseries of daily average streamflow since 1979. This is based on the ERA-5 historical data product. It is updated at the beginning of each year when the ERA-5 data for the previous year is available.\n",
        "- **Return Periods** estimates the 2, 5, 10, 20, 50, and 100 year return period for the river based on the historic simulation on the river and the Gumbel Type 2 distribution of flood events.\n",
        "- **Daily Averages** returns a times eries with 366 entries, one for each day of the year including leap day. This is the average simulated flow on each day of the year and is a view of seasonal trends on that river. \n",
        "- **Monthly Averages** returns a time series with 12 entries, one for each month of the year. This is the average of all simulated daily values for that month. This is another view of seasonal trends."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zwGEdm0X4VuA"
      },
      "source": [
        "hist = geoglows.streamflow.historic_simulation(reach_id)\n",
        "rperiods = geoglows.streamflow.return_periods(reach_id)\n",
        "day_avg = geoglows.streamflow.daily_averages(reach_id)\n",
        "mon_avg = geoglows.streamflow.monthly_averages(reach_id)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Syd_HueA4VuC"
      },
      "source": [
        "# Visualizing data\n",
        "You just gathered forecasted and historical simulation streamflow data for the river you chose by latitude and longitude. While this is useful for many applications, a good place to start is with visualizations. \n",
        "\n",
        "The geoglows package has tools for graphing all the data you can get from the GEOGloWS ECMWF Streamflow model. Each of the following cells will turn the data you requested into a Plotly figure and then show it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3am0DtCr_zFQ"
      },
      "source": [
        "# The title dictionary allows you to add additional information to the title block of the resulting graphs\n",
        "title = {'Reach ID': reach_id}"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lcs2HdCL4VuF"
      },
      "source": [
        "# Statistical summary of the forecasted flows\n",
        "forecast_figure = geoglows.plots.forecast_stats(stats, rperiods, titles=title)\n",
        "forecast_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Aj9rtmUf4VuH"
      },
      "source": [
        "# View the previously saved forecasts\n",
        "records_figure = geoglows.plots.forecast_records(records, rperiods, titles=title)\n",
        "records_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "HrE4htUw4VuI"
      },
      "source": [
        "# View each of the forecasts individually\n",
        "ensembles_figure = geoglows.plots.forecast_ensembles(ensembles, rperiods, titles=title)\n",
        "ensembles_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WJlBhbA24VuK"
      },
      "source": [
        "# Historically simulated flow (ERA-5)\n",
        "historic_figure = geoglows.plots.historic_simulation(hist, rperiods, titles=title)\n",
        "historic_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m_82tcZU4VuM"
      },
      "source": [
        "# Processing the historical data into a daily average flow\n",
        "day_figure = geoglows.plots.daily_averages(day_avg, titles=title)\n",
        "day_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2ty7bSg_LrtP"
      },
      "source": [
        "# Processing the historical data into a monthly average flow\n",
        "mon_figure = geoglows.plots.monthly_averages(mon_avg, titles=title)\n",
        "mon_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pq2vzug04VuP"
      },
      "source": [
        "# Flow Duration Curve (derived from the ERA-5 data)\n",
        "flow_duration_figure = geoglows.plots.flow_duration_curve(hist, titles=title)\n",
        "flow_duration_figure.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N6-m_5w34VuR"
      },
      "source": [
        "# View the probabilities table\n",
        "prob_table = geoglows.plots.probabilities_table(stats, ensembles, rperiods)\n",
        "display(HTML(prob_table))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e49Lj4bA4VuT"
      },
      "source": [
        "# View the probabilities table\n",
        "rperiods_table = geoglows.plots.return_periods_table(rperiods)\n",
        "display(HTML(rperiods_table))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B7_Oy_9ONxQ1"
      },
      "source": [
        "## Saving data\n",
        "\n",
        "The results from the GEOGloWS Model are stored as DataFrames using the `pandas` package. You can use the `.to_csv()` method of `pandas` DataFrames to save your resulting information to a csv file or use other similar methods. From the Google Collaboratory environment, this can be downloaded to your personal computer using the `files.download` command.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O3-OqJNvK5-O"
      },
      "source": [
        "# Saving DataFrame to CSV file and downloading from the Google notebook instance\n",
        "stats.to_csv('stats.csv')\n",
        "files.download('stats.csv')\n",
        "\n",
        "hist.to_csv('hist.csv')\n",
        "files.download('hist.csv')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AK4ArWAo4VuU"
      },
      "source": [
        "# What's next?\n",
        "Go learn more about this tool at http://geoglows.readthedocs.io and more about GEOGloWS at https://www.geoglows.org."
      ]
    }
  ]
}

Hi Dr. Hales,

The visualization functions, such as:

# Statistical summary of the forecasted flows
forecast_figure = geoglows.plots.forecast_stats(stats, rperiods, titles=title)
forecast_figure.show()

appear not to be working correctly at the moment. Request the minor bug to be fixed.

Sharing some error msgs on the above code snippet below:

AttributeError                            Traceback (most recent call last)
Cell In[21], [line 2](vscode-notebook-cell:?execution_count=21&line=2)
      [1](vscode-notebook-cell:?execution_count=21&line=1) # View the previously saved forecasts
----> [2](vscode-notebook-cell:?execution_count=21&line=2) records_figure = geoglows.plots.forecast_records(records, rperiods, titles=title)
      [3](vscode-notebook-cell:?execution_count=21&line=3) records_figure.show()

AttributeError: module 'geoglows._plots' has no attribute 'forecast_records'

---

AttributeError                            Traceback (most recent call last)
Cell In[30], [line 2](vscode-notebook-cell:?execution_count=30&line=2)
      [1](vscode-notebook-cell:?execution_count=30&line=1) # View the previously saved forecasts
----> [2](vscode-notebook-cell:?execution_count=30&line=2) records_figure = geoglows.plots.forecast_records(records, rperiods)
      [3](vscode-notebook-cell:?execution_count=30&line=3) records_figure.show()

AttributeError: module 'geoglows._plots' has no attribute 'forecast_records'