How to analyze time series using TSTool software (overview)

-owf-howto-tstool-analyze-timeseries.md

How to Analyze Time Series using TSTool Software

by Steve Malers, Open Water Foundation, last updated 2016-02-16

Index of Open Water Foundation gists

Summary

This gist provides a document explaining how to analyze time series using TSTool software. In this case time series are related to hydrology. Rather than repeat information, it is recommended that the documentation be reviewed. TSTool command files corresponding to the documentation are provided for download. Consistent with gist conventions, all files are located in the same folder, although in a normal data processing workflow an appropriate use of folders and naming convention would be implemented to manage input, processing, and results.

Prerequisites

1. The example requires installation of the TSTool software
2. A basic understanding of the TSTool software is helpful.
3. It is assumed that a "real" text editor is installed to edit files, for example Notepad++.

Contents

Files in this gist used for input are as follows:

• -owf-howto-tstool-analyze-timeseries.md - example summary (this file)
• HowToAnalyzeTimeSeriesUsingTSTool.docx (and pdf) - documentation that forms the bulk of the information presented here
• *.TSTool - TSTool command files to run examples
• .gitattributes and .gitignore - these are used to manage files in the repository

.gitattributes

#common settings that generally should always be used with your language specific settings

# Auto detect text files and perform LF normalization
# http://davidlaing.com/2012/09/19/customise-your-gitattributes-to-become-a-git-ninja/
* text=auto

#
# The above will handle all files NOT found below
#

# Documents
*.doc	 diff=astextplain
*.DOC	 diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot  diff=astextplain
*.DOT  diff=astextplain
*.pdf  diff=astextplain
*.PDF	 diff=astextplain
*.rtf	 diff=astextplain
*.RTF	 diff=astextplain
*.md text
*.adoc text
*.textile text
*.mustache text
*.csv text
*.tab text
*.tsv text
*.sql text
*.TSTool text

# Database
*.dbf binary

# Graphics
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.tif binary
*.tiff binary
*.ico binary
*.svg binary
*.eps binary

.gitignore

# Ignore files in repository
!.gitignore
*.log
*.dv
*.xlsx
*.kml

01-RetrievePrecipitationFromACIS-initial.TSTool

# Read all daily precipitation time series for the full historical period
# - reading all time series for Larimer County and/or for full period works but is slow
# - instead, read a subset for example a shorter period, or Summit County, which has less stations
# ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08069 - Larimer, CO",InputStart="2010-01-01",InputEnd="2014-02-14")
ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08117 - Summit, CO")
# Save the precipitation time series to a local DateValue file.
# This file can be read by other command files for faster processing.
# A local copy of time series data also is a data archive to document work.
WriteDateValue(OutputFile="precip.dv")

01-RetrievePrecipitationFromACIS-withKML-outputTable.TSTool

# Read all daily precipitation time series for the full historical period
# - reading all time series for Larimer County and/or for full period works but is slow
# - instead, read a subset for example a shorter period, or Summit County, which has less stations
# ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08069 - Larimer, CO",InputStart="2010-01-01",InputEnd="2014-02-14")
ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08117 - Summit, CO")
# Save the precipitation time series to a local DateValue file.
# This file can be read by other command files for faster processing.
# A local copy of time series data also is a data archive to document work.
WriteDateValue(OutputFile="precip.dv")
# Create a KML with station coordinates in order to visualize station locations in Google Earth
WriteTimeSeriesToKml(OutputFile="precip-stations.kml",Name="Precipitation Stations",Description="Precipitation stations from RCC ACIS",LongitudeProperty="longitude",LatitudeProperty="latitude")
# Create a table containing time series properties and save to Excel.
# The table can also be written to a CSV file if Excel is not available.
CopyTimeSeriesPropertiesToTable(TableID="PrecipitationStations",TableTSIDColumn="TSID",TableTSIDFormat="%L")
NewExcelWorkbook(OutputFile="PrecipitationStations.xlsx",Worksheets="PrecipitationStations",KeepOpen=True)
WriteTableToExcel(TableID="PrecipitationStations",OutputFile="PrecipitationStations.xlsx",ExcelAddress="A1",ExcelColumnNames=FirstRowInRange,KeepOpen=False,ColumnWidths="Default:Auto")

01-RetrievePrecipitationFromACIS-withKML.TSTool

# Read all daily precipitation time series for the full historical period
# - reading all time series for Larimer County and/or for full period works but is slow
# - instead, read a subset for example a shorter period, or Summit County, which has less stations
# ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08069 - Larimer, CO",InputStart="2010-01-01",InputEnd="2014-02-14")
ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08117 - Summit, CO")
# Save the precipitation time series to a local DateValue file.
# This file can be read by other command files for faster processing.
# A local copy of time series data also is a data archive to document work.
WriteDateValue(OutputFile="precip.dv")
# Create a KML with station coordinates in order to visualize station locations in Google Earth
WriteTimeSeriesToKml(OutputFile="precip-stations.kml",Name="Precipitation Stations",Description="Precipitation stations from RCC ACIS",LongitudeProperty="longitude",LatitudeProperty="latitude")

01-RetrievePrecipitationFromACIS.TSTool

# Read all daily precipitation time series for the full historical period
# - reading all time series for Larimer County and/or for full period works but is slow
# - instead, read a subset for example a shorter period, or Summit County, which has less stations
# ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08069 - Larimer, CO",InputStart="2010-01-01",InputEnd="2014-02-14")
# ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08069 - Larimer, CO",InputStart="2015-01-01",InputEnd="2016-02-14")
ReadRccAcis(DataStore="RCC-ACIS",DataType="pcpn",Interval="Day",Where1="FIPS County;Matches;08117 - Summit, CO")
# Save the precipitation time series to a local DateValue file.
# This file can be read by other command files for faster processing.
# A local copy of time series data also is a data archive to document work.
WriteDateValue(OutputFile="precip.dv")

02-ReadDateValueFile-AnalyzePrecipitation.TSTool

# This example shows how to read a previously downloaded DateValue file as input and then analyze
# Processing is faster because the data do not need to be downloaded again
# Read the precipitation time series previously retrieved from ACIS
ReadDateValue(InputFile="precip.dv")
# Cumulate the precipitation starting on Oct 1 to represent water year precipitation
Cumulate(Reset="10-01")

03-ReadStationsFromExcelList-RetrieveTimeSeries-AnalyzePrecipitation-cumulate.TSTool

# Read the list of time series from the previously created Excel file, which contains the list of precipitation stations
ReadTableFromExcel(TableID="PrecipitationStations",InputFile="PrecipitationStations.xlsx",Worksheet="PrecipitationStations",ExcelColumnNames=FirstRowInRange,ColumnExcludeFilters="TSID:",Comment="#",ExcelDateTimeColumns="valid_daterange1,valid_daterage2")
# Read the corresponding time series from RCC-ACIS web services
ReadTimeSeriesList(TableID="PrecipitationStations",LocationColumn="IDWithLocType",DataType="pcpn",Interval="Day",DataStore="RCC-ACIS",Alias="%L-%T",IfNotFound=Warn)
# Cumulate the precipitation starting on Oct 1 to represent water year precipitation
Cumulate(Reset="10-01")

03-ReadStationsFromExcelList-RetrieveTimeSeries.TSTool

# Read the list of time series from the previously created Excel file, which contains the list of stations
ReadTableFromExcel(TableID="PrecipitationStations",InputFile="PrecipitationStations.xlsx",Worksheet="PrecipitationStations",ExcelColumnNames=FirstRowInRange,ColumnExcludeFilters="TSID:",Comment="#",ExcelDateTimeColumns="valid_daterange1,valid_daterage2")
# Read the corresponding time series from RCC-ACIS web services
ReadTimeSeriesList(TableID="PrecipitationStations",LocationColumn="IDWithLocType",DataType="pcpn",Interval="Day",DataStore="RCC-ACIS",Alias="%L-%T",IfNotFound=Warn)

HowToAnalyzeTimeSeriesUsingTSToolSoftware.docx

HowToAnalyzeTimeSeriesUsingTSToolSoftware.pdf