Skip to content

Instantly share code, notes, and snippets.

@sckott
Last active August 29, 2015 14:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sckott/440222f040a3444057ad to your computer and use it in GitHub Desktop.
Save sckott/440222f040a3444057ad to your computer and use it in GitHub Desktop.

The NOAA NCDC API is great (avail in ncdc*() functions in rnoaa), but it has a relatively low rate limit (Default: 1000 per day, can ask for up to 10K per day limit). If you need a lot of data, you may need to take a different approach. One way to tackle this is via their FTP server. The isd() and related functions work with the NOAA FTP server.

Here's an example of how you can get data from ISD.

NOTE: ISD data is let's just say, not fun to parse, so it still needs more work, but is somewhat operational for now.

load package

library("rnoaa")

Get station ids

stations <- isd_stations()
head(stations)
#>   usaf  wban station_name ctry state icao lat lon elev_m    begin      end
#> 1 7005 99999   CWOS 07005                  NA  NA     NA 20120127 20120127
#> 2 7011 99999   CWOS 07011                  NA  NA     NA 20111025 20121129
#> 3 7018 99999   WXPOD 7018                   0   0   7018 20110309 20130730
#> 4 7019 99999   CWOS 07019                  NA  NA     NA 20130625 20130703
#> 5 7025 99999   CWOS 07025                  NA  NA     NA 20120127 20120127
#> 6 7026 99999   WXPOD 7026   AF              0   0   7026 20120713 20141120

Probably use ncdc_stations() to find stations, then search for Ids in the FTP station ids data.frame

ncdc_stations(locationid='ZIP:44720') # let's target row 5, id='GHCND:USW00014895'
#> $meta
#> $meta$totalCount
#> [1] 8
#> 
#> $meta$pageCount
#> [1] 25
#> 
#> $meta$offset
#> [1] 1
#> 
#> 
#> $data
#>                  id elevation                                 name
#> 1       COOP:330058     368.2 AKRON CANTON REGIONAL AIRPORT, OH US
#> 2       COOP:331254     345.0                    CANTON 5 N, OH US
#> 3 GHCND:USC00331254     345.0                    CANTON 5 N, OH US
#> 4 GHCND:USC00335750     335.3                    NEW BERLIN, OH US
#> 5 GHCND:USW00014895     368.2 AKRON CANTON REGIONAL AIRPORT, OH US
#> 6       NEXRAD:KCLE     232.6                     CLEVELAND, OH US
#> 7       NEXRAD:KPBZ     361.2                    PITTSBURGH, PA US
#> 8        WBAN:14895     368.2 AKRON CANTON REGIONAL AIRPORT, OH US
#>   elevationUnit datacoverage longitude    mindate latitude    maxdate
#> 1        METERS       0.9861 -81.43333 1948-07-01 40.91667 2014-07-01
#> 2        METERS       1.0000 -81.40000 1893-07-01 40.86667 1948-09-01
#> 3        METERS       1.0000 -81.40000 1948-05-01 40.86667 1948-09-30
#> 4        METERS       1.0000 -81.45000 1893-01-01 40.90000 1913-08-31
#> 5        METERS       1.0000 -81.43333 1948-07-01 40.91667 2014-12-30
#> 6        METERS       0.9500 -81.86000 1995-01-05 41.41306 2015-01-01
#> 7        METERS       0.9500 -80.21833 1995-01-19 40.53167 2015-01-01
#> 8        METERS       1.0000 -81.43333 1948-07-01 40.91667 2014-12-31
#> 
#> attr(,"class")
#> [1] "ncdc_stations"

Find stations, e.g.

stations[grep("14895", stations$wban),]
#>         usaf  wban                  station_name ctry state icao    lat
#> 19837 725210 14895 AKRON-CANTON REGIONAL AIRPORT   US    OH KCAK 40.917
#> 28487 999999 14895    AKRON AKRON-CANTON AIRPORT   US    OH KCAK 40.917
#>           lon elev_m    begin      end
#> 19837 -81.433  368.2 19730101 20141209
#> 28487 -81.433  376.7 19480701 19721231

Get data

(res <- isd(usaf="725210", wban="14895", year=1986))
#> <ISD Data>
#> Size: 10358 X 79
#> 
#>    total_chars usaf_station wban_station     date time date_flag latitude
#> 1         0233       725210        14895 19860101 0000         C   +40917
#> 2         0083       725210        14895 19860101 0100         D   +40917
#> 3         0083       725210        14895 19860101 0200         D   +40917
#> 4         0170       725210        14895 19860101 0300         D   +40917
#> 5         0098       725210        14895 19860101 0400         D   +40917
#> 6         0087       725210        14895 19860101 0437         4   +40917
#> 7         0120       725210        14895 19860101 0500         D   +40917
#> 8         0109       725210        14895 19860101 0520         4   +40917
#> 9         0342       725210        14895 19860101 0600         C   +40917
#> 10        0265       725210        14895 19860101 0700         C   +40917
#> ..         ...          ...          ...      ...  ...       ...      ...
#> Variables not shown: longitude (chr), type_code (chr), elevation (chr),
#>      call_letter (chr), quality (chr), wind_direction (chr),
#>      wind_direction_quality (chr), wind_code (chr), wind_speed (chr),
#>      wind_speed_quality (chr), ceiling_height (chr),
#>      ceiling_height_quality (chr), ceiling_height_determination (chr),
#>      ceiling_height_cavok (chr), visibility_distance (chr),
#>      visibility_distance_quality (chr), visibility_code (chr),
#>      visibility_code_quality (chr), temperature (chr), temperature_quality
#>      (chr), temperature_dewpoint (chr), temperature_dewpoint_quality
#>      (chr), air_pressure (chr), air_pressure_quality (chr), REM.remarks
#>      (chr), REM.identifier (chr), REM.length_quantity (chr), REM.comment
#>      (chr), AG1.precipitation (chr), AG1.discrepancy (chr),
#>      AG1.est_water_depth (chr), GF1.sky_condition (chr), GF1.coverage
#>      (chr), GF1.opaque_coverage (chr), GF1.coverage_quality (chr),
#>      GF1.lowest_cover (chr), GF1.lowest_cover_quality (chr),
#>      GF1.low_cloud_genus (chr), GF1.low_cloud_genus_quality (chr),
#>      GF1.lowest_cloud_base_height (chr),
#>      GF1.lowest_cloud_base_height_quality (chr), GF1.mid_cloud_genus
#>      (chr), GF1.mid_cloud_genus_quality (chr), GF1.high_cloud_genus (chr),
#>      GF1.high_cloud_genus_quality (chr), KA1.extreme_temp (chr),
#>      KA1.period_quantity (chr), KA1.max_min (chr), KA1.temp (chr),
#>      KA1.temp_quality (chr), EQD.observation_identifier (chr),
#>      EQD.observation_text (chr), EQD.reason_code (chr), EQD.parameter
#>      (chr), MD1.atmospheric_change (chr), MD1.tendency (chr),
#>      MD1.tendency_quality (chr), MD1.three_hr (chr), MD1.three_hr_quality
#>      (chr), MD1.twentyfour_hr (chr), MD1.twentyfour_hr_quality (chr),
#>      MW1.first_weather_reported (chr), MW1.condition (chr),
#>      MW1.condition_quality (chr), EQD.observation_identifier.1 (chr),
#>      EQD.observation_text.1 (chr), EQD.reason_code.1 (chr),
#>      EQD.parameter.1 (chr), EQD.observation_identifier.2 (chr),
#>      EQD.observation_text.2 (chr), EQD.reason_code.2 (chr),
#>      EQD.parameter.2 (chr)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment