Skip to content

Instantly share code, notes, and snippets.

@CamonZ
Created January 8, 2017 03:41
Show Gist options
  • Save CamonZ/df07da5fcf3f3443ea36ddce50f539b8 to your computer and use it in GitHub Desktop.
Save CamonZ/df07da5fcf3f3443ea36ddce50f539b8 to your computer and use it in GitHub Desktop.
1. Download CSV with rasters you want from USGS.
2. Convert CSV to JSON with something like http://www.csvjson.com/
3. Run script with `ruby usgs_downloader.rb path_to_json_file`
4. Sit back.
5. ???
6. Receive slope analysis
#! /usr/bin/env ruby
require 'json'
class USGSDownloader
def self.process(filename='')
if filename.empty?
puts "You must pass the path to a json file as argument"
return -1
end
unless File.exists?(filename)
puts "Source file does not exists"
return -1
end
new.process(rasters_metadata(filename))
end
def self.rasters_metadata(filename)
JSON.parse(File.read(filename))
end
def process(metadatas)
puts "Downloading #{metadatas.length} rasters\n\n"
metadatas.each do |meta|
download_raster(meta)
unzip_raster(meta)
reproject_raster
analyze_reprojection
delete_zip(meta)
delete_tempdir_contents
end
end
private
def download_raster(meta)
`wget #{download_url(meta)}`
end
def unzip_raster(meta)
`unzip #{zip_filename(meta)} -d #{tempdir}`
end
def delete_zip(meta)
`rm #{zip_filename(meta)}`
end
def reproject_raster
puts "Reprojecting raster for: #{raster_name}"
`gdalwarp -s_srs EPSG:4269 -t_srs EPSG:4326 #{raster_filename} #{reprojected_filename}`
end
def analyze_reprojection
puts "Analyzing raster for: #{raster_name}"
`gdaldem slope #{reprojected_filename} #{results_dir}/#{raster_name}_slope.tif`
end
def delete_tempdir_contents
`rm -r #{tempdir}/*`
end
def tempdir
Dir.mkdir('/tmp/usgs_dataset') unless Dir.exists?('/tmp/usgs_dataset')
'/tmp/usgs_dataset'
end
def download_url(meta)
meta["downloadURL"]
end
def results_dir
Dir.mkdir("#{ENV['PWD']}/analysis") unless Dir.exists?("#{ENV['PWD']}/analysis")
"#{ENV['PWD']}/analysis"
end
def raster_filename
Dir.glob("#{tempdir}/*.img")[0]
end
def reprojected_filename
"#{tempdir}/#{raster_name}_4326.tif"
end
def raster_name
Regexp.new(/(n\d{2}w\d{3})/).match(raster_filename)[1]
end
def zip_filename(meta)
Regexp.new(/IMG\/(.+\.zip)$/).match(download_url(meta))[1]
end
def raster_filename
Dir.glob("/tmp/usgs_dataset/*.img")[0]
end
end
USGSDownloader.process(ARGV[0])
[
{
"sourceId": "581d213ae4b08da350d52d30",
"extent": "1 x 1 degree",
"sourceOriginName": "gda",
"boundingBox": "{minY:30.9994444449061,minX:-90.0005555551941,maxY:32.0005555552955,maxX:-88.9994444448046}",
"sizeInBytes": 376281338,
"bestFitIndex": 0,
"format": "IMG",
"downloadURL": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/13/IMG/USGS_NED_13_n32w090_IMG.zip",
"previewGraphicURL": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/13/IMG/USGS_NED_13_n32w090_IMG_thumb.jpg",
"prettyFileSize": "358.85 MB",
"datasets": "[National Elevation Dataset (NED) 1/3 arc-second,National Elevation Dataset (NED)]",
"title": "USGS NED 1/3 arc-second n32w090 1 x 1 degree IMG 2016",
"sourceOriginId": 8539703,
"lastUpdated": "2016-11-12",
"dateCreated": "2016-11-04",
"metaUrl": "https://www.sciencebase.gov/catalog/item/581d213ae4b08da350d52d30",
"sourceName": "ScienceBase",
"publicationDate": "2016-10-07"
},
{
"sourceId": "581d213ce4b08da350d52d89",
"extent": "1 x 1 degree",
"sourceOriginName": "gda",
"boundingBox": "{minY:31.9994444447062,minX:-117.000555555194,maxY:33.0005555550957,maxX:-115.999444444805}",
"sizeInBytes": 153988106,
"bestFitIndex": 0,
"format": "IMG",
"downloadURL": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/13/IMG/USGS_NED_13_n33w117_IMG.zip",
"previewGraphicURL": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/13/IMG/USGS_NED_13_n33w117_IMG_thumb.jpg",
"prettyFileSize": "146.85 MB",
"datasets": "[National Elevation Dataset (NED) 1/3 arc-second,National Elevation Dataset (NED)]",
"title": "USGS NED 1/3 arc-second n33w117 1 x 1 degree IMG 2016",
"sourceOriginId": 8268718,
"lastUpdated": "2016-11-12",
"dateCreated": "2016-11-04",
"metaUrl": "https://www.sciencebase.gov/catalog/item/581d213ce4b08da350d52d89",
"sourceName": "ScienceBase",
"publicationDate": "2016-07-11"
}
]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment