Skip to content

Instantly share code, notes, and snippets.

@nateklaiber
Last active August 29, 2015 14:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nateklaiber/40d5b614e9e5f76a84e7 to your computer and use it in GitHub Desktop.
Save nateklaiber/40d5b614e9e5f76a84e7 to your computer and use it in GitHub Desktop.
Dover Utilities Electric Usage Information
API_HOST='http://utilities.theklaibers.com'
CACHE_HOST='localhost:11211'
/log/*.log
/.ruby-version
/.ruby-gemset
/.env
electric-usage-information
source 'https://rubygems.org'
gem 'faraday'
gem 'faraday_middleware', git: "https://github.com/lostisland/faraday_middleware.git"
gem 'terminal-table'
gem 'dotenv'
gem 'multi_json'
gem 'memcached'
gem 'faraday-http-cache'
GIT
remote: https://github.com/lostisland/faraday_middleware.git
revision: 9a49b369c39cbef5525f4fda7e01cf8f5eb09e24
specs:
faraday_middleware (0.9.0)
faraday (>= 0.7.4, < 0.10)
GEM
remote: https://rubygems.org/
specs:
dotenv (1.0.2)
faraday (0.9.1)
multipart-post (>= 1.2, < 3)
faraday-http-cache (0.4.2)
faraday (~> 0.8)
memcached (1.8.0)
multi_json (1.10.1)
multipart-post (2.0.0)
terminal-table (1.4.5)
PLATFORMS
ruby
DEPENDENCIES
dotenv
faraday
faraday-http-cache
faraday_middleware!
memcached
multi_json
terminal-table

Electric Usage Information

I publish our electric usage information for visualizations. I recently prepared this in accordance with a rate increase.

I wanted to display:

  • A sortable table of the data.
  • A simple cost over time visualization of the data.

This is comprised of a few things:

  • HTML rendering of the page itself.
  • Usage of Google Visualization libraries to render the table and chart.
  • Simplified API endpoints to return the resource as is needed in different representations.

I wanted the data to be available even if JavaScript was disabled. You can disable JavaScript and see the data in a raw HTML table as well as links to view the data as a CSV file.

UI

The interface is separated into different concerns:

  • HTML that includes the necessary data and links
  • JavaScript applied. Using Google's Visualization libraries, we can inspect the DOM and pass corresponding Accept headers to retrieve the data necessary as DataTable or other needs for the sortable table and charts. The client side code does not construct any models out of data. It requests the data as necessary and passes it to the google.visualization.DataTable. The only rules declared are UI related, in terms of height, width, formats, legends, etc.
  • API endpoints that respond as necessary. The use cases here are very specific, but could be altered to take query string or body parameters and return different data sets. For example, the endpoints could be modified to support sorting, filtering, or grouping.

There is a loose coupling of these layers and they can easily be replaced or re-used.

API Examples

The data used in this page is readily available via some endpoints.

Some things to note:

  • The data set was small enough that I didn't include pagination.
  • I only needed to work with a single endpoint, so I didn't embed links to support navigation.
  • The data is returned and a response code of 200 Successful.
  • If you try and request a representation that is not supported, you will retrieve a 406 Not Acceptable
  • Each of these endpoints could easily be hidden behind HTTP Authorization schemes like Basic or Bearer token.

We have a single resource, http://utilities.theklaibers.com/historical/billing that has 4 different representations.

Retrieve the data as a CSV

Console Output
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: text/csv'
Console Output (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: text/csv' -H 'Accept-Encoding: gzip'
Save to a local file
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: text/csv' -o usage.csv
Save to a local file (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: text/csv' -H 'Accept-Encoding: gzip' -o usage.csv.gz

Retrieve the data as a JSON

Console Output
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json'
Console Output (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json' -H 'Accept-Encoding: gzip'
Save to a local file
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json' -o usage.json
Save to a local file (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json' -H 'Accept-Encoding: gzip' -o usage.json.gz

Retrieve the data as a JSON Table (Google Sortable Table)

Console Output
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+table'
Console Output (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+table' -H 'Accept-Encoding: gzip'
Save to a local file
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+table' -o usage.json
Save to a local file (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+table' -H 'Accept-Encoding: gzip' -o usage.json.gz

Retrieve the data as a JSON Chart (Google Chart)

Console Output
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+chart'
Console Output (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+chart' -H 'Accept-Encoding: gzip'
Save to a local file
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+chart' -o usage.json
Save to a local file (compressed)
curl 'http://utilities.theklaibers.com/historical/billing' -H 'Accept: application/json+chart' -H 'Accept-Encoding: gzip' -o usage.json.gz
#!/usr/bin/env ruby
module Response
module Models
class UsageRecords
include Enumerable
def initialize(collection)
@collection = Array(collection)
end
def each(&block)
record_collection.each(&block)
end
def total_cost
self.inject(0) { |sum,record| sum += record.total_cost; sum }
end
private
def record_collection
@collection.map { |r| UsageRecord.new(r) }
end
end
class UsageRecord
include Comparable
def initialize(attributes={})
@attributes = attributes
end
def <=>(other)
self.billing_date <=> other.billing_date
end
def billing_date
begin
DateTime.parse(@attributes['billing_date'])
rescue
nil
end
end
def billing_days
@attributes['billing_days']
end
def usage
@attributes['usage']
end
def cost
@attributes['cost']
end
def kwh_tax
@attributes['kwh_tax']
end
def pca
@attributes['pca']
end
def total_cost
@attributes['total_cost']
end
end
end
end
require 'dotenv'
Dotenv.load!
require 'multi_json'
require 'logger'
require 'faraday'
require 'faraday_middleware'
require 'faraday/http_cache'
require 'memcached'
require 'terminal-table'
cache_host = ENV.fetch('CACHE_HOST')
api_host = ENV.fetch('API_HOST')
cache_store = Memcached::Rails.new(cache_host)
logger = Logger.new(File.expand_path('../log/development.log', __FILE__))
request_logger = Logger.new(File.expand_path('../log/requests.log', __FILE__))
cache_logger = Logger.new(File.expand_path('../log/cache.log', __FILE__))
logger.info("Connecting to %s" % [api_host])
ConnectionError = Class.new(StandardError)
connection = Faraday.new(url: api_host) do |conn|
conn.use(:http_cache, store: cache_store, logger: cache_logger)
conn.response(:logger, request_logger)
conn.adapter(Faraday.default_adapter)
end
connection.headers['User-Agent'] = 'NK/CLI'
connection.headers['Accept'] = 'application/json'
begin
usage_request = connection.get('/historical/billing')
rescue Faraday::ConnectionFailed
msg = "Could not connect to %s" % [api_host]
logger.error(msg)
raise ConnectionError.new(msg)
end
case(usage_request.status)
when 200
logger.info("Retrieving Usage...")
usage_response = MultiJson.load(usage_request.body)
usage_data = usage_response.fetch('data', [])
usage_records = Response::Models::UsageRecords.new(usage_data)
money_formatter = ->(amount) do
("$%.2f" % [amount])
end
date_formatter = ->(date) do
date.strftime("%B %Y")
end
row_display = ->(record,counter) do
[(counter+1), date_formatter.call(record.billing_date), record.billing_days, record.usage, money_formatter.call(record.cost), money_formatter.call(record.kwh_tax), money_formatter.call(record.pca), money_formatter.call(record.total_cost)]
end
headings = ['#', 'Billing Date', 'Billing Days', 'Usage', 'Cost', 'KWH Tax', 'PCA', 'Total Cost']
rows = usage_records.sort.reverse.each_with_index.map { |r,i| row_display.call(r,i) }
table = Terminal::Table.new(title: 'Electric Usage Data', headings: headings, rows: rows)
puts "\n"
puts "Total Spent: $%.2f" % [usage_records.total_cost]
puts "\n"
puts table.to_s
grouped_by_year = usage_records.group_by { |r| r.billing_date.year }
year_row_display = ->(record) do
[record[0], money_formatter.call(record[1].inject(0) { |sum,record| sum += record.total_cost; sum })]
end
headings = ['Year', 'Total Cost']
rows = grouped_by_year.map { |r| year_row_display.call(r) }
table = Terminal::Table.new(title: 'Usage by Year', headings: headings, rows: rows)
puts table.to_s
else
msg = "Could not retrieve Usage: %s" % [usage_request.inspect]
logger.error("Could not retrieve Usage: %s" % [usage_request.inspect])
puts msg
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment