Skip to content

Instantly share code, notes, and snippets.

@dinomite
Created October 12, 2015 20:16
Show Gist options
  • Save dinomite/df6e8565573ebb8d415f to your computer and use it in GitHub Desktop.
Save dinomite/df6e8565573ebb8d415f to your computer and use it in GitHub Desktop.
495/95 express lanes pricing scraper
#!/usr/bin/env ruby
require 'open-uri'
require 'json'
direction = 'Southbound';
origin = '180SO';
destination = '220SD';
date = '10/09/2015';
@curl_cmd = "curl 'https://www.expresslanes.com/historic-rate-data' -H 'Cookie: data=%7B%7D; digest=b9a6341bf276298b85e14530344aeb95a29b39bb; expire=1445866842; __utmt=1; __utma=269898021.1470859880.1444574061.1444574061.1444657244.2; __utmb=269898021.1.10.1444657244; __utmc=269898021; __utmz=269898021.1444574061.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' -H 'Origin: https://www.expresslanes.com' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: https://www.expresslanes.com/historic-rates' -H 'X-curl_cmded-With: XMLHttpcurl_cmd' -H 'Connection: keep-alive' --data 'date=DATE+TIME&direction=DIRECTION&origin=ORIGIN&destination=DESTINATION' --compressed -k";
@curl_cmd = @curl_cmd.sub("DIRECTION", URI::encode(direction));
@curl_cmd = @curl_cmd.sub("ORIGIN", URI::encode(origin));
@curl_cmd = @curl_cmd.sub("DESTINATION", URI::encode(destination));
@curl_cmd = @curl_cmd.sub("DATE", URI::encode(date));
def execute(time)
cmd = @curl_cmd.sub("TIME", URI::encode(time))
response = JSON.parse(`#{cmd} 2>/dev/null`)
matches = response['content'].scan(/<span class="rate">\$(\d+\.[^<]+)<\/span>/)
if matches.length > 0
cost = matches[-1][0]
puts "#{time} - #{cost}"
end
end
for halfday in ['AM', 'PM'] do
for hour in 1..12
if hour < 10
hour = "0#{hour}"
end
for tens in 0..5
time = "#{hour}:#{tens}0 #{halfday}"
execute(time)
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment