Skip to content

Instantly share code, notes, and snippets.

@westwickfarrow
Created November 18, 2016 10:39
Show Gist options
  • Save westwickfarrow/5505f2e396d36d8468cf26943c8112f3 to your computer and use it in GitHub Desktop.
Save westwickfarrow/5505f2e396d36d8468cf26943c8112f3 to your computer and use it in GitHub Desktop.
require 'rubygems'
require 'capybara'
require 'capybara/dsl'
require 'capybara/rspec'
require 'capybara/poltergeist'
require 'csv'
require 'pry'
include Capybara::DSL
Capybara.register_driver :poltergeist do |app|
Capybara::Poltergeist::Driver.new(app, {js_errors:false, port:44678+rand(5), phantomjs_options:['--proxy-type=none', '--ignore-ssl-errors=yes'], timeout:180, default_wait_time:180, window_size:[1920, 1080]})
end
Capybara.default_selector = :xpath
agents = [
"Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36",
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2226.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.4; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2224.3 Safari/537.36"
]
# Use XPath as the default selector for the find method
@session = Capybara::Session.new(:poltergeist)
@session.driver.headers = { 'User-Agent' => agents.sample }
(0..10).each do |i|
@session.visit "https://www.fast.com"
sleep 60
@doc = Nokogiri::HTML(@session.html)
results = @doc.xpath("//div[@class='speed-results-container succeeded']").text
units = @doc.xpath("//div[@class='speed-units-container succeeded']").text
final = "#{ Time.now };#{ results };#{ units }"
puts final
CSV.open("fast-com-#{ ARGV[0] }.csv", "ab") do |csv|
csv << [Time.now, results, units]
end
sleep 300
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment