Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Collect UA's from UserAgentString. Store in Redis.
require "ohm"
require "typhoeus"
require "nokogiri"
Ohm.connect
class UserAgent < Ohm::Model
attribute :user_agent
index :user_agent
end
ua_list = []
ua_list << Nokogiri::HTML(Typhoeus::Request.get("http://useragentstring.com/pages/Internet%20Explorer/").body).xpath('//li').map {|item| item.content }.keep_if {|item| item =~ /MSIE [7-9]+.0/}
ua_list << Nokogiri::HTML(Typhoeus::Request.get("http://useragentstring.com/pages/Firefox/").body).xpath('//li').map {|item| item.content }.keep_if {|item| item =~ /Firefox\/[3-9]+\./ && item =~ /en-US/}
ua_list << Nokogiri::HTML(Typhoeus::Request.get("http://useragentstring.com/pages/Chrome/").body).xpath('//li').map {|item| item.content }.keep_if {|item| item =~ /Chrome\/[4-9]+\./ && item =~ /en-US/}
ua_list << Nokogiri::HTML(Typhoeus::Request.get("http://useragentstring.com/pages/Safari/").body).xpath('//li').map {|item| item.content }.keep_if {|item| item =~ /Version\/[4-9]+\./ && item =~ /en-US/}
ua_list.flatten!.each {|ua| UserAgent.create(:user_agent => ua)}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.