Skip to content

Instantly share code, notes, and snippets.

@donfanning
Forked from abacha/share_crawler.rb
Created August 15, 2018 12:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save donfanning/29605308c19b766974fada5408e67f69 to your computer and use it in GitHub Desktop.
Save donfanning/29605308c19b766974fada5408e67f69 to your computer and use it in GitHub Desktop.
require_relative "xpath_crawler"
require_relative "parser"
module ShareCrawler
class << self
def get(crawler)
xpath_crawler = XPathCrawler.new(crawler["url"])
parsed = { "warning" => 0 }
parsed["value"] = Parser.parse_value(xpath_crawler.parse(crawler["xpath_value"]))
parsed["date"] = Parser.parse_date(xpath_crawler.parse(crawler["xpath_date"]))
unless crawler["xpath_name"].nil?
name = xpath_crawler.parse(crawler["xpath_name"])
parsed["warning"] = 1 if name != crawler["display_name"]
end
if (!crawler["last_value"].nil? &&
((crawler["last_value"]/parsed["value"]*100)-100).abs > crawler["max_ratio"])
parsed["warning"] = 1
end
if parsed["value"] == 0 || parsed["value"].nil? || parsed["date"].nil?
parsed["warning"] = 1
end
parsed
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment