Skip to content

Instantly share code, notes, and snippets.

@tmtmtmtm
Created February 4, 2017 08:26
Show Gist options
  • Save tmtmtmtm/295a78f47080fd046f629999c922c137 to your computer and use it in GitHub Desktop.
Save tmtmtmtm/295a78f47080fd046f629999c922c137 to your computer and use it in GitHub Desktop.
generate QuickStatements for ensuring Philippine elections have correct "instance of"
#!/usr/bin/ruby
require 'pry'
require 'scraped'
require 'wikidata/fetcher'
INSTANCE_OF = 'P31'
PPE = 'Q7185370'
PSE = 'Q28333164'
PHE = 'Q28649370'
class AddWikidataIds < Scraped::Response::Decorator
def body
Nokogiri::HTML(super).tap do |doc|
links = doc.css('#bodyContent a[href*="/wiki"][title]').reject { |a| a[:title].include? ':' }
wdids = WikiData.ids_from_pages(language, links.map { |a| a[:title] }.uniq)
links.each { |a| a[:wikidata] = wdids[a[:title]] }
end.to_s
end
private
def language
URI.parse(url).hostname.split('.').first
end
end
class ElectionTemplate < Scraped::HTML
decorator AddWikidataIds
field :presidential do
table.xpath('.//th[.="Presidential elections"]/following-sibling::td//a/@wikidata').map(&:text)
end
field :senate do
table.xpath('.//th[.="Senate elections"]/following-sibling::td//a/@wikidata').map(&:text)
end
field :house do
table.xpath('.//th[.="House elections"]/following-sibling::td//a/@wikidata').map(&:text)
end
private
def table
noko.css('div.navbox table')
end
end
def command(*list)
puts list.join("\t")
end
url = 'https://en.wikipedia.org/wiki/Template:Philippine_elections'
page = ElectionTemplate.new(response: Scraped::Request.new(url: url).response)
page.presidential.each { |e| command(e, INSTANCE_OF, PPE) }
page.senate.each { |e| command(e, INSTANCE_OF, PSE) }
page.house.each { |e| command(e, INSTANCE_OF, PHE) }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment