public
Last active

Fetch the Queen's new-years speeches into a SQLite3 database... WHY WOULDN'T YOU?

  • Download Gist
da_queen.rb
Ruby
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
# Fetch the Queen's new-years speeches into a SQLite3 database... WHY WOULD YOU?
 
require "rubygems"
require "open-uri"
require "hpricot"
require "dm-core"
 
class Speech
include DataMapper::Resource
property :id, Serial
property :year, String
property :body, Text
end
 
DataMapper.setup(:default,
"sqlite3:///#{Dir.pwd}/data.db")
DataMapper.auto_migrate!
 
class String
def utfy(from='ISO-8859-1')
require "iconv"
Iconv.conv('utf-8', from, self)
end
end
 
index = Hpricot(open("http://kongehuset.dk/publish.php?dogtag=k_dk_aktuelt_taler"))
links = (index / "div#content a").select { |a| a.html =~ /tale/i }
 
links.each do |tale|
h = Hpricot(open(tale.get_attribute("href")))
year, body = [tale.html.utfy,
(h / "div#content").to_html.chop.utfy.gsub!(/(<[^>]*>)/s,"")]
Speech.new(:year => year, :body => body).save
end

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.