Skip to content

Instantly share code, notes, and snippets.

@holysugar
Created July 22, 2011 02:54
Show Gist options
  • Save holysugar/1098798 to your computer and use it in GitHub Desktop.
Save holysugar/1098798 to your computer and use it in GitHub Desktop.
戦コレDB のデータを CSV に落とす。
#!/usr/bin/env ruby
# coding: utf-8
require 'active_support/all'
require 'open-uri'
require 'nokogiri'
require 'cgi'
require 'ir_b'
def parse(no, body)
doc = Nokogiri::HTML(body)
text = doc.text
sub, name = doc.to_s.scan(/<\/span>(?:<br>)?(?:\[(.*)?\])?(.*?)<span/)[2]
return unless name
name = CGI.unescapeHTML(name)
type = doc.search('span')[1].text rescue nil
type_a = { '炎' => 'R', '水' => 'B', '風' => 'G' }[type]
num = text.scan(/No\.(\d+)/)[0][0].to_i
gender = text.scan(/♂|♀/)[0]
rare = text.scan(/S?(?:レア|ノーマル)\+?/)[0]
rare = rare.sub(/レア/,"R").sub(/ノーマル/,"N") unless rare.empty?
magic = text.scan(/魔力(\d+)/).join
atk, matk = text.scan(/攻撃力(\d+)⇒(\d*)/)[0]
deff, mdeff = text.scan(/防御力(\d+)⇒(\d*)/)[0]
skill = doc.at("div.skill span a").try(:text)
[no, type, num, gender, rare, sub, name, magic, atk, deff, matk, mdeff, skill, type_a]
end
def direct(no)
$stderr.puts "getting http://enedo.velvet.jp/senkore/sk_data.php?no=#{no}"
body = open("http://enedo.velvet.jp/senkore/sk_data.php?no=#{no}", 'r:cp932'){|f| f.read.encode("utf-8") }
parse(no, body)
end
def output_csv(from, to, out = $stdout)
to ||= from
(from..to).each_with_index do |no, idx|
sleep(rand(5) + 5) unless idx == 0
result = direct(no)
out.puts result.join(',') if result
end
end
def main(args)
case args.length
when 0
from = 1
to = 890
when 1
from = args[0].to_i
to = nil
else
from = args[0].to_i
to = args[1].to_i
end
from = 1 if from.zero?
to = from if to.nil? || to.zero?
output_csv(from, to)
end
main(ARGV)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment