Skip to content

Instantly share code, notes, and snippets.

@jc00ke

jc00ke/big_xml.rb

Created May 16, 2014 06:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jc00ke/c3eec3d0cabecc8c3e3f to your computer and use it in GitHub Desktop.
Save jc00ke/c3eec3d0cabecc8c3e3f to your computer and use it in GitHub Desktop.
Download ZIP
Raw
big_xml.rb
require 'nokogiri'
require 'benchmark'
d = File.open("big.xml"){ |f| f.read }
s = Nokogiri::XML(d)
def iterfun(s)
s.xpath("./*").each do |c|
iterfun(c)
end
end
t = Benchmark.realtime do
iterfun(s)
end
puts "done in #{t}"
Raw
generate_big_xml.rb
File.open("big.xml", "w") do |f|
f.puts "<root>"
100.times do |j|
f.puts " <foo>#{j}"
1000.times do |i|
f.puts " <bar>#{i}</bar>"
end
f.puts " </foo>"
end
f.puts "</root>"
end
Raw
normal-run.txt
$> ./bin/rbx big_xml_benchmark.rb big.xml
done in 28.391707
Thread 1: total running time: 28.502019778s
% cumulative self self total
time seconds seconds calls ms/call ms/call name
------------------------------------------------------------
53.40 15.23 15.23 567 26.86 26.86 GC.collect_young
4.66 1.33 1.33 100102 0.01 0.01 Nokogiri::XML::XPathContext.new
3.96 1.19 1.13 100102 0.01 0.01 Nokogiri::XML::XPathContext#evaluate
3.32 18.24 0.95 200204 0.00 0.09 Array#map
3.21 1.23 0.92 300306 0.00 0.00 Array#initialize
2.26 9.32 0.64 100102 0.01 0.09 Nokogiri::XML::Node#extract_params
1.72 27.26 0.49 100102 0.00 0.27 Nokogiri::XML::Node#xpath
1.55 3.75 0.44 100102 0.00 0.04 Nokogiri::XML::Node::xpath<150> {}
1.54 15.40 0.44 400408 0.00 0.04 Rubinius::Mirror.reflect
1.26 1.17 0.36 200206 0.00 0.01 Array#each
1.19 1.81 0.34 100102 0.00 0.02 Hash.[]
1.08 0.36 0.31 100103 0.00 0.00 Array#include?
1.04 2.93 0.30 100102 0.00 0.03 Array#initialize_copy
1.02 0.35 0.29 100102 0.00 0.00 Hash#__setup__
1.00 28.39 0.29 100102 0.00 0.28 Nokogiri::XML::NodeSet#each
0.92 3.08 0.26 100102 0.00 0.03 Nokogiri::XML::Node#namespaces
0.83 0.29 0.24 300325 0.00 0.00 BasicObject#!
0.80 4.11 0.23 100102 0.00 0.04 Array#reverse
0.77 0.70 0.22 100102 0.00 0.01 Nokogiri::XML::Node::extract_params<925> {}
0.76 0.25 0.22 100101 0.00 0.00 Nokogiri::XML::NodeSet#[]
0.69 28.39 0.20 100102 0.00 0.28 Integer#upto
0.60 0.21 0.17 200204 0.00 0.00 Array#first
0.60 3.23 0.17 100102 0.00 0.03 Kernel#dup
0.60 0.35 0.17 100102 0.00 0.00 Hash#initialize
0.59 28.39 0.17 100102 0.00 0.28 Object#iterfun
0.50 0.17 0.14 200204 0.00 0.00 Array#last
0.48 1.18 0.14 100102 0.00 0.01 Hash.new_from_associate_array
0.44 0.94 0.13 100103 0.00 0.01 Class#new
0.43 28.39 0.12 100101 0.00 0.28 Nokogiri::XML::NodeSet::each<236> {}
0.41 0.13 0.12 400408 0.00 0.00 Rubinius::Mirror::Array#tuple
0.39 1.16 0.11 100102 0.00 0.01 Enumerable#find
0.38 0.18 0.11 100102 0.00 0.00 Rubinius::Type.check_convert_type
0.35 0.10 0.10 1 99.58 99.58 Nokogiri::XML::Document.read_memory
0.35 0.16 0.10 100102 0.00 0.00 Nokogiri::XML::XPathContext#register_namespaces
0.34 0.46 0.10 100102 0.00 0.00 Hash.allocate
0.32 0.10 0.09 400408 0.00 0.00 Rubinius::Mirror#initialize
0.30 0.23 0.08 100102 0.00 0.00 Nokogiri.uses_libxml?
0.28 14.91 0.08 600613 0.00 0.02 Class#allocate
0.26 0.12 0.07 1201255 0.00 0.00 Kernel#equal?
0.26 0.11 0.07 1001271 0.00 0.00 Fixnum#<
0.23 0.08 0.07 300303 0.00 0.00 Nokogiri::XML::Node#document
0.23 0.07 0.07 100102 0.00 0.00 Nokogiri::XML::Node#namespace_scopes
0.21 0.07 0.06 200203 0.00 0.00 Nokogiri::XML::Document#decorate
0.21 0.76 0.06 100102 0.00 0.01 Enumerable::find<416> {}
0.21 0.10 0.06 1001266 0.00 0.00 Fixnum#+
52 methods omitted
97 methods called a total of 16,421,041 times
Raw
with-hamt.txt
$> ./bin/rbx -Xhash.hamt big_xml_benchmark.rb
done in 26.097106
Thread 1: total running time: 26.208010089000002s
% cumulative self self total
time seconds seconds calls ms/call ms/call name
------------------------------------------------------------
55.46 14.54 14.54 548 26.54 26.54 GC.collect_young
4.33 1.14 1.14 100102 0.01 0.01 Nokogiri::XML::XPathContext.new
3.96 1.07 1.04 100102 0.01 0.01 Nokogiri::XML::XPathContext#evaluate
3.44 1.21 0.90 300306 0.00 0.00 Array#initialize
3.32 19.28 0.87 200204 0.00 0.10 Array#map
2.48 5.89 0.65 100102 0.01 0.06 Nokogiri::XML::Node#extract_params
1.71 24.94 0.45 100102 0.00 0.25 Nokogiri::XML::Node#xpath
1.55 14.70 0.41 400408 0.00 0.04 Rubinius::Mirror.reflect
1.52 3.36 0.40 100102 0.00 0.03 Nokogiri::XML::Node::xpath<150> {}
1.51 1.19 0.40 200206 0.00 0.01 Array#each
1.45 0.69 0.38 100102 0.00 0.01 Hash.[]
1.38 0.41 0.36 100103 0.00 0.00 Array#include?
1.34 0.38 0.35 100101 0.00 0.00 Nokogiri::XML::NodeSet#[]
1.23 0.73 0.32 100102 0.00 0.01 Array#initialize_copy
1.03 1.93 0.27 100102 0.00 0.02 Array#reverse
0.92 0.29 0.24 300306 0.00 0.00 Array#first
0.81 26.10 0.21 100102 0.00 0.26 Nokogiri::XML::NodeSet#each
0.76 1.79 0.20 100102 0.00 0.02 Nokogiri::XML::Node#namespaces
0.71 26.10 0.18 100102 0.00 0.26 Integer#upto
0.65 0.70 0.17 100102 0.00 0.01 Nokogiri::XML::Node::extract_params<925> {}
0.64 26.10 0.17 100102 0.00 0.26 Object#iterfun
0.62 1.01 0.16 100102 0.00 0.01 Kernel#dup
0.58 0.18 0.15 200204 0.00 0.00 Array#last
0.46 26.10 0.12 100101 0.00 0.26 Nokogiri::XML::NodeSet::each<236> {}
0.43 0.13 0.11 400408 0.00 0.00 Rubinius::Mirror::Array#tuple
0.41 1.20 0.11 100102 0.00 0.01 Enumerable#find
0.38 0.10 0.10 1 99.22 99.22 Nokogiri::XML::Document.read_memory
0.35 0.13 0.09 100102 0.00 0.00 Nokogiri::XML::XPathContext#register_namespaces
0.35 0.11 0.09 100121 0.00 0.00 BasicObject#!
0.33 0.10 0.09 400408 0.00 0.00 Rubinius::Mirror#initialize
0.32 0.22 0.08 100102 0.00 0.00 Nokogiri.uses_libxml?
0.26 0.11 0.07 1101153 0.00 0.00 Kernel#equal?
0.24 0.07 0.06 100102 0.00 0.00 Nokogiri::XML::Node#namespace_scopes
0.24 14.22 0.06 500511 0.00 0.03 Class#allocate
0.23 0.10 0.06 1001271 0.00 0.00 Fixnum#<
0.23 0.09 0.06 1001266 0.00 0.00 Fixnum#+
0.21 0.07 0.06 300303 0.00 0.00 Nokogiri::XML::Node#document
0.21 0.75 0.06 100102 0.00 0.01 Enumerable::find<416> {}
0.21 0.08 0.06 600881 0.00 0.00 Kernel#kind_of?
0.20 0.82 0.05 100102 0.00 0.01 Rubinius::Type.object_initialize_dup
0.20 0.09 0.05 100102 0.00 0.00 Nokogiri::VersionInfo.instance
0.20 0.06 0.05 200203 0.00 0.00 Nokogiri::XML::Document#decorate
0.20 0.06 0.05 100102 0.00 0.00 Array#reverse!
0.20 0.07 0.05 100110 0.00 0.00 Module#===
0.19 0.05 0.05 100102 0.00 0.00 Nokogiri::VersionInfo#libxml2?
43 methods omitted
88 methods called a total of 14,919,490 times
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment