Skip to content

@kmile /xml_parser.rb
Created

Embed URL

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
A small nokogiri xml reader DSL.
# A small DSL for helping parsing documents using Nokogiri::XML::Reader. The
# XML Reader is a good way to move a cursor through a (large) XML document fast,
# but is not as cumbersome as writing a full SAX document handler. Read about
# it here: http://nokogiri.org/Nokogiri/XML/Reader.html
#
# Just pass the reader in this parser and specificy the nodes that you are interested
# in in a block. You can just parse every node or only look inside certain nodes.
#
# A small example:
#
# Xml::Parser.new(Nokogiri::XML::Reader(open(file))) do
# inside_element 'User' do
# for_element 'Name' do puts "Username: #{inner_xml}" end
# for_element 'Email' do puts "Email: #{inner_xml}" end
#
# for_element 'Address' do
# puts 'Start of address:'
# inside_element do
# for_element 'Street' do puts "Street: #{inner_xml}" end
# for_element 'Zipcode' do puts "Zipcode: #{inner_xml}" end
# for_element 'City' do puts "City: #{inner_xml}" end
# end
# puts 'End of address'
# end
# end
# end
#
# It does NOT fail on missing tags, and does not guarantee order of execution. It parses
# every tag regardless of nesting. The only way to guarantee scope is by using
# the `inside_element` method. This limits the parsing to the current or the named tag.
# If tags are encountered multiple times, their blocks will be called multiple times.
require 'nokogiri'
module Xml
class Parser
def initialize(node, &block)
@node = node
@node.each do
self.instance_eval &block
end
end
def name
@node.name
end
def inner_xml
@node.inner_xml.strip
end
def is_start?
@node.node_type == Nokogiri::XML::Reader::TYPE_ELEMENT
end
def is_end?
@node.node_type == Nokogiri::XML::Reader::TYPE_END_ELEMENT
end
def attribute(attribute)
@node.attribute(attribute)
end
def for_element(name, &block)
return unless self.name == name and is_start?
self.instance_eval &block
end
def inside_element(name=nil, &block)
return if @node.self_closing?
return unless name.nil? or (self.name == name and is_start?)
name = @node.name
depth = @node.depth
@node.each do
return if self.name == name and is_end? and @node.depth == depth
self.instance_eval &block
end
end
end
end
@dimitko

This is the most amazing Ruby code I seen (and used) for this year. Thanks a ton, mate!

@kmile
Owner

You're very welcome, glad you've found a use for it :)

@dimitko

Your code brings to the XML Pull approach the one thing it seriously lacks -- readability.

With it, you just need to isolate the "for_element" / "inside_element" starting statements with one newline on the top and the bottom, and the XML Pull parsing code becomes a charm to read. Plus, the indenting as you go further and further in is intuitive as we are parsing an XML (which is also hierarchical).

In short, I love you. :)

@dimitko

BTW, I removed the "name", "inner_xml" and "attribute" methods and just added a "method_missing" handler:

def method_missing(sym, *args, &block)
  @node.send sym, *args, &block
end

If there are security concerns I might end up limiting the method names which will get propagated to the node, but for now this works well enough.

Considering of doing a mini-gem and sharing it here on GitHub. My laziness will have the final word. :)

@kmile
Owner

I decided to write this DSL after I looked at my 300 line XML reader code and could not understand what was going on even a day later. Turns out using this code it got easier to read and maintain, faster, and less prone to errors as well!

The reason I wanted to define the inner_xml explicitly is because of the strip. If there are newlines and indenting in the XML, you also get them in your result which is usually not what you want.

Otherwise the method_missing is a nice addition if you end up hooking into the other node properties a lot as well.

My laziness had the final word for me creating this gist instead of a full fledged gem or project ;)

@dimitko

BTW, do you know a faster replacement for the "instance_eval(&block)"? I did some profiling and I didn't like the results in this regard. ;)

@joshuaflanagan

This is very helpful, thanks a lot. I fixed a bug that occurs when inside_element matches on a self-closing tag:

def inside_element(name=nil, &block)
  return if @node.self_closing? # there is nothing inside this element, get out now
  return unless name.nil? or (self.name == name and is_start?)
@kmile
Owner

Ah, I have not taken self-closing elements into account. You're right, thanks!

@dimitko

Just a quick question -- I haven't been able to decipher what the method "state" returns? Does anyone have any idea of what is the possible set of values it might return?

@kmile
Owner

This appears to be from libxml2. It seems that it references the XmlTextReaderMode enum:

Enum xmlTextReaderMode {
    XML_TEXTREADER_MODE_INITIAL = 0
    XML_TEXTREADER_MODE_INTERACTIVE = 1
    XML_TEXTREADER_MODE_ERROR = 2
    XML_TEXTREADER_MODE_EOF = 3
    XML_TEXTREADER_MODE_CLOSED = 4
    XML_TEXTREADER_MODE_READING = 5
}

But I cannot tell for sure without looking at the source what these states/modes mean, or if this is a complete list.

@santuxus

That's a really nice and useful gist.
Thanks!

@cmartin81

Very nice code! Thanks a lot!

@lesterz

Is there any more documentation or examples on how to use this anywhere? I'm having a hard time instantiating classes inside for_element. Keep getting NoMethodErrors...

@joonty

This really is fantastic - excellent work!

@gal-at-aljazeera

I just woke up in the middle of the night envisioning something like this.

And it already exists.

Good work.

@nicka

OMG! This is awesome!!!! +1

@twigbranch

inner_xml doesn't seem to unescape & -- what's the recommended way to do this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.