Skip to content

Instantly share code, notes, and snippets.

@quadrophobiac
Last active September 13, 2015 19:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save quadrophobiac/f4f16c44aed3adf939eb to your computer and use it in GitHub Desktop.
Save quadrophobiac/f4f16c44aed3adf939eb to your computer and use it in GitHub Desktop.
Suitability of Lazy Enumeration in CSVlint
# ref - http://blog.honeybadger.io/using-lazy-enumerators-to-work-with-large-files-in-ruby/
# The Lazy Enumerator class contains a number of built in collection and mapping methods
# e.g. ```map, flat_map, select, reject, grep, zip, take, take_while, drop, drop_while, cycle```
# the basic way that lazy evaluation works with the above methods is like so
enum.lazy.select {|variable| (conditional_eval_of_variable) } # returns an array where the eval returns true
# in pseudo code fashion I would imagine it would proceed like this
CSVlintValidator.select { |validation| output(@ErrorCollector) if !validation.valid? } # returns object or array for the rows which raise messages
# validation.valid? could be rewritten in Csvlint::ErrorCollector to be errors.empty? && warnings.empty?
# ~ && info_messages.empty? or the output could only happen when an error arises, informing the user that there is already
# one error detected and do they wish to continue evaluating in 'lazy' fashion
# the idea above would be that the output to console (or on website) would be updated everytime the ErrorCollectors instance
# ~ variables increment
CSVlintValidator.select { |validation| output(@ErrorCollector) if !validation.valid? }
# the main impediment I forsee regarding this is that currently the Validation procedure returns two instances at present
@data # which is a duplicate of every row that has been parsed
Csvlint::ErrorCollector # (by proxy) an included class that contains the @errors, @warnings and @info_messages instance vars
# As noted in TODOs @datas purpose is unclear. I assume that @data is populated so that
# ~ the dialect checking feature which is a part of the rails interface to the gem can work from some local memory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment