ismasan

# A Pipeline extension to process steps concurrently
# Example
# class ConcurrentPipeline < Pipeline
#   include ConcurrentProcessing
# end
#
# MyPipeline = ConcurrentPipeline.new do |pl|
#   pl.step ValidateInput
#
#   # These steps run concurrently

hopsoft

ActiveRecord ETL

I created this to help me run benchmarks/comparisons against Universal ID, but it could serve as the foundation for a robust ETL data pipeline... and it's less than 70 LOC right now! 🤯 🚀

It handles the extract and transform parts of an ETL process and supports the following options:

• only - specify which attributes to include