Skip to content

Instantly share code, notes, and snippets.

@infectious
Created July 5, 2013 08:43
Show Gist options
  • Save infectious/5933004 to your computer and use it in GitHub Desktop.
Save infectious/5933004 to your computer and use it in GitHub Desktop.
## Question: I want to use this same ETL for a few different dimensions. How can I use a parameter (cols) to employ the 'columns' option in the load block (line 28/29 below)?
parameter :today do
8.day.ago.to_date
end
parameter :dimension do
"placement"
end
parameter :cols do
case dimension.downcase
when "ad" then :id, :name, :type, :width, :height
when "placement" then :id, :name, :start_date, :end_date, :budget
when "video" then [:id, :name, :duration]
else [:id, :name]
end
end
extract :ARC do
from "tubemogul/PlayTime_*_GEN-*_ST-#{today.ymd}_END-#{today.ymd}_DIM-#{filename}.tsv.gz"
separator "\t"
end
load :PDW do
into :"tubemogul__#{dimension.downcase}s"
insert_ignore
#columns cols # cannot get to work
#columns [:id, :name, :start_date, :end_date, :budget] #works
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment