My background is writing code and one of the most useful things I learnt was [Don't Repeat Yourself][wikipedia], this was something I read in a book called the [Pragmatic Programmer][pragprog] which I would thoroughly recommend to anyone today who has anything to do with software.
What interests me now is how we should relate this to the Business Intelligence, Data Integration and Analytics worlds, and one conversation I had yesterday reminded me of several other conversations that I have had over the years: how can we automate ETL and Business Intelligence meta data generation?
A traditional Data Warehousing project will follow some or all of these steps, whether they do it in repeated [Agile][wikipedia 2] sprints, or one big [Waterfall Model][wikipedia 3]:
- elicit, document and validate requirements.
- design logical and then physical model.
- indentify source data attribute(s) and transformations for target model.
- write ETL packages to move code through whichever data warehouse architecture you a