DataFeed
Class : Tracking each datafeedLogSummary
Class : Tracking the log for each datafeed. (part of DataFeed object)
- Instead of having code spread across
core/generic
,core/stages
and the python files inside them, move all the class related abstractions to a single filedatafeed.py
and the "transformations" & "actions" level code into a single folder (can be divided by stage if need be). - For the datafeeds, each one of them named after the feed (eg. "avent", "bos") has a package created in the
{root_dir}/datafeeds
directory. If custom logic is present for that feed, a file named based on the stage is created and the logic is written there. - All pure functions (functions that return the same output for the same input) should be put in
utils
package.
{root_dir}