lazyflow wishlist

lazyflow wishlist

Transactions!

Updates should occur in a breadth-first order through the graph instead of depth first.
- Formalize the transformation from lazyflow.graph.Graph() into some pure DAG.
- Determine a topological ordering for the DAG.
- Configure the operators IN ORDER according to the topological ordering.  That is, a change to an upstream output doesn't affect its downstream partners immediately.  Instead, the change is merely noted and the downstream partner's operator is reconfigured in turn according to the DAG topological order.

Also, a clear specification for when it's okay to execute an input slot:
- Do we need a clearer distinction between setup time and execution time?
- For example, it's common to read an input "value" slot setting (e.g. Slicer index, etc.) during setupOutputs, but technically this could trigger an upstream execution (if the slot's value is a non-trivial computation instead of a simple value).  Is it an error to trigger a graph computation like this during setup?  Or is it okay?  How can we make it clearer? Use different input slot types for "value" a.k.a. setup-time slots and "execution" slots?

Better signal mechanism:
- Return subscription tokens for automatic cleanup
- Built-in support for bind? (like pyqtSignal)

Eliminate special 'value slot' case.
- And eliminate the 'backpropagate_values' unholiness along with it.
- Support 0-dimensional slots just like numpy.

Eliminate support for different rtypes (for now)

Partner access forwarding (i.e. all downstream slots store the source callable instead of going through N partner "call" operators on every slot access).  If an upstream slot is reconnected, it's downstream slots need to be notified, but that penalty is paid during graph setup, not execution.

Requests:
- Cancellable wait() for 'foreign' threads
- Find a way to provide the "direct execute" optimization without defeating the priority scheme.

Kill the RequestExecutionWrapper!
Also, remove the broken execute/setup locking logic.
Add support for graph state (version).  Make requests aware of it somehow (via a subclass? or a special callable?)

Needed concepts/operators:
- Split request blocking and data caching into two seperate opeators
- Create blocked array caches as a composite of the two operators
- Pass optimal blocking via metadata

Generalized operator wrapper? (Allow double-wrapping)

Use weak references for Operator.parent.
Related: Clear ownership semantics for "add-on" operators (e.g. opSubRegions, channels slicers, op5ifyers, etc.).  When/how do those get cleaned up?

Better names: Rename partner/partners to source/sinks or upstream/downstream or something.

Explicit numba support.
- Perhaps a "compile" function that auto-updates an entire operator and its children to replace members with jitted members based on their current types.

Awareness of request RAM usage and CPU load
- Via: Usage sampling?  Slot meta-data?  Request init parameter?

Simplifications: Eliminate features that steepen learning curve without providing useful abstractions
- MetaDict could probably be replaced with a plain dict or maybe defaultdict(lambda: None)
- Value slots (mentioned above).
- Slot.get() vs __call__() vs __getitem__()
- Magical slot definitions in the class scope even though they are actually instance variables.