Computation Graph:
Instantiating Variable objects as well as Ops objects adds to a graph of computation that is tracked by tf.
After building your computation graph, where nodes are Variables or Ops or Constants, you can retrieve the executed and resulting value of any of the nodes in the graph.
The difference between Ops and Variables:
- Ops take Tensor inputs and return Tensor outputs, they have no internal state.
- Variables can have changing internal state.
So for example:
- you can instantiate one variable (initialized to 0, and thereby add it to computation graph)
- you can instantiate one constants (and thereby add it to the computation graph)
- you can create an op that adds constant and variable
- you can create an op assigns the previous result to the variable
You can then loop over retrievals of the assignment op and thereby add and store a running sum. Looping over retrievals of the adding op will not cause the result to be stored in the internal state of the variable, as only the computation required to retrieve the value of a node is executed.