This document presents the IR as of October 17th 2018. Future changes like mutability might render parts of this document outdated.
PyTorch uses an SSA-based IR, which is built of multiple entities:
Graphis generally the outermost container for the program representation. At the moment all programs are mostly pure (modulo special operators like
prim::PythonOp), but this will change in the future.
- Then, there are
Blocks, which you can treat as functions. They have a list of inputs and outputs, and a (topologically ordered) list of
Graphhas a top-level
Blockwhich is (using C lingo) like a
Nodes, in turn, represent calls to functions (with possibly multiple arguments and returns).
- Finally, every single intermediate value that appears in the program is represented using a
Value- the root
Blocks have a list of input values, and every
Nodetakes them as inputs, and returns some more as outputs. Every Value has a