| test |
Today, Swift AD has two separate transforms that produce forward mode derivatives (JVPs) and reverse mode derivatives (VJPs).
We can generalize this to a single transform that produces a single "derivative function" that can evaluate derivatives in both modes.
The "derivative function" is generic over a "derivative type" that determines the derivative mode.
The advantages over the existing system are:
| import TensorFlow | |
| // MARK: - Protocols | |
| protocol Vector { | |
| func scaled(by factor: Float) -> Self | |
| func adding(_ other: Self) -> Self | |
| static var zero: Self { get } | |
| } |
| import TensorFlow | |
| // MARK: Example function and its transformed version. | |
| func cube(_ x: Float) -> Float { | |
| return x.product(x).product(x) | |
| } | |
| func cubeT<A: WrapsFloat>(_ x: A) -> A { | |
| return x.product(x).product(x) |
| Don't use this. ArrayReader here is better: https://groups.google.com/a/tensorflow.org/g/swift/c/Xo5YmLIt12s/m/OM8n6J4TAQAJ |
| import TensorFlow | |
| let g = gradient(at: Dense()) { m in m.callForward(Tensor<Float>(0)) } | |
| print(g) |