marcrasi

Compile Time Constant Expressions for Swift

• Proposal: SE-NNNN
• Authors: Marc Rasi, Chris Lattner
• Review Manager: TBD
• Status: Awaiting implementation

Swift-evolution thread: https://forums.swift.org/t/pitch-compile-time-constant-expressions-for-swift/12879

Introduction

marcrasi

test

marcrasi

Unifying the differentiation modes

Today, Swift AD has two separate transforms that produce forward mode derivatives (JVPs) and reverse mode derivatives (VJPs).

We can generalize this to a single transform that produces a single "derivative function" that can evaluate derivatives in both modes.

The "derivative function" is generic over a "derivative type" that determines the derivative mode.

The advantages over the existing system are:

marcrasi

import TensorFlow

// MARK: - Protocols

protocol Vector {
  func scaled(by factor: Float) -> Self
  func adding(_ other: Self) -> Self
  static var zero: Self { get }
}

marcrasi

import TensorFlow

// MARK: Example function and its transformed version.

func cube(_ x: Float) -> Float {
  return x.product(x).product(x)
}

func cubeT<A: WrapsFloat>(_ x: A) -> A {
  return x.product(x).product(x)

marcrasi

Don't use this. ArrayReader here is better: https://groups.google.com/a/tensorflow.org/g/swift/c/Xo5YmLIt12s/m/OM8n6J4TAQAJ

marcrasi

marcrasi

marcrasi

extension Array where Element: Differentiable {
    /// Views the array as the differentiable product manifold of `Element` with itself `count` times.
    public struct DifferentiableView: Differentiable {
        /// The array that we are viewing.
        public var base: [Element]

        /// Construct a view of the given array.
        public init(_ base: [Element]) { self.base = base }
        
        // MARK: - Differentiable conformance.

marcrasi