Migrating FIRRTL Transforms from inputForm/outputForm to dependency API

forms2dep.md

Introduction

Hello! Hopefully you saw this linked to in a deprecation warning and want to migrate your code. Great! We want to make it as easy as possible, so if you liked this, please feel free to say so on the chisel-users mailing list :)

What are Forms?

Forms were a way to describe, in the FIRRTL compiler, the state of the circuit regarding which IR nodes exist within the circuit. As a circuit is transformed to eventually emit Verilog (or another backend), more complex IR nodes are rewritten to simpler IR nodes.

While useful, we made a bad decision to require custom transformations to use these forms as the mechanism by which the compiler schedules transformations. These forms were too clunky and lacked the fine-grain necessary to properly describe the dependency relationship between transformations.

Our goal, if you implemented your own custom transform, is to migrate from the inputForm/outputForm API to our newer Dependency API.

What is the new Dependency API?

Now, each transform can declare which other transformations (or set of transformations) it depends on. In addition, it can declare which transformations it invalidates. This enables the FIRRTL compiler to figure out if it needs to rerun transforms. For example, suppose your custom transform requires deduplication to have occurred, but also introduces UnknownTypes into the circuit. Your implementation of the Dependency API would look something like this:

import firrtl.passes.InferTypes
import firrtl.transforms.DedupModules
class MyCustomTransform extends Transform {
  override def prerequisites: List[Dependency[Transform]] = List(Dependency(DedupModules))
  override def invalidates(xform: Transform): Boolean = xform match {
    case InferTypes => true
    case _    => false
  }
  override def execute(cs: CircuitState): CircuitState = ???
}

Migration

We recommend you take the time to understand the transform you are migrating and implement the Dependency API properly. All FIRRTL transformations in the firrtl repo have been upgraded, and you can use them as an examples. In addition, you can checkout the dependency API tests.

The migration will require removing two functions def inputForm = ... and def outputForm = ... and implementing two functions override def prerequisites: List[Dependency[Transform]] = ... and override def invalidates(xform: Transform): Boolean = ....

Running scalafix

//TODO!!! // Talk about change to set default value for inputForm/outputForm // Talk about adding scalafix to your project // Talk about running scalafix // Rename "Firrtl.scala" to something like "Forms2Dep.scala"

> scalafix --rules=https://raw.githubusercontent.com/freechipsproject/firrtl/reduce-warnings/scalafix/rules/src/main/scala/fix/Firrtl.scala

Implementing prerequisites

Next, we need to create the transform's prerequisites. There is a one-to-one map from inputForm to prerequisites:

• firrtl.LowForm -> firrtl.stage.Forms.LowForm
• firrtl.MidForm -> firrtl.stage.Forms.MidForm
• firrtl.HighForm -> firrtl.stage.Forms.Deduped
• firrtl.ChirrtlForm -> Nil

One caveat is if this transform requires being run after LowForm optimizations. In general this behavior is undesirable because it will always require optimization passes to be run. This makes optimizations no longer optional, which is bad. However, if you have no choice, then use the following relationship to migrate:

• firrtl.LowForm -> firrtl.stage.Forms.LowFormOptimized

If your inputForm was UnknownForm, then you need to understand the context where it was being used, and determine the prerequisites accordingly. Note that all forms in firrtl.stage.Forms._ are just collections of transformations. You can inspect that file to get a better idea of the transforms which make up the FIRRTL compiler.

Concrete example if your inputForm was override def inputForm = firrtl.LowForm

+ override def prerequisites: List(Dependency[DedupModules]) = firrtl.stage.Forms.LowForm

Implementing invalidates

The invalidates function is more complicated, as it depends on both inputForm, outputForm, and the behavior of the transformation you are migrating.

Case 1: inputForm == outputForm == UnknownForm

It all boils down to what your transformation does. If it does nothing but emit stuff, then it doesn't touch the CircuitState and thus it invalidates nothing:

- override def inputForm = UnknownForm
- override def outputForm = UnknownForm
+ override def invalidates(xform: Transform): Boolean = false

If your transform does a bunch of stuff, you need to understand what needs to be rerun to lower and complex IR nodes you reintroduced. If you aren't sure, the heavy-handed solution (and safest solution) is to invalidate everything:

- override def inputForm = UnknownForm
- override def outputForm = UnknownForm
+ override def invalidates(xform: Transform): Boolean = true

Case 2: inputForm == outputForm != UnknownForm

In this case, your transform LIKELY does not invalidate any other transformations, because the inputForm/outputForm API didn't really let you rerun anything. Thus, its likely you don't invalidate anything

- override def inputForm = [[[FORM]]]
- override def outputForm = [[[FORM]]]
+ override def invalidates(xform: Transform): Boolean = false

If your transformation manually runs a bunch of transformations at the end to patch up the CircuitState, its possible to remove those and instead add them to your invalidates function. E.g., if your transformation runs DedupModules at the end, your invalidates could look like:

- override def inputForm = [[[FORM]]]
- override def outputForm = [[[FORM]]]
+ override def invalidates(a: Transform): Boolean = a match {
+   case _: firrtl.transforms.DedupModules => true
+   case _ => false
+ }

Case 3: inputForm != outputForm, and outputForm is 'higher' than inputForm

For faster and less conservative invalidation, consider if you know more about what this transform does. If so, you can selectively invalidate the transforms you know need to be rerun. For example if you invalidate deduplication (and want it to be rerun) you can do the following:

- override def inputForm = ...
- override def outputForm = ...
+ override def invalidates(a: Transform): Boolean = a match {
+   case _: firrtl.transforms.DedupModules => true
+   case _ => false
+ }

To get identical behavior at the cost of runtime performance, you can do one of the following, depending on the value of outputForm:

- override def inputForm = ...
- override def outputForm = HighForm
+ override def invalidates(a: Transform): Boolean =
+   (Forms.VerilogOptimized -- Forms.MinimalHighForm).contains(Dependency.fromTransform(a))

- override def inputForm = ...
- override def outputForm = MidForm
+ override def invalidates(a: Transform): Boolean =
+   (Forms.VerilogOptimized -- Forms.MidForm).contains(Dependency.fromTransform(a))

Final Diff

Overall, your diff should look like the following with the ... filled in properly!

- override def inputForm = ...
- override def outputForm = ...
+ override def prerequisites: List[Dependency[Transform]] = ...
+ override def invalidates(a: Transform): Boolean = ...

Hopefully that was relatively painless! If not, please submit an issue here or ask questions on gitter.