Now that Gecko uses Servo's parallel style system, we want to work outward from there. The next phase could be parallel frame construction, and after that, parallel layout using Servo layout code.
We want to ship this work incrementally (every six weeks), without needing to replace the entire layout system at once. There are a few ways we can convert some parts of layout to use parallel Servo/Rust code, while other parts are still using the current Gecko implementations:
At a high level, Servo and Gecko divide layout into the same basic top-down and bottom-up passes through the flow/frame tree. We could convert one of Gecko's passes to be parallel while leaving others unchanged.
For example, the bottom-up pass to assign intrinsic widths could be the first one to parallelize. In Servo, this is combined with frame construction, so this especially makes sense if we are also parallelizing frame construction. (Note: Gecko does this layout lazily, while Servo lacks this optimization. We might need a heuristic to determine when it's better to use Gecko's lazy behavior versus Servo's eager parallel behavior. This heuristic could be useful for Servo too.)
We could also use parallel Servo-based traversals on certain parts of the frame tree, and existing Gecko traversal code for frames that the Servo code can't yet handle correctly.
For example, we might want to use Servo for laying out block boxes and flex boxes, but use Gecko for inline layout and table layout. (Block is most critical for parallel gains; flex and grid are also becoming important. Inline and table are less important, and Servo's inline and table layout are far from complete/correct.)
One way to implement this would be to add a flow type to Servo that is just a wrapper for a Gecko frame. This would act like "replaced content"; to the Servo code, the flow is a black box, and it just calls Gecko to lay out its contents. (This could be a useful concept for extending Servo layout in other ways too, beyond Gecko integration.)
To make this integration more feasible, we want to make some changes to Servo's parallel traversal code, especially how it handles floats.
Currently, Servo tries to determine ahead of time (in a separate pass before the main layout passes) which flows might be impacted by floats. During the main layout pass, if a flow has children that might be impacted by floats, it lays them out recursively in serial. Its other children are placed in a work queue that is processed in paralled. This is complex and has been a source of bugs, and may not be optimal for performance.
Instead, we'd like to change the main layout pass so that each node lays out all of its children recursively, then does any post-order layout calculations on itself. Impact from floats will be detected more dynamically: Until we reach a child that has floats in its sub-tree, children are processed in a parallel loop. Once we reach a float, any children following it are processed in serial until we reach a child that "clears" or extends beyond any floats. Then we can resume the parallel loop until the next child containing a float, and so on.
This will make Servo's main layout pass simpler, and closer to a traditional layout engine. It will separate sub-tree traversals more cleanly, which should help it delegate some sub-trees to Gecko layout code. (It should also help with correctness and maintainance work.)