Looking into the .NET inliner's brain

inline.md

I've dug through the .NET 5 JIT in an attempt to understand how it determines whether a method should be inlined or not (so I know whether to spam method inlining attributes)

There's a lot of info. I'll strip out most of it so I can leave in the important parts (keep in mind this is my own understanding so some parts may be incorrect. If you have a correction, let me know)

The JIT has different "policies" for inlining that decide in different ways whether or not a method should be inlined. This gist attempts to break down the default policy, which is the one you should be concerned with most of the time

(Keep in mind that at this stage, all methods are still in IL form)

The JIT takes the following precautions before it even thinks about inlining a call to a method regardless of any attributes it has:

• If the method is virtual, do not inline
• If the method calls itself (i.e it is recursive), do not inline
• If the method has 32 locals or takes 32 arguments, do not inline
• If the maximum stack size of the method is more than 16 bytes, do not inline
• If the method has more than 5 blocks, do not inline
• If a call to the method is nested too deeply, do not inline the call. The maximum nest depth by default is 20

(There's some other precautions but they likely will not be your concern)

If the method is still viable, the JIT continues.

The first thing the JIT does is look at the method's attributes. If there's a [MethodImpl(MethodImplOptions.AggressiveInlining)] attribute on the method, it is automatically a candidate to be inlined (a lot of people think it just "suggests" to the JIT it should inline the method and that it may not do it anyways, but more accurately it makes the JIT absolutely inline the method unless the JIT believes there'd be a catastrophic issue with it. In most cases, if you ask for an aggressive inlining, the method will just constantly get inlined, so make sure you know what you're doing)

If there's no force inline attribute, the JIT will assess the method. The JIT will look at the method's IL/Code size in bytes (not the instruction count!). If it's 16 bytes or less, the method becomes a candidate to be inlined. Otherwise, if the size is within 100 bytes, the method becomes a candidate that is pending a profitability evaluation (more on this in a second). If the size is larger than 100 bytes, the method will never be inlined.

If a method is a candidate that is awaiting evaluation (i.e code size is between 17 and 100 bytes), the JIT determines the "profitability" of it being inlined (that is, whether it's beneficial at all) in respect to the speed/size tradeoff of the program.

If the called method is still viable, the JIT continues:

• The JIT estimates the size of the native code it'll generate if it compiles the called method to native instructions
• The JIT then estimates the size of the native code of the caller method. The estimate is bumped up by several factors such as (but not limited to) whether or not the caller is an instance method, and the types of the caller method's parameters. In general, the more parameters the caller method has, the bigger its estimated native size.
• The JIT then determines a "benefit multiplier" to the method being inlined, which is just a double.
  • If the method is an instance constructor, add 1.5 to the multiplier
  • If the method is from a promotable struct, add 3.0 to the multiplier (your struct is a promotable struct if it is less than 32 bytes in size, has minimum 1 field and maximum 4 fields. If your struct uses a custom layout, it must be aligned correctly and have no overlapping fields. If your struct uses a custom layout and it is an HFA type, it is not promotable. For some reason, structs that are only 1 float field are not promoted ever)
  • If the method looks like a wrapper method (i.e its body is just a call followed by a return), add 1f to the multiplier
  • If the method takes arguments that are constant / do not change, add 1.0 to the multiplier
  • If most of what the method does is just load and store instructions, add 3.0 to the multiplier
  • If the method has an argument used in a range check, add 0.5 to the multiplier
  • If the method has an argument that does not change which is used in a conditional check, add 3.0 to the multiplier
  • Add an extra 3.0 if we're in Pre-JIT stage? I think?
  • The JIT treats methods as either UNUSED, RARE, BORING, WARM, LOOP, HOT in terms of how frequently they are called. RARE will completely discard any of the previous points and force the multiplier to be 1.3. BORING adds 1.3. WARM adds 2.0. LOOP and HOT add 3.0. RARE is for methods that reside in catch clauses, next to throws, etc. Most methods will not be considered RARE.
  • I believe it's possible to explicitly tell the JIT to bump the multiplier by a constant amount (for what i assume is platform-specific optimization)
  • Debug builds have an inline stress factor that can dramatically increase the multiplier by 10.0
• The JIT now computes a value called the threshold using (int)(EstimatedSizeOfCALLERMethod * CalculatedMultiplier) as the final heuristic to be used.
• If the estimated size of the target method to be inlined is greater than the threshold, it is not inlined. Otherwise, the inline is deemed profitable, and an inline is scheduled.