Custom Intrinsics Groups

0000-custom-intrinsics-groups.md

• Feature Name: custom_instrinsics_groups
• Start Date: (fill me in with today's date, YYYY-MM-DD)
• RFC PR: (leave this empty)
• Rust Issue: (leave this empty)

Summary

Right now, all intrinsics and functionality for interacting with non-rust code are provided by the compiler. Both intrinsics and external ABIs are provided through extern "XXX" blocks. With this RFC, we enable compiler plugins to provide their own intrinsic group, which can be used for custom code generation additions to the rust programming language.

Motivation

My original motivation for doing this is the rust-cpp project. I needed a way to connect into the rust process late enough that specialization had already occurred, so that I could correctly generate the code for generic invocations of C++ code. This would also be useful for extending the language with support for intrinsics which are not supported by the rust compiler, without resorting to inline assembly, among potentially other things.

Detailed design

NOTE: I have not thought this out nearly enough for this RFC to actually be accepted, it is currently just a stepping stone towards figuring something out. (A.K.A. This is a horribly under-specified RFC)

This feature already requires compiler plugins, which are gated behind the plugin feature gate. It would add another feature gate intrinsics_groups which would track this particular plugin feature.

Plugin developers would, in the plugin_registrar function, call the method Registry::register_intrinsic_group(name: String, handler: Box<IntrinsicHandler>).

The programmer, in the code using the plugin, would then be able to define an extern block. This block would use the string "{}-intrinsics" (where {} is the name provided in register_intrinsic_group). This naming convention is chosen to avoid any of these custom intrinsic groups conflicting with future ABIs which rust wishes to support in the future.

extern "CUSTOM_NAME-intrinsics" {
    fn some_intrinsic<S>(some_argument: S, some_other_argument: int);
}

These functions could then be called exactly like any other intrinsic or extern-declared function. They would require an unsafe block to call.

The IntrinsicHandler trait would look something like the following:

trait IntrinsicHandler {
    fn validate_decl(&mut self, /* XXX Rust Context, Intrinsic Declaration AST node */);

    fn handle_call(&mut self,
                   /* XXX Rust Context, Function Decl, and Generic Substitutions */,
                   /* XXX Codegen Context, and ExprRefs for arguments */) -> ExprRef;
}

During semantic analysis, declarations inside the custom intrinsic group would be validated by the validate_decl callback. This is intended to allow the plugin author to reject incorrect intrinsic declarations quickly and easily.

During codegen, when the code generator wishes to generate the LLVM Expression node for a call to one of these intrinsics, it would instead invoke the handle_call function on the intrinsic handler. This function will be provided by the plugin author, and implement the intrinsic.

Drawbacks

This would add some complexity to the compiler in exchange for a questionable benifit. In addition, there may be better ways to enable plugin developers to generate custom code which handles generics.

In addition, unlike previous forms of plugins, this plugin makes it's usage infectious. If a library uses a plugin which adds intrinsics, in the general case, consumers of that library will also need the plugin installed and enabled. This is because the consumers of the library may call a generic function which needs to be specialized, and that specialization may invoke one of these custom intrinsics.

Alternatives

I haven't been able to come up with another solution for solving the problem of exposing specialization to plugins. That doesn't mean that there isn't a much better one, however.

The impact of not implementing this is that plugins won't have access to custom code generation on specific specialization paths, which makes some types of compiler plugins impossible.

Unresolved questions

Unfortunately I don't know enough about how compiler internals work to actually figure out how this would be implemented, or even what the arguments to the callback methods on IntrinsicHandler would be.

In addition, I have not considered fully what the wider impact of this RFC would be on other parts of the rust plugin ecosystem.

Should there also be a validation step running on every call site before specific code generation? This might be necessary for intrinsics like transmute, which perform validation on their types beyond just the generic type parameters. This could possibly be done by allowing the code generation logic to report errors.

I feel like a good metric of whether this API will actually be useful would be to see if most/all of "rust-intrinsics" could actually be implemented using it. if it cannot, then the API is probably not general enough.