This is a set of features to make Pin easier to use.
The reason Pin is so difficult to use is that it does not have any of the syntactic sugar or support of ordinary reference types, because it is a pure library type. We implemented it this way in 2018 to minimize the set of changes we were making to Rust as part of the async/await "minimum viable product." However, with some relatively non-invasive changes to the language, the useability of pinned references could be brought close to parity with unpinned references.
Two new reference operators are added &pinned and &pinned mut, which evaluate to pinned references (Pin<&T> and Pin<&mut T> respectively).
If the type of the value at the place referenced implements Unpin, these behave exactly like the other reference operators. If the type of the value does not implement Unpin, these move and invalidate the value except for other pinned reference operators. If it is not possible to move the value in that case (because it is not owned), this is an error. If the arguments of these is a value and not a place, a temporary is created, like the other reference operators.
If the place is the target of a pinned reference, this is permitted as a reborrowing operation.
If the place is a field behind a pinned reference (a "pinned projection"), this can be permitted if the type permits pinned projections. See the #[pinned_fields] attribute section.
These operators serve multiple functions currently served by library APIs:
- They replace stack pinning macros when they take a value.
- They replace
Pin::newforUnpinvalues. - They replace
Pin::as_mutby supporting re-borrowing.
Method calls will can automatically insert a pinned reference operator, including as a re-borrow, using the same rules they use for unpinned references. In other words, this line of the reference will change:
Then, for each candidate T, add &T and &mut T to the list immediately after T.
To add the pinned reference operators as permissible operators.
The combination of this and the above feature will make it possible for pinned references to be used without specifying it as an extra step, but with the compiler enforcing its requirements; this makes them equally as ergonomic as other references. For example, consider this definition of Stream:
trait Stream {
type Item;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>>;
// Note! This is not the `Self: Unpin` version of next, instead it takes a pinned reference
fn next(self: Pin<&mut Self>) -> Next<'_, Self> { ... }
}This next adapter can be used without explicitly pinning:
let mut stream1: impl Stream;
// inserts a `&pinned mut` operator and stream1 is pinned
// you can keep calling next or other pinned methods but not move it
stream1.next().await
stream1.next().await
let mut stream2: impl Stream + Unpin;
// inserts a `&pinned mut` operator but stream2 is unpinned;
// you can move it freely again later:
stream2.next().await
stream2.next().await
owning_function(stream2);This eliminates the last use of pinning in "end user facing" APIs: trying to interact repeatedly with a stream without for await loop.
The combination of these features brings us close to parity with the original Move proposal, except that the "pinning after address is taken" effect only applies to APIs that explicitly take pinned references, as opposed to any API that takes a reference. This is better in some ways, because it allows you to reference values without pinning them.
It is totally safe to assign to a pinned mutable reference, but currently you have to use the Pin::set API. It should just be possible to assign to a pinned place the same way you can assign to a normal mutable reference:
let x: Pin<&mut Option<T>>;
*x = None;Note that you can even do this to a field projection; whether that field is pinned or unpinned, it is always safe to reassign it through a pinned pointer:
self: Pin<&mut T> where T has a foo field:
self.foo = Foo::new();Special syntax can support pinned method receiver arguments, just like normal method receivers:
trait Future {
type Output;
fn poll(&pinned mut self, cx: &mut Context<'_>) -> Poll<Self::Output>;
}
trait Stream {
type Item;
fn poll_next(&pinned mut self, cx: &mut Context<'_>) -> Poll<Option<Self::Item>>;
}This is more ergonomic in itself than self: Pin<&mut Self>, but it has another key advantage. Right now, if you call a method on the receiver that requires mutablility (like Pin::as_mut), you must declare self mutable. If you don't, you get a warning when you add the mut. This syntax can be treated as mutable but not emit the warning if the mutability isn't used.
This brings pinned methods up to parity with normal by-reference methods, and avoids additional touch points in which the user gets an error or warning and needs to edit their code to satisfy the compiler.
The final piece of improving the experience of implementing a type which uses pinning (like a future or a stream) would be an attribute to permit pinned projections to fields:
#[pinned_fields]
struct Foo {
bar: Bar,
}With this attribute, it is safe to pin project from a Pin<&mut Foo> to a Pin<&mut Bar>. The attribute imposes two additional requirements on the type:
- Its only
Unpinimplementation must be the default auto trait inherited implementation; it cannot have an explicitUnpinimplementation. - If it has its own
Dropimplementation, it must be declared with a pinned receiver, instead of a normal mutable receiver:
impl Drop for Foo {
fn drop(&pinned mut self) { ... }
}This second rule can be enforced because you already aren't allowed to call Drop::drop, so it's possible to have some types that permit a different signature on this method.
An additional attribute can be applied to fields to make them not pinned; these instead permit normal projections from the pinned interface, rather than pinned projections:
#[pinned_fields]
struct Foo {
bar: Bar,
#[unpinned] baz: Baz,
}Combining all these features, the experience of implementing a type involving Pin can be made much more similar to the ordinary experience. Here is an example of an implementation of Join, with all of the uses of pin projections and implicit pinned reference operators highlighted:
#[pinned_fields]
struct Join<F1: Future, F2: Future> {
fut1: F1,
fut2: F2,
#[unpinned] res1: Option<F1::Output>,
#[unpinned] res2: Option<F2::Output>,
}
impl<F1: Future, F2: Future> Future for Join<F1, F2> {
type Output = (F1::Output, F2::Output);
fn poll(&pinned mut self, cx: &mut Context<'_>) -> Poll<Self::Output> {
// unpinned immutable projection:
if self.res1.is_none() {
// pinned projection and reborrow:
if let Poll::Ready(res) = self.fut1.poll(cx) {
// unpinned projection and reborrow:
self.res1 = Some(res);
}
}
// unpinned immutable projection:
if self.res2.is_none() {
// pinned projection and reborrow:
if let Poll::Ready(res) = self.fut2.poll(cx) {
// unpinned projection and reborrow:
self.res2 = Some(res);
}
}
// unpinned immutable projection:
if self.res1.is_some() && self.res2.is_some() {
// unpinned projection and reborrow:
let res1 = self.res1.take().unwrap();
// unpinned projection and reborrow:
let res2 = self.res2.take().unwrap();
return Poll::Ready((res1, res2))
} else {
return Poll::Pending
}
}
}This attribute should work properly with enums and allow projections through destructuring as well. Here's another version of join using an enum to save space:
#[pinned_fields]
enum MaybeDone<F: Future> {
Done(#[unpinned] Option<F::Output>),
NotDone(F),
}
impl<F: Future> MaybeDone<F> {
fn poll(&pinned mut self, cx: &mut Context<'_>) {
// pinned projection via match ergonomics; fut: &pinned mut F
if let MaybeDone::NotDone(fut) = self {
if let Poll::Ready(res) = fut.poll(cx) {
// pinned reference assignment:
*self = MaybeDone::Done(Some(res));
}
}
}
}
#[pinned_fields]
struct Join<F1, F2> {
fut1: MaybeDone<F1>,
fut2: MaybeDone<F2>,
}
impl<F1: Future, F2: Future> Future for Join<F1, F2> {
type Output = (F1::Output, F2::Output)
fn poll(&pinned mut self, cx: &mut Context<'_>) -> Poll<Self::Output> {
// pinned projection and reborrow
self.fut1.poll(cx);
// pinned projection and reborrow
self.fut2.poll(cx);
// pinned projection and reborrow
match (&pinned mut self.fut1, &pinned mut self.fut2) {
// unpinned projection
(MaybeDone::Done(res1), MaybeDone::Done(res2)) =>
Poll::Ready((res1.take().unwrap(), res2.take().unwrap())),
_ => Poll::Pending
}
}
}This is undoubtedly the most "unergonomic" part of the pin APIs, so its worth asking if it would be possible to avoid this attribute if one were willing to make breaking changes to Rust, without abandoning Pin entirely. You could get rid of the attribute if you did two things:
- Make
Unpinunsafe to implement likeSendandSync. Implementing it for a type which you pin project through would usually be undefined behavior. - Make
Droptake&pinned mut selfevery time. This would avoid the issue with being able to move out of the mutable reference in the destructor.
Both of these are breaking changes, but they would allow you to avoid #[pinned_fields].
However, you could not avoid some sort of #[unpinned] attribute with Pin. It's just the case that there is a meaningful difference between fields you project to pinned, and fields you project to unpinned. This isn't necessarily a type-based distinction: you could project unpinned to a type that doesn't implement Unpin, because it isn't being pinned yet. For example, in the above Join implementations, its possible to write join futures that have other !Unpin futures as their output; those are not meant to be pinned by the pinned reference to the Join future, because you aren't polling them yet. You need to have some way of distinguishing which of these kinds of projections is permitted through this type.
The original Move proposal had a big problem with this, but we just didn't encounter it because we abandoned the proposal for backward incompatibility reasons before we got there. Consider Option<F::Output> in the above examples, where F::Output: ?Move. Is Option<F::Output>: Move or not? If it isn't, we can't take the output as above; but if it is, we can't project through Option to immoveable types. On closer inspection, I now think the original Move proposal was fatally flawed, even if it weren't backward incompatible. It really is helpful to distinguish between references which are pinned and references which are not pinned.
Another syntactic option would be to instead require a #[pinned] attribute on the fields you want to pin project to; this would then enforce the same requirements as the #[pinned_fields] attribute. The types above would then look like this:
struct Join<F1: Future, F2: Future> {
#[pinned] fut1: F1,
#[pinned] fut2: F2,
res1: Option<F1::Output>,
res2: Option<F2::Output>,
}
enum MaybeDone<F: Future> {
Done(Option<F::Output>),
NotDone(#[pinned] F),
}
struct Join<F1: Future, F2: Future> {
#[pinned] fut1: MaybeDone<F1>,
#[pinned] fut2: MaybeDone<F2>,
}This is a matter of taste, the semantics are the same.
This is far less important than any of the above features, but one additional feature to consider would be new deref operator traits for pinned dereferencing. These would be implemented by Box, for example:
let mut boxed_fut: Pin<Box<dyn Future<Output = ()>>>;
// pinned mutable dereference:
boxed_fut.poll(&mut cx)This would avoid needing to use Pin::as_mut to convert Pin<Box<T>> to Pin<&mut T>. Code that needs to do this seems pretty rare, so it may not be worth the trouble of a new operator trait.
The interface would look like this, and would be implemented for at least Pin<Box<T>>:
trait DerefPinned: Deref {
fn deref_pinned(&mut self) -> Pin<&mut Self::Target>;
}An immutable equivalent of DerefPinned is not necessary, because you can always immutable deref a pinned pointer; using that, you can go from Pin<P> to Pin<&P::Target> with the previous change to method resolution.
Another change that wouldn't have much effect on ergonomics, but may improve learnability, would be to add type synonyms for Pin<&T> and Pin<&mut T> that look like other reference types: &pinned T and &pinned mut T.
This wouldn't involve eliminating Pin entirely, because you still need Pin<Box<T>>. These two types would need to unify completely with the library-defined version of them. I'm not convinced this is a necessary change, but it might help make pin "feel" more native when learning about it than it does right now.
At first glance, these changes all seem backwards compatible and undisruptive. Possibly there are edge cases in method resolution that crater would encounter.
Some new keyword would need to be reserved in an edition for the pinned reference operators. I chose pinned to be less disruptive than pin, which is the name of the std module and many constructors.
This set of changes makes pinned reference much easier to work with, but does nothing to solve the "pinned iterator" problem: the problem that iterator as currently defined doesn't support self-referential generators.
One option is still to only implement iterator for pinned references to the generator type, instead of the generator type itself. With the automatic borrowing in method resolution, this might be made to work without the user having to explicitly pin the generators. This should be explored as the most backward compatible option.
Another option of course would be to change the definition of Iterator across an edition, so that it takes pinned references like it certainly should. This would not be trivial to do, but could be planned for 2027.
The third option is to deprecate Pin and move to some sort of Move-based solution, which would support iterators that enter the pinned typestate without being behind a distinct pinned reference type. This would be by far the least backward compatible option, because it would require adding a new auto trait or ?Trait (neither of which is a backward compatible change) and it would require disruptive changes to the existing async ecosystem as Pin is deprecated.
For a while, I was interested in the Move-based solution on the grounds that a similarly disruptive change would be needed to add linear types or unforgettable types, in the name of supporting scoped tasks and session types. However, looking at the relatively minimal changes needed to make Pin easier to use, I think sticking with Pin to represent the pinned type state is a far better solution.