I want to write about what it means to be a Rustacean. To cut to the chase, I think that being a Rustacean is about a lot more than using Rust. It's about an approach to problem solving that is based on kindness and deep curiosity -- and I think that approach is the key to all Rust has accomplished.
Moreover, I feel that we should extend those same values of kindness and curiosity to the ecosystem at large. We should encourage experimentation, particularly in young areas where we don't have everything figured out. I think this is particularly applicable to the nascent async ecosystem -- but I'm getting ahead of myself.
Let's go back in time a bit. As many folks know, Rust went through a long -- and quite public -- period of evolution and design. I started work on Rust in 20111, shortly before the 0.1 release, but it had already been an active project for some time by then.
The Rust of yore was very different from today's Rust in almost every particular:
- no borrow checker, minimal use of ownership types
- no trait system
- lots of concepts built-in to the language, including a green threaded runtime
- a simple garbage collector based on reference counting
Still, despite all the change that has happened in the interim, it still feels to me that Rust has "stayed true to itself". This is because Rust was not defined by some technical detail, but rather by its ideals:
- uncompromised efficiency
- safety and correctness
- a culture that emphasize kindness, deep curiosity, and research
Of these three ideals, I think that the final bullet -- kindness and curiosity -- is the most important. After all, it's clear that, left to their own devices, efficiency and safety tend to pull in opposite directions. Peak efficiency is easiest to achieve by giving users control. Safety on the other hand requires limits.
Overcoming this inherent tension requires talking, thought, and exploration. It means working with one another closely. It often requires finding (and fixing) the flaws in a bunch of otherwise good designs. And this kind of exploration simply cannot be done without kindness and curiosity.
Rust has always had a code of conduct. The CoC says a lot of things, but I remember that when I first came to Rust, I found the following bullets very inspirational:
- Please be kind and courteous. There’s no need to be mean or rude.
- Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
- Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
I highlighted three points from the CoC, but I think the final point contained an interesting suggestion:
If you have solid ideas you want to experiment with, make a fork and see how it works.
You can already see the kernel of an idea here: sometimes, the best way to test and prove out an idea is to go and try it.
Now, when this sentence talks about "fork", it of course means a GitHub fork. But I think that the underlying sentiment is often true in a bigger sense. If somebody has a project that you like, but there are some fundamental choices you'd like to change, you have two choices:
- you can try to convince them you are correct;
- or, you can build your own thing.
Both have their pros and cons, but there is nothing wrong with the latter. Oftentimes, starting a new project is the easiest way to prove out an idea. If it works, you might merge it back, but you might also just run with it. A lot of the problems in life are caused by people trying to stay together when they really ought to part ways.
One of the key decisions we made in the history of Rust was to lean hard on cargo, our package manager. We saw that a having a rich standard library made for a great user experience, but that it could unintentionally hamper the development of an ecosystem. It also implied a large maintenance burden for us. We thought it would be better for everyone if the standard library was lean and the ecosystem was rich and active. I think this bet has been proven out many times over.
It may seem like competition and collaboration are opposites. But I don't think so. If you can find a way to have good relations with your competitors, they have a lot to offer -- nobody quite knows your space the way that they do!
This is part of the reason that we in the Rust community have always tried to be very careful in how we talk about other languages. Sure, we are competing with Go, C++, and JavaScript -- in the sense that individual projects have to choose a language to be written in. But there are many projects out there. And Go, C++, and JavaScript have a lot to offer us! For example, I've looked at Go's scheduler routines for inspiraton when hacking on Rayon. And C++ offers the basis for so many of Rust's designs. JavaScript has a lot of lessons for us around async code -- not to mention that it gave rise to WASM, which I hope will be a fruitful avenue for Rust in the future.
This is not to say it's easy. After all, you are competitors for a reason. There must be things you like about your project better than the other, and you should talk about those. But you can do that in a way that is calm and fair, and you can acknowledge that there are reasons one might prefer a different design.
It's worth pointing out, as an aside, that this idea of "competing without malice" applies equally well within a project as across projects. The whole premise of Collaborative Summary Documents, for example, is to try and shift the dialog from a thread where people are competing to prove their point to a collaboration on exploring the design space. When we do this process, it's not like people forget what they prefer. It's just that they try to put that aside.
One thing that can help to enable "competition without malice" is to create standards -- often in the standard library. Standard interfaces mean that customers can migrate cleanly between competitors. Suddenly, it's not so important that people pick your product at first -- after all, now, if they find that they don't like the competition, they can always migrate over to your system later. Moreover, standard interfaces means that people can write libraries and tools that work with all system, and such those people don't have to "pick a side" at all.
So we need common interfaces, but designing these common interfaces is hard. The details matter. If you make the wrong choice, you can unintentionally prohibit people from doing what they want to do. For this reason, it often doesn't make sense to design a common interface until you've let the ecosystem build up enough to have some competitors, so that you can get some idea of the design space involved.
Designing common interfaces for Rust also has another challenge. As the first (major) language to embrace ownership and borrowing as type-system concepts, Rust is carving out new ground. This often means that the interfaces that have evolved from other languages don't work so well in Rust.
By now, I've seen several people observe that there are tensions in the async space. The current locus of those complaints are the two major async runtimes: async-std and tokio. But these tensions go further back.
Honestly, I think you can trace these tensions back to the very beginning of Rust. Remember how I said that, when I came to Rust in 2011, it included a green-threaded runtime? We ultimately decided to split that out in RFC #230, but those who were around at the time can attest that this conversation was very divisive. I don't think it's an exaggeration to say that we came close to a hostile fork around the issue. The discussions around the design of the Future trait and its associated types (Waker, Context, etc) haven't been that much easier.
Part of this is that we are forging very new ground here. The
poll-based design of our Future trait is novel -- it is not the same
as what, e.g., libraries like Finagle did. Our async-await syntax
looks similar to JavaScript, but the "lazy semantics" are very
different under the hood. These changes are the key to making Rust's
futures zero-cost. But they also mean that there are a lot of
unknowns in terms of how to tweak the design, which in turn
exacerbates the challenges of reaching consensus. Combine that with a
RFC process in need of some tweaking and you've got a recipe
for bad feeling.
Here is the good news: as of this next release, we will have stabilized the core, foundational pieces of Rust's async story. This is a huge enabler, and I think it's going to be a major turning point in the history of Rust. Just using what exists today, it already means that people can write things like future combinators that are independent of any particular runtime. In many cases, it should also be possible to create frameworks that are configurable to switch between runtimes.
The bad news is that we've still got a number of core concepts left to
stabilize. It seems clear we need a standard trait for streams, along
with language facilities to make writing streams/iterators easier.
We're going to need traits for things like AsyncRead. Each of these
steps that we take is going to enable more and more experimentation in
the ecosystem around it.
But if we're going to do that job well, we're going to need experimentation. We're exploring new territory here, and we need as many people as possible carving paths through that wilderness.
In that light, I'd like to put forward that I am glad to see competition around the async runtime space! I think tokio is a great project. It has been around a while and has a very mature set of features. It is fast and getting faster. I often hear people tell me at Rust meetups how great tokio is. I myself love to point out how tokio (and hyper!) are the top of the tech-empower benchmarks2.
But I also think async-std is bringing a lot of innovations to the table. This starts with their idea to mirror the API of Rust's standard library as closely as they can -- I'd like to see them finish that, and I'd like to compare that to tokio's APIs, and understand which differences are important, and which are not. async-std is also doing a lot of clever things at the lower levels too. I'd like to see where that goes.
Ultimately, I expect that both projects will learn from one another and improve. And that's to say nothing of projects like embrio-rs, targeting embedded devices, or projects like actix, which add an actor-like interface. I'm sure that they too will have a lot to teach us.
Of course, there are costs to having multiple standards. Nobody wants to split the async ecosystem in two, and having too many choices can be confusing. But I would rather that we address this by enabling projects to interoperate smoothly, and by standardizing key interfaces so that it becomes easier and easier to write code that is agnostic about the underlying runtime.
(More generally, particularly in exploration phases, we should look for opportunities to allow multiple projects to flower, without causing undue problems for users. I think we often get stuck in RFCs trying to find the "perfect" solution, when it might've been better to first ship the building blocks, and then let people experiment with what the final product should look like.)
It's worth pointing out that these challenges are not specific to async. The same trends apply in many domains. For example, as I was revising this post, I saw this reddit thread discussing the reclutch library, which seems to be hitting on some similar themes. (I was happy to see a number of folk encouraging experimentation, however.)
At this point, I think what makes the most sense is for us to give all the runtimes space to develop. Async-await syntax isn't even stable yet! Let's give these runtimes some time to pursue their own, independent visions and see what comes of it. I believe this will pay off: both because we'll have stronger, better runtimes to choose from, and because -- as we move forward with future standardizations -- we'll have a broader range of experience to draw upon.
So let me leave you with a call to action: Let's explore this space together, and let's build something great.
Footnotes
-
Here is a cute picture of me and my daughter taken at the time, along with the Mozilla dinosaur from the old offices on Castro Street. ↩
-
As ever, I would advise you to take benchmarks with a grain of salt. Though it's very interesting to compare hyper's relatively clean benchmark harness with some of the, um, rather hacky harnesses of the competitors. Go hyper! ↩