Skip to content

Instantly share code, notes, and snippets.

Created Dec 8, 2016
What would you like to do?

MOSS report

Background (previous state of code)

Before the grant, Mio was the only asynchronous I/O library for Rust. The goal of Mio was to provide a zero cost abstraction interface on top of the operating system's asynchronous I/O functions. It did this by providing an event loop. I/O handles, such as TCP sockets, would be registered with the event loop and readiness notifications would then be delivered once the socket becomes ready to read data from or write data to.

However, Mio's goal of being a zero cost abstraction meant that it could only provide limited features on top of the operating system's facilities. Mio's API, while an idiomatic Rust API, was low level. A user of Mio operated directly on the I/O handles and there wasn't much in the way of a framework provided for organizing code consuming Mio.

The most common feature request that I received was to provide some way for libraries using Mio to be able to share a single event loop. For example, if both a DNS client and an HTTP client are implemented using Mio, then the user of both the DNS client and the HTTP client should be able to run a single event loop (and thus a single thread). Besides this feedback, I also received a good amount of feedback stating that Mio was hard to use compared to solutions that exist for other languages.

Being able to share an event loop was not possible due to the fact that sockets registered with Mio must provide a token. This token is included in all readiness notifications triggered by Mio. This enables the developer to associate a readiness notification with the I/O handle that the notification is related to. Each event loop has a single token space, so any library built on top of Mio would risk having tokens conflict with other libraries sharing the same event loop.

What you applied to do, and how much for

I applied for a $30,000 grant to build a high-level API for Mio. The amount requested was to enable me to spend three full months on the project. The specific goals were as follows:

  • Provide a ‘Service’ trait that allows describing RPC-style network requests.
  • Use a Future based API to represent asynchronous results.
  • Provide building blocks to implement TCP services on top of Mio's core API.
  • Implement some simple services using this new API for demonstration purposes.

The idea was to provide a layer on top of the existing Mio API that would resolve issues encountered when trying to use Mio. I also had a much more ambitious goal of making it as easy as possible to write high performance network clients and servers in Rust. To accomplish this, I needed to provide facilities for managing complex asynchronous I/O code as well as provide a library that handled all the associated "boilerplate".

There are a number of strategies for writing manageable asynchronous code. I chose to use futures. I had already written a future library for Rust: Eventual. It was not yet ready for production use. My goal was to get it to such a point and then use it as a building block for building a future based I/O library.

The next step was to use the future library and Mio to provide a higher level API. In other words, there would still be TCP sockets, but instead of having to register the sockets with the event loop, get readiness notifications, then perform non-blocking reads / writes on the socket handles, the new higher level API would only have read / write functions and those functions would return futures representing the completion of the operation.

Once the future based API existed, my plan was to take a look at all the boilerplate that is common in asynchronous I/O protocol implementations (HTTP, DNS, SQL, MongoDB, etc...) and to extract as much as possible into reusable building blocks. I wanted it to be possible for library authors to only have to focus on what was unique about the protocol they were implementing instead of having to implement the same things over and over again. A key requirement to do this was to standardize an API that library authors would provide: the Service trait.

What you did, and whether it was exactly the same as what you planned

I am very pleased with what was achieved thanks to the MOSS grant.

I started to build out the new API as described above. As the code started to grow, it became clear that the scope of the new API really needed to exist as a dedicated project instead of part of Mio. This is how Tokio came to exist. Extracting the higher level API into Tokio also enabled me to reduce the Mio API to be entirely a "portable epoll" library.

While I was working on the early version of Tokio, Alex and Aaron of the Rust core team expressed interest in the project and started to get involved. There were a lot of discussions around the future library (Eventual). I originally wrote Eventual before Rust reached 1.0 and the language was quite different. We revisited a lot of the design of the future library and end up writing one from scratch.

Tokio itself was built out as a set of smaller libraries that targeted the various points in the I/O stack. As of today, the primary components are Tokio Core, Tokio Service, and Tokio Proto.

Tokio Core is the library that combines Mio and futures. It provides TCP and UDP primitives as well as a reactor (event loop) that allows multiplexing and isolating how code runs on a single reactor. A library can be written to use a Tokio reactor without having to be concerned about any other libraries running on the same reactor. This solves the problems that users had with Mio.

Tokio Service is an abstraction that handles asynchronous request / response exchanges. The concept is heavily inspired from Finagle and the "Your Server as a Function" paper. The idea is that by reducing a server (and client) API to a single function of a request to a future of a response, a lot of the boilerplate can be extracted. The goal here is that the community can work to create these reusable components as they are needed. One example of a reusable component is the timeout service middleware. It is a little bit of code that aborts a request if the response takes too long to generate. The code is here and also depends on tokio timer. Tokio Service is mostly useful for application level developers.

Tokio Proto includes builders useful for implementors of protocols. It handles a number of cases out of the box such as multiplexing, pipelining, and streaming bodies. This library is mostly useful for protocol level developers. For example, Hyper provides an HTTP library and is working to migrate to use Tokio. Hyper is using Tokio Proto and is able to delete 1,500 lines of code while gaining additional features and robustness.

There are also a number of repositories in the tokio-rs Github organization that serve as examples for how to use Tokio. tokio-minihttp is a working, but minimal HTTP implementation. tokio-line is a client / server implementation of a simple line based protocol. I also wrote a MiniDB, a toy distributed database to demonstrate how to use Tokio in a real application.

Tokio is being built as a set of loosely coupled modules for now. In the future, there may be a Tokio meta package that pulls in all of these components, but for now we are waiting to see how users end up using Tokio in practice.

I consider the current state of Tokio to have surpassed the goals set by the MOSS grant, but of course there is still quite a bit of work to do.

How long it took, and whether that was what you expected

I originally budgeted for three months of full time work. I ended up spending approximately four months of full time work followed by about 16-20 hours / week on an on going basis. However, I achieved the goals outlined as part of the MOSS grant before the three months had elapsed. The rest of the time was spent on iterating, improving the design, and polish work.

Before I submitted the MOSS grant application, I had already put a lot of thought into what the design of the higher level API would look like. I already had researched similar projects for other languages. This let me have a good idea as to the amount of work needed to achieve the outlined goals. No code was written before the MOSS grant work begain though.

When Alex and Aaron from the Rust core team joined the project, a lot more time was spent on design discussions than I had originally anticipated. These design discussions definitely ended up paying for themselves in terms of a better end result.

What positive outcomes you have seen from the work being done (uses of the code, people appreciating new features, community growth etc.)

There has definitely been a lot of positive community feedback. Most of it so far as been from people indicating that they were very excited that Tokio was happening because asynchronous I/O is important to them. The community around Tokio has been growing at a strong pace. Since the release, there have been numerous issues and PRs submitted on GitHub. There are currently 234 people registered on the Gitter channel that participate in conversations and ask questions.

What’s most exciting about our community is people are already trying to build things using Tokio. Some of these projects include:

  • Hyper is rewriting their core to use Tokio.
  • Tarpc, an RPC framework, is built on Tokio.
  • AgilData built a MySQL proxy with Tokio.
  • TRust-DNS is migrating to use Tokio.
  • NPM is planning to use Tokio for internal services.

There are also many more projects under active development. As our community grows to include more types of developers, we are looking forward to see what they create with the new enhancements we will be releasing to the Tokio library over the next coming months.

What is the long term sustainability plan

As of today, the Tokio project is sustainable. A core set of functionality has been created that is valuable. A core team, consisting of Aaron Turon, Alex Crichton, and myself have all budgeted for time on an ongoing basis to maintain the project and continue development. The community is growing and there are people coming out who are interested in participating in the project. New features will be developed and as companies start depending on Tokio, I believe that it will be possible to find funding from these companies.

The goal of the second MOSS grant is to enable development that would significantly accelerate both Tokio adoption and Rust adoption. The focus of the second MOSS grant wouldn't be on the Tokio core itself. There are currently enough resources to manage that. The second MOSS grant is to focus on an HTTP layer on top of Tokio, which is the biggest missing component for the majority of potential Tokio + Rust users.

I believe that by building an HTTP specific abstraction on top of Tokio Service that is similar to Ruby's Rack, Node.JS's Express, and Java's Servlet as well as providing a robust HTTP 2.0 implementation, Tokio + Rust will be a very attractive solution for many people who may not be considering Rust at all today.

In other words, Tokio is sustainable today and will evolve on a modest adoption curve. The MOSS grant would enable the undertaking of a larger project that will be able to significantly accelerate the adoption of Tokio & Rust.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment