README.md

Libprocess Developer Guide

Note This Developer Guide is Work in Progress.

The library libprocess provides high level elements for an actor programming style with asynchronous message-handling and a variety of related basic system primitives. Its API and implementation are written in C++.

Introduction

The design of libprocess is inspired by Erlang, a language that implements the actor model.

As the name already suggests, one of the libprocess core concepts is a Process. This is a single threaded, independent actor which communicates with other processes, locally and remotely, by sending and receiving HTTP requests and responses.

At a higher level, functional composition of processes is facilitated using futures and promises.

Overview

Table of Contents

• Processes and the Asynchronous Pimpl Pattern
• Futures and Promises
• HTTP
• Testing
• Miscellaneous Primitives

---

Processes and the Asynchronous Pimpl Pattern

A process is an actor, effectively a cross between a thread and an object.

Creating/spawning a process is very cheap (no actual thread gets created, and no thread stack gets allocated).

Each process has a queue of incoming events that it processes one at a time.

Processes provide execution contexts (only one thread executing within a process at a time so no need for per process synchronization).

delay

delay instead of dispatching for execution right away, it allows it to be scheduled after a certain time duration.

dispatch

dispatch schedules a method for asynchronous execution.

ID

Generates a unique identifier string given a prefix. This is used to provide PID names.

PID

A PID provides a level of indirection for naming a process without having an actual reference (pointer) to it (necessary for remote processes).

---

Futures and Promises

The Future and Promise primitives are used to enable programmers to write asynchronous, non-blocking, and highly concurrent software.

A Future acts as the read-side of a result which might be computed asynchronously. A Promise, on the other hand, acts as the write-side "container". We'll use some examples to explain the concepts.

First, you can construct a Promise of a particular type by doing the following:

using namespace process;

int main(int argc, char** argv)
{
  Promise<int> promise;

  return 0;
}

A Promise is not copyable or assignable, in order to encourage strict ownership rules between processes (i.e., it's hard to reason about multiple actors concurrently trying to complete a Promise, even if it's safe to do so concurrently).

You can get a Future from a Promise using the Promise::future() method:

using namespace process;

int main(int argc, char** argv)
{
  Promise<int> promise;

  Future<int> future = promise.future();

  return 0;
}

Note that the templated type of the future must be the exact same as the promise, you can not create a covariant or contravariant future. Unlike Promise, a Future can be both copied and assigned:

using namespace process;

int main(int argc, char** argv)
{
  Promise<int> promise;

  Future<int> future = promise.future();

  // You can copy a future.
  Future<int> future2 = future;

  // You can also assign a future (NOTE: this future will never
  // complete because the Promise goes out of scope, but the
  // Future is still valid and can be used normally.)
  future = Promise<int>().future();

  return 0;
}

The result encapsulated in the Future/Promise can be in one of four states: PENDING, READY, FAILED, DISCARDED. When a Promise is first created the result is PENDING. When you complete a Promise using the Promise::set() method the result becomes READY:

using namespace process;

int main(int argc, char** argv)
{
  Promise<int> promise;

  Future<int> future = promise.future();

  promise.set(42);

  CHECK(future.isReady());

  return 0;
}

NOTE: CHECK is a macro from gtest which acts like an assert but prints a stack trace and does better signal management. In addition to CHECK, we've also created wrapper macros CHECK_PENDING, CHECK_READY, CHECK_FAILED, CHECK_DISCARDED which enables you to more concisely do things like CHECK_READY(future) in your code. We'll use those throughout the rest of this guide.

TODO(benh):

• Using Future and Promise between actors, i.e., dispatch returning a Future
• Promise::fail()
• Promise::discard() and Future::discard()
• Future::onReady(), Future::onFailed(), Future::onDiscarded()
• Future::then(), Future::repair(), Future::after
• defer
• Future::await()

HTTP

libprocess provides facilities for communicating between actors via HTTP messages. With the advent of the HTTP API, HTTP is becoming the preferred mode of communication. Let's set up an empty HTTP process so that we can work through some examples. In these examples, we'll assume we have access to the process::http namespace:

using namespace process;
using namespace process::http;

class HttpProcess : public Process<HttpProcess>
{
public:
  HttpProcess() {}

protected:
  virtual void initialize() {}
};

class Http
{
public:
  Http() : process(new HttpProcess())
  {
    spawn(process.get());
  }

  ~Http()
  {
    terminate(process.get());
    wait(process.get());
  }

  Owned<HttpProcess> process;
};

route

route installs an HTTP endpoint onto a process. Let's install one onto an instance of our new Http process:

Http http_process;
http_process.route("/testing", None(), [] (Request request) {
  std::string arg = request.query;
  doImpressiveThings(arg);
  return OK();
});

Now we can do something like: $ curl localhost:1234/testing?value=42

get

get will hit an HTTP endpoint with a GET request and return a Future containing the response. We can pass it either a libprocess UPID or a URL. First, let's hit our endpoint locally using http_process's PID:

Future<Response> future = get(http_process.process->self(), "testing");

Or let's assume our serving process has been set up on a remote server and we want to hit its endpoint. We'll construct a URL for the address and then call get:

URL url = URL("http", "some.hostname", 80, "/testing");

Future<Response> future = get(url);

post and requestDelete

The post and requestDelete functions will similarly send POST and DELETE requests to an HTTP endpoint. Their invocation is analogous to get.

Connection

A Connection represents a connection to an HTTP server. connect can be used to connect to a server, and returns a Future containing the Connection. Let's open a connection to a server and send some requests:

Future<Connection> connect = connect(url);

connect.await();

Connection connection = connect.get();

Request request;
request.method = "GET";
request.url = url;
request.body = "Amazing prose goes here.";
request.keepAlive = true;

Future<Response> response = connection.send(request);

It's also worth noting that if multiple requests are sent in succession on a Connection, they will be automatically pipelined.

Testing

---

Miscellaneous Primitives

async

Async defines a function template for asynchronously executing function closures. It provides their results as futures.