Docker Container Reaper in Ponylang (a learning exercise)

reaper.pony

use "collections"
use "json"
use "net"
use "net/http"
use "time"

actor Main
  """
  This is my learning exercise for the Pony language (ponylang.org). It is a
  port of a Java application that:

    * Maintains a Map of ID to Timestamp.
    * Periodically scans the entire Map and kills the Docker containers
      corresponding to those IDs with a Timestamp that is too old.
    * Clients can query the content of the Map via an HTTP GET request.
    * Clients can add/update an ID in the Map with an HTTP PUT request. The ID
      should be atomically inserted or updated in the Map with the current time.
  """
  let defaultHost: String = "0.0.0.0"
  let defaultPort: String = "8080"
  let defaultLimit: USize = 100

  new create(env: Env) =>
    serve(env, env.root)

  fun serve(env: Env, rootAuth: AmbientAuth) =>
    let manager : ManagedIds = ManagedIds
    Server(
      where auth = NetAuth.create(rootAuth)
      ,     host = try env.args(1) else defaultHost end
      ,     service = try env.args(1) else defaultPort end
      ,     limit = try env.args(2).usize() else defaultLimit end
      ,     handler = HttpHandler(manager)
      ,     notify = InfoListener(env)
      ,     logger = CommonLog(env.out)
      )

  fun serve(env: Env, rootAuth: None) =>
    env.err.print("No authority to run")


actor ManagedIds
  """
  This actor owns the Map and runs the periodic scan-and-kill (not yet
  implemented).
  """

  let _map: Map[String, Timestamp] = _map.create()
    .add("foo", Timestamp.now())
    .add("bar", Timestamp.now())

  be list(fn: {ref(String)} iso) =>
    var rows: List[String] = List[String]
    for (key, ts) in _map.pairs() do
      var date = ts.formatISO8601()
      var row = JsonObject.from_map(Map[String, JsonType]
        .add("id", key)
        .add("expiry", date)
      ).string(where pretty_print=true)
      rows.push(row)
    end
    (consume fn)("blah") // TODO: haven't worked out the string join yet


class HttpHandler

  let _manager: ManagedIds

  new val create(manager: ManagedIds) =>
    _manager = manager

  fun apply(request: Payload) if request.method == "GET" =>
    """
    Handle GET requests
    """
    _manager.list(
      object iso
        var _request: Payload = consume request
        fun ref apply(rows: String) =>
          let response: Payload = Payload.response()
          response.add_chunk(rows)
          response.add_chunk("\n")
          let r: Payload = _request = Payload.request()
          (consume r).respond(consume response)
      end)

  /*
  fun apply(request: Payload) if request.method == "PUT" =>
    """
    Handle PUT requests
    """
    //_managedIDs.insert(request.url.path, Timestamp.now())
    //_managedIDs = _managedIDs.add(request.url.path, Timestamp.now())

    let response = Payload.response(StatusOK)
    (consume request).respond(consume response)
  */

  fun apply(request: Payload) =>
    """
    Handle all other HTTP methods and return 405 Not Allowed
    """
    let response: Payload = Payload.response(StatusMethodNotAllowed)
    (consume request).respond(consume response)



class InfoListener
  let _env: Env

  new iso create(env: Env) =>
    _env = env

  fun ref listening(server: Server ref) =>
    try
      (let host, let service) = server.local_address().name()
      _env.out.print("Listening on " + host + ":" + service)
    else
      _env.out.print("Couldn't get local address.")
      server.dispose()
    end

  fun ref not_listening(server: Server ref) =>
    _env.out.print("Failed to listen.")

  fun ref closed(server: Server ref) =>
    _env.out.print("Shutdown.")


class val Timestamp
  let _seconds: I64
  let _nanos: I64

  new val now() =>
    (_seconds, _nanos) = Time.now()

  fun formatISO8601(): String val =>
    Date(_seconds, _nanos).format("%FT%TZ")

Error:
/Users/nbartlett/Projects/reaper-pony/reaper.pony:67:23: argument not a subtype of parameter
      request.respond(response)
                      ^
    Info:
    /Users/nbartlett/Projects/reaper-pony/reaper.pony:66:21: Payload iso! is not a subtype of Payload iso: iso! is not a subtype of iso
          let response: Payload = Payload.response()
                        ^

The error is related to trying to use the iso in your lambda capture (in particular, the request parameter).

I haven't done anything with iso's and lambda's before (and its the end of day here) so I'm not sure the correct way to address that.

you'll get past your first error by doing:

      request.respond(consume response)

which will lead you to this...

      request.respond(consume response)
                     ^
    Info:
    /Users/sean/Dropbox/Private/Code/pony/repear/s.pony:67:7: receiver type: this->Payload iso!
          request.respond(consume response)
          ^
    /usr/local/lib/pony/0.2.1-1137-gaa273a3/packages/net/http/payload.pony:101:3: target type: Payload iso
      fun iso respond(response': Payload) =>
      ^
    /Users/sean/Dropbox/Private/Code/pony/repear/s.pony:57:22: Payload tag is not a subtype of Payload iso: tag is not a subtype of iso
      fun apply(request: Payload ) /* if request.method == "GET" */ =>

request needs to be an iso to call respond but inside that capture you have a this->Payload iso!

I dont think you can do what you want because request has to be an iso and the capture in the lambda means that its an alias to a Payload and that violates the capabilites. You can send that fn to multiple actors and violate the iso capability. And, its not letting you do that.

The error after Sean's fix is caused by this line:

request.respond(consume response)

Here request is being aliased (that's what the iso! in the error means). It's being passed as an implicit parameter to the respond method. It needs to be consumed with:

(consume request).respond(consume response)

But doing that gives this error:

main.pony:67:8: consume must take 'this', a local, or a parameter
      (consume request).respond(consume response)

This is because the request variable is passed in the environment of the closure. It can't be consumed. The reason it can't be consumed is because then the variable in the environment is pointing to nothing. It has no type. It needs to be a valid Payload, not a consumed Payload. This is one of those times where an object literal shows what is happening better than a lambda. Replacing it with an object literal is code like:

 _manager.list(object iso
                          var _request: Payload  = consume request
                          fun ref apply(rows: String) =>
                             let response: Payload = Payload.response()
                             let r: Payload = _request = Payload.request()
                             (consume r).respond(consume response)
                             None
                       end)

This code is now valid. Notice we consume the request when assigning it to our local _request field. We can't just consume that later in the respond call because that would leave _request in an invalid state. So we have to use destructive read. This will replace _request with an empty payload while at the same time assigning it to something we can consume safely.

This brings a further compile error;

main.pony:61:19: argument not a subtype of parameter
    _manager.list(object iso
              ^
    Info:
    main.pony:63:21: ref method is not a subtype of box method
                        fun ref apply(rows: String) =>
                        ^
    main.pony:63:21: object literal iso is not a subtype of {(String)} val: method 'apply' has an incompatible signature
                        fun ref apply(rows: String) =>

This error is a bit easier. Our apply method in the object literal is a ref so we need to change the type of it to say that:

 be list(fn: {ref(String)} iso) =>

Notice the ref. This annotation makes the apply method change from the default box to a ref. See closures in Pony.

I've also changed the type of fn to use the iso capability. We're passing an iso object in. This means it needs to be consume when called too:

(consume fn)("blah")

BTW, you can still use a lambda instead of the object literal, I just find it a bit easier to work out what is happening when investigating errors. The equivalent lambda is:

_manager.list(recover iso
                lambda ref(s:String)(_request = consume request) =>
                  let response: Payload = Payload.response()
                  let r: Payload = _request = Payload.request()
                  (consume r).respond(consume response)
                end
              end)

Thanks Sean and Chris, this is excellent!

I have updated the code in the Gist as per your suggestion, and it does compile and work. However, is this really idiomatic Pony code?? Isn't there an easier way?

Chris, yes I noticed that I couldn't consume request inside the lambda for the reason you stated. I tried to do this:

_manager.list(recover iso lambda(s:String)(consume request) =>

But that gave the error: syntax error: expected capture after. I'm not sure why I can't just do a consume inside the capture parentheses...

However, is this really idiomatic Pony code?? Isn't there an easier way?

For whatever it's worth, I consider the current HTTP server code in the pony standard library to have some significant usability issues, including the ones you've run into. It's in need of a redesign by RFC, but no one has stepped up to the task just yet. I might take it on in a few weeks after I've cleared some other work off my plate.

@jemc: Thanks, I'm glad it's not just me!