A content negotiated resource in Freya

gistfile1.fs

open System
open System.Text
open Freya.Core
open Freya.Machine
open Freya.Machine.Extensions.Http
open Freya.Machine.Router
open Freya.Router
open Freya.Types.Http
open Microsoft.Owin.Hosting

(* Representation/Negotiation

   Freya doesn't impose any choice of serializer or encoding on you,
   as it's usually best left to the developer to decide what their app
   needs. We'll assume you've got some functions that will serialize
   your resources to JSON and XML in this case.

   We write a represent function which looks at the results of the
   content negotiation, given as an instance of the Specification type.
   We're only interested in the negotiation of MediaTypes in this case.
   The results returned are lists of acceptable MediaTypes, ordered by
   client preference.

   Our function returns a Representation (actually a Freya<Representation>
   as everything is a function in Freya!) which describes what this data is
   and how it's represented. Here we only care that we've encoded the data
    as UTF-8, and we've serialized it in a specific way.

   This is the kind of code you only write once per app in general. *)

let json _ =
    """{ "Hello": "World" }"""

let xml _ =
    """<Hello>World</Hello>"""

let represent res spec =
    let serialized, mediaType =
        match spec.MediaTypes with
        | Free -> json res, MediaType.Json
        | Negotiated (m :: _) when m = MediaType.Json -> json res, MediaType.Json
        | Negotiated (m :: _) when m = MediaType.Xml -> xml res, MediaType.Xml
        | _ -> failwith "Representation Failure"

    Freya.init
        { Data = Encoding.UTF8.GetBytes serialized
          Description =
            { Charset = Some Charset.Utf8
              Encodings = None
              MediaType = Some mediaType
              Languages = None } }

(* Resources

   We're creating a very simple resource here, all we're providing is a
   handler, which will be called when the server wants to return a
   200 OK response, and some configuration - in this case, which
   MediaTypes are supported for this resource. Again - the MediaTypes are
   actually given as a function - although they're static here, it is possible
   to change things like this at runtime.
   
   The simple fact of providing MediaTypes in this way means that the request
   will now be negotiated when an Accept header is present (this is a
   very declarative configuration model).

   You probably wouldn't specify MediaTypes on every resource in a real
   app - you'd define one or more resources containing common properties
   like this, and then include them within resources. Resources with
   Freya.Machine are completely nestable in this sense.

   You would likely do the same with the "using http" statement here, which
   tells the resource that it should use the HTTP processing state
   machine. *)

let exampleMediaTypes =
    Freya.init [
        MediaType.Json
        MediaType.Xml ]

let exampleHandler =
    represent "Hello World"

(* Allowed/Exists

   Supporting extra logic is optional, but you can add in extra logical checks
   (in fact overriding the default results of these checks) by adding new
   elements to the resource.

   In this case we've added two very simple decision
   functions (Freya<bool>) which just look at an untyped query string for
   their data. Of course normally you'd want to actualy be checking if the
   resource existed, or there was some token in the request which matched a
   requirement for being allowed to obtain a resource.

   These could be written more succinctly, but this way is clearer. In many
   cases you may also wish to only evaluate one of these functions once,
   though it's used in multiple places (per request) and you can do that
   by adding |> Freya.memo as we've done here with "exampleAllowed" - this
   will now be evaluated at-most-once per request. *)

let exampleAllowed =
    freya {
        let! query = Freya.getLens Request.query

        return query <> "forbidden" } |> Freya.memo

let exampleExists =
    freya {
        let! query = Freya.getLens Request.query

        return query <> "missing" }

let exampleResource =
    freyaMachine {
        using http

        allowed exampleAllowed
        exists exampleExists
        mediaTypesSupported exampleMediaTypes
        handleOk exampleHandler } |> FreyaMachine.toPipeline

(* Routing

   We need to hook our resource up to a path, so we use the Freya.Router
   to do this. Just a simple root resource for this example. *)

let exampleRouter =
    freyaRouter {
        resource "/" exampleResource } |> FreyaRouter.toPipeline

(* App

   We want to turn our router in to an OWIN AppFunc, so we can run it
   using any OWIN compatible server. *)

let exampleApp =
    exampleRouter |> OwinAppFunc.ofFreya

(* Katana

   We'll use Katana to host our little example app. It'll need an app
   type. *)

type ExampleApp () =
    member __.Configuration () =
        exampleApp

(* Entry

   Finally, we'll spin up our server as a quick and dirty console
   app on localhost for testing. *)

[<EntryPoint>]
let main _ =
    let _ = WebApp.Start<ExampleApp> "http://localhost:8080"
    let _ = Console.ReadLine ()
    0

Thank you for providing this example!

At this point, I understand the conneg part (the represent function), but I'm struggling with understanding how conditional handling would work.

The handleOk function seems to set up exampleHandler to always return a representation. How would I go about receiving and validating input, and e.g. return 403 (Forbidden) or 404 (Not Found) if the input was invalid?

Now updated with a very trivial example of adding extra logic to the resource (in this case 403/404 support). Hopefully it makes sense with the explanation! I've purposely written it in the most verbose style, as it's easiest to read to begin with. All of these condense to one-liners if you like more use of operators, but there's no requirement to do so!

One thing I haven't done here is override any more handlers - this is just letting Freya deal with how to return 403, 404, etc. (which is usually fine if you're serving up an API). You could also add "handleNotFound" and "handleForbidden" if you really needed to use your own presentation of errors.

Perhaps validation can be done based exclusively on the incoming request, but a 404 response will typically be based on the internal state of the application; e.g. a request coming in for /users/1235, but there's no user with the ID 1235 in the application's (persistent) store.

How would you model something like that?

How about a POST operation where you'd return 200 (OK), 201 (Created), or 202 (Accepted) upon success, but perhaps 500 (Internal Server Error) if something catastrophic happened while you attempted to persist the resource in a database?

Essentially, what I'm after is something like Scott Wlaschin's Railway Oriented Programming, where you get out of the HTTP framework (Freya) as quickly as possible, do whatever you need to do, and return the Either result, and then back in the HTTP framework (Freya) translate that Either value to an appropriate HTTP response.

In terms of the 404, yes you'll almost always need access to some state. Of course that depends slightly on how the rest of your app works, but I've been working with simply passing the state to the relevant function, so that when partially applied it results in a Freya<bool> or similar. So in a case where you had some function which need access to users, or similar, you might have function let userExists users = ... and then when constructing your resource, pass the resources needed through so exists exampleExists becomes something like exists (exampleExists users). It seems to work quite well in my experience, and is effectively inversion of control in a sane functional form (subjective opinion alert! :) ).

In terms of returning 500 in the event of catastrophes - I'm actually planning on making that change so that any function that throws will result in a 500. This needs a little thought to avoid edge cases, so I haven't done so yet - your opinion on that one is more than welcome. As you say, we could encode that as a specific sum type (Either, or similar) rather than (or as well as) throwing, which might be a good solution. It would be a relatively straightforward change while we're still pre-1.0 I think.

Opinions welcome!

Thank you for your answer. The approach of using Freya<bool> with various functions makes me a bit uneasy. What if I have a bunch of possible return values, like:

isValid = Freya<true>
isPossible = Freya<true>

//...

allowed isValid
exists isPossible

Both are true... I might have four or seven of these, but I can't do a single pattern match on the possible outcomes? I find that difficult to reason about.

As an example, in my talk Look, No Mocks! Functional TDD with F#, I end up composing an ASP.NET Web API controller like this:

[<CLIMutable>]
type ReservationRendition = {
    Date : string
    Name : string
    Email : string
    Quantity : int }

type ReservationsController(imp) =
    inherit ApiController()
    member this.Post(rendition : ReservationRendition) =
        match imp rendition with
        | Failure(ValidationError msg) -> 
            this.BadRequest msg :> IHttpActionResult
        | Failure CapacityExceeded ->
            this.StatusCode HttpStatusCode.Forbidden :> IHttpActionResult
        | Success () -> this.Ok() :> IHttpActionResult

I find the ability to delegate all functionality to a function (imp) that knows nothing about HTTP very powerful, but it means that there has to be a reasonably sane way of going back and forth between these worlds.

I see where you're coming from, the machine-style approach is a little more abstracted than that. Essentially the effectively "railway" part of things is within the machine framework (which is simply a decision graph, or state machine, where the developer can choose to answer certain questions in the graph themselves, rather than accept the default answers).

I've found that the resource definition, plus the functions for implementing the decisions, tend to be the HTTP "boundary" in this sense. The decision functions are essentially usually very thin functions transforming the result of a function which is part of a general (non-HTTP aware) API, in to whatever is needed to answer a graph question. As an example, see this API: https://github.com/freya-fs/freya/blob/master/tests/Freya.Machine.Tests/Api.fs which is part of our tests.

That's a wrapper (and a verbose one) around an API which has no knowledge of HTTP, etc. and which simply transforms results of using that API to and from appropriate decisions and handlers within resources.

In your example of composition, you have some types of failures you're expecting the framework to handle, and some which you're handling yourself (for example, the capacity exceeded case which you map to a 403). The Freya machine style takes a fairly different approach - each question may result in continuing on, or returning one type of failure, whereas imp seems likely to have quite a few separate concerns - validating, checking system capacity, etc. I might be wrong on that, please correct me!

I suppose to some extent it becomes a matter of taste. One thing I would say is that re-use through composition is actually pretty high with the Freya machine approach. While we only have one resource in this example, we'd usually have rather more, and at that stage can start composing aspects of them. For example:

let secured context =
    freyaMachine {
        authorized ...
        allowed ... }

let valid =
    freyaMachine {
        entityLengthValid ...
        processable ... }

let resource1 =
    freyaMachine {
        including valid
        handleOk ... }

let resource2 context =
    freyaMachine {
        including (secured context)
        including valid
        handleOk ... }

In this way we can start to compose APIs out of reusable chunks of logic, parameterised as we need. I think when it comes to reasoning about the results, I understand your concern. I've personally found it to be OK (so far) but I will freely admit that I haven't built a system this way with hundreds of resources yet.

This applies equally to success mode decisions/configuration too. Supplying functions for things like eTag and lastModified will perform correctly and composably under all of the conditions that an HTTP request might result in - which, done manually, is potentially quite a headache.

Anyway, sorry that's a bit of a ramble, hopefully a slightly interesting one! Obviously a lot of this is quite subjective, and I'm always interested in different opinions. I hope that this level of abstraction in Freya (and a higher level in the works) will result in a safe, productive way to produce complex systems in a simple fashion, but I'm nowhere near innocent enough to believe it's perfect, so welcome your thoughts!

Perhaps I'll just need to understand that other way of thinking, but so far, I don't get it. While I'm all for composability and elimination of duplication, I don't understand how you can model any sort of resource where the response depends on what happens during the request.

Perhaps I misunderstand, but doesn't handleOk unconditionally return 200 (OK)? Isn't that the case for e.g. handleCreated as well (although, obviously, the status code is different)?

In one of my recent projects, I helped design and build RESTful services that integrate with a quite unpredictable and coarse-grained legacy system. This legacy system exposes lots of operations that violate Command Query Separation, so in a single operations we'd often ask it to do something, and it'd subsequently return some data. Depending on the returned data, the resulting HTTP status code might be in the 200 range, the 400 range, or the 500 range. However, as a REST service developer, I have no chance knowing this beforehand - I'll need to react to whatever comes back from the legacy system, and take it from there.

It's still not clear to me how I can do that with Freya.

---

BTW, it's true that imp in my example above seems to have many separate concerns, but it's only because you see it from the consumer side. It's actually composed of small-grained functions:

let imp = 
    Validate.reservation
    >> Rop.bind (Capacity.check 10 SqlGateway.getReservedSeats)
    >> Rop.map SqlGateway.saveReservation

The strength of this approach is that imp has the return type Result<unit, Error>, so if, in my Controller, I don't match all possible cases of that return type, I get a compiler warning or error. That's a strong way of ensuring that developers don't forget to handle some corner case.

This also means that I'd be sorry to have to go back to exception throwing/handling for those sorts of scenarios.

Yes I see where you're coming from. You're right in saying that handleOk, handle* etc. return a specific status code (well, you can override them, but it would be odd to do so). I think I'd probably describe it as an inversion of where the logic sits.

In your approach, you're attempting some action, and then interpreting the result of that to see what you should return. So if you succeeded, some kind of 2xx, failed because of the client, 4xx, etc.

The Freya machine model (and indeed general machine models, it's very similar in WebMachine, Liberator, or other similar approaches) is to model things differently. When defining a resource, I tend to start with the happy path. Define what handleOk means, in the case where everything is fine.

Next, could it fail in some particular way? So, could this resource ever not exist? If there's ever any chance of that (if it's a collection it always exists perhaps, but if this resource represents some item it doesn't if the ID isn't found for example), then I'll implement the exists decision. Is it possible that the data coming in may not be in the correct form? If so, I'll implement the processable decision, where I might validate that the JSON is syntactically correct for a POSTed representation (as an example).

Could this fail because the user is not allowed to do something? Then I'll implement allowed, and so on.

In cases where I'm actually modifying something, that'll be happening in an action - so for example, I may implement doPost, or doPut, which again form a branch in the decision graph - or may do.

In essence, yes handleOk will always return 200 OKor equivalent - but it will only ever be invoked if everything is actually fine and that's a correct response. If something wasn't right, I would have branched off in the decision graph long before (hopefully!).

In cases where you are working with legacy systems it's obviously trickier - sometimes you'd need to attempt the action quite early in the process if it's coarse-grained and non-modifiable, and refer back to it in your decisions. So i the case of your API as given, you might see something like this:

let result =
    freya {
        return! imp request } |> Freya.memo

let exampleAllowed =
    freya {
        let! res = result

        match res with
        | Failure CapacityExceeded -> return false
        | _ -> return true }

let exampleProcessable =
    freya {
        let! res = result

        match res with
        | Failure ValidationError _ -> return false
        | _ -> return true }

let resource =
    freyaMachine {
        allowed exampleAllowed
        processable exampleProcessable
        ... }

In this case we attempt an action and then implement our decisions based on the result (and due to the call to Freya.memo, we only attempt that action once). Of course, if you have control over your backend completely, you might optimise it so that you can actually make fine-grained calls in those decisions to your backend, but you don't have to if you can't for any reason. I've glossed over how to invoke imp in that implementation, but it generally isn't awkward.

One other thing - yes in the case I'm describing, you would lose some level of certainty that all of your error cases have been handled, as you can't rely on exhaustiveness of sum type handling. I'm not sure I can find an obvious answer to that with this model at this stage.

EDIT to add: I've found that encoding all possible outcomes of actions past a certain level of complexity as DUs becomes quite taxing and difficult to maintain in places. Not to say that it isn't desirable where practical though, in your example it seems a good approach.

Wow, thanks, it's beginning to click in place for me now - thank you for your patience with me!

So, when you define the resource using the freyaMachine computation expression, you pass various Freya<bool> values to allowed, authorized, exists, etc. In what order of precedence are they evaluated? Is the order they are added in the freyaMachine computation expression important?

Oh no problem - I don't mind at all. It's going to help me write docs, because knowing what's not obvious even to experienced smart people is quite hard when you're too close to the code! It's been fantastically helpful to me, so thanks for your time too, it really makes a difference.

The order is defined by the HTTP graph, which is one of the things that will be part of the documentation - and is also part of the debugging tools (though they're not quite prime time yet). However, being able to see that is pretty helpful, it's top pf my list.

The order in the computation expression isn't relevant - the computation expression is essentially building up a map of configuration and functions, which is then "compiled" to an actual graph instance which can be executed at runtime. So you don't have to worry about ordering, and you can set any combination of functions/configuration without worrying how they interact.

This is an out of date version of the reference [https://github.com/freya-fs/freya.documentation/blob/master/references/machine-custom-operations.md](before I made the graph system more powerful and extensible) which I'll be updating this week to reflect what's there now. With that and a diagram of the graph, it should be a lot clearer to visualise!

WebMachine have something like this as a diagram for their graph: https://raw.githubusercontent.com/wiki/basho/webmachine/images/http-headers-status-v3.png

Our graph isn't quite the same, and is also complicated by the fact that our graph actually works on an extension model. The absolute default graph in a resource is Start --> Finish unless you say something like using http which will modify that graph at runtime (in the "compilation" step). That's how we currently implement CORS - you would be using http and using httpCors, which will extend the graph with HTTP support and then CORS support where that's needed (and again, luckily, ordering in the computation expression doesn't matter here).

You can even write your own extensions to add processing steps to the graph if needed (for example, you could write an extension which would introduce WebDAV support). However, that is definitely not for the faint-hearted at this point, and it should never be needed if people just want to do simple and accurate HTTP work.