Flexibase RFC

flexibase.md

Flexibase

Synopsis

Service-oriented architecture is a scheme by which a system is built out of separate, encapsulated components that communicate as services.

This mirrors the concept of microservices, except where microservices communicate between processes, and often between hosts, SOA services can be installed in-process.

Flexibase is a language-agnostic specification for a service-oriented architecture, but expands upon the principle by demanding certain facilities be met.

• A Flexibase implementation must be configurable such that an application developer can easily select services without having to write complex or tedious bootstrapping code.
• Components to a Flexibase application must communicate in very consistent ways. Flexibase (or an extension) defines service types, providing interfaces and protocols that an actual service must conform to.
• Components to a Flexibase system are not necessarily services, in the sense that they would not be called upon to perform actions by arbitrary parts of the system. Instead, the components may passively identify themselves as consuming a protocol whose actual controller may or may not be part of the system.
• Flexibase not only allows for the definition of behaviour but also of data types. Data types are defined along with the functional specification for service types, and there is a defined core service for registration and interrogation.

Often, when writing an application, the developer must first choose a paradigm of user interface. There exist frameworks for web applications - both browser-side and server side - for mobile devices, for desktop environments, for terminal applications...

The problem with this approach is that the framework you select is the same technology that defines how your modular code is actually put together. However carefully you try to modularise your code you must always write some sort of connector between your framework and your business logic, and any community code you choose to install must be interfaced with in a specific way.

For example, if you choose to write a web application, you must interface with your database in the way it prescribes. You probably will also install an authentication package and a session store for login, and then maybe try to integrate those with LDAP or OAuth2, or hack in 2FA later. Then, 2 years down the line, you realise that business logic has made its way into your web controllers and the only way to turn this into a cron job is to either refactor your entire application or create a script that fires mock requests at a copy of your app.

This makes as much sense as choosing a test framework first and then forcing all your code to be written in terms of that.

Flexibase does not require you to decide what you are writing before you write it. Instead, it encourages you to write services that can be employed by any application, and defers the user interface concerns to whatever point you choose to implement it.

For example, you want to write a web application. You start a Flexibase application and select a module that provides an auth service, and a module that provides session storage. Then you write a module that provides the webapp service to Flexibase and now you have a UI. Later you realise that you've put business logic in your web controller, but because you wrote that code in terms of Flexibase you can trivially move it to a new service. Now your cron job can construct the same Flexibase application without the UI and just run the code with sensible data.

A Flexibase application is driven by putting Minds in a Hive.

Terms

Application

The application is the single executable that will comprise multiple Minds. Each application will define its own Minds by configuring the Hive; an application is therefore identified by this set of Minds.

Although each Mind can itself be configured, this configuration is not considered pertinent to the identity of the application, but rather is drawn from the environment in which the application runs.

Component

Generic term for a module added to the system. A component contains a Mind and some Hats, and may provide one or more services. The term also encompasses all the other code in the component that is not relevant to the Hive or its functionality.

Hat

A Mind installed into the Hive wears one or many Hats. Each Hat is a unit of behaviour provided by that Mind. Some Hats may be active; these implement services. Passive Hats are used by Services to identify Minds that can provide behaviour or information.

The term is named after the colloquial expression, "To wear one's [occupation] hat". referring to the notion that an individual may be able to function in many capacities but is at any one time only "wearing the hat" of one of them.

Hive

The core object, properly known as the Hive, is the central system of Flexibase. The Hive is constructed by the application developer defining a configuration for it, and the configuration defines the Minds used and the uses to which they are put.

Mind

A Mind is any object that can be installed in a Hive. The Mind can define any form of configuration necessary for its functionality and may require certain Services to exist.

Service

A service is a semantic name for a class of behaviour, whose usage is defined by consensus. Some services are defined by Flexibase itself but any service can be added to the Hive.

Conflicting services are unlikely: a second service would be designed to implement the same interface as the first one in a different way. The interface to services therefore needs to be strongly considered up front.

A Service is a Mind that implements the service; at runtime, the term refers to the Mind configured to provide the service. Multiple Minds may implement the same service but only one may provide it at runtime.

Hive

Flexibase defines a core object whose implementation details are left vague to accommodate the differences in languages.

We call the core object the Hive because it sounds cool. The concept of a hive mind also helps us conceptualise the structure of the application.

Availability

The Hive is essentialy defined as a singleton object; but this is in the sense that each application needs only one. A developer may build a new one if they choose to, but only in a way that does not infringe on the global one.

The Hive should be modified and controlled by means of a globally-available interface. It is preferred that the global object is immutable; this is especially important after initialisation.

The Hive and its Minds should work together to ensure the knowledge of their structure is encapsulated within them and their communications. That is to say, if someone constructs a new Hive for whatever reason, nothing done via that Hive should ever interface with the global one.

Construction

If the language permits, the Hive should be constructable by means of a configuration object. This object will specify the Minds to use, define their constructor parameters, and tell the Hive which Mind provides each service.

It is preferable to allow the Hive to construct the Minds using the data passed in, than to pass in pre-constructed Minds. This ensures that we cannot have a Hive in an unusable state. It also gives us an opportunity to initialise Minds with the Hive they are installed in.

A compiled language may present a challenge constructing objects at configuration time, since it may not be possible to load classes at runtime. The precise mechanism by which a Hive is constructed is left up to the necessities of the implementation, but it is a requirement that the Hive may not exist for use by the application in an inconsistent state.

Irrespective of construction strategy, the configuration object must also specify which Mind is used to provide each service. This applies even if only one Mind is able to provide the service; we don't want to accidentally provide a service we didn't intend to.

Check

Check is the primary hook for ensuring Hive consistency. Every Mind may declare dependencies on services or, rarely, other Minds. At check time, the configuration is compared with the dependencies, and anything not provided causes an error.

Initialisation

The Hive is not viable until it has been initialised. This step iterates through the constructed Minds and gives them the opportunity to perform further actions now that they are instantiated and part of a Hive.

This is the first point at which the passive behaviour of Hats is valuable. The initialisation step of certain Minds may involve asking the Hive for any other Mind that implements a specific interface: in our world, this means wearing a particular Hat.

Once every Mind has successfully run its initialisation then the Hive is considered viable.

Specification

The Hive MUST be available at any point in the code.

Any function called on a Mind by the Hive MUST receive the Hive.

The Hive MUST provide itself to any function it calls on a Mind.

The Mind MUST use the provided Hive and never call out to the global one.

The Hive MUST be constructible.

The Hive MUST be configurable. The Hive SHOULD be able to construct Minds based on this configuration. The Hive MAY accept pre-constructed Minds but is encouraged not to.

The Hive MUST NOT provide a service unless it is explicitly configured to do so.

The Hive MUST NOT be considered viable before declared constraints are satisfied.

The Hive MUST cause a fatal error if declared constraints are not satisfied.

The Hive MUST cause a fatal error if a service is requested that cannot be provided.

The Hive MUST initalise each Mind before it is considered viable. This initialisation stage MUST pass the Hive to the initialisation functions. The Hive MUST have constructed all configured Minds and registered all configured services.

new(Config config) -> Hive

Constructs a new Hive from the given Config; the Config type is not specified in this document to allow for interpretation.

service(Str name) -> Hat

Returns the Hat that provides the given service.

hats(Str name) -> List[Hat]

Returns a list of all Hats in the Hive by the given name.

mind(Str name) -> Mind

Returns the Mind called this. Calling this couples your Mind to another Mind and is strongly discouraged.

Minds

Minds are core objects in the component architecture. They provide an interface into the behaviour your component provides, and exposes services and further interfaces with which other Minds can access their functionality.

Each Mind is identified by a name. There is no restriction on the naming scheme but it is recommended that it follows the identifier convention of the host language. This name is intended to be set at the point the Mind is added to the Hive, so that if for some reason two Minds have the same name, one can be overridden.

The Mind must wear at least one Hat. Hats are either passive or they implement a service. A Hat that implements a service will be named the same as the service, in context of the Mind. More on that later.

Minds can declare reliance on services. This is normally done to ensure other Minds are initialised first. It also allows for an early sanity check if you know that your Mind cannot function without another service being available. For example, you might declare a dependency on a database connection, or you might be writing a component whose sole purpose is to provide an extension to another one.

When a Mind is installed into a Hive its dependencies are checked. Following this, the Mind is initialised with the completed Hive. At this stage, it is guaranteed that all other Minds are available. It is also guaranteed that dependencies are initialised.

Specification

A Mind MUST wear at least one Hat.

A Mind MUST return an instantiated Hat when provided with its name.

The name of a Mind MUST be configurable, and MUST have a default value.

A Mind MAY provide one or more services.

A Mind SHOULD NOT hold a reference to the Hive it was installed in, unless the implementor can guarantee that this Hive is never rendered obsolete.

A Mind MUST throw an exception if initialisation fails for any reason.

Str name = 'defaultname'

Holds the name of the Mind. Must be defined.

hat(Str name) -> Hat

Returns the Hat by this name. Service name is the same as Hat name so there is no equivalent service defined.

hats() -> List[Str]

Returns a list of names of Hats that this Mind wears.

services() -> List[Str]

Returns a list of service names available on this Mind.

initialise(Hive)

Runs any necessary initialisation. Default behaviour is no-op. No return value is expected; a failure to initialise should be fatal.

Hats

Hats are how everything ultimately communicates, so obviously they're down here at the end of the document instead of somewhere near the beginning.

A Hat provides a unit of behaviour and comes in two types: active and passive. There is essentially no difference between a passive and an active Hat, except how it is used.

Defining Hat types

A Hat's type is determined by how it is used, which really means how it is intended to be used. An important consideration is that a Hat is only useful if something is expecting it to exist.

Services

The first type of Hat implements a service. When a service is requested from the Hive, the Hive looks up the configured Mind to provide that service. It then requests the corresponding Hat from that Mind. Only one of these Hats will be in active duty at any one time.

There is a problem in defining an interface to a service, which is that a service is something that can be replaced by an alternative implementation. That means that any component that implements a service should be implementing an interface that is defined somewhere more generic than that component; otherwise, the first component gains the monopoly on the interface and interoperability is sacrificed.

In practice, this is fine; with sufficient care applied to the design of an interface, anything expecting that interface will continue to work if the implementation is replaced.

Some services will be defined by the core of Flexibase, and these will be the sorts of things considered common in many applications. (Not all services need be provided in any given application!)

Passive Hats

The second type of Hat provides passive behaviour. For this type, all Hats of the same name will be discovered in the Hive. Often, this type of Hat will provide an extension to another service. Thus, the service, when it performs its duty, will want to find all such extensions.

The interface to this Hat type is defined by the service to which it is relevant. As with the service definitions, the interface to its passive behaviour should be defined independently of the implementation that consumes them. This way, a replacement implementation can interface with the other Minds already in the Hive, already wearing those Hats.

Specification

A Hat MUST implement an interface. This interface MUST exist somewhere, but the location of this interface is specific to the type.

The Hive MUST return a Hat when asked for a service. The caller MUST NOT use any methods not defined by the interface.

Mind mind

A private handle on the Mind wearing this hat.

Semantic Types

Flexibase defines a system by which components can register their internal types in an externally semantic way. This means that the internal details of a particular implementation of a service (the stuff in a component that is not a Hat or a Mind) can be hidden while still allowing other services to interact with the objects inside it.

The semantic nature of these semantic types refers to the principle that any service can deal with any object by its semantic type name, knowing that, irrespective of the implementation detail, it will always deal with "the thing that it thought it was going to be". Which is to say, if you ever asked for, e.g., an object representing a user, you will always get an object representing a user, irrespective of which Mind it is that responds to the request.

The second advantage of this semantic typing system is that Minds can expose behaviour associated with types, without having to know whether anything is actually providing that type. As with behaviour, the specification for a type exists independently of the components that implement them, meaning any Mind can wear a Hat that passively makes use of any semantic type names from any specification.

Example

Alice is writing a normal web app, with password authentication. She wants each user to be able to edit their own profile, and she wants administrators to be able to associate various other data with the users as well.

She writes a Mind that wears the webapp Hat, allowing the generic webrunner component to find her application and expose it to her httpd.

She installs the auth service by selecting the Password Auth component, and hooks it up to her login page by converting the OpenAPI schema into an HTML form. She sends the posted data back to the service and handles the response accordingly.

She also installs the objectparams service, which initially has no effect. To enable it, she puts the objectparams::extender Hat on her Mind.

The :: in this name is a convention brought in from various languages. While not strictly necessary to adhere to the naming convention, it is preferred.

This new Hat is used to expose a list of semantic type names, and associate them with OpenAPI schemata. The objectparams service will discover this Hat and register these schemata against the types.

Finally, she installs the User administration UI component, which is not a Hive component, but a component for her web app framework that understands how to use the Hive. This component is designed to administrate any object that calls itself auth::user. When the User page is displayed, the component asks the objectparams service for all parameters for the auth::user semantic type; and this responds with the OpenAPI schema that Alice programmed onto her objectparams::extender Hat. It also responds with any other schemata that other Minds might have added to the auth::user type to support their own behaviour.

The component converts these schemata into HTML form components, adding their fields to the form it was already going to draw for the core component. When the form is saved, the posted data are despached back to the components that initially defined them.

Specification

Flexibase implementations MUST support a semantic type registry system.

The Hive SHOULD use a type method to retrieve information about a type by name. An alternative method name MAY be used, with justification.

The Hive MUST check all Hats for their exposed types.

Service specifications MAY define one or more semantic types.

Service specifications MUST define on each semantic type at least sufficient attributes to comprise a unique key.

Service specifications SHOULD use OpenAPI schemata in the documentation to define the types, even if it is just an example schema. They MAY use plain English to define the attributes and leave the specific schema up to the implementation.

Type names MUST have at least one level of namespace, identifying the scope to which the type belongs.

Namespaces SHOULD be relevant to the service name where possible.

Implementations of a service MUST expose ALL of the types defined by the specification.

Implementations MUST NOT expose any types defined by a different service. (This is avoided by sensible use of namespaces.)

Implementations MUST use the OpenAPI schema format to define the fields.

Implementations MAY expose types not documented by the service specification.

Implementations MUST use, for these extra types, a namespace relevant to the specific implementation.

services.md

Flexibase Core Service Specification

This document specifies core services recommended for a Flexibase system to be considered complete. None of these services is strictly required, but it is intended as a guideline for base functionality.

Terms and document layout

Each second-level heading (except this one) defines a service using its human name. Each service's definition has an English prologue explaining the purpose and design of the service.

This is followed by a specification with several paragraphs, each defining a feature of the service. These use the familiar MUST, MUST NOT, MAY, MAY NOT, SHOULD, and SHOULD NOT qualifiers from RFCs.

The first of these will specify the name of the service within the Hive. The name of the Mind itself is irrelevant; an implementation detail. However, unless there is some reason not to, this might as well be the same as the service name.

It is not recommended to implement more than one of these services on the same Mind. This is to prevent unnecessarily large amounts of constructor parameters (and to separate those concerns).

After these paragraphs the interface is defined. In this section the term type is used to define a data type used within the rest of the interface definition. This is not to be confused with the term Types used later on.

Following the lists of data types is a list of interface functions. Each function's parameters and return type is formally defined here, plus an English paragraph defining the purpose and behaviour of the function.

The interface specification may be followed by a Types section, defining types contained within the service that are actually exposed to the Hive by means of the type registry. Usually these types will be tied to one of the types used in the interface definition but this is not necessarily always the case.

Function specifications

Each interface function is specified thus:

function-name[(parameter-list)] -> return-type

Where return-type is TypeName; parameter-list is:

parameter[, parameter ...]

And parameter is:

TypeName parameter-name[?][= default-value]

A ? denotes an optional parameter with no default. = default-value denotes an optional parameter with a default. The presence of neither denotes a required parameter.

TypeName will either be some hopefully recognisable basic data type (Str, Num, Bool, etc), or one of the data types defined above the function specification in type headings.

To allow for differences in language capabilities and conventions, these are usually kept intentionally vague.

It is also not prescribed whether the implementation uses keyword arguments or positional arguments. Implementers are free to select whichever language feature makes most sense.

A note on data types

Some data types are mentioned without fanfare. I will list them here.

type OpenAPI

Any data structure that contains an OpenAPI v3 schema. With OpenAPI schemata being JSON objects, this would represent whatever a "deserialised JSON object" turns out to be in the host language.

A note on semantic Types

Each Types section defines a set of semantic types that the service should register with Flexibase's type registry. In each, we use :: to separate namespaces. This is a convention found in several languages but SHOULD NOT be translated into the convention for the implementation's language.

The reason for this is that if we ensure that types are named the same across languages, we can theoretically have Flexibase applications communicate through a connector interface that farms off any given service to another Flexibase.

As a result, we can expect the name of these types to leave one Flexibase system and arrive in another, potentially written in a different language. This means that the naming of the types follows Flexibase conventions, not local language conventions.

Authentication

Authentication is simply the requirement that a set of credentials be verified. We prescribe no structure to these credentials.

Upon a verification attempt the service must respond with a structured response. We do prescribe this structure.

The auth service therefore

• Accepts arbitrary data
• Returns a data structure defining a response

We want to support an arbitrary method of authentication. This includes various sources of user/password authentication, password-free authentication, and multi-factor authentication.

Therefore, we define various alternative responses for the authentication attempt, all of which can be encoded in the object returned.

We also define a pre-auth step, allowing the auth system to seed the login UI with data, for example the current value of an OTP.

In each auth step (including preauth) we communicate to the caller two pieces of information: a schema for data that must be produced for the next step, and a payload for data that must be returned to the next step verbatim.

Every auth service is expected to respond to preauth with a schema for the data structure this auth type requires. It is unlikely that an auth system would work without accepting auth data!

Every auth service is expected to also respond to auth, with an a structured response object. This object may return another schema - rinse and repeat!

Each auth service should concern itself with a single type of authentication. A Flexibase application can string together authentication forms by defining its own authentication service that forwards auth calls to one service, produces a token, and returns the structure required by the second service.

Here's a pseudocode example:

pre-auth()
    -> [custom auth service]
        -> password-auth.pre-auth()
        <- schema{user, password}
    <- schema{user, password}
auth(preauth-data)
    -> [custom auth service]
        -> password-auth.auth(preauth-data)
        <- success
        -> 2fa.pre-auth()
        <- schema{code}, payload{challenge}
        generate session token
        store 2fa challenge in session
    <- schema{code}, payload{session-token}
auth(code, session-token)
    -> [custom auth service]
        recover 2fa challenge from session
        -> 2fa.auth(code, challenge)
        <- success
    <- success
logged in!

Specification

The service in the Hive MUST be auth.

An auth service MUST implement a pre-auth method, providing an OpenAPI schema for the initial request.

An auth service MUST implement an auth method, accepting data conforming to the schema returned by pre-auth.

The auth method MAY return another schema instead of a success or failure.

The pre-auth method and auth methods MAY return payload data.

Calling code MUST allow the 'next step' response to be returned.

Calling code MUST send the payload data back in the next auth request.

The auth method MUST know what payload to expect and fail if it is not provided.

The auth system SHOULD use exceptions to indicate that the caller made an error; returning from the auth method SHOULD only indicate success, bad credentials, or next step. (This is a SHOULD because not all languages implement an exception system.)

type AuthData

An arbitrary data structure. Used to represent data collected from the user as part of a login attempt.

type AuthResponse

A key/value type with the following properties:

AuthResult result: Any identifiable data type (e.g. an enumeration) to determine whether the result was one of "success", "failure", or "next step".

"Next step" tells the system that to authenticate, another attempt must be made with further data. The schema and probably payload will be used by the login system to perform the next step.

OpenAPI schema: An OpenAPI schema defining the data that this step requires from the user.

struct payload: Data that must be sent verbatim back to the system when auth is requested.

The use of properties in this specification is an implementation detail. Some languages may prefer to use subtypes of AuthResponse to differentiate between the three auth result options.

type AuthResult

Anything that is meaningful in the host language to identify the result of the authentication attempt. The simplest example is just a 3-element subset of Str. A more robust example would be three separate data types: AuthResponseSuccess isa AuthResponse. As usual, practicality in context of the host language should influence this decision.

preauth -> AuthResponse

The entrypoint into an auth system is to call preauth. This returns the initial OpenAPI schema, and possibly payload, to begin a login attempt.

auth(AuthData data, struct payload?) -> AuthResponse

Perform authentication. payload contains any data provided to the login system in the AuthResponse from before.

This will loop as long as AuthResponse indicates "next step"!

Types

• auth::user - The user object that represents the logged-in user (or user attempting to log in).

Fundamentally, a user has no information that we can define at this point, so this user object does not "exist" in so many words. However, simply defining the name of a type is sufficient for other systems to make use of it, so we do.

Session

The session service simply supplies serialised assurance of state. This is done by means of a token and some form of storage.

The service can either generate a token (for example, on a success response from the auth service), or validate that a token is legitimate. It can also be requested to terminate a token.

Tokens are active within realms. A realm is simply any arbitrary identifier for a, well, realm of behaviour. This ensures that a correct session token from one realm is not considered correct in another. A default realm is supplied if not specified.

Two realms in the same system, for example, may be auth and cookie; one to remain logged in, and another to store cookie data. Different realms may have different rules on token expiry.

The service will associate a supplied payload with the token it generates, and allow access to this payload via the token. Users of the service should take care to only store as little data as possible, to avoid the sorts of problems that can come with serialising oversized payloads.

Specification

The service MUST be called session within the Hive.

The session service MUST provide an interface to generate a token.

The generator function MUST require a payload parameter.

The generator function MUST accept an optional realm parameter.

The service MUST store the provided payload, and return a token to identify it.

The token itself MUST be cryptographically secure.

The session service MUST provide an interface to retrieve stored data.

The retrieval function MUST accept a token of the same type as the generator function returns.

The retrieval function MUST accept an optional realm parameter.

The retrieval function MUST return functionally identical data to those provided by the user of the generator function.

Hopefully obviously, the returned data MUST be that data that was provided when the generataor function returned the supplied token.

The retrieval function MUST NOT return any data if the token is considered expired.

The retrieval function MUST NOT return any data if the token exists in storage but against a different realm.

The retrieval function SHOULD use an exception to indicate that the token is invalid.

The service MUST implement a function to test the validity of a token.

If no realm is supplied, a default one is provided; the function MUST NOT provide a means to search across realms.

The session service SHOULD implement a timeout feature. This means that if a session token is used after a certain amount of real time has passed, even if it is valid it will be considered invalid. Requirements marked * only apply to session services that implement timeouts.

* The generator function MUST accept an optional timeout parameter. This will default to the timeout set by the realm provided in the realm parameter, or of course the default realm if this is also not provided. The returned token will not be available after this much time has passed.

* Functions MUST allow for sessions not to time out at all. Common values to indicate this are 0, -1, or some variation on the theme of Nil.

* The retrieval function MUST reset the timer for the timeout function on retrieval. (The timeout function is based on access time, not creation time.)

* The retrieval function MUST return the same response for a timed-out token as for an invalid one.

* The service configuration SHOULD allow each realm to define a default timeout.

type SessionToken

Common representations of crypto-secure data include hex and base64, both of which are subsets of the String type. Implementations may wish to use something consistent, e.g. 32-character hex strings.

Implementations should bear in mind that some uses of these tokens will leave the system, like session cookies. They should be short enough to be used in most common situations without causing issues.

type SessionData

An arbitrary, serialisable data structure.

store(SessionData data, Number timeout?, Str realm = 'default') -> SessionToken

Store this data and return a session token. timeout may be interpreted as relevant to the language in question; some languages use integers and assume milliseconds; some languages assume seconds but allow decimals; and everything in between.

realm is described above and is used to partition data to avoid cross-pollination of separate concerns. While the argument is optional, a default realm must exist.

retrieve(SessionToken token, Str realm = 'default') -> SessionData

Retrieve the data associated with this token. if the token is invalid (either does not exist, or has timed out), returns no value. May throw an exception to indicate invalidity of token.

The timeout of a token is based on its last access time so on a successful access this function must reset the access time.

checkToken(SessionToken token, Str realm = 'default') -> Bool

Test the validity of this token in the given realm. Return True if the token is valid, or else False.