Skip to content

Instantly share code, notes, and snippets.

@icewind1991
Last active December 17, 2015 22:49
Show Gist options
  • Save icewind1991/5685072 to your computer and use it in GitHub Desktop.
Save icewind1991/5685072 to your computer and use it in GitHub Desktop.

New Architecture

The Server

The server object functions as the global scope, it holds the various core components, manages plugins and routed and handles incomming requsts

\OC\Server###

Implementes \OC\Hooks\Emitter

  • registerPlugin(\OC\Plugin $plugin)
  • registerRoute(\OC\Route $route)
  • getUserManager(): \OC\User\Manager
  • getFilesystem(): \OC\Files\Node\Root
  • getDB(): \OC\DB
  • getConfig(): \OC\Config
  • getJobManager(): \OC\BackgroundJob\JobList
  • getLog(): \OC\Log
  • getL10N(): OC_L10N
  • getSession(): \OC\Session\Session

Components

The system is broken down in various components which provides the core api's

  • User management \OC\User\Manager
  • Filesystem \OC\Files\Node\Root
  • Database \OC\DB
  • Config \OC\Config
  • Jobs \OC\BackgroundJob\JobList
  • Log \OC\Log
  • Router
  • L10N
  • Session

Hooks

When a component is created on of the following events are emitted

  • getDB(\OC\DB $db)
  • getFilesystem(\OC\Files\Node\Root $filesystem)
  • getUserManager(\OC\User\Manager $manager)
  • getConfig(\OC\Config $config)
  • getJobManager(\OC\BackgroundJob\JobList $manager)
  • getLog(\OC\Log $log)
  • getL10N(\OC_L10N $l10n)
  • getSession(\OC\Session\Session $session)

These hooks can be used by apps to make changes to the components, such as registering backends or adding additional hooks before a request is handled.

Making the changes component in the hooks instead of making them during the loading of the apps, will make the loading of the apps quicker and only trigger changes for components that are actually used, if the filesystem component isn't used during a request no time will be wasted on initializing storage backends. This removes the need of only loading a subset of the apps which can lead to unexpected behaviour.

Requests

Holds the data from the request made by the server, request header, body, cookies, etc. HTTPFoundation from Symphony is used as base.

\OC\Request\Request

extends \Symfony\Component\HttpFoundation\Response

  • getBodyParser(): \OC\Request\BodyParser (The parser will be selected based on the Content-Type of the request)

\OC\Request\BodyParser

  • getParsed(): mixed
  • getRaw(): string

\OC\Request\JsonParser

Inherits \OC\Request\BodyParser

  • getParsed(): mixed

\OC\Request\XmlParser

Inherits \OC\Request\BodyParser

  • getParsed(): SimpleXMLElement

Responses

Holds the data for the response to be send to the client, headers, body, etc. Various reponse types are represented by subclass (JSON, HTML, etc)

\Symfony\Component\HttpFoundation\Response is used for managing the responses.

Symfony provides the follwing sub classes for specific response types:

  • \Symfony\Component\HttpFoundation\JsonResponse
  • \Symfony\Component\HttpFoundation\StreamedResponse
  • \Symfony\Component\HttpFoundation\BinaryFileResponse

\OC\Response\XML

inherits \Symfony\Component\HttpFoundation\Response

  • setData(SimpleXMLElement $xml)

\OC\Response\Page

Create a response based on a Page object, inherits \Symfony\Component\HttpFoundation\Response

  • addScript(string $app, string $script)
  • addStyle(string $app, string $script)
  • addNavigationEntry(\OC\Page\Navigation\Entry $entry)
  • setContent(string $html)

\OC\Response\Template

Create a response based on a template file, inherits \OC\Resonse\Page

  • setTemplate(string $app, string $template)
  • set(string $key, mixed $value)

\OC\Response\EventSource

Create an eventsource response, inherits \Symfony\Component\HttpFoundation\Response.

  • open()
  • write(string $type, mixed $data)
  • close()

\OC\Response\File

Download a file from the oc virtual filesystem, sets content-length, content-type, etc header and writes the file content inherits \Symfony\Component\HttpFoundation\Response. Use \Symfony\Component\HttpFoundation\BinaryFileResponse for files from the local filesystem.

  • writeFile(\OC\Files\Node $file)

\OC\Response\Exception

Created when a route throws an unhandled exception inherits \Symfony\Component\HttpFoundation\Response

  • setException(Exception $exception)

Pages

A Page represents an html page that is send to the browser, it manages it's navigation, scripts, styles and content

\OC\Page

  • getNavigation(): \OC\Navigation\Navigation
  • addScript(string $app, string $script)
  • getScripts(): string[]
  • clearScripts()
  • addStyle(string $app, string $script)
  • getStyles(): string[]
  • clearStyles()
  • setContent(string $html)

\OC\Page\Navigation\Navigation

Manages the list of navigation entries and the active entry

  • addEntry(\OC\Page\Navigation\Entry $entry)
  • setActive(\OC\Page\Navigation\Entry $entry)
  • getEntries(): \OC\Page\Navigation\Entry[]

\OC\Page\Navigation\Entry

  • __construct(string $app, $page)
  • getApp(): string
  • getPage(): string

Plugins

\OC\Plugin

Plugins manipulate the bahaviour of all requests made (think express middleware), example plugin are auth handelers, file viewers and js/css minifiers

  • handleRequest(\OC\Request\Request $request, \OC\Server $server)
  • handleResponse(\OC\Response\Response $response, \OC\Server $server)

Routes

Routes are responseible of creating a Response for a Request, apps can register Routes in a server and the server handles calling the correct Route for each request made.

\OC\Route

Inherits Symfony\Component\Routing\Route

  • callCheck() check for CSRF headers, throws an exception if not set
  • checkLoggedIn() throws an exception if not logged in
  • checkAdmin() throws an exception if not logged in
  • execute(\OC\Request\Request $request): \OC\Response\Response

Lifecycle of a request

  1. Initialize the Server
  2. Initialize the apps
  3. Parse the request from the server into a Request object
  4. Run handleRequest on all registered plugin
  5. Handle the request using the router, generating a Response object for the Request
  6. Run handleResponse on all registered plugin
  7. Send the response to the client

Legacy apps

Legacy apps mostly follow the following lifecycle for a request

  1. Initialize owncloud (load lib/base, which also initializes the apps)
  2. Get information from the request ($_GET, $_POST)
  3. Create an OC_Template object for the response
  4. Set template variables, add scripts, navigation, etc using OC_Util and OC_App
  5. Print the response using OC_Template->printPage()

When a legacy app is run against a new server OC_Template, OC_Util and OC_App function as the main compatibility layers, the OC_Template instance will internally keep a Template Response object, adding scripts, navigation, etc using OC_Util and OC_App will be redirected to the Response object. When OC_Template->printPage() is called, handleResponse is called on all registered plugin before sending the response

This way legact apps will follow the same request lifecycle as apps build against the new api

Advantages

Testing

Because both the Request and Response are objects and the entire lifecycle is controlled by the Server it becomes much easier to test the code. To test the behaviour of an app you can simple create a mock Request, pass it to the app and check the Response returned.

With the seperation of logic into various Plugins it becomes easy to test that logic, to test Basic Auth you create a mock Request with the right headers set, pass it trough the Basic Auth plugin and check if the login is successfull.

Performance

Because the functionality of apps is seperated over the Plugins, Routes and Hooks, the Server has a more fine-grained controll over what parts of the apps get activated, if the user system isn't used during a request, the user backends won't be loaded.

Examples

Some pseudocode examples of how apps in interact with the Server in the new architecture

Add a user backend

$server->listen('\OC\Server', 'getUserManager', function($manager){
    $manager->registerBackend(new MyUserBackend());
}

Minify js/css files

class MinifierPlugin extends \OC\Plugin {
    public function handleResponse($response){
        if ($response instanceof \OC\Response\Page) {
            $scripts = $response->getScripts();
            $minifiedScript = minifyAndConcat($scripts);
            $response->clearScripts();
            $response->addScript('minifier', $minifiedScript)
        }
    }
}
@tanghus
Copy link

tanghus commented May 31, 2013

As mentioned on IRC much of this has already been implemented for the App Framework and we might as well integrate the different parts to not have different implementations doing the same thing:

  • Request is implemented as an immutable object with array access to the different methods. Currently instantiated in the DIContainer
  • Response has all the mentioned subclasses except EventSource.
  • Request/Response handlers are implemented via a request dispatcher which calls middleware objects to do e.g. access control and deal with exceptions.

What's missing is:

  • General initialization
    • autoloader
    • setting server/PHP variables
    • various server checks
    • ?
  • Initialization of user backends.
  • Authentication (can be implemented as middleware).
  • App loading (including filesystems).
  • ?

Since it is per definition an App framework, it cannot be a drop in replacement, but let's find out which parts that could be used.

@BernhardPosselt
Copy link

Goes into the right direction IMO. How do you wire things up? is the getDB method static? Is the database connection a singleton?

As for userbackends where we got a plugin architecture, IOC is not really gonna work because all plugins need to be in place before you can use it. IMO we should use well defined files like the routes.php file in appinfo/ that can be loaded when initalizing the plugins

The Plugin stuff is basically middleware, maybe name it like this :) also you want to let middlware handle exceptions that are thrown in the controller, so you can do security stuff for instance.

What about going REST from the beginning? basically its just aweful as hell but PHP allows you to read only once from php://input which will be a horrible source of errors.

Btw i really like that idea of splitting the requests into own classes like \OC\Request\GET, \OC\Request\POST. Polymorphism ftw :)

Another thing: the eventsource stuff MUST NOT open a connection in the constructor. This is bad practice and makes injection nearly impossible. I need another layer in between because of this.

@icewind1991
Copy link
Author

Goes into the right direction IMO. How do you wire things up? is the getDB method static? Is the database connection a singleton?

Nothing is static, all the components are singletons, while loading the apps there will be an instance of the Server in scope for the app to work with.

As for userbackends where we got a plugin architecture, IOC is not really gonna work because all plugins need to be in place before you can use it. IMO we should use well defined files like the routes.php file in appinfo/ that can be loaded when initalizing the plugins

My idea was to use the hooks that are emitted when constructing a Component for that, while loading the app, the app should only register it's hooks, classes, plugins and routes, non of the logic from an app should be executed in that step. That way we have plenty of control over the order in which things are executed

The Plugin stuff is basically middleware, maybe name it like this :) also you want to let middlware handle exceptions that are thrown in the controller, so you can do security stuff for instance.

Middleware might be a better name yes, handling exceptions sounds good.

What about going REST from the beginning?

Not sure what you mean by this

Another thing: the eventsource stuff MUST NOT open a connection in the constructor. This is bad practice and makes injection nearly impossible. I need another layer in between because of this.

I'm aware of this problem

@BernhardPosselt
Copy link

Dont try to use singletons, rather create one instance in the server method and let apps or classes request that instance.

About the hooks: not sure how this can work without global state. Id rather use convention over configuration. That way its also easier to debug since you know where the backend configuration is.

REST: I mean not only providing a GET and POST response but also one for PUT, DELETE and if possible (not sure if symfony routing supports it PATCH) cc @DeepDiver1975

@icewind1991
Copy link
Author

@Raydiation

Dont try to use singletons, rather create one instance in the server method and let apps or classes request that instance.

Yes, I mean one instance per server

About the hooks: not sure how this can work without global state. Id rather use convention over configuration. That way its also easier to debug since you know where the backend configuration is.

The server acts as a "global" state, I think using hooks makes it easier to control when things are executed then loading specific files for each app.

REST: I mean not only providing a GET and POST response but also one for PUT, DELETE and if possible (not sure if symfony routing supports it PATCH)

Makes sense

@tanghus
Copy link

tanghus commented Jun 1, 2013

Id rather use convention over configuration.

For us uninitialized you will have to elaborate on what that statement means ;)

Wrt. REST that is as far as I can tell a sub-issue so to say, to be dealt with by the router. Not relevant in this context.

The server acts as a "global" state

That's how I see it too. When initializing an app it gets a reference to the server object. It will act kinda like the API instances works in app framework, and keep references to often used variables that in the static API would have been retrieved by yet another db query.
This can drastically reduce the server load and open db connections. I can't tell if it could potentially introduce race conditions or dead-locks though?

And let's not lock us on to any certain design pattern for the patterns sake :)

@Raydiation @icewind1991

@BernhardPosselt
Copy link

Maybe use open() for the event source instead of start (since you use close() to close it and not end() )

The Page stuff looks like you want a proper Templating language. All these kinds of things are easy solvable with template blocks for instance.

About the minifying stuff: most frameworks use something like a pipeline for these things, see

@bartv2
Copy link

bartv2 commented Jun 26, 2013

Great idea, I think the app interface should be more data driven, more using xml like in owncloud/core#1235

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment