Skip to content

Instantly share code, notes, and snippets.

@pheix

pheix/Raku-Advent-2020.md

Last active Nov 30, 2020
Embed
What would you like to do?
Raku web templating engines: boost up the parsing performance 

Raku web templating engines: boost up the parsing performance

Modern Raku web templating engines

A templating engine basically provides tools for effective metadata interpolation inside static files (templates). At web application runtime, the engine parses and replaces variables with actual content values. Finally client gets a HTML page generated from the template, where all metadata (variables, statements, expressions) has been proceed.

Raku ecosystem has a few modern templating engines: Template::Mojo (last commit on 12 Jun 2017), Template::Mustache (last commit on 25 Jul 2020 — it's alive!), Template6 (last commit on 20 Nov 2020 - active maintenance), Template::Classic (last commit on 11 Apr 2020), Template::Toolkit (by @DrForr, unfortunately it idles now) and HTML::Template (last commit on 28 Oct 2016).

Also there is the handy multi-module adapter Web::Template — a simple abstraction layer, providing a consistent API for different template engines.

What engine should you choose? My criteria was: the project should be alive and be the part of Rakudo Star Bundle distributive. Well, Template::Mustache is the chosen one 😇.

What are web templates?

Web templates are HTML documents with additional metadata (markup), that's going to be processed by templating engine — in simple templates metadata is presented by variables (e.g., Template6 templating engine interpolates variable to [% varname %] and Template::Mustache to {{ varname }}). After the web template is processed all metadata variables are replaced with actual content values.

Сomposite web template includes the links (bindings) to another templates. For example canonical Mustache templates could perform import with {{> template_name }} (see Partials). By the way, it's possible to use links at imported template, so recursive partials are accepted.

A web template with logic (inline programs) uses extended meta markup. We can write simple layout management programs right inside the template. Template6 engine successfully «executes» constructions like [% for item in list %][% item %]\n[% end %] or [% if flag %]foo[% else %]bar[% end %].

Regular practice for the most of web applications — templating with simple and composite templates. We use only variables and import dependencies (header, footer, comments block, feedback form, etc...). All extended logic (that could be implemented with inline programs) should be excluded as much as possible, or put onto web application layer.

In this article we will consider the most trivial case — web template with variables (no imports, no inline logic).

Performance

As I have mentioned below, my criterias to choose a templating engine were the project support and accessibility. But the in real life actually the important criteria is performance. Well, no matter the project is very much alive or is included into all known distros — if the client waits a few second for page rendering, we have to seek another module or solution.

So, to test the performance i have used the Pheix CMS embedded template as a one of the most trivial. In my opinion — if the templating engine will easily deal with it, we can go to the next step of testing — for example, inline programs.

Notes: the template has 14 variables and 2 of them are using the extended Mustache syntax {{{ varname }}}. Triple curly braces are are telling the engine to skip the escaping inside the content block we are replacing into.

The test suite is based on the processing script, where the render() method from helper module Pheix::View::TemplateM is used. The sources are quite simple and as close as possible to documentation guidelines. We are profiling the render() method and measuring the execution time (no compiling, loading, object initialization, etc… time is considered). Also we use automated bash helper to run tmpl-mustache.raku in loop for 100 iterations and count average run time.

Tests are performed on MacBook Unibody Core2Duo 2.6 GHz, 8Gb RAM platform (consider it like the mid-performance VPS).

Result (unit — second): mustache render time: 1.8555348.

In other words, if the web application works as the classic CGI instance (no caching, no available workers, no proxies — every run is from the scratch), the request will be rendered at least in 2 seconds (network latency + duty cycle + templating [+ server resource throttling]). Actually this time may be more than 10 seconds (a few parallel clients case) — absolutely bad.

The same test is performed for Template6. Result (unit — second): template6 render time: 0.5481035. This module is x3(!!!) faster. Sources: template and script.

First optimization

The first idea on optimization was: «ok, it seems that the considered modules are heavy for this task — let's write something simple». And I have implemented my own render() method, based on generic regular expressions.

What's about simple regular expression

method fast_render(Str :$template is copy, :%vars) returns Str {
    for %vars.keys -> $key {
        if $key ~~ /tmpl_timestamp/ {
            $template ~~ s:g/ \{\{\{?: <$key>: \}\}\}?: /%vars{$key}/;
        }
        else {
            $template ~~ s/ \{\{\{?: <$key>: \}\}\}?: /%vars{$key}/;
        }
    }

    $template;
}

Sources: script, helper module.

Result (unit — second): regexpr render time: 0.2721529. This is a 2x increase in performance compared to Template6 and 6x increase compared to Template::Mustache.

Second optimization

The next idea was: «well, let's parse HTML template to the tree and replace/substitute required blocks». I have used the XML module for this task.

This approach requires a little bit tricky template: as the template is parsed to the tree, we need to address the blocks to interpolate. In case of Template6 or Template::Mustache we use meta markup, but it fails on XML validation.

Well, I add the XML markup into the basic template instead of meta markup:

  • specific tags: <pheixtemplate variable="tmpl_pagetitle"></pheixtemplate>;
  • specific attributes:
    • <script src="resources/skins/akin/js/api.js" pheix-timestamp-to="src">;</script> — this means the timestamp value will be concatenated to string from src attribute;
    • <meta name="keywords" content="" pheix-variable-to="content" pheix-variable="tmpl_metakeys" /> — this means the tmpl_metakeys value will be inserted to content attribute;

Also specific tags should have pre-built HTML code and bindings to existed tree nodes, specific attributes just have to be defined — this is done so straight-forward at initialization step:

my %tparams =
    title   => {
        name    => 'tmpl_pagetitle',
        new     => make-xml('title', "This is the page title"),
        existed => Nil,
        value   => q{}
    },
    mkeys   => {
        name    => 'tmpl_metakeys',
        new     => Nil,
        existed => Nil,
        value   => 'This is meta.keywords data'
    },
    ...
;

for %tparams.keys -> $k {
    %tparams{$k}<existed> =
        $xml.root.elements(:TAG<pheixtemplate>, :variable(%tparams{$k}<name>), :RECURSE, :SINGLE);

    if !%tparams{$k}<existed> {
        %tparams{$k}<existed> = $xml.root.elements(:pheix-variable(%tparams{$k}<name>), :RECURSE, :SINGLE);
    }
}

Timestamps blocks are collected with:

my @timestampto = $xml.root.elements(:pheix-timestamp-to(* ~~ /<[a..z]>+/), :RECURSE);

And processed by (as trivial as possible):

for (@timestampto) {
    my Str $attr = $_.attribs<pheix-timestamp-to>;

    if $attr {
        $_.set($attr, ($_.attribs{$attr} ~ q{?} ~ now.Rat));

        %report<timestamps>++;
    }
}

Variable tags are processed with:

for %tparams.keys -> $k {
    if %tparams{$k}<new> {
        %tparams{$k}<existed>.parent.replace(%tparams{$k}<existed>, %tparams{$k}<new>);

        %report<variables>++;
    }
    else {
        my Str $attr = %tparams{$k}<existed>.attribs<pheix-variable-to>;

        if $attr {
            %tparams{$k}<existed>.set($attr, %tparams{$k}<value>);

            %report<variables>++;
        }
    }
}

Yeah! Let's run the script:

$ raku html-2-xml.raku

# processing time: 0.15519139
# added 6 timestamps
# replaced 8 variables

Average result on 100 iterations (unit — second): xml render time: 0.1550534. This is a 2x increase in performance compared to custom RegExpr, x4 increase compared to Template6 and 12x increase compared to Template::Mustache 🤯.

HTML::Template

Canonical usage (following guideline)

Just for fun I have measured the performance of the old and forgotten HTML::Template module. Test sources: template, script (toggle comments Pheix::View::TemplateH to Pheix::View::TemplateH2), helper module.

Average result on 100 iterations (unit — second): htmltmpl render time: 0.1911648. Well, it's quite fast out-of-the-box.

Make it ~ x100 faster

HTML::Template module provides simple grammar, so the third idea was «yep, let's parse HTML template into the variable (according the given grammar) at initialization stage and substitute at runtime».

Test sources: template, script (toggle comments Pheix::View::TemplateH2 to Pheix::View::TemplateH), helper module.

Average result on 100 iterations (unit — second): htmltmpl render time: 0.0021661. This is a 100x increase in performance compared to canonical usage 🎉 🎉 🎉.

Need more boost?!

Modern web development techniques involve the backend templating engine load balancing.

The common technique is based on idea of distributing template rendering task between the server and the client: depending on server load, the template is fully rendered by backend or backend just does the fast generation. It pulls the template file and concatenates its content with the data to be replaced (represented in JSON). Usually this data is added to end of template as the JavaScript code inside the <script></script> tag.

On the next step JavaScript template engine (for example, RIOT.js) does the full page rendering right inside the client's browser.

This approach could be effective in case we use Template::Mustache as the primary templating engine on backend. Template variables for server-side rendering are marked as {{ varname }} or {{{ varname }}}, client-side rendering variables are marked as { props.varname }. So, this is the way we get the consistency of template source code and basic semantic integrity.

Conclusion

Why does the latency matter?

Client's point of view: nobody likes slow websites.

Developer's point of view: if we will minimize latencies and bottlenecks at routine tasks — and template rendering is the such one, — we will free the space for resource-intensive or slow technologies.

For example, blockchain. This humble research was inspired by the development of Pheix web content management system with data storing on Ethereum blockchain. Some of the ideas outlined here are put into code of public β-version — it will be released by the end of this year.

Summary

All sources considered in this post are available at https://gitlab.com/pheix-research/templates. The final scores are:

1. htmltmpl pre-parse render time: 0.0021661
2. xml render time:                0.1550534
3. htmltmpl native render time:    0.1911648
4. regexpr render time:            0.2721529
5. template6 render time:          0.5481035
6. mustache render time:           1.8555348

Raku driven web applications

I'm sure, we can use Raku as web programming language. We can create fast, reactive and scalable Raku driven backends. On other side it requires a little bit more practice and time: combining different techniques and approaches we can get more performance improvements. The sad things — the ecosystem is still raw and when we want to use some module in our project, we should profile it, compare with analogs, maybe fork and tweak. The optimistic things — we can get our Raku driven web application work fast, so it can be released to production.

@JJ

This comment has been minimized.

Copy link

@JJ JJ commented Nov 29, 2020

  1. Links all over. For instance, "templating engine"
  2. Probably don't need a dash in HTML-page
  3. DrForr's has been adopted by the community. And there's also Template::Classic, by Chloe Kekoa, which I particularly like. Checkout the typo in Mustashe
  4. No dash in Web-template. That's processing → that's going to be processed
  5. "Web-template with logic" possibly could be rewritten as "a web template with logic", or "web templates with logic".
  6. "choose templating engine" → choose a templating engine
  7. "are pointing the engine " → are telling the engine
  8. "inside content block, we are replacing to." → inside the content block we are replacing into
  9. "if web-application works via classic CGI1.1, the every request " → don't use dashes after web, ever. Also, "the" web application, and you lost me with CGI1.1. Maybe you are saying that the content is going to be served on the fly? Maybe also add a link.
  10. Why do you use и?
    11: that's → that the
    12 own → my own
    13 XML module → the XML module
    14 concatinated → concatenated. I don't quite follow what you are doing in this section. You might want to elaborate it a bit more.
    15 "old and forgotten" → the old and forgotten
    16 "none" → no one or nobody
    17 released at → released by

That's it, I think, Please send me your wordpress ID and tell me if you prefer I'd upload it to the site, or you want to do it yourself.

@pheix

This comment has been minimized.

Copy link
Owner Author

@pheix pheix commented Nov 29, 2020

@JJ, thanks a lot for feedback! I will cook the update today.

@pheix

This comment has been minimized.

Copy link
Owner Author

@pheix pheix commented Nov 29, 2020

@JJ, thanks a lot for feedback! I will cook the update today.

🆕 🔄 Updated!

@JJ, I'm @pheix at wordpress. Of course, I can post the article by myself. If something won't be clear during the upload process, I will contact you!

@pheix

This comment has been minimized.

Copy link
Owner Author

@pheix pheix commented Nov 30, 2020

@JJ, I have been uploaded the article to https://raku-advent.blog (post id 726) and polish layout as much as possible. Now it's in Drafts with Pending review status. Please check it out.

@JJ

This comment has been minimized.

Copy link

@JJ JJ commented Nov 30, 2020

@JJ

This comment has been minimized.

Copy link

@JJ JJ commented Nov 30, 2020

OK, scheduled in principle for Dec 8th. You can still make changes if you want. Thanks!

@pheix

This comment has been minimized.

Copy link
Owner Author

@pheix pheix commented Nov 30, 2020

Yeess! 👍 😎
Thank you for assistance!

@JJ

This comment has been minimized.

Copy link

@JJ JJ commented Nov 30, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment