Create a gist now

Instantly share code, notes, and snippets.

Lightning Fast WordPress: Caddy+Varnish+PHP-FPM

README

This gist assumes you are migrating an existing site for www.example.com — ideally WordPress — to a new server — ideally Ubuntu Server 16.04 LTS — and wish to enable HTTP/2 (backwards compatibile with HTTP/1.1) with always-on HTTPS, caching, compression, and more. Although these instructions are geared towards WordPress, they should be trivially extensible to other PHP frameworks, other FastCGI backends, and even non-FastCGI backends (using proxy in lieu of fastcgi in the terminal Caddyfile stanza).

Quickstart: Use your own naked and canonical domain names instead of example.com and www.example.com and customize the Caddyfile and VCL provided in this gist to your preferences!

These instructions target Varnish Cache 4.1, PHP-FPM 7.0, and Caddy 0.9.4. (I'm using MariaDB 10.1 as well, but that's not relevant to this guide.)

Architecture

What's the point of all this?

Why are they so many layers?

Why Caddy?

The primary goal of this architecture is to massively and immediately improve any given WordPress site with a quick and easy upgrade to HTTP/2, always-on HTTPS, and robust caching and compression middleware. To that end, we can envision a stack that looks something like:

                                                    +-----------+
          Request +->                               |  Dynamic  |
                                                 +-->  Content  |
+----------+     +----------+     +-----------+  |  |           |
|          +----->          +-----> Cache &   <--+  +-----------+
|  Client  |     | SSL/Edge |     | Compress  |
|          <-----+          <-----+           <--+  +-----------+
+----------+     +----------+     +-----------+  |  |  Static   |
                                                 +--+  Assets   |
         <-+ Response                               |           |
                                                    +-----------+

Varnish enables caching and compression of static assets as well as dynamic content and is the best in its class, so let's plug that in. This leaves us wanting for implementations of the following components:

  • SSL termination and certificate management
  • Request sanitization and XSS attack mitigation*
  • Request and error logging
  • IP-based whitelisting and rate-limiting
  • URL rewriting (in request headers)
  • URL canonicalization (in response bodies)
  • Static asset serving
  • Hand-off to Varnish via reverse proxy
  • Hand-off to PHP-FPM via FastCGI

Caddy does (*almost) all of these things right out of the box. It makes the "happy path" very happy. And although HAProxy might be better at the edge, and NGINX might have more features, Caddy is a nice fit for this use-case. Performance is still an open question, but so far it's held up under load quite capably for me.

Our architecture starts to take shape as we fill in the boxes:

                                                    +-----------+
          Request +->                               | FastCGI   |
                                                 +--> (PHP-FPM) |
+----------+     +----------+     +-----------+  |  |           |
|          +----->          +----->           <--+  +-----------+
|  Client  |     |  Caddy   |     |  Varnish  |
|          <-----+          <-----+           <--+  +-----------+
+----------+     +----------+     +-----------+  |  |  Static   |
                                                 +--+  Assets   |
        <-+ Response                                |           |
                                                    +-----------+

Ah, but there's a problem: Varnish can't talk to FastCGI or serve static assets from disk. We need to add at least one more component:

                                                             +-----------+
        Request +->                                          | FastCGI   |
                                                          +--> (PHP-FPM) |
+----------+   +---------+   +-----------+   +---------+  |  |           |
|          +--->         +--->           +--->         <--+  +-----------+
|  Client  |   |  Caddy  |   |  Varnish  |   |  Caddy  |
|          <---+         <---+           <---+         <--+  +-----------+
+----------+   +---------+   +-----------+   +---------+  |  |  Static   |
                                                          +--+  Assets   |
       <-+ Response                                          |           |
                                                             +-----------+

             |--------Distribution---------|--------Origination----------|                                                              

This should (hopefully) illustrate why we need so many different layers. It may help to think of the left side of the stack as content distrbution, and the right side of the stack as content origination. Keeping these two halves separate makes it easy to scale when you're ready by letting you swap in a content distribution network (CDN)—which is essentially caching and compression as a service with anycast and SSL termination—while keeping the same origin:

                Hypothetical Future State
                =========================

                                              +-----------+
        Request +->                           | FastCGI   |
                                           +--> (PHP-FPM) |
+----------+   +----------+   +---------+  |  |           |
|          +--->          +--->         <--+  +-----------+
|  Client  |   |   CDN    |   |  Caddy  |
|          <---+          <---+         <--+  +-----------+
+----------+   +----------+   +---------+  |  |  Static   |
                                           +--+  Assets   |
       <-+ Response                           |           |
                                              +-----------+

             |-Distribution-|--------Origination----------|                                                              

Alternatively, if you don't go the CDN route, you have the ability to add more Caddy, Varnish, and/or PHP-FPM instances distributed and load-balanced across multiple servers.

HOWTO

Follow these steps to configure your server.

Firewall

Set up your firewall, minimally:

# ufw disable
# ufw reset
# ufw limit OpenSSH
# ufw allow to any port 80,443 proto tcp
# ufw enable

Filesystem

Make a directory for your WordPress installation (e.g. /var/www/example.com/wordpress) and log files (e.g. /var/www/example.com/logs). Your Caddyfile can go in this directory structure (e.g. /var/www/example.com/Caddyfile) or you can build a Caddyfile "repo" somewhere in /etc (e.g. /etc/caddy) and go nuts with the import directive if you have multiple sites.

Follow the normal process to migrate your WordPress site to its new home directory and restore the database. These steps are well-documented elsewhere and so are not covered here. (Don't install Apache, Nginx, PHP, or any other services; we'll cover that below!)

Packages

Minimally, install php7.0-fpm and varnish from apt. Copy default.vcl from this gist into /etc/varnish. It should work as-is in this basic configuration, and is essentially a blank slate for future customization using the expressive Varnish Configuration Language.

Be sure to install the appropriate PHP extension(s) for your database; for example, php7.0-mysql for MySQL/MariaDB. I found I needed to install php7.0-curl and php7.0-mbstring as well. YMMV. Don't forget to reload the php7.0-fpm service whenever you add/remove extensions.

Start and enable the php7.0-fpm and varnish services. Optionally stop and disable the varnishlog and varnishncsa services.

Caddy

At the time of this writing, the caddy-filter and ipfilter extensions provided by caddyserver.com are not sufficiently up-to-date and are missing important bug fixes and features. Follow the instructions to build Caddy from source and include the ratelimit, filter, and ipfilter addons.

You'll need to write a service unit file and install it to /etc/systemd/system; one generated by antoiner77.caddy v2.0.1 is included in this gist for reference.

You'll need to decide how Caddy will acquire its SSL certificate. If you don't already have an SSL certificate for the site and can afford a few minutes of downtime, the easiest method by far is to simply point the domain name at the new server, wait for the TTL to expire, and let Caddy's Automatic HTTPS feature apply the http-01 and/or tls-sni-01 challenge(s) at startup. Be warned that you might get temporarily blocked by Let's Encrypt if something isn't configured correctly and it looks like you're spamming their systems.

Copy the Caddyfile from this gist to wherever you've decided to keep it and customize it for your site:

  • Set the canonical domain name for each stanza.
  • Set the email address for your Let's Encrypt account (Caddy will create the account for you).
  • Decide whether you want to support older TLS 1.0 clients or not.
  • Update directory paths to reflect your WordPress installation and log directories.
  • Choose log rotation strategies for access.log and error.log.
  • Update the ipfilter block with your allowed IP address range(s).
  • If new user signup is open to the public, remove /wp-login.php from the ipfilter directive.
  • If you need XML-RPC, remove or comment out the statements dealing with X-Pingback and xmlrpc.php.
  • Adjust the filter rule to correctly rewrite your non-canonical URLs to the canonical URL. For example, rewrite http://example.org to https://www.example.com.
  • Adjust the caching policy for (truly) static assets.

Start and enable the caddy service when all's said and done.

WordPress

Most WordPress optimization guides suggest unsetting cache control headers such as Cache-Control, ETag, Last-Modified, Expires, etc. Instead, we're going to set them and let Varnish serve posts, pages, and attachments as slowly-changing dynamic content. Yes, you read that correctly: your WordPress site is now a glorified static site generator!

  • Change your WordPress and Site Addresses to https:// in wp-config.php, or by using the wp-cli tool.
  • Apply the wp-config.php snippets provided in this gist.
  • Install and activate the Add Headers and Cache Control plugins.
  • After the site is up-and-running, you can (and should!) browse it with the error console open to see which remote assets (e.g. analytics tracking scripts) are still being loaded via http://, and take the appropriate action(s) to update those links to https://. If you have a larger site, there are more efficient methods outside the scope of this guide.
  • Force your adminstrator users to change their passwords over HTTPS!

If you have a lot of dynamic fragments and/or pages on your site, for example a shopping cart or PHP-based discussion forum, caching may not work correctly out of the box. Fortunately, there are ways to vary the cache based on the logged-in user's session cookie or other factors. (TODOC.) If you'd rather not deal with this part of the puzzle, simply deactivate the Add Headers and Cache Control plugins.

Notes

  • The ports and Varnish instance can be shared between Caddyfiles. Caddy will select vhosts using the Host header, and Varnish will vary its cache by the Host header. (You may not want to re-use the Varnish instance if you have some gnarly, site-specific VCL, but that's another question entirely.)
  • Optionally update/override the ExecStart statement in varnish.service to bind to localhost, i.e. -a localhost:6081.
  • Don't forget to hit up Google and Bing Webmaster Tools to change the preferred URL to https://, upload a new site map, etc.
  • I've also had good luck with the HTTP Headers plugin, to add Strict-Transport-Security, X-Frame-Options, etc.

TODO

Open Questions

  • Is there a way to safely apply a global ratelimit?
  • Will Varnish 5.0 allow us to use HTTP/2 all the way down to FastCGI?
  • What more can we do to secure a typical WordPress installation, particularly .php files in wp-includes and wp-content?

FAQ

Why is this a weird, rambly gist?
It started out as a few code snippets and grew into a huge thing. I think it should probably be a blog post, a repo, an Ansible role, etc. I'll develop it further if there's interest.
[Unit]
Description=Caddy HTTP/2 web server
Documentation=https://caddyserver.com/docs
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service
[Service]
Restart=on-failure
; User and group the process will run as.
User=www-data
Group=www-data
; Letsencrypt-issued certificates will be written to this directory.
Environment=CADDYPATH=/etc/ssl/caddy
; Always set "-root" to something safe in case it gets forgotten in the Caddyfile.
ExecStart=/usr/local/bin/caddy -log stdout -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp
ExecReload=/bin/kill -USR1 $MAINPID
; Limit the number of file descriptors; see `man systemd.exec` for more limit settings.
LimitNOFILE=1048576
; Unmodified caddy is not expected to use more than that.
LimitNPROC=64
; Use private /tmp and /var/tmp, which are discarded after caddy stops.
PrivateTmp=true
; Use a minimal /dev
PrivateDevices=true
; Hide /home, /root, and /run/user. Nobody will steal your SSH-keys.
ProtectHome=true
; Make /usr, /boot, /etc and possibly some more folders read-only.
ProtectSystem=full
; … except /etc/ssl/caddy, because we want Letsencrypt-certificates there.
; This merely retains r/w access rights, it does not add any new. Must still be writable on the host!
ReadWriteDirectories=/etc/ssl/caddy
; The following additional security directives only work with systemd v229 or later.
; They further retrict privileges that can be gained by caddy. Uncomment if you like.
; Note that you may have to add capabilities required by any plugins in use.
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
www.example.com {
tls hostmaster@example.com {
# Android 4 and IE 8-10 only support TLS 1.0.
# Change tls1.0 to tls1.1 (or remove this block entirely) for better privacy.
protocols tls1.0 tls1.2
}
log / /var/www/example.com/logs/access.log "{combined}" {
rotate {
size 100 # rotate after 100 MB
age 14 # keep log files for 14 days
keep 10 # keep at most 10 log files
}
}
ipfilter /wp-admin /wp-login.php {
rule allow
ip 1:2:3:4::/56
strict
}
ipfilter /wp-admin/admin-ajax.php {
rule block
}
ratelimit /wp-login.php 4 7 minute
proxy / localhost:6081 {
transparent
}
}
# This intermediate (:2020) stanza modifies the response object of the terminal
# (:2021) stanza before it's cached in Varnish. Its main purpose is to rewrite
# http links in your site to https, assuming you are going live with https for
# the first time as a part of this migration. My site had http links everywhere
# and it wasn't easily fixed by changing the site URL or adding a plugin. YMMV.
#
# This operation may seem expensive, but the impact is limited because the
# results will be cached in Varnish.
#
# The `filter' directive can't coexist with the `fastcgi' directive because it
# results in a Content-Length of zero, which is why we need a separate stanza.
# (It's broken with `proxy', too, but there's a work-around we can use.) At
# some point, when these issues are fixed, I will revisit this configuration.
# If you don't need to rewrite links, you can simply remove this stanza and
# change the port of the terminal stanza to 2020. If you still want to remove
# the X-Pingback header, add this directive to the new :2020 stanza:
#
# header / -X-Pingback
#
www.example.com:2020 {
bind localhost
tls off
proxy / localhost:2021 {
# The Host header gets unset coming into this vhost, so re-set it here.
# Other headers such as X-Forwarded-Proto will be passed-thru as-is.
header_upstream Host {host}
# XML-RPC is disabled, so delete the header that points to it.
header_downstream -X-Pingback
# Let Varnish re-count the bytes because `filter' does the wrong thing. :c
header_downstream -Content-Length
}
filter rule {
content_type (?:text|javascript)
search_pattern https?://(?:www\.)?example\.(?:com|net|org)
replacement https://www.example.com
}
}
www.example.com:2021 {
bind localhost
tls off
errors {
log /var/www/example.com/logs/error.log {
size 100 # rotate after 100 MB
age 14 # keep log files for 14 days
keep 10 # keep at most 10 log files
}
}
root /var/www/example.com/wordpress
# Protect secrets against misconfiguration.
internal /wp-config.php
# Disable XML-RPC (assuming you're not using it).
internal /xmlrpc.php
# Set caching directives for static content. Caddy will automatically add
# Last-Modified and ETag headers for files on disk. Neat!
header /wp-content Cache-Control "public, max-age=2592000, s-maxage=86400"
header /wp-includes Cache-Control "public, max-age=2592000, s-maxage=86400"
rewrite {
if {path} not_match ^/wp-admin
to {path} {path}/ /index.php?_url={uri}
}
fastcgi / /run/php/php7.0-fpm.sock php
}

2017/01/10

  • Add note about options for adding the filter to wp-config.php.
  • Improve the rewrite rule slightly.
  • Oops, s-max-age should be s-maxage!

2017/01/09

  • Add note/link for HTTP Headers plugin.
  • Varnish and Caddy port sharing tested and confirmed working!
  • Raise the ratelimit on /wp-login.php based on real-world experience.

2017/01/07

  • Add /wp-admin to ipfilter and /wp-config.php to internal.
  • Remove /wp-admin from ratelimit.
  • Move ratelimit to initial stanza and change parameters.
  • Remove now-unnecessary realip directive.

2017/01/06

  • Initial publication.
vcl 4.0;
backend default {
.host = "localhost";
.port = "2020";
}
sub vcl_backend_response {
if (beresp.http.content-type ~ "text|javascript|json") {
set beresp.do_gzip = true;
}
}
// Insert *before* requiring wp-settings.php
if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && 'https' === $_SERVER['HTTP_X_FORWARDED_PROTO']) {
$_SERVER['HTTPS'] = 'on';
}
// Insert *after* requiring wp-settings.php
// Alternatively, you can add this to your child theme's functions.php, or into its own (mu) plugin.
add_filter('addh_options', 'my_addh_options');
function my_addh_options($options) {
// We only want the last-modified header, so disable these other ones.
$options['add_etag_header'] = false;
$options['add_expires_header'] = false;
$options['add_cache_control_header'] = false;
return $options;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment