Create a gist now

Instantly share code, notes, and snippets.

Embed
What would you like to do?
The perfect Varnish configuration for Joomla, WordPress & other CMS based websites

The perfect Varnish configuration for Joomla, WordPress & other CMS based websites

IMPORTANT: Read this before implementing one of the configuration files below (for either Varnish 3.x or 4.x+).

USE: Replace the contents of the main Varnish configuration file located in /etc/varnish/default.vcl (root server access required - obviously) with the contents of the configuration you'll use (depending on your Varnish version) from the 2 examples provided below.

IMPORTANT: The following setup assumes a 3 minute (180 sec) cache time. You can safely increase this to 5 mins for less busier sites or drop it to 1 min or even 30s for high traffic sites.

This configuration requires an HTTP Header and a user cookie to identify if a user is logged in a site, in order to bypass caching overall (see how it's done in the Joomla section). If your CMS provides a way to add these 2 requirements, then you can use this configuration to speed up your site or entire server. You can even exclude the domains you don't want to cache if you're looking to use it in a multi-site setup.

JOOMLA & VARNISH

Since Joomla v3.6, all you need to do to have Joomla play nicely with Varnish is add your exclusion points (URLs). Just have a look at the 2 blocks starting with "Exclude the following paths..." below in the Varnish configurations.

Or you can install this handy plugin from JoomlaWorks https://www.joomlaworks.net/downloads/?f=plg_urlnormalizer-v1.3_j1.5-3.x.zip which covers all Joomla versions from 1.5 to 3.x and extends/improves the built-in changes introduced in Joomla v3.6 (as stated above).

If you're using Joomla before version 3.6 and you don't wish to use the plugin above, you need to do the following: This Varnish configuration makes use of a custom HTTP header plus a user cookie to determine whether some user is logged in or not inside Joomla. To insert the HTTP header, simply append the following code block in your template's "index.php" file, right after the line:

defined('_JEXEC') or die;

...and make sure you set the $cookieDomain value:

// Make Joomla Varnish-friendly [START]
$cookieDomain = 'domain.tld'; // Replace "domain.tld" with your "naked" domain

$getUserState = JFactory::getUser();

if ($getUserState->guest) {
    JResponse::allowCache(true);
    JResponse::setHeader('X-Logged-In', 'False', true);
    if($_COOKIE["userID"]){
        setcookie("userID", "", time() - 3600, '/', $cookieDomain, 0);
    }
} else {
    JResponse::allowCache(true);
    JResponse::setHeader('X-Logged-In', 'True', true);
    if(!isset($_COOKIE["userID"])){
        setcookie("userID", $getUserState->id, 0, '/', $cookieDomain, 0);
    }
}
// Make Joomla Varnish-friendly [FINISH]

IMPORTANT: If you use K2 (getk2.org) in your Joomla site, simply set the "Cookie Domain" option in the K2 parameters ("Advanced" tab) and all the above will be automatically enabled for your entire Joomla site.

HOW TO HANDLE FRONTEND LOGINS (e.g. for use with member areas, forums etc.)

It is important for you to understand that since Joomla (in a very amateur way) uses session cookies for any user (even guests) supposedly for additional security (debatable), Varnish cannot work with Joomla out-of-the-box.

If you installed Varnish without any modification to its configuration besides the cache time, it could not properly cache Joomla content because of the session cookies Joomla uses for both guest and logged in visitors. To bypass Joomla's behaviour, we must additionally set Varnish to strip any cookies set by Joomla, except for a specific one (userID). For even better control, we also set a custom HTTP header (X-Logged-In), which we have Varnish check on all requests. All this is explained how to integrate into Joomla via your template in the code sample above.

However, if we want Varnish to allow frontend logins in Joomla, without breaking Joomla (because we strip its session cookies), we must explicitly tell Varnish which entry pages (=login pages) not to cache. Such a page could be for example the default Joomla login form (e.g. with an alias "login"). In the 2 Varnish exclusion lists defined in the configurations below, we would add "^/login" to make sure Varnish completely switches off when a user visits this page. In that case, Joomla's session cookie gets set and the form can be submitted normally, passing all Joomla security checks (aka a token exists as a hidden form input). Same goes for any page in Joomla that requires user input: a contact form, a newsletter signup form, a forum, comments and so on. So the solution to keep in mind is simple:

  • If the action requires the user to login first (e.g. a forum), we must create a specific/unique page for users to login first. Once they log in, Varnish switches off completely and then a user can post in the forum or write comments or use a contact form as if Varnish did not exist. If the user continues to browse the site while logged in, Varnish will be completely off ONLY for this user. If the user logs out, Varnish will kick back in.

  • If the action does not require a user to be logged in first, e.g. a contact form, we simply exclude the contact form's URL from Varnish, in which case -again- Varnish will switch off completely and the user will be able to submit the form passing the Joomla security checks. If the user browses anywhere else in the site, Varnish will kick back in.

WORDPRESS/WOOCOMMERCE & VARNISH

New exceptions have been added (on July 9th, 2018) to properly filter user generated content (and user logins) in WordPress and WooCommerce. This results in more refined exlusions and thus better caching.

Updated on July 9th, 2018

##############################################################################################
### The perfect Varnish 3.x configuration for Joomla, WordPress & other CMS based websites ###
##############################################################################################
######################
#
# UPDATED on July 9th, 2018
#
# Configuration Notes:
# 1. Default dynamic content caching set to 180s.
# Do a search for "180" and replace with the new value in seconds.
# Stale cache is served for up to 12 hours.
# 2. Make sure you update the "backend default { ... }" section with the correct IP and port
#
######################
# Default backend definition. Set this to point to your content server.
backend default {
.host = "127.0.0.1"; # UPDATE this only if the web server is not on the same machine
.port = "8080"; # UPDATE 8080 with your web server's (internal) port
}
sub vcl_recv {
/*
# Blocks
if (req.http.user-agent ~ "^$" && req.http.referer ~ "^$") {
return (synth(204, "No content"));
}
if (req.http.user-agent ~ "(ahrefs|bingbot|domaincrawler|dotbot|mj12bot|semrush)") {
return (synth(204, "Bot blocked"));
}
# If we host multiple domains on a server, here you can list the domains you DO NOT want to cache
# The first check matches both naked & "www" subdomains. Use the second for non generic subdomains.
if (
req.http.host ~ "(www\.)?(domain1.com|domain2.org|domain3.net)" ||
req.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)"
) {
return (pass);
}
*/
# LetsEncrypt Certbot passthrough
if (req.url ~ "^/\.well-known/acme-challenge/") {
return (pass);
}
# Forward client's IP to the backend
if (req.restarts == 0) {
if (req.http.X-Real-IP) {
set req.http.X-Forwarded-For = req.http.X-Real-IP;
} else if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
# httpoxy
unset req.http.proxy;
# Non-RFC2616 or CONNECT which is weird.
if (
req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE"
) {
return (pipe);
}
# We only deal with GET and HEAD by default
if (req.request != "GET" && req.request != "HEAD") {
return (pass);
}
# === URL manipulation ===
# First remove the Google Analytics added parameters, useless for our backend
if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");
set req.url = regsub(req.url, "\?&", "?");
set req.url = regsub(req.url, "\?$", "");
}
# Strip hash, server doesn't need it.
if (req.url ~ "\#") {
set req.url = regsub(req.url, "\#.*$", "");
}
# Strip a trailing ? if it exists
#if (req.url ~ "\?$") {
# set req.url = regsub(req.url, "\?$", "");
#}
# === Generic cookie manipulation ===
# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
# Check for the custom "X-Logged-In" header (used by K2 and other apps) to identify
# if the visitor is a guest, then unset any cookie (including session cookies) provided
# it's not a POST request.
if(req.http.X-Logged-In == "False" && req.request != "POST") {
unset req.http.Cookie;
}
# === DO NOT CACHE ===
# Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
if (
req.http.Authorization ||
req.http.Authenticate ||
req.http.X-Logged-In == "True" ||
req.http.Cookie ~ "userID" ||
req.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
req.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
) {
#set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set req.http.Pragma = "no-cache";
return (pass);
}
# Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
# In Joomla specifically, you are advised to create specific entry points (URLs) for users to
# interact with the site (either common user logins or even commenting), e.g. make a menu item
# to point to a user login page (e.g. /login), including all related functionality such as
# password reset, email reminder and so on.
if(
req.url ~ "^/addons" ||
req.url ~ "^/administrator" ||
req.url ~ "^/cart" ||
req.url ~ "^/checkout" ||
req.url ~ "^/component/banners" ||
req.url ~ "^/component/socialconnect" ||
req.url ~ "^/component/users" ||
req.url ~ "^/connect" ||
req.url ~ "^/contact" ||
req.url ~ "^/login" ||
req.url ~ "^/logout" ||
req.url ~ "^/lost-password" ||
req.url ~ "^/my-account" ||
req.url ~ "^/register" ||
req.url ~ "^/signin" ||
req.url ~ "^/signup" ||
req.url ~ "^/wc-api" ||
req.url ~ "^/wp-admin" ||
req.url ~ "^/wp-login.php" ||
req.url ~ "^\?add-to-cart=" ||
req.url ~ "^\?wc-api="
) {
#set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set req.http.Pragma = "no-cache";
return (pass);
}
# Don't cache ajax requests
if(req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
#set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set req.http.Pragma = "no-cache";
return (pass);
}
# === STATIC FILES ===
# Properly handle different encoding types
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elseif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elseif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unknown algorithm (aka crappy browser)
remove req.http.Accept-Encoding;
}
}
# Remove all cookies for static files & deliver directly
if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset req.http.Cookie;
return (lookup);
}
return (lookup);
}
sub vcl_fetch {
/*
# If we host multiple domains on a server, here you can list the domains you DO NOT want to cache
# The first check matches both naked & "www" subdomains. Use the second for non generic subdomains.
if (
bereq.http.host ~ "(www\.)?(domain1.com|domain2.org|domain3.net)" ||
bereq.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)"
) {
set beresp.uncacheable = true;
return (hit_for_pass);
}
*/
# Don't cache 50x responses
if (
beresp.status == 500 ||
beresp.status == 502 ||
beresp.status == 503 ||
beresp.status == 504
) {
return (hit_for_pass);
}
# === DO NOT CACHE ===
# Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
# In Joomla specifically, you are advised to create specific entry points (URLs) for users to
# interact with the site (either common user logins or even commenting), e.g. make a menu item
# to point to a user login page (e.g. /login), including all related functionality such as
# password reset, email reminder and so on.
if(
bereq.url ~ "^/addons" ||
bereq.url ~ "^/administrator" ||
bereq.url ~ "^/cart" ||
bereq.url ~ "^/checkout" ||
bereq.url ~ "^/component/banners" ||
bereq.url ~ "^/component/socialconnect" ||
bereq.url ~ "^/component/users" ||
bereq.url ~ "^/connect" ||
bereq.url ~ "^/contact" ||
bereq.url ~ "^/login" ||
bereq.url ~ "^/logout" ||
bereq.url ~ "^/lost-password" ||
bereq.url ~ "^/my-account" ||
bereq.url ~ "^/register" ||
bereq.url ~ "^/signin" ||
bereq.url ~ "^/signup" ||
bereq.url ~ "^/wc-api" ||
bereq.url ~ "^/wp-admin" ||
bereq.url ~ "^/wp-login.php" ||
bereq.url ~ "^\?add-to-cart=" ||
bereq.url ~ "^\?wc-api="
) {
#set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set beresp.http.Pragma = "no-cache";
return (hit_for_pass);
}
# Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
if (
bereq.http.Authorization ||
bereq.http.Authenticate ||
bereq.http.X-Logged-In == "True" ||
bereq.http.Cookie ~ "userID" ||
bereq.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
bereq.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
) {
#set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set beresp.http.Pragma = "no-cache";
return (hit_for_pass);
}
# Don't cache ajax requests
if(beresp.http.X-Requested-With == "XMLHttpRequest" || bereq.url ~ "nocache") {
#set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set beresp.http.Pragma = "no-cache";
return (hit_for_pass);
}
# Don't cache backend response to posted requests
if (bereq.request == "POST") {
return (hit_for_pass);
}
# Ok, we're cool & ready to cache things
# so let's clean up some headers and cookies
# to maximize caching.
# Check for the custom "X-Logged-In" header to identify if the visitor is a guest,
# then unset any cookie (including session cookies) provided it's not a POST request.
if(beresp.http.X-Logged-In == "False" && bereq.request != "POST") {
unset beresp.http.Set-Cookie;
}
# Unset the "etag" header (suggested)
unset beresp.http.etag;
# Unset the "pragma" header
unset beresp.http.Pragma;
# Allow stale content, in case the backend goes down
set beresp.grace = 12h;
# This is how long Varnish will keep cached content
set beresp.ttl = 180s;
# Modify "expires" header - https://www.varnish-cache.org/trac/wiki/VCLExampleSetExpires
#set beresp.http.Expires = "" + (now + beresp.ttl);
# If your backend server does not set the right caching headers for static assets,
# you can set them below (uncomment first and change 604800 - which 1 week - to whatever you
# want (in seconds)
#if (req.url ~ "\.(ico|jpg|jpeg|gif|png|bmp|webp|tiff|svg|svgz|pdf|mp3|flac|ogg|mid|midi|wav|mp4|webm|mkv|ogv|wmv|eot|otf|woff|ttf|rss|atom|zip|7z|tgz|gz|rar|bz2|tar|exe|doc|docx|xls|xlsx|ppt|pptx|rtf|odt|ods|odp)(\?[a-zA-Z0-9=]+)$") {
# set beresp.http.Cache-Control = "public, max-age=604800";
#}
if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset beresp.http.set-cookie;
set beresp.do_stream = true;
}
# We have content to cache, but it's got no-cache or other Cache-Control values sent
# So let's reset it to our main caching time (180s as used in this example configuration)
# The additional parameters specified (stale-while-revalidate & stale-if-error) are used
# by modern browsers to better control caching. Set these to twice & four times your main
# cache time respectively.
# This final setting will normalize cache-control headers for CMSs like Joomla
# which set max-age=0 even when the CMS' cache is enabled.
if (beresp.http.Cache-Control !~ "max-age" || beresp.http.Cache-Control ~ "max-age=0" || beresp.ttl < 180s) {
set beresp.http.Cache-Control = "public, max-age=180, stale-while-revalidate=360, stale-if-error=43200";
}
return (deliver);
}
sub vcl_deliver {
/*
# Send a special header for excluded domains only
# The if statement can be identical to the ones in the vcl_recv() and vcl_fetch() functions above
if (
req.http.host ~ "(www\.)?(domain1.com|domain2.org|domain3.net)" ||
req.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)"
) {
set resp.http.X-Domain-Status = "EXCLUDED";
}
# Enforce redirect to HTTPS for specified domains only
if (
req.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)" &&
req.http.X-Forwarded-Proto !~ "(?i)https"
) {
set resp.http.Location = "https://" + req.http.host + req.url;
set resp.status = 302;
}
*/
# Send special headers that indicate the cache status of each web page
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
set resp.http.X-Cache-Hits = obj.hits;
} else {
set resp.http.X-Cache = "MISS";
}
return (deliver);
}
###############################################################################################
### The perfect Varnish 4.x+ configuration for Joomla, WordPress & other CMS based websites ###
###############################################################################################
######################
#
# UPDATED on July 9th, 2018
#
# Configuration Notes:
# 1. Default dynamic content caching set to 180s.
# Do a search for "180" and replace with the new value in seconds.
# Stale cache is served for up to 12 hours.
# 2. Make sure you update the "backend default { ... }" section with the correct IP and port
#
######################
# Varnish Reference:
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and https://www.varnish-cache.org/trac/wiki/VCLExamples for more examples.
# Marker to tell the VCL compiler that this VCL has been adapted to the new 4.0 format
vcl 4.0;
# Imports
import std;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "127.0.0.1"; # UPDATE this only if the web server is not on the same machine
.port = "8080"; # UPDATE 8080 with your web server's (internal) port
}
sub vcl_recv {
/*
# Blocks
if (req.http.user-agent ~ "^$" && req.http.referer ~ "^$") {
return (synth(204, "No content"));
}
if (req.http.user-agent ~ "(ahrefs|bingbot|domaincrawler|dotbot|mj12bot|semrush)") {
return (synth(204, "Bot blocked"));
}
# If we host multiple domains on a server, here you can list the domains you DO NOT want to cache
# The first check matches both naked & "www" subdomains. Use the second for non generic subdomains.
if (
req.http.host ~ "(www\.)?(domain1.com|domain2.org|domain3.net)" ||
req.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)"
) {
return (pass);
}
*/
# LetsEncrypt Certbot passthrough
if (req.url ~ "^/\.well-known/acme-challenge/") {
return (pass);
}
# Forward client's IP to the backend
if (req.restarts == 0) {
if (req.http.X-Real-IP) {
set req.http.X-Forwarded-For = req.http.X-Real-IP;
} else if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
# httpoxy
unset req.http.proxy;
# Normalize the query arguments
set req.url = std.querysort(req.url);
# Non-RFC2616 or CONNECT which is weird.
if (
req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE"
) {
return (pipe);
}
# We only deal with GET and HEAD by default
if (req.method != "GET" && req.method != "HEAD") {
return (pass);
}
# === URL manipulation ===
# First remove the Google Analytics added parameters, useless for our backend
if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");
set req.url = regsub(req.url, "\?&", "?");
set req.url = regsub(req.url, "\?$", "");
}
# Strip hash, server doesn't need it.
if (req.url ~ "\#") {
set req.url = regsub(req.url, "\#.*$", "");
}
# Strip a trailing ? if it exists
#if (req.url ~ "\?$") {
# set req.url = regsub(req.url, "\?$", "");
#}
# === Generic cookie manipulation ===
# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
# Check for the custom "X-Logged-In" header (used by K2 and other apps) to identify
# if the visitor is a guest, then unset any cookie (including session cookies) provided
# it's not a POST request.
if(req.http.X-Logged-In == "False" && req.method != "POST") {
unset req.http.Cookie;
}
# === DO NOT CACHE ===
# Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
if (
req.http.Authorization ||
req.http.Authenticate ||
req.http.X-Logged-In == "True" ||
req.http.Cookie ~ "userID" ||
req.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
req.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
) {
#set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set req.http.Pragma = "no-cache";
return (pass);
}
# Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
# In Joomla specifically, you are advised to create specific entry points (URLs) for users to
# interact with the site (either common user logins or even commenting), e.g. make a menu item
# to point to a user login page (e.g. /login), including all related functionality such as
# password reset, email reminder and so on.
if(
req.url ~ "^/addons" ||
req.url ~ "^/administrator" ||
req.url ~ "^/cart" ||
req.url ~ "^/checkout" ||
req.url ~ "^/component/banners" ||
req.url ~ "^/component/socialconnect" ||
req.url ~ "^/component/users" ||
req.url ~ "^/connect" ||
req.url ~ "^/contact" ||
req.url ~ "^/login" ||
req.url ~ "^/logout" ||
req.url ~ "^/lost-password" ||
req.url ~ "^/my-account" ||
req.url ~ "^/register" ||
req.url ~ "^/signin" ||
req.url ~ "^/signup" ||
req.url ~ "^/wc-api" ||
req.url ~ "^/wp-admin" ||
req.url ~ "^/wp-login.php" ||
req.url ~ "^\?add-to-cart=" ||
req.url ~ "^\?wc-api="
) {
#set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set req.http.Pragma = "no-cache";
return (pass);
}
# Don't cache ajax requests
if(req.http.X-Requested-With == "XMLHttpRequest" || req.url ~ "nocache") {
#set req.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set req.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set req.http.Pragma = "no-cache";
return (pass);
}
# === STATIC FILES ===
# Properly handle different encoding types
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
# No point in compressing these
unset req.http.Accept-Encoding;
} elseif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elseif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unknown algorithm (aka crappy browser)
unset req.http.Accept-Encoding;
}
}
# Remove all cookies for static files & deliver directly
if (req.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
return (hash);
}
sub vcl_backend_response {
/*
# If we host multiple domains on a server, here you can list the domains you DO NOT want to cache
# The first check matches both naked & "www" subdomains. Use the second for non generic subdomains.
if (
bereq.http.host ~ "(www\.)?(domain1.com|domain2.org|domain3.net)" ||
bereq.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)"
) {
set beresp.uncacheable = true;
return (deliver);
}
*/
# Don't cache 50x responses
if (
beresp.status == 500 ||
beresp.status == 502 ||
beresp.status == 503 ||
beresp.status == 504
) {
return (abandon);
}
# === DO NOT CACHE ===
# Exclude the following paths (e.g. backend admins, user pages or ad URLs that require tracking)
# In Joomla specifically, you are advised to create specific entry points (URLs) for users to
# interact with the site (either common user logins or even commenting), e.g. make a menu item
# to point to a user login page (e.g. /login), including all related functionality such as
# password reset, email reminder and so on.
if(
bereq.url ~ "^/addons" ||
bereq.url ~ "^/administrator" ||
bereq.url ~ "^/cart" ||
bereq.url ~ "^/checkout" ||
bereq.url ~ "^/component/banners" ||
bereq.url ~ "^/component/socialconnect" ||
bereq.url ~ "^/component/users" ||
bereq.url ~ "^/connect" ||
bereq.url ~ "^/contact" ||
bereq.url ~ "^/login" ||
bereq.url ~ "^/logout" ||
bereq.url ~ "^/lost-password" ||
bereq.url ~ "^/my-account" ||
bereq.url ~ "^/register" ||
bereq.url ~ "^/signin" ||
bereq.url ~ "^/signup" ||
bereq.url ~ "^/wc-api" ||
bereq.url ~ "^/wp-admin" ||
bereq.url ~ "^/wp-login.php" ||
bereq.url ~ "^\?add-to-cart=" ||
bereq.url ~ "^\?wc-api="
) {
#set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set beresp.http.Pragma = "no-cache";
set beresp.uncacheable = true;
return (deliver);
}
# Don't cache HTTP authorization/authentication pages and pages with certain headers or cookies
if (
bereq.http.Authorization ||
bereq.http.Authenticate ||
bereq.http.X-Logged-In == "True" ||
bereq.http.Cookie ~ "userID" ||
bereq.http.Cookie ~ "joomla_[a-zA-Z0-9_]+" ||
bereq.http.Cookie ~ "(wordpress_[a-zA-Z0-9_]+|wp-postpass|comment_author_[a-zA-Z0-9_]+|woocommerce_cart_hash|woocommerce_items_in_cart|wp_woocommerce_session_[a-zA-Z0-9]+)"
) {
#set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set beresp.http.Pragma = "no-cache";
set beresp.uncacheable = true;
return (deliver);
}
# Don't cache ajax requests
if(beresp.http.X-Requested-With == "XMLHttpRequest" || bereq.url ~ "nocache") {
#set beresp.http.Cache-Control = "private, max-age=0, no-cache, no-store";
#set beresp.http.Expires = "Mon, 01 Jan 2001 00:00:00 GMT";
#set beresp.http.Pragma = "no-cache";
set beresp.uncacheable = true;
return (deliver);
}
# Don't cache backend response to posted requests
if (bereq.method == "POST") {
set beresp.uncacheable = true;
return (deliver);
}
# Ok, we're cool & ready to cache things
# so let's clean up some headers and cookies
# to maximize caching.
# Check for the custom "X-Logged-In" header to identify if the visitor is a guest,
# then unset any cookie (including session cookies) provided it's not a POST request.
if(beresp.http.X-Logged-In == "False" && bereq.method != "POST") {
unset beresp.http.Set-Cookie;
}
# Unset the "etag" header (suggested)
unset beresp.http.etag;
# Unset the "pragma" header
unset beresp.http.Pragma;
# Allow stale content, in case the backend goes down
set beresp.grace = 12h;
# This is how long Varnish will keep cached content
set beresp.ttl = 180s;
# Modify "expires" header - https://www.varnish-cache.org/trac/wiki/VCLExampleSetExpires
#set beresp.http.Expires = "" + (now + beresp.ttl);
# If your backend server does not set the right caching headers for static assets,
# you can set them below (uncomment first and change 604800 - which 1 week - to whatever you
# want (in seconds)
#if (req.url ~ "\.(ico|jpg|jpeg|gif|png|bmp|webp|tiff|svg|svgz|pdf|mp3|flac|ogg|mid|midi|wav|mp4|webm|mkv|ogv|wmv|eot|otf|woff|ttf|rss|atom|zip|7z|tgz|gz|rar|bz2|tar|exe|doc|docx|xls|xlsx|ppt|pptx|rtf|odt|ods|odp)(\?[a-zA-Z0-9=]+)$") {
# set beresp.http.Cache-Control = "public, max-age=604800";
#}
if (bereq.url ~ "^[^?]*\.(7z|avi|bmp|bz2|css|csv|doc|docx|eot|flac|flv|gif|gz|ico|jpeg|jpg|js|less|mka|mkv|mov|mp3|mp4|mpeg|mpg|odt|ogg|ogm|opus|otf|pdf|png|ppt|pptx|rar|rtf|svg|svgz|swf|tar|tbz|tgz|ttf|txt|txz|wav|webm|webp|woff|woff2|xls|xlsx|xml|xz|zip)(\?.*)?$") {
unset beresp.http.set-cookie;
set beresp.do_stream = true;
}
# We have content to cache, but it's got no-cache or other Cache-Control values sent
# So let's reset it to our main caching time (180s as used in this example configuration)
# The additional parameters specified (stale-while-revalidate & stale-if-error) are used
# by modern browsers to better control caching. Set these to twice & four times your main
# cache time respectively.
# This final setting will normalize cache-control headers for CMSs like Joomla
# which set max-age=0 even when the CMS' cache is enabled.
if (beresp.http.Cache-Control !~ "max-age" || beresp.http.Cache-Control ~ "max-age=0" || beresp.ttl < 180s) {
set beresp.http.Cache-Control = "public, max-age=180, stale-while-revalidate=360, stale-if-error=43200";
}
return (deliver);
}
sub vcl_deliver {
/*
# Send a special header for excluded domains only
# The if statement can be identical to the ones in the vcl_recv() and vcl_fetch() functions above
if (
req.http.host ~ "(www\.)?(domain1.com|domain2.org|domain3.net)" ||
req.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)"
) {
set resp.http.X-Domain-Status = "EXCLUDED";
}
# Enforce redirect to HTTPS for specified domains only
if (
req.http.host ~ "(subdomain.domain4.tld|othersubdomain.domain5.tld)" &&
req.http.X-Forwarded-Proto !~ "(?i)https"
) {
set resp.http.Location = "https://" + req.http.host + req.url;
set resp.status = 302;
}
*/
# Send special headers that indicate the cache status of each web page
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
set resp.http.X-Cache-Hits = obj.hits;
} else {
set resp.http.X-Cache = "MISS";
}
return (deliver);
}
@Basilakis

This comment has been minimized.

Show comment
Hide comment
@Basilakis

Basilakis Feb 28, 2016

Good Work Fotis, thanks a lot for sharing

Good Work Fotis, thanks a lot for sharing

@humblehumanbeing

This comment has been minimized.

Show comment
Hide comment
@humblehumanbeing

humblehumanbeing May 26, 2016

I am a little bit confused:
If I would like content/images etc to have cached for 6hrs is must-be changed value "2m" with all occurrences

I am a little bit confused:
If I would like content/images etc to have cached for 6hrs is must-be changed value "2m" with all occurrences

@gerardo15

This comment has been minimized.

Show comment
Hide comment
@gerardo15

gerardo15 Jul 7, 2016

Hello! im trying to make this work with the latest version of Varnish and i get tons of error messages.

Hello! im trying to make this work with the latest version of Varnish and i get tons of error messages.

@fevangelou

This comment has been minimized.

Show comment
Hide comment
@fevangelou

fevangelou Sep 20, 2016

@sural98 if you want to enforce longer caching, you should use cache headers in your CMS or app.

@gerardo15 Indeed the latest Varnish (4.1+) does not work with the above setup. I'll be updating the setup today...

Owner

fevangelou commented Sep 20, 2016

@sural98 if you want to enforce longer caching, you should use cache headers in your CMS or app.

@gerardo15 Indeed the latest Varnish (4.1+) does not work with the above setup. I'll be updating the setup today...

@fevangelou

This comment has been minimized.

Show comment
Hide comment
@fevangelou

fevangelou Sep 21, 2016

Configuration updated with a common preface file and 2 separate files for Varnish 3.x and 4.x or newer.

Owner

fevangelou commented Sep 21, 2016

Configuration updated with a common preface file and 2 separate files for Varnish 3.x and 4.x or newer.

@ssnobben

This comment has been minimized.

Show comment
Hide comment
@ssnobben

ssnobben Sep 22, 2016

This configuration works with Joomla 3.6.+ for Varnish 4.0 but not Varnish 5.0 I guess?

This configuration works with Joomla 3.6.+ for Varnish 4.0 but not Varnish 5.0 I guess?

@chris001

This comment has been minimized.

Show comment
Hide comment
@chris001

chris001 Oct 1, 2016

Would be nice to try the 4.x+ config to confirm it to be compatible with 5.0 because binary packages are available for download to install on rpm and deb linux distros here https://repo.varnish-cache.org/pkg/

chris001 commented Oct 1, 2016

Would be nice to try the 4.x+ config to confirm it to be compatible with 5.0 because binary packages are available for download to install on rpm and deb linux distros here https://repo.varnish-cache.org/pkg/

@theret

This comment has been minimized.

Show comment
Hide comment
@theret

theret Oct 17, 2016

Hey,
Sorry fo the ignorance... How exactly do I configure the file (Varnish 4) for Joomla 3.6 sites? Can you please show an example how to exclude in the code?

Thanks ;)

theret commented Oct 17, 2016

Hey,
Sorry fo the ignorance... How exactly do I configure the file (Varnish 4) for Joomla 3.6 sites? Can you please show an example how to exclude in the code?

Thanks ;)

@fevangelou

This comment has been minimized.

Show comment
Hide comment
@fevangelou

fevangelou Dec 1, 2016

@ALL For #Joomla specifically, I have a plugin to assist with better HTTP headers (makes controlling caching even better). Here's the link for the plugin: http://www.joomlaworks.net/downloads/?f=plg_urlnormalizer-v1.1_j1.5-3.x.zip

By installing the above plugin, you can set a different cache time for your homepage and internal pages. Varnish will respect the cache times you set because it "reads" the HTTP headers generated by the plugin.

A WordPress version is in the works as well.

Owner

fevangelou commented Dec 1, 2016

@ALL For #Joomla specifically, I have a plugin to assist with better HTTP headers (makes controlling caching even better). Here's the link for the plugin: http://www.joomlaworks.net/downloads/?f=plg_urlnormalizer-v1.1_j1.5-3.x.zip

By installing the above plugin, you can set a different cache time for your homepage and internal pages. Varnish will respect the cache times you set because it "reads" the HTTP headers generated by the plugin.

A WordPress version is in the works as well.

@alen12345

This comment has been minimized.

Show comment
Hide comment
@alen12345

alen12345 Feb 21, 2017

set req.url = std.querysort(req.url);

seems to break Joomla pagination. Can anyone confirm?

Or are the SEF links breaking pagination together with varnish?

alen12345 commented Feb 21, 2017

set req.url = std.querysort(req.url);

seems to break Joomla pagination. Can anyone confirm?

Or are the SEF links breaking pagination together with varnish?

@rodz37

This comment has been minimized.

Show comment
Hide comment
@rodz37

rodz37 Mar 10, 2017

Hi guys
I copy and pasted this into default.vcl file but it's giving me: Error 503 Service Unavailable

Guru Meditation:
XID: 32773

I am testing with Joomla 3.7 Beta3, on Nginx.

rodz37 commented Mar 10, 2017

Hi guys
I copy and pasted this into default.vcl file but it's giving me: Error 503 Service Unavailable

Guru Meditation:
XID: 32773

I am testing with Joomla 3.7 Beta3, on Nginx.

@nigelbpeck

This comment has been minimized.

Show comment
Hide comment
@nigelbpeck

nigelbpeck Mar 30, 2017

According to the documentation 50x responses are not cached by default:

Only the following status codes will be cached by default:

200: OK
203: Non-Authoritative Information
300: Multiple Choices
301: Moved Permanently
302: Moved Temporarily
304: Not modified
307: Temporary Redirect
410: Gone
404: Not Found

So why does the 4.0 config have this in vcl_backend_response:

# Don't cache 50x responses
if (
    beresp.status == 500 ||
    beresp.status == 502 ||
    beresp.status == 503 ||
    beresp.status == 504
) {
    return (abandon);
}

Also, abandon will set the status to 503, which doesn't seem desirable. So if this is going to be done, something like this would be better.

Perhaps I'm missing something, interested to hear your thoughts. Thanks for the awesome code it's been very helpful.

nigelbpeck commented Mar 30, 2017

According to the documentation 50x responses are not cached by default:

Only the following status codes will be cached by default:

200: OK
203: Non-Authoritative Information
300: Multiple Choices
301: Moved Permanently
302: Moved Temporarily
304: Not modified
307: Temporary Redirect
410: Gone
404: Not Found

So why does the 4.0 config have this in vcl_backend_response:

# Don't cache 50x responses
if (
    beresp.status == 500 ||
    beresp.status == 502 ||
    beresp.status == 503 ||
    beresp.status == 504
) {
    return (abandon);
}

Also, abandon will set the status to 503, which doesn't seem desirable. So if this is going to be done, something like this would be better.

Perhaps I'm missing something, interested to hear your thoughts. Thanks for the awesome code it's been very helpful.

@nigelbpeck

This comment has been minimized.

Show comment
Hide comment
@nigelbpeck

nigelbpeck Apr 10, 2017

On line 35 of the Varnish 4 config, why does X-Forwarded-For get overwritten with just the X-Real-IP? It seems a shame to lose any other IP addresses that are listed in there, and it should already include the X-Real-IP? Thanks.

On line 35 of the Varnish 4 config, why does X-Forwarded-For get overwritten with just the X-Real-IP? It seems a shame to lose any other IP addresses that are listed in there, and it should already include the X-Real-IP? Thanks.

@Sulpher

This comment has been minimized.

Show comment
Hide comment
@Sulpher

Sulpher May 5, 2017

Hi Fotis,
The solution works perfect, but the issue appears when I enable SSL and the Joomla needs to be patched.
I used this solution: joomla/joomla-cms#7902

Now Joomla works with Varnish and SSL, but the change linked above breaks all hardcoded relative URL in a page, the most common being the named anchors such as Go to top or similar.

Would be nice if more people will confirm the problem with Varnish + SSL and submit an issue in Joomla issues tracker / github.

Sulpher commented May 5, 2017

Hi Fotis,
The solution works perfect, but the issue appears when I enable SSL and the Joomla needs to be patched.
I used this solution: joomla/joomla-cms#7902

Now Joomla works with Varnish and SSL, but the change linked above breaks all hardcoded relative URL in a page, the most common being the named anchors such as Go to top or similar.

Would be nice if more people will confirm the problem with Varnish + SSL and submit an issue in Joomla issues tracker / github.

@chris001

This comment has been minimized.

Show comment
Hide comment
@chris001

chris001 May 23, 2017

@Sulpher
Could you provide the steps to recreate the issue with the unpatched code and explain why the issue occurs?

And the steps to recreate the problem after the patch from joomla/joomla-cms#7902 ...when and why the patched code work, and doesn't work with html "fragments" (hash code at the end of the URL) ?

@sharphosting
You're right, X-Forwarded-For X-Real-IP X-Forwarded-Proto and X-Forwarded-Port are important to get right. and not lose information since many incoming site visitors are coming thru proxies.

chris001 commented May 23, 2017

@Sulpher
Could you provide the steps to recreate the issue with the unpatched code and explain why the issue occurs?

And the steps to recreate the problem after the patch from joomla/joomla-cms#7902 ...when and why the patched code work, and doesn't work with html "fragments" (hash code at the end of the URL) ?

@sharphosting
You're right, X-Forwarded-For X-Real-IP X-Forwarded-Proto and X-Forwarded-Port are important to get right. and not lose information since many incoming site visitors are coming thru proxies.

@lupetalo

This comment has been minimized.

Show comment
Hide comment
@lupetalo

lupetalo Oct 26, 2017

It breaks Wordpress admin

lupetalo commented Oct 26, 2017

It breaks Wordpress admin

@Mr-Anonymous

This comment has been minimized.

Show comment
Hide comment
@Mr-Anonymous

Mr-Anonymous Nov 21, 2017

Hi Fotis,

I have been reading through a lot of varnish configuration for Joomla and so far yours has been by far the most transparent and sounds like an ideal approach to mitigate the issues with Joomla + Varnish. So, first of all, thank you for sharing this solution.

I do have a question on your approach here. I see that you are disabling Varnish cache completely for Logged-in users and only serving the cache for guest users. My question is, if the website only has a handful of pages for members, then whats the point in disabling the cache completely when user logs in? For example, in your Joomla Works site, I see the pages that really require the member login status check are:

"^/members"
"^/forum"

and maybe couple of other pages like contact-us, etc. But most of the pages in your site are the same for both guests and members and doesn't change whether they are logged in or not. Pages such as:

Home
"^/Extensions"
"^/joomla-templates"
"^/about"
a few others

My website has a similar set-up where 70% of the site menus are the exact same for both guests and logged in users and only 30% pages which consists of Forum and Member pages requires authentication check.

So my question is:

  1. Can't we completely remove the req.http.X-Logged-In check from the Varnish configuration above and serve the same cache for both logged in users and guests for all pages except for the ones that are listed in excluded paths such as the forum and members urls?

  2. If I do this, is that logically flawed or will it cause more mix-ups with accidental user cookies being cached or something else?

  3. If this approach can be done, then is it just about removing ALL cookies from all requests and backend response for all cacheable pages for it work?

Pros: Both guests and logged in users can enjoy the fast cached page loads for most pages that doesn't change.

Cons: Joomla will not know the logged in user's session is continued when they are browsing the cached pages. But that can be mitigated by increasing session lifetime.

I have completely restructured my site now (on Joomla 3.8.2) with this approach in mind and now only 2 main menu links are for members and rest of the others are same for all. So I was thinking if something like this can be achieved with varnish instead.

What do you think of this approach? Is it possible or risky to mix-up?

Thanks,
Neel.

Hi Fotis,

I have been reading through a lot of varnish configuration for Joomla and so far yours has been by far the most transparent and sounds like an ideal approach to mitigate the issues with Joomla + Varnish. So, first of all, thank you for sharing this solution.

I do have a question on your approach here. I see that you are disabling Varnish cache completely for Logged-in users and only serving the cache for guest users. My question is, if the website only has a handful of pages for members, then whats the point in disabling the cache completely when user logs in? For example, in your Joomla Works site, I see the pages that really require the member login status check are:

"^/members"
"^/forum"

and maybe couple of other pages like contact-us, etc. But most of the pages in your site are the same for both guests and members and doesn't change whether they are logged in or not. Pages such as:

Home
"^/Extensions"
"^/joomla-templates"
"^/about"
a few others

My website has a similar set-up where 70% of the site menus are the exact same for both guests and logged in users and only 30% pages which consists of Forum and Member pages requires authentication check.

So my question is:

  1. Can't we completely remove the req.http.X-Logged-In check from the Varnish configuration above and serve the same cache for both logged in users and guests for all pages except for the ones that are listed in excluded paths such as the forum and members urls?

  2. If I do this, is that logically flawed or will it cause more mix-ups with accidental user cookies being cached or something else?

  3. If this approach can be done, then is it just about removing ALL cookies from all requests and backend response for all cacheable pages for it work?

Pros: Both guests and logged in users can enjoy the fast cached page loads for most pages that doesn't change.

Cons: Joomla will not know the logged in user's session is continued when they are browsing the cached pages. But that can be mitigated by increasing session lifetime.

I have completely restructured my site now (on Joomla 3.8.2) with this approach in mind and now only 2 main menu links are for members and rest of the others are same for all. So I was thinking if something like this can be achieved with varnish instead.

What do you think of this approach? Is it possible or risky to mix-up?

Thanks,
Neel.

@fevangelou

This comment has been minimized.

Show comment
Hide comment
@fevangelou

fevangelou Jul 9, 2018

@Mr-Anonymous Sorry, but for some reason, GitHub does not send notifications for comments on Gists... Anyway, to answer your questions:

  1. This header is used to detect the state of a user. Otherwise we'd have to allow cookies for all URLs and thus effectively kill the point of caching as Varnish creates cache objects combining the cookie value and URL. So a different set of cache objects per user.
  2. Use the config as is. It's battle tested on very large deployments (aka sites with millions of monthly users) and the exclusions are the same as on Engintron (albeit adapted for Nginx). Engintron is used on more than 50,000 servers worldwide.
  3. This is done already. In fact, the new version I uploaded today strip even more stuff from the request body.
Owner

fevangelou commented Jul 9, 2018

@Mr-Anonymous Sorry, but for some reason, GitHub does not send notifications for comments on Gists... Anyway, to answer your questions:

  1. This header is used to detect the state of a user. Otherwise we'd have to allow cookies for all URLs and thus effectively kill the point of caching as Varnish creates cache objects combining the cookie value and URL. So a different set of cache objects per user.
  2. Use the config as is. It's battle tested on very large deployments (aka sites with millions of monthly users) and the exclusions are the same as on Engintron (albeit adapted for Nginx). Engintron is used on more than 50,000 servers worldwide.
  3. This is done already. In fact, the new version I uploaded today strip even more stuff from the request body.
@centminmod

This comment has been minimized.

Show comment
Hide comment
@centminmod

centminmod Jul 10, 2018

@fevangelou might want to move the config into an actual github repo so you can get notifications :)

@fevangelou might want to move the config into an actual github repo so you can get notifications :)

@fevangelou

This comment has been minimized.

Show comment
Hide comment
@fevangelou

fevangelou Jul 10, 2018

@centminmod True - in the meantime, I've found Giscus >> https://giscus.co

Owner

fevangelou commented Jul 10, 2018

@centminmod True - in the meantime, I've found Giscus >> https://giscus.co

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment