ETag & Last-Modified set from Rails

etag-and-last-modified-set-from-rails.md

Serving from NFS via Nginx with ETag & Last-Modified set by Nginx

$ curl -s -v http://assets-origin.dev.gov.uk/media/599ef676759b7414919c2e7b/tmp.txt
*   Trying 10.1.1.254...
* TCP_NODELAY set
* Connected to assets-origin.dev.gov.uk (10.1.1.254) port 80 (#0)
> GET /media/599ef676759b7414919c2e7b/tmp.txt HTTP/1.1
> Host: assets-origin.dev.gov.uk
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 24 Aug 2017 17:26:17 GMT
< Content-Type: text/plain
< Content-Length: 29
< Connection: close
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Last-Modified: Thu, 24 Aug 2017 15:53:24 GMT
< Vary: Accept-Encoding
< Content-Disposition: inline; filename="tmp.txt"
< Cache-Control: private
< ETag: "599ef674-1d"
< X-Frame-Options: DENY
< Accept-Ranges: bytes
< Strict-Transport-Security: max-age=31536000
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, OPTIONS
< Access-Control-Allow-Headers: origin, authorization
< 
Thu 24 Aug 2017 16:53:17 BST
* Closing connection 0

Proxying to S3 via Nginx with ETag & Last-Modified set by S3

$ curl -s -v http://assets-origin.dev.gov.uk/media/599ef676759b7414919c2e7b/tmp.txt?proxy_to_s3_via_nginx=true
*   Trying 10.1.1.254...
* TCP_NODELAY set
* Connected to assets-origin.dev.gov.uk (10.1.1.254) port 80 (#0)
> GET /media/599ef676759b7414919c2e7b/tmp.txt?proxy_to_s3_via_nginx=true HTTP/1.1
> Host: assets-origin.dev.gov.uk
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 24 Aug 2017 17:27:55 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 29
< Connection: close
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Cache-Control: no-cache
< Last-Modified: Thu, 24 Aug 2017 15:53:42 GMT
< ETag: "724f4c9aaec837d8264b4ba710b16ca1"
< Cache-Control: max-age=86400, public
< Content-Disposition: inline; filename="tmp.txt"
< Accept-Ranges: bytes
< X-Frame-Options: DENY
< Strict-Transport-Security: max-age=31536000
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, OPTIONS
< Access-Control-Allow-Headers: origin, authorization
< 
Thu 24 Aug 2017 16:53:17 BST
* Closing connection 0

Serving from NFS via Nginx with ETag & Last-Modified set from Rails

$ curl -s -v http://assets-origin.dev.gov.uk/media/599ef676759b7414919c2e7b/tmp.txt
*   Trying 10.1.1.254...
* TCP_NODELAY set
* Connected to assets-origin.dev.gov.uk (10.1.1.254) port 80 (#0)
> GET /media/599ef676759b7414919c2e7b/tmp.txt HTTP/1.1
> Host: assets-origin.dev.gov.uk
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 24 Aug 2017 17:14:05 GMT
< Content-Type: text/plain
< Content-Length: 29
< Connection: close
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Content-Disposition: inline; filename="tmp.txt"
< Cache-Control: private
< ETag: "599ef674-1d"
< Last-Modified: Thu, 24 Aug 2017 15:53:24 GMT
< Accept-Ranges: bytes
< Strict-Transport-Security: max-age=31536000
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, OPTIONS
< Access-Control-Allow-Headers: origin, authorization
< 
Thu 24 Aug 2017 16:53:17 BST
* Closing connection 0

Proxying to S3 via Nginx with ETag & Last-Modified set from Rails

$ curl -s -v http://assets-origin.dev.gov.uk/media/599ef676759b7414919c2e7b/tmp.txt?proxy_to_s3_via_nginx=true
*   Trying 10.1.1.254...
* TCP_NODELAY set
* Connected to assets-origin.dev.gov.uk (10.1.1.254) port 80 (#0)
> GET /media/599ef676759b7414919c2e7b/tmp.txt?proxy_to_s3_via_nginx=true HTTP/1.1
> Host: assets-origin.dev.gov.uk
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Thu, 24 Aug 2017 17:13:25 GMT
< Content-Type: text/plain; charset=utf-8
< Content-Length: 29
< Connection: close
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Vary: Accept-Encoding
< Cache-Control: max-age=0, private, must-revalidate
< Last-Modified: Thu, 24 Aug 2017 15:53:24 GMT
< ETag: "599ef674-1d"
< Cache-Control: max-age=86400, public
< Content-Disposition: inline; filename="tmp.txt"
< Accept-Ranges: bytes
< Strict-Transport-Security: max-age=31536000
< Access-Control-Allow-Origin: *
< Access-Control-Allow-Methods: GET, OPTIONS
< Access-Control-Allow-Headers: origin, authorization
< 
Thu 24 Aug 2017 16:53:17 BST
* Closing connection 0

media_controller.diff

diff --git a/app/controllers/media_controller.rb b/app/controllers/media_controller.rb
index b9aad2c..9235eb5 100644
--- a/app/controllers/media_controller.rb
+++ b/app/controllers/media_controller.rb
@@ -21,11 +21,15 @@ def download
         elsif proxy_to_s3_via_nginx?
           url = Services.cloud_storage.presigned_url_for(asset)
           headers['X-Accel-Redirect'] = "/cloud-storage-proxy/#{url}"
+          headers['ETag'] = %{"#{sprintf('%x-%x', File.mtime(asset.file.path), File.size(asset.file.path))}"}
+          headers['Last-Modified'] = File.mtime(asset.file.path).httpdate
           render nothing: true
         elsif proxy_to_s3_via_rails?
           body = Services.cloud_storage.load(asset)
           send_data(body.read, **AssetManager.content_disposition.options_for(asset))
         else
+          headers['ETag'] = %{"#{sprintf('%x-%x', File.mtime(asset.file.path), File.size(asset.file.path))}"}
+          headers['Last-Modified'] = File.mtime(asset.file.path).httpdate
           send_file(asset.file.path, disposition: AssetManager.content_disposition.type)
         end
       end

nginx-vhost.conf

upstream asset-manager.dev.gov.uk-proxy {
  server localhost:3037;
}

server {
  server_name asset-manager.dev.gov.uk private-asset-manager.dev.gov.uk;

  listen 80;

  # More stuff here ...

  location ~ /raw/(.*) {
    internal;
    
    # ** Set response headers based on values from Rails response headers **
    add_header ETag $upstream_http_etag;
    add_header Last-Modified $upstream_http_last_modified;

    alias /var/apps/asset-manager/uploads/assets/$1;
  }

  # ** Store values from Rails response headers **
  set $etag_from_rails $upstream_http_etag;
  set $last_modified_from_rails $upstream_http_last_modified;

  location ~ /cloud-storage-proxy/(.*) {
    # Prevent requests to this location from outside nginx
    internal;

    # ** Set response headers based on values stored from Rails response headers **
    add_header ETag $etag_from_rails;
    add_header Last-Modified $last_modified_from_rails;

    # More stuff here ...

    # Download the file and send it to client
    proxy_pass $download_url;
  }
}