Skip to content

Instantly share code, notes, and snippets.

@raucao
Last active September 8, 2020 01:29
Show Gist options
  • Save raucao/c7a982226d08acd4e041 to your computer and use it in GitHub Desktop.
Save raucao/c7a982226d08acd4e041 to your computer and use it in GitHub Desktop.
Nginx proxy to S3
location ~* ^/s3/(.*) {
set $bucket '<REPLACE WITH YOUR S3 BUCKET NAME>';
set $aws_access '<REPLACE WITH YOUR AWS ACCESS KEY>';
set $aws_secret '<REPLACE WITH YOUR AWS SECRET KEY>';
set $url_full "$1";
set_by_lua $now "return ngx.cookie_time(ngx.time())";
set $string_to_sign "$request_method\n\n\n\nx-amz-date:${now}\n/$bucket/$url_full";
set_hmac_sha1 $aws_signature $aws_secret $string_to_sign;
set_encode_base64 $aws_signature $aws_signature;
resolver 172.31.0.2 valid=300s;
resolver_timeout 10s;
proxy_http_version 1.1;
proxy_set_header Host $bucket.s3.amazonaws.com;
proxy_set_header x-amz-date $now;
proxy_set_header Authorization "AWS $aws_access:$aws_signature";
proxy_buffering off;
proxy_intercept_errors on;
rewrite .* /$url_full break;
proxy_pass http://s3.amazonaws.com;
}
@insekticid
Copy link

in case you have error: Invalid date format header, expected to be in ISO8601, RFC1123 or RFC1123Z time format.

replace cookie_time -> http_time

https://github.com/openresty/lua-nginx-module#ngxhttp_time

@aarati-K
Copy link

aarati-K commented May 2, 2019

I tried using this s3 proxy solution, but every time I get 403 AccessDenied.

(I have made just one modification. Instead of set_by_lua directive, I've used set_formatted_gmt_time $now "%a, %e %b %Y %H:%M:%S +0000"; I've followed this to make sure that the time format is correct)

I make requests to the nginx server as follows

import requests
r = requests.get("http://localhost:80/s3/key_name")
print(r.status_code)

Is there something I am missing? Thanks in advance!

EDIT: So it looks like this config uses AWS signature version 2, which has now been deprecated for S3. Only signature version 4 is supported. More information here.

Copy link

ghost commented Aug 29, 2019

Same issue here with minio

        location /media  {
            set $bucket           'media';
            set $aws_access       'AKIAIOSFODNN7EXAMPLE';
            set $aws_secret       'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY';
            set $url_full         "$1";
            set $aws_signature    '';
            set_by_lua $now       "return ngx.http_time(ngx.time())";
            set $string_to_sign   "$request_method\n\n\n\nx-amz-date:${now}\n/$url_full/$bucket";
            set_hmac_sha1          $aws_signature $aws_secret $string_to_sign;
            set_encode_base64      $aws_signature $aws_signature;
            resolver_timeout       10s;
            proxy_http_version     1.1;
            proxy_set_header       x-amz-date $now;
            proxy_set_header       Authorization "AWS $aws_access:$aws_signature";
            proxy_buffering        off;
            proxy_intercept_errors on;
            rewrite .* /$url_full break;
            proxy_pass             http://backend-data;
            proxy_set_header       Host $http_host;
            add_header X-Content-Type-Options nosniff;
        }

I get the following error: The request signature we calculated does not match the signature you provided. Check your key and signing method.

@sathishkumarpkd
Copy link

getting this error on compiling
nginx: [emerg] unknown "aws_signature" variable
please help!

@burgew
Copy link

burgew commented Sep 8, 2020

I installed OpenResty, in order to set_by_lua, set_hmac_sha1 and others, and the problem I've run into is that the $url_full variable is not recognized by the rewrite command, so the URL is rewritten as just "/". This has the result of discarding the specific object URL and just trying to do a list on the S3 bucket, itself. Is there a way to get the rewrite directive to understand previously declared variables, rather than just the $1, $2, ... that are built into the rewrite directive?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment