Skip to content

Instantly share code, notes, and snippets.

@Voronenko
Created March 9, 2018 13:08
Show Gist options
  • Save Voronenko/fb9953c0ceb62ce0e0d2d79f8f8349ac to your computer and use it in GitHub Desktop.
Save Voronenko/fb9953c0ceb62ce0e0d2d79f8f8349ac to your computer and use it in GitHub Desktop.
nginx proxy private s3 bucket
worker_processes 1;
daemon off;
error_log /dev/stdout info;
pid /usr/local/var/nginx/nginx.pid;
events {
worker_connections 1024;
}
http {
default_type text/html;
access_log /dev/stdout;
sendfile on;
keepalive_timeout 65;
proxy_cache_path /tmp/ levels=1:2 keys_zone=s3_cache:10m max_size=500m
inactive=60m use_temp_path=off;
server {
listen 8080;
location /s3/ {
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Authorization '';
proxy_set_header Host yanpy.dev.s3.amazonaws.com;
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-server-side-encryption;
proxy_hide_header x-amz-server-side-encryption;
proxy_hide_header Set-Cookie;
proxy_ignore_headers Set-Cookie;
proxy_intercept_errors on;
add_header Cache-Control max-age=31536000;
proxy_pass http://yanpy.dev.s3.amazonaws.com/;
}
location /s3_cached/ {
proxy_cache s3_cache;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Authorization '';
proxy_set_header Host yanpy.dev.s3.amazonaws.com;
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-server-side-encryption;
proxy_hide_header x-amz-server-side-encryption;
proxy_hide_header Set-Cookie;
proxy_ignore_headers Set-Cookie;
proxy_cache_revalidate on;
proxy_intercept_errors on;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_lock on;
add_header Cache-Control max-age=31536000;
add_header X-Cache-Status $upstream_cache_status;
proxy_pass http://yanpy.dev.s3.amazonaws.com/;
}
}
}
@voltechs
Copy link

How does this work with a private bucket if you're setting the Authorization header to ''?

@Voronenko
Copy link
Author

Perhaps title is quite misleading.

For private you are about to combine location as

        location ~ '/' {
            resolver                  8.8.8.8 valid=300s;
            resolver_timeout          10s;

            set $key "${PREFIX}$1";
            set $bucketname "${BUCKET}";

            set $aws_custom_secret "${BUCKET_CUSTOM_SECRET}";

            proxy_set_header User-Agent $aws_custom_secret;
            proxy_buffering off;

            # we need to set the host header here in order to find the bucket
            proxy_set_header Host $bucketname.s3.eu-central-1.amazonaws.com;
            rewrite ^(.*)/$ $1/index.html break;
            rewrite ^(.*/[^./]+)$ $1/index.html break;

            proxy_pass https://$bucketname.s3.eu-central-1.amazonaws.com;
        }

as support it with aws policy , validating secret string

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "enforce-tls-requests-only",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::YOURBUCKETNAME/*",
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        },
        {
            "Sid": "bucket_web_serving_with_header",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOURBUCKETNAME/*",
            "Condition": {
                "StringLike": {
                    "aws:UserAgent": "YOURSUPERDUPERSECRETSTRING"
                }
            }
        }
    ]
}

In that scenario your bucket remains private, but some trusted process, like your nginx is able to proxy and serve files from it.
Quite dummy, but for some stuff, like allure reports website , etc - acceptable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment