Using (and debugging) Pre-signed S3 URLS.md

Using (and debugging) Pre-signed S3 URLS

Architecture - A Brief Overview

A Pre-signed URL is used to allow untrusted users to temporary access to private S3 resources. (Example: upload an image to a private S3 bucket).

The URL is a string consists of:

1. AWS URL of the object to be accessed, and
2. A signed base64 string describing the HTTP request:

• HTTP verb (i.e. PUT)
• S3 operation type (i.e. putObject)
• S3 object key
• Your Access Key ID

This URL is then signed by your AWS Secret Access Private Key (See code example below), yeilding the final pre-signed URL. The one-time signed URL is then ready for use.

How it Works

Understanding this section is important for debugging.

Step 1: Signature match

When the URL is accessed, S3 looks at your HTTP request (i.e. PUT mybucket.s3.amazonaws.com/nazgul.png), extracts the Access Key ID and calculates what the signature should look like. It then compares this with the actual signature contained in your request. If the signatures match, execution continues.

If the signatures don't exactly match, (Say, the URL was signed for PUT but you tried POST; you're tring to access the wrong object key, etc), the action will fail with SignatureDoesNotMatch.

❗️ S3 cannot tell you what part of the request is being used incorrectly, all it knows is that the signature does not match. Carefully review every part of your request an dmake that everythign matches.

Step 2: IAM Permission Check

If the signature passes, S3 will temporarily sign in as the IAM user who signed the URL and will execute with their permissions. If the signing IAM user does not have permission to perform the requested action, the request will fail with AccessDenied. Permission can be granted by adding a rule to the S3 bucket's Permissions > Bucket Policy , or by adding an IAM Security Policy granting access to the bucket to the IAM user.

If the IAM user permission check passes, the action will be performed. The signed URL will be immediately expired to prevent reuse.

The IAM user that generates the URL must have the S3 permissions to access the bucket (The S3 permissions getObject, putObject, listBucket, etc.)

If the resource doesn't exist (i.e., GET on a nonexistant file), S3 will also fail with AccessDenied.

Code Example - Generating Signed S3 URLs with AWS Lambda

Setup S3

1. Create the S3 bucket.
2. Add the CORS policy from the Appendix in Permissions > CORS Configuration. This will allow JavaScript in the browser to talk to S3.

Setup Lambda

4. Create a new Node.js Lambda function
5. Paste in the code from the Appendix, customizing it with your bucket name and anything else you may want to add.
6. Expose it to the public web using API Gateway (In the Lambda console of your function, click + Add Trigger and select API Gateway). Copy the gateway URL
7. Locate the Execution Role. Click to visit it in the IAM console.
8. In the IAM console of the execution role, create a new Policy by clicking Attach Policy > Create Policy. In the wizard, grant access to the S3 bucket we've just created.

✅ Lambda is ready to start signing URLs!

Try:

curl <gateway-url>?filekey=mysecretfile.jpeg&contentType=image%2Fjpeg
# => Should return a signed URL!

Appendix 1: Lambda function

var AWS = require('aws-sdk');
var s3 = new AWS.S3({
  signatureVersion: 'v4',
});

const HTTP_STATUS = {
 OK: 200
}

exports.handler = (event, context) => {
    try {
     if(!event.queryStringParameters) throw `Missing required parameters "filekey" and "contentType"`;
     if(!event.queryStringParameters.contentType) throw `Missing parameter "contentType" Got ${event.queryStringParameters.contentType}`;
      if(!event.queryStringParameters.filekey) throw `Missing parameter "filekey" Got ${event.queryStringParameters.filekey}`;

     const url = s3.getSignedUrl('putObject', {
         Bucket: 'your-bucket',
         Key: event.queryStringParameters.filekey,
         ContentType: event.queryStringParameters.contentType,
         ACL: 'public-read',
         Expires: 600, // URL expiry time in seconds
       });

     var responseCode = 200;

     if (event.queryStringParameters.httpStatus) {
         console.log("Received http status override: " + event.queryStringParameters.httpStatus);
         responseCode = event.queryStringParameters.httpStatus || HTTP_STATUS.OK;
     }

     var responseBody = {
         signed_url: url,
     };

     var response = {
         statusCode: responseCode,
         headers: {
             "Access-Control-Allow-Origin": "*"
         },
         body: JSON.stringify(responseBody)
     };

    return response;
  } catch(error) {
    return {
       statusCode: 400,
       headers: {
           "Access-Control-Allow-Origin": "*"
       },
       body: JSON.stringify(error)
     };
  }
};

Appendix 2; Bucket Policy

This policy make the bucket contents public

{
    "Version": "2012-10-17",
    "Id": "Policy1564527997207",
    "Statement": [
        {
            "Sid": "Stmt1564527321604",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::your-bucket/*"
        }
    ]
}

Appendix 3: CORS Configuration

Allows

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>http://127.0.0.1:8080</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Further Reading

• S3 signed URL reference (it's short ;) )