Skip to content

Instantly share code, notes, and snippets.

@imbradbrown
Last active November 30, 2022 06:20
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save imbradbrown/df9e75e1202efacf70f0 to your computer and use it in GitHub Desktop.
Save imbradbrown/df9e75e1202efacf70f0 to your computer and use it in GitHub Desktop.
Uploading to Amazon S3 from curl with Server Side Encrpytion - Customer Provided Key used. Note that this uses the Amazon Access Keys which should be used with care.
#!/bin/bash
## file to upload.
S3_UPLOAD_FILE=some/path/file.txt
## Specify the bucket name here. This can be found in the S3 console
S3_BUCKET=bucket name here
## The desired path relative to the root of the bucket. All folders must be forward slash '/' separated
S3_DESTINATION_FILE=folder/folder2/file.txt
## The Admin Access Key for the account found on https://console.aws.amazon.com/iam/home#security_credential
S3_KEY=Amazon Admin Access Key
## The Admin Access Secret Key for the account found on https://console.aws.amazon.com/iam/home#security_credential
S3_SECRET= Secret Key Here
## the content type to post.
S3_CONTENT_TYPE="application/octet-stream"
## The date formatted in GMT
S3_DATE="$(LC_ALL=C date -u +"%a, %d %b %Y %X %z")"
## calculate the MD5 of the file to upload and BASE64 encode it
S3_MD5SUM="$(openssl md5 -binary < ${S3_UPLOAD_FILE} | base64)"
S3_SSEC_ALGORITHM=AES256
## The Server Side Encryption - Customer Provided Key to use. This must be 32 bytes in length
S3_SSEC_KEY=00000000000000000000000000012345
## Base64 encode the SSE-C key. This is used as the x-amz-server-side-encryption-customer-key
S3_ENCRYPTION_KEY="$(echo -n ${S3_SSEC_KEY} | openssl enc -base64)"
## MD5 hash the SSE-C key. Base64 the result. This is used as the x-amz-server-side-encryption-customer-key-MD5
S3_ENCRYPTION_MD5="$(echo -n ${S3_SSEC_KEY} | openssl dgst -md5 -binary | openssl enc -base64)"
## S3 validates the request by checking the data passed in is in a specific order. See http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-authentication-HTTPPOST.html
S3_SIGNATURE="$(printf "PUT\n$S3_MD5SUM\n$S3_CONTENT_TYPE\n$S3_DATE\nx-amz-server-side-encryption-customer-algorithm:$S3_SSEC_ALGORITHM\nx-amz-server-side-encryption-customer-key:$S3_ENCRYPTION_KEY\nx-amz-server-side-encryption-customer-key-md5:$S3_ENCRYPTION_MD5\n/$S3_BUCKET/$S3_DESTINATION_FILE" | openssl sha1 -binary -hmac "$S3_SECRET" | base64)"
## Send the actual curl
curl -v -T ${S3_UPLOAD_FILE} https://$S3_BUCKET.s3.amazonaws.com/${S3_DESTINATION_FILE} -H "Date: ${S3_DATE}" -H "Authorization: AWS ${S3_KEY}:${S3_SIGNATURE}" -H "Content-Type: ${S3_CONTENT_TYPE}" -H "Content-MD5: ${S3_MD5SUM}" -H "x-amz-server-side-encryption-customer-algorithm:${S3_SSEC_ALGORITHM}" -H "x-amz-server-side-encryption-customer-key:${S3_ENCRYPTION_KEY}" -H "x-amz-server-side-encryption-customer-key-MD5:${S3_ENCRYPTION_MD5}"
@antoweb
Copy link

antoweb commented Feb 4, 2022

Hello, thanks for this script..works good. But how to download the file? I have tryed curl without -T but receive error

@antoweb
Copy link

antoweb commented Feb 5, 2022

UPDATE:
According to this documentation: https://docs.aws.amazon.com/it_it/AmazonS3/latest/userguide/specifying-s3-c-encryption.html

For get are only necessary 3 headers, so the curl for download become:
curl -v https://BUCKETNAME.s3.amazonaws.com/filename -H "x-amz-server-side-encryption-customer-algorithm:AES256" -H "x-amz-server-side-encryption-customer-key:KEYGENERATEDBASE64" -H "x-amz-server-side-encryption-customer-key-MD5:HASHMD5OFTHEKEY" -o filename

@antoweb
Copy link

antoweb commented Nov 28, 2022

Hello my comment is for get (download) for put +upload here is the documentation
https://docs.aws.amazon.com/it_it/AmazonS3/latest/userguide/ServerSideEncryptionCustomerKeys.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment