Last active
November 30, 2022 06:20
-
-
Save imbradbrown/df9e75e1202efacf70f0 to your computer and use it in GitHub Desktop.
Uploading to Amazon S3 from curl with Server Side Encrpytion - Customer Provided Key used. Note that this uses the Amazon Access Keys which should be used with care.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
## file to upload. | |
S3_UPLOAD_FILE=some/path/file.txt | |
## Specify the bucket name here. This can be found in the S3 console | |
S3_BUCKET=bucket name here | |
## The desired path relative to the root of the bucket. All folders must be forward slash '/' separated | |
S3_DESTINATION_FILE=folder/folder2/file.txt | |
## The Admin Access Key for the account found on https://console.aws.amazon.com/iam/home#security_credential | |
S3_KEY=Amazon Admin Access Key | |
## The Admin Access Secret Key for the account found on https://console.aws.amazon.com/iam/home#security_credential | |
S3_SECRET= Secret Key Here | |
## the content type to post. | |
S3_CONTENT_TYPE="application/octet-stream" | |
## The date formatted in GMT | |
S3_DATE="$(LC_ALL=C date -u +"%a, %d %b %Y %X %z")" | |
## calculate the MD5 of the file to upload and BASE64 encode it | |
S3_MD5SUM="$(openssl md5 -binary < ${S3_UPLOAD_FILE} | base64)" | |
S3_SSEC_ALGORITHM=AES256 | |
## The Server Side Encryption - Customer Provided Key to use. This must be 32 bytes in length | |
S3_SSEC_KEY=00000000000000000000000000012345 | |
## Base64 encode the SSE-C key. This is used as the x-amz-server-side-encryption-customer-key | |
S3_ENCRYPTION_KEY="$(echo -n ${S3_SSEC_KEY} | openssl enc -base64)" | |
## MD5 hash the SSE-C key. Base64 the result. This is used as the x-amz-server-side-encryption-customer-key-MD5 | |
S3_ENCRYPTION_MD5="$(echo -n ${S3_SSEC_KEY} | openssl dgst -md5 -binary | openssl enc -base64)" | |
## S3 validates the request by checking the data passed in is in a specific order. See http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-authentication-HTTPPOST.html | |
S3_SIGNATURE="$(printf "PUT\n$S3_MD5SUM\n$S3_CONTENT_TYPE\n$S3_DATE\nx-amz-server-side-encryption-customer-algorithm:$S3_SSEC_ALGORITHM\nx-amz-server-side-encryption-customer-key:$S3_ENCRYPTION_KEY\nx-amz-server-side-encryption-customer-key-md5:$S3_ENCRYPTION_MD5\n/$S3_BUCKET/$S3_DESTINATION_FILE" | openssl sha1 -binary -hmac "$S3_SECRET" | base64)" | |
## Send the actual curl | |
curl -v -T ${S3_UPLOAD_FILE} https://$S3_BUCKET.s3.amazonaws.com/${S3_DESTINATION_FILE} -H "Date: ${S3_DATE}" -H "Authorization: AWS ${S3_KEY}:${S3_SIGNATURE}" -H "Content-Type: ${S3_CONTENT_TYPE}" -H "Content-MD5: ${S3_MD5SUM}" -H "x-amz-server-side-encryption-customer-algorithm:${S3_SSEC_ALGORITHM}" -H "x-amz-server-side-encryption-customer-key:${S3_ENCRYPTION_KEY}" -H "x-amz-server-side-encryption-customer-key-MD5:${S3_ENCRYPTION_MD5}" |
UPDATE:
According to this documentation: https://docs.aws.amazon.com/it_it/AmazonS3/latest/userguide/specifying-s3-c-encryption.html
For get are only necessary 3 headers, so the curl for download become:
curl -v https://BUCKETNAME.s3.amazonaws.com/filename -H "x-amz-server-side-encryption-customer-algorithm:AES256" -H "x-amz-server-side-encryption-customer-key:KEYGENERATEDBASE64" -H "x-amz-server-side-encryption-customer-key-MD5:HASHMD5OFTHEKEY" -o filename
Hello my comment is for get (download) for put +upload here is the documentation
https://docs.aws.amazon.com/it_it/AmazonS3/latest/userguide/ServerSideEncryptionCustomerKeys.html
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello, thanks for this script..works good. But how to download the file? I have tryed curl without -T but receive error