Skip to content

Instantly share code, notes, and snippets.

@adrianbartyczak
Created October 24, 2019 01:54
Show Gist options
  • Star 11 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save adrianbartyczak/1a51c9fa2aae60d860ca0d70bbc686db to your computer and use it in GitHub Desktop.
Save adrianbartyczak/1a51c9fa2aae60d860ca0d70bbc686db to your computer and use it in GitHub Desktop.
A signature creator for AWS signature version 4
#!/usr/bin/env bash
#
# File:
# aws-signature-creator.sh
#
# Description:
# A signature creator for AWS signature version 4
#
# References:
# https://czak.pl/2015/09/15/s3-rest-api-with-curl.html
#
readonly AWS_ACCESS_KEY_ID='<your_access_key_id>'
readonly AWS_SECRET_ACCESS_KEY='<your_secret_access_key>'
readonly AWS_SERVICE='s3'
readonly AWS_REGION='us-east-1'
readonly AWS_S3_BUCKET_NAME='<your_bucket_name>'
readonly AWS_SERVICE_ENDPOINT_URL="\
${AWS_S3_BUCKET_NAME}.${AWS_SERVICE}.amazonaws.com"
# Create an SHA-256 hash in hexadecimal.
# Usage:
# hash_sha256 <string>
function hash_sha256 {
printf "${1}" | openssl dgst -sha256 | sed 's/^.* //'
}
# Create an SHA-256 hmac in hexadecimal.
# Usage:
# hmac_sha256 <key> <data>
function hmac_sha256 {
key="$1"
data="$2"
printf "${data}" | openssl dgst -sha256 -mac HMAC -macopt "${key}" | \
sed 's/^.* //'
}
readonly CURRENT_DATE_DAY="$(date -u '+%Y%m%d')"
readonly CURRENT_DATE_TIME="$(date -u '+%H%M%S')"
readonly CURRENT_DATE_ISO8601="${CURRENT_DATE_DAY}T${CURRENT_DATE_TIME}Z"
readonly HTTP_REQUEST_METHOD='GET'
readonly HTTP_REQUEST_PAYLOAD=''
readonly HTTP_REQUEST_PAYLOAD_HASH="$(printf "${HTTP_REQUEST_PAYLOAD}" | \
openssl dgst -sha256 | sed 's/^.* //')"
readonly HTTP_CANONICAL_REQUEST_URI='/video_clips/0940.m3u8'
readonly HTTP_CANONICAL_REQUEST_QUERY_STRING=''
readonly HTTP_REQUEST_CONTENT_TYPE='application/x-www-form-urlencoded'
readonly HTTP_CANONICAL_REQUEST_HEADERS="\
content-type:${HTTP_REQUEST_CONTENT_TYPE}
host:${AWS_SERVICE_ENDPOINT_URL}
x-amz-content-sha256:${HTTP_REQUEST_PAYLOAD_HASH}
x-amz-date:${CURRENT_DATE_ISO8601}"
# Note: The signed headers must match the canonical request headers.
readonly HTTP_REQUEST_SIGNED_HEADERS="\
content-type;host;x-amz-content-sha256;x-amz-date"
readonly HTTP_CANONICAL_REQUEST="\
${HTTP_REQUEST_METHOD}
${HTTP_CANONICAL_REQUEST_URI}
${HTTP_CANONICAL_REQUEST_QUERY_STRING}
${HTTP_CANONICAL_REQUEST_HEADERS}\n
${HTTP_REQUEST_SIGNED_HEADERS}
${HTTP_REQUEST_PAYLOAD_HASH}"
# Create the signature.
# Usage:
# create_signature
function create_signature {
stringToSign="AWS4-HMAC-SHA256
${CURRENT_DATE_ISO8601}
${CURRENT_DATE_DAY}/${AWS_REGION}/${AWS_SERVICE}/aws4_request
$(hash_sha256 "${HTTP_CANONICAL_REQUEST}")"
dateKey=$(hmac_sha256 key:"AWS4${AWS_SECRET_ACCESS_KEY}" \
"${CURRENT_DATE_DAY}")
regionKey=$(hmac_sha256 hexkey:"${dateKey}" "${AWS_REGION}")
serviceKey=$(hmac_sha256 hexkey:"${regionKey}" "${AWS_SERVICE}")
signingKey=$(hmac_sha256 hexkey:"${serviceKey}" "aws4_request")
printf "${stringToSign}" | openssl dgst -sha256 -mac HMAC -macopt \
hexkey:"${signingKey}" | awk '{print $2}'
}
readonly SIGNATURE="$(create_signature)"
readonly HTTP_REQUEST_AUTHORIZATION_HEADER="\
AWS4-HMAC-SHA256 Credential=${AWS_ACCESS_KEY_ID}/${CURRENT_DATE_DAY}/\
${AWS_REGION}/${AWS_SERVICE}/aws4_request, \
SignedHeaders=${HTTP_REQUEST_SIGNED_HEADERS};x-amz-date, Signature=${SIGNATURE}"
curl -X "${HTTP_REQUEST_METHOD}" -v \
"https://${AWS_SERVICE_ENDPOINT_URL}${HTTP_CANONICAL_REQUEST_URI}" \
-H "Authorization: ${HTTP_REQUEST_AUTHORIZATION_HEADER}" \
-H "content-type: ${HTTP_REQUEST_CONTENT_TYPE}" \
-H "x-amz-content-sha256: ${HTTP_REQUEST_PAYLOAD_HASH}" \
-H "x-amz-date: ${CURRENT_DATE_ISO8601}"
@ManfredServoy
Copy link

Somehow, create_signature returns an empty string for me

@sksube
Copy link

sksube commented Feb 9, 2020

Could you please help how to upload a file to S3 bucket using similar method?

@ManfredServoy
Copy link

I figured out something interesting:
This script works just fine on Linux, however, if you want to run it on Mac OS X/terminal, you would need to change line 83 into
hexkey:"${signingKey}" | awk '{print $0}'

@ambanmba
Copy link

In the first 24 hours of creating a bucket, this script is likely to fail unless you change line 19 to:
${AWS_S3_BUCKET_NAME}.${AWS_SERVICE}-${AWS_REGION}.amazonaws.com"

Refer this issue: https://aws.amazon.com/premiumsupport/knowledge-center/s3-http-307-response/

@ambanmba
Copy link

@ManfredServoy, even more interesting is that it works as is on some Macs with Catalina and not with others... setting it to $0 as you have done seems to make it work on all Macs. Really curious what makes some Macs susceptible.

@dmartinezbello
Copy link

Could you please help how to upload a file to S3 bucket using similar method?

@ambanmba
Copy link

Could you please help how to upload a file to S3 bucket using similar method?

https://gist.github.com/ambanmba/6ec9f2ba1afdc07711b2cc4aa63eabcf

That's the one I use to push stuff to S3. Just keep in mind that the credentials in IAM need to allow WRITE.

@jamini87
Copy link

jamini87 commented Jan 7, 2022

Thanks. It is great.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment