Skip to content

Instantly share code, notes, and snippets.

@davidejones
Last active July 17, 2024 17:34
Show Gist options
  • Save davidejones/d05f51df75e659111227 to your computer and use it in GitHub Desktop.
Save davidejones/d05f51df75e659111227 to your computer and use it in GitHub Desktop.
curl get file from private s3 with iam role
#!/bin/bash
instance_profile=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/`
aws_access_key_id=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep AccessKeyId | cut -d':' -f2 | sed 's/[^0-9A-Z]*//g'`
aws_secret_access_key=`curl http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | grep SecretAccessKey | cut -d':' -f2 | sed 's/[^0-9A-Za-z/+=]*//g'`
token=`curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/${instance_profile} | sed -n '/Token/{p;}' | cut -f4 -d'"'`
file="somefile.deb"
bucket="some-bucket-of-mine"
date="`date +'%a, %d %b %Y %H:%M:%S %z'`"
resource="/${bucket}/${file}"
signature_string="GET\n\n\n${date}\nx-amz-security-token:${token}\n/${resource}"
signature=`/bin/echo -en "${signature_string}" | openssl sha1 -hmac ${aws_secret_access_key} -binary | base64`
authorization="AWS ${aws_access_key_id}:${signature}"
curl -s -H "Date: ${date}" -H "X-AMZ-Security-Token: ${token}" -H "Authorization: ${authorization}" "https://s3-us-west-1.amazonaws.com/${resource}" -o "output.deb"
{
"Version": "2012-10-17",
"Id": "Policy1441048292119",
"Statement": [
{
"Sid": "Stmt1441048289544",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::123456789012:role/your_role_name",
]
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::some-bucket-of-mine/*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::some-bucket-of-mine",
"arn:aws:s3:::some-bucket-of-mine/*"
]
},
]
}
@kwent
Copy link

kwent commented Feb 24, 2018

How to perform the same in an ECS Container with a role attached ? Thanks

@afunsten
Copy link

afunsten commented Aug 9, 2018

@Cricen
Copy link

Cricen commented Dec 8, 2019

Thank you for this! Have you happen to updated this to work with V4?

@kocoten1992
Copy link

kocoten1992 commented Dec 24, 2021

Hello, this work 99% of the time, however, occationally I got this:

aws authentication requires a valid date or x-amz-date header

No idea where that coming from, anyone else got that ?

P/s: OMG, one machine doesn't have English as main language LC_ALL=en_EN.utf8 date +'%a, %d %b %Y %H:%M:%S %z', Merry Christmas everyone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment