Skip to content

Instantly share code, notes, and snippets.

@treyperrone

treyperrone/20240217_aws_orgs_shared_cmk_test.md

Last active February 21, 2024 19:17
Show Gist options
  • Save treyperrone/7417a34c4d8e85d2f8724aef4a02de98 to your computer and use it in GitHub Desktop.
Save treyperrone/7417a34c4d8e85d2f8724aef4a02de98 to your computer and use it in GitHub Desktop.
Download ZIP
20240217_aws_orgs_shared_cmk_test
Raw
20240217_aws_orgs_shared_cmk_test.md

2024-02-21

  • the lab account policy was not in the prior comments, its the plain jane vanilla default one generated at KMS key creation:
{
    "Id": "key-consolepolicy-3",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::590183926393:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        }
    ]
}
  • updated the image factory acct MRK KMS key policy to single SID for simplicity:
{
    "Version": "2012-10-17",
    "Id": "key-consolepolicy-3",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::058264393352:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow use of the KMS key for organization",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:Encrypt",
                "kms:ReEncrypt*",
                "kms:GetKeyPolicy",
                "kms:CreateGrant",
                "kms:ListGrants"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "o-vxxxxxx2l6n"
                }
            }
        }
    ]
}

2024-02-17

  • setup brand new aws acct for orgs/management

    • create child acct image-factory xxxxxxxx3352

    • create child acct for lab20240217 xxxxxx6393

      • enable default EBS encryption manually at acct level

    • acct image-factory

      • create shared MRK KMS key (but only in us-east-1)
        • left policy default so only usable within ami acct to start
      • create simple al2023 ami using the above MRK key
        • share that ami via console to the lab20240217 account6393

    • lab20240217 account6393

      • able to see the new shared ami
      • attempt launch #1, FAILS and terminates
        • logs unclear in cloudtrail but im dumb here need to learn
      • modify the MRK key policy ONLY on image-factory account3352 and add the org policy statement from https://repost.aws/knowledge-center/kms-key-organization-account to the default one
        • NOTE - did not run a manual aws cli grant-key command to child account at this time
{
    "Id": "key-consolepolicy-3",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::xxxxxxxx3352:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {  
            "Sid": "Allow use of the KMS key for organization",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:Encrypt",
                "kms:ReEncrypt*",
                "kms:GetKeyPolicy"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "o-vxxxx2l6n"
                }
            }
        }
    ]
}

  • attempt launch #2 via console, successful!

  • attempt launch via CLI run-instance for faster changes...

    • attempt launch #1 just call instance - successful

  • attempt launch #2 call instance and set encryption OFF for EBS - (our default ebs encryption should also prevent that anyways but a sanity check) - FAILS AS EXPECTED

An error occurred (InvalidBlockDeviceMapping) when calling the RunInstances operation: Cannot create unencrypted device /dev/xvda from encrypted snapshot

  • create a local CMK to test launch (instead of default AWS EBS key)

  • attempt launch #3 via cli and call new local CMK key vs default ebs - successful

aws ec2 run-instances \
  --image-id ami-0b2e9fef7a3024ce6 \
  --count 1 \
  --instance-type t4g.nano \
  --security-group-ids sg-0afe189dbba7fc8df \
  --subnet-id subnet-06bda357a91d1e5c1 \
  --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=cli-test-shared-ami-$(date '+%Y-%m-%d_%H-%M')}]" \
  --ebs-optimized \
  --credit-specification CpuCredits=standard \
  --no-associate-public-ip-address \
  --iam-instance-profile Name=_ec2-ssm \
  --block-device-mappings "DeviceName=/dev/xvda,Ebs={Encrypted=true,KmsKeyId=arn:aws:kms:us-east-1:xxxxxxxx6393:key/5c48313e-7831-436a-bb5e-d96b26d6d58b}"
  • attempt launch #4 via cli and call MRK key vs default ebs - FAILS WHICH WE WANT; we do not want app teams re-using the KMS key from image-factory!
aws ec2 run-instances \
  --image-id ami-0b2e9fef7a3024ce6 \
  --count 1 \
  --instance-type t4g.nano \
  --security-group-ids sg-0afe189dbba7fc8df \
  --subnet-id subnet-06bda357a91d1e5c1 \
  --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=cli-test-shared-ami-$(date '+%Y-%m-%d_%H-%M')}]" \
  --ebs-optimized \
  --credit-specification CpuCredits=standard \
  --no-associate-public-ip-address \
  --iam-instance-profile Name=_ec2-ssm \
  --block-device-mappings "DeviceName=/dev/xvda,Ebs={Encrypted=true,KmsKeyId=arn:aws:kms:us-east-1:xxxxxxxx3352:key/mrk-7700fxxxxxxxxxxxx900d857022b3878}"

ASG testing

  • test an ASG #1 - this fails as expected to what we have seen in past per the prior org sharing blog post there is a statement at bottom:

Note: The aws:PrincipalOrgID global condition context key can't be used to restrict access to an AWS service principal. AWS services that invoke an API call are made from an internal AWS account that is not part of the AWS Organization.

aws kms create-grant \
  --key-id arn:aws:kms:us-east-1:xxxxxxxx3352:key/mrk-7700fxxxxxxxxxxxx900d857022b3878 \
  --grantee-principal arn:aws:iam::$(aws sts get-caller-identity \
  --query Account --output text):role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling \
  --operations "Encrypt" "Decrypt" "ReEncryptFrom" "ReEncryptTo" "GenerateDataKey" "GenerateDataKeyWithoutPlaintext" "DescribeKey" "CreateGrant"
  • This action cannot be performed without an update to our MRK key policy, error will be like such (using bash sub to get local caller account as we have to essentially do this across hundreds of accts and multiple regions??):
[cloudshell-user@ip-10-130-93-105 ~]$ aws kms create-grant   --key-id arn:aws:kms:us-east-1:xxxxxxxx3352:key/mrk-7700fxxxxxxxxxxxx900d857022b3878   --grantee-principal arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling   --operations "Encrypt" "Decrypt" "ReEncryptFrom" "ReEncryptTo" "GenerateDataKey" "GenerateDataKeyWithoutPlaintext" "DescribeKey" "CreateGrant"

An error occurred (AccessDeniedException) when calling the CreateGrant operation: User: arn:aws:sts::xxxxxxxx6393:assumed-role/aw24-OrganizationAccountAccessRole/tp24 is not authorized to perform: kms:CreateGrant on this resource because the resource does not exist in this Region, no resource-based policies allow access, or a resource-based policy explicitly denies access
  • Now we will update the MRK policy in image-factory account3352 and create grant again NOTE: only doing createGrant now, we may want to add other grant perms such as listGrant (we could make it list and apply if not found via some script/lambda?)
{
    "Id": "key-consolepolicy-3",
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM User Permissions",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::xxxxxxxx3352:root"
            },
            "Action": "kms:*",
            "Resource": "*"
        },
        {  
            "Sid": "Allow use of the KMS key for organization",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "kms:Decrypt",
                "kms:DescribeKey",
                "kms:Encrypt",
                "kms:ReEncrypt*",
                "kms:GetKeyPolicy"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "o-vxxxx2l6n"
                }
            }
        },
        {
            "Sid": "Allow attachment of persistent resources for organization",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "kms:CreateGrant"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "o-vxxxx2l6n"
                }
            }
         }
    ]
}
  • now lets grant again in the child acct:
[cloudshell-user@ip-10-130-93-105 ~]$ aws kms create-grant \
>   --key-id arn:aws:kms:us-east-1:xxxxxxxx3352:key/mrk-7700fxxxxxxxxxxxx900d857022b3878 \
>   --grantee-principal arn:aws:iam::$(aws sts get-caller-identity \
>   --query Account --output text):role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling \
>   --operations "Encrypt" "Decrypt" "ReEncryptFrom" "ReEncryptTo" "GenerateDataKey" "GenerateDataKeyWithoutPlaintext" "DescribeKey" "CreateGrant"
{
    "GrantToken": "AQpAZTZmYjE0NTAzZTA4MzE1ZmY0NGJjYWE2MTM0M2RjY2ZhNjAyYjc1OGRkZGQ0MjhhMTdmZWQyMTdkNjBiODQyNyKWAgEBAgB45vsUUD4IMV_0S8qmE0Pcz6YCt1jd3UKKF_7SF9YLhCcAAADtMIHqBgkqhkiG9w0BBwaggdwwgdkCAQAwgdMGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMrYcjhkq1EXq0sJwdAgEQgIGlR_cPtWqNdUE3btd1Ts5615rBgYknL55cZ7S-aAmuY-JGt-_28TIRQ1r8eyrGuGHE4meo_Dj7aS2U-fSHEqLlb65b7D45-Y5VaCmOKP7hHlkchMwG2ymYATS82lb_cjctwAMLMYN5n1bkIiMuYrNaogz9vnh4uBqnj7B_4S_NGGQLOAsL_am2a5EmAjJvRSI6qn2t-Vw7Jo0HXN89h_00lGBL66MAKiCldqWbInRJ3VpAw0Q1Rxkc_Hwc8tPCujz1Ss4jbBctkg",
    "GrantId": "a576a59b227449dd5a40c3443547191cfc7c1cf2d3c2ba3cf54ace236c172d92"
}
  • Now autoscaling group in the child account was able to launch instance

thoughts for more testing and application?

  • add more grant perms for list if we want to make a smart scripts in children accts to check and apply (in theory applying once the config should stick unless someone does a retire or revoke grant right?)

  • we probably have to loop all regions on the grant unless there's a method to wildcard the region in the command (it seems like based on quick read of docs it wont allow that?)

  • create alias for keys - may make things easier?

  • are there any other SCPs or other configs unique to our env that could block things?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment