Steps to setup a ROSA secure config demo

rosa-demo-secure-setup.md

Secure Demo Setup

This assumes you have a ROSA cluster and you have added an OIDC provider for the cluster. We will walk through the steps to auth ROSA pods to access specific S3 buckets. Instead of creating and distributing your AWS credentials to the containers or using the Amazon EC2 instance’s role, we will create a IAM Policy and Role (with federated identity) and associate it to some of the built-in ROSA service accounts.

Setup temp AWS creds

aws configure --profile rosa-demo

....answer questions....

export AWS_PROFILE=rosa-demo

Create a custom policy for S3 Access

export S3_CUSTOM_BUCKET=sagemaker-us-east-1-546584748567

export S3_FULL_POLICY_ARN=arn:aws:iam::aws:policy/AmazonS3FullAccess

cat <<EOF > /tmp/s3-policy.json
{
 "Version": "2012-10-17",
 "Statement": [
     {
         "Sid": "Statement",
         "Effect": "Allow",
         "Action": [
             "s3:ListBucket",
             "s3:GetObject"
         ],
         "Resource": [
             "arn:aws:s3:::$S3_CUSTOM_BUCKET/*",
             "arn:aws:s3:::$S3_CUSTOM_BUCKET"
         ]
     }
 ]
}
EOF

S3_CUSTOM_POLICY_ARN=$(aws iam create-policy --policy-name ROSADemoS3Access \
   --policy-document file:///tmp/s3-policy.json \
   --query 'Policy.Arn' --output text)
 echo $S3_CUSTOM_POLICY_ARN

Create a new role and ROSA service account

(ref: https://docs.aws.amazon.com/eks/latest/userguide/associate-service-account-role.html)

Get your AWS account info:

export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

Get your cluster OIDC info:

export OIDC_PROVIDER=$(oc get authentication.config.openshift.io cluster -o json | jq -r .spec.serviceAccountIssuer | sed -e "s/^https:\/\///")

The trust policy defines the rules around who can assume this new role:

cat <<EOF > /tmp/trustpolicy.json
{
  "Version": "2012-10-17",
  "Statement": [
 {
   "Effect": "Allow",
   "Principal": {
     "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
   },
   "Action": "sts:AssumeRoleWithWebIdentity",
   "Condition": {
     "StringEquals": {
       "${OIDC_PROVIDER}:sub": [
         "system:serviceaccount:*:builder"
       ]
     }
   }
 }
  ]
}
EOF

Create the new role:

S3_ROLE=$(aws iam create-role \
   --role-name "ROSADemoS3AccessRole" \
   --assume-role-policy-document file:///tmp/trustpolicy.json \
   --query "Role.Arn" --output text)

echo $S3_ROLE

Attach the role:

aws iam attach-role-policy \
   --role-name "ROSADemoS3AccessRole" \
   --policy-arn $S3_CUSTOM_POLICY_ARN

Annotate the local namespace's service account(s) that will run pods needing S3 access:

oc annotate serviceaccount builder eks.amazonaws.com/role-arn=arn:aws:iam::$AWS_ACCOUNT_ID:role/ROSADemoS3AccessRole

oc annotate serviceaccount default eks.amazonaws.com/role-arn=arn:aws:iam::$AWS_ACCOUNT_ID:role/ROSADemoS3AccessRole

Alternatively, create a new service account, annotate that, and reference it in your deployments:

cat <<EOF > /tmp/new-service-account.json
apiVersion: v1
kind: ServiceAccount
metadata:
  name: new-s3-service-account
  namespace: default
EOF

oc apply -f /tmp/new-service-account.json

oc annotate -n default serviceaccount new-s3-service-account eks.amazonaws.com/role-arn=arn:aws:iam::$AWS_ACCOUNT_ID:role/ROSADemoS3AccessRole