AWS CMS policy mapping using jq and a netcat listener

README.md

Using aws sdk's CSM to obtain an iam policy statement

Based on work in https://github.com/iann0036/iamlive Using the data in map.json from https://github.com/iann0036/sdk-iam-map

I wanted to be able to use this same type of functionality without requiring yet another tool. As I already have jq included in all of my tooling I chose to use it for simplicity -- although the language itself is not always so simple to grok.

This script will allow you to execute your commands as arguments to the script. As it is setting environment variables we are able to gain insights into any process that is built using the aws sdk's, which include the cli, cli2, and terraform.

An example of an unauthenticated call to get-caller-identity will provide.

$ map-iam.sh aws sts get-caller-identity

An error occurred (ExpiredToken) when calling the GetCallerIdentity operation: The security token included in the request is expired
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sts:GetCallerIdentity"
      ],
      "Resource": "*"
    }
  ]
}

Similarly, an authenticated call will then show:

$ PAGER=cat map-iam.sh aws sts get-caller-identity
{
    "UserId": "AIDASAMPLEUSERID:arledesma",
    "Account": "123456789012",
    "Arn": "arn:aws:sts::123456789012:assumed-role/AWSReservedSSO_AdministratorAccess_b69091cd2e3a57cc/arledesma"
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sts:GetCallerIdentity"
      ],
      "Resource": "*"
    }
  ]
}

Describing the regions in ec2, filtered to endpoints containing the string us:

$ PAGER=cat map-iam.sh aws ec2 describe-regions --filters "Name=endpoint,Values=*us*"
{
    "Regions": [
        {
            "Endpoint": "ec2.us-east-1.amazonaws.com",
            "RegionName": "us-east-1",
            "OptInStatus": "opt-in-not-required"
        },
        {
            "Endpoint": "ec2.us-east-2.amazonaws.com",
            "RegionName": "us-east-2",
            "OptInStatus": "opt-in-not-required"
        },
        {
            "Endpoint": "ec2.us-west-1.amazonaws.com",
            "RegionName": "us-west-1",
            "OptInStatus": "opt-in-not-required"
        },
        {
            "Endpoint": "ec2.us-west-2.amazonaws.com",
            "RegionName": "us-west-2",
            "OptInStatus": "opt-in-not-required"
        }
    ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeRegions"
      ],
      "Resource": "*"
    }
  ]
}

Organization cloudtrail with s3 bucket and kms key in separate account

$ PAGER=cat map-iam.sh aws cloudtrail describe-trails --include-shadow-trails
{
    "trailList": [
        {
            "Name": "company-organization-trail",
            "S3BucketName": "cloudtrail-123456789013-company-organization-trail-00001",
            "IncludeGlobalServiceEvents": true,
            "IsMultiRegionTrail": true,
            "HomeRegion": "us-west-2",
            "TrailARN": "arn:aws:cloudtrail:us-west-2:123456789012:trail/company-organization-trail",
            "LogFileValidationEnabled": true,
            "KmsKeyId": "arn:aws:kms:us-west-2:123456789013:key/00000000-dead-beef-cafe-000000000001",
            "HasCustomEventSelectors": true,
            "HasInsightSelectors": false,
            "IsOrganizationTrail": true
        }
    ]
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudtrail:DescribeTrails"
      ],
      "Resource": "*"
    }
  ]
}

Some will fail as the iam map is not yet complete. Failures may show up as either a service name value of null in the action or possibly an incorrect service name. An example of a failure can be seen with the stepfunctions service:

$ PAGER=cat map-iam.sh aws stepfunctions list-activities
{
    "activities": []
}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "null.listactivities"
      ],
      "Resource": "*"
    }
  ]
}

The map.json is only downloaded once with this script. As it is updated it may be a good idea to simply remove the file iann0036-sdk-iam-map.json from your TMPDIR (probably /tmp) and let the script download a new one. i.e. rm -f /tmp/iann0036-sdk-iam-map.json

map-iam.sh

#!/bin/bash
# https://github.com/arledesma

readonly IAM_MAP_URL="https://raw.githubusercontent.com/iann0036/sdk-iam-map/main/map.json";
readonly IAM_MAP_FILE="${TMPDIR:-/tmp}/iann0036-sdk-iam-map.json";

export AWS_CSM_ENABLED=true;
export AWS_CSM_HOST=127.0.0.1;
export AWS_CSM_PORT=31000;

function ensure_iam_map_exists {
  local -r map_url="$1";
  local -r map_file="$2";
  # if we do not have a file named map.json then get it from ian0036/sdk-iam-map
  if [ ! -f "$map_file" ];
  then
    # just being safe - remove anything that is named map.json
    /bin/rm -rf "$map_file" &>/dev/null || true;
    # use iann0036's iam service map that is extracted from the js sdk
    wget -O "$map_file" "$map_url";
  fi
}

function start_listener {
  local -r csm_output="$1";
  # there are too many versions of nc to ensure a proper max timeout so we
  # background the netcat listener with timeout using a 30 second limit
  timeout 30 sh -c "nc -l -u -p 31000 | tee $csm_output" &>/dev/null &
}

function execute_command {
  # make your aws calls.  the environment variables will be picked up by the sdk as well so you are not limited to the cli
  # terragrunt plan
  # terraform plan
  # aws sts get-caller-identity;
  (
    eval "$@";
  )

  sleep 1;
}

function kill_listener {
  # kill the nc listener
  # currently we only have only 1 backgrounded process so this logic will need to be expanded if it gets more complicated
  kill %1;
}

function parse_csm {
  local -r csm_output="$1";
  local -r map_file="$2";

  # slurp the csm output - [ndjson](http://ndjson.org/) format
  # assign iann0036/sdk-iam-map to iam_map
  # attempt to find the $Service.$Api key in sdk_method_iam_mappings
  # fallback to finding the $Service key in sdk_service_mappings 
  # using the same logic as 
  jq --slurp --argfile iam_map "$map_file" <"$csm_output" '
    [
      .[] |
        $iam_map["sdk_method_iam_mappings"]["\(.Service).\(.Api)"] // [{ "action": "\($iam_map["sdk_service_mappings"][.Service]).\(.Api | ascii_downcase)" }]
    ] |
    flatten |
    unique_by(.action) |
    {
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Action": [.[].action],
        "Resource": "*"
      }]
    }';
}

function main {
  local -r iam_map_url="$IAM_MAP_URL";
  local -r iam_map_file="$IAM_MAP_FILE";
  local -r tmpdir="${TMPDIR:-/tmp}";
  local -r csm_tmp="$(mktemp --tmpdir="$tmpdir")";
  
  # shellcheck disable=SC2064
  trap "/bin/rm -f $csm_tmp" exit;

  ensure_iam_map_exists "$iam_map_url" "$iam_map_file";
  start_listener "$csm_tmp";
  execute_command "$@";
  kill_listener;
  parse_csm "$csm_tmp" "$iam_map_file";
}

main "$@";