AWS Incident Response Playbook (Jupyter Notebook)

incident_response_helpers.py

import boto3
import time
from datetime import datetime, timedelta

def execute_log_query(log_group, query, days_to_search):
    start_time = int((datetime.today() - timedelta(days=days_to_search)).timestamp())
    end_time=int(datetime.now().timestamp())
    client = boto3.client('logs')
    start_query_response = client.start_query(logGroupName=log_group,startTime=start_time,endTime=end_time,queryString=query,)
    query_id = start_query_response['queryId']
    print ('Running...')
    while True:
        response = client.get_query_results(queryId=query_id)
        if response['status'] != 'Running': break    
        time.sleep(3)
    print (response['status'])
    return response

def convert_dictionary_to_object(d):
    o = {}
    for f in d:
        o[f['field']] = f['value']
    return o

incident_response_playbook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Incident Response Playbook with Jupyter - AWS IAM \n",
    "\n",
    "## Authors\n",
    "\n",
    "- Byron Pogson, Solutions Architect, AWS\n",
    "- Ben Potter, Security Lead, Well-Architected, AWS\n",
    "\n",
    "## Contents\n",
    "\n",
    "1. [Getting Started](#getting_started)\n",
    "2. [Access Key Investigation](#access_key)\n",
    "3. [User Investigation](#user)\n",
    "4. [Role Investigation](#role)\n",
    "5. [Containment](#containment)\n",
    "6. [Interesting API Requests](#other_investigations)\n",
    "\n",
    "# 1. Getting Started <a name=\"getting_Started\"></a>\n",
    "\n",
    "## 1.1 Understanding Jupyter Notebooks\n",
    "\n",
    "Juypyter notebooks are a tool that allow us to combine instructions, easily editable incident response process, and the code we need to investigate a potential incident.\n",
    "\n",
    "A notebook is made up of a series of \"cells\". These cells allow us to enter discrete pieces of content which can be executed. Each cell has a type of content and can be executed, for example markdown or code. Read the instructions for each cell, modify the variables or even the code and click Run.\n",
    "\n",
    "## 1.2 Prerequisites\n",
    "\n",
    "Python 3  \n",
    "Python modules required:\n",
    "[Jupyter](https://jupyter.org/install) for the runbook itself, [Boto3](https://boto3.amazonaws.com) AWS SDK for Python, [Pandas](https://pandas.pydata.org/) for output.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import required Python modules\n",
    "import boto3\n",
    "import pandas as pd\n",
    "import json\n",
    "from datetime import datetime, timedelta\n",
    "import time\n",
    "import incident_response_helpers as helpers\n",
    "import platform\n",
    "\n",
    "# Set Pandas column width\n",
    "pd.set_option('max_colwidth', 800)\n",
    "pd.set_option('max_rows', 500)\n",
    "\n",
    "#Prints Python version, 3.7.x recommended    \n",
    "print ('Python:  ' + platform.python_version())\n",
    "\n",
    "# Prints Boto 3 version\n",
    "print ('Boto 3:  ' + boto3.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Find your CloudTrail Log Group Name\n",
    "\n",
    "[Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html) can be used to monitor, store, and access your log files from many different sources including AWS CloudTrail. The following API call lists all your log groups so you can find yours, then enter the name you used for your log group in the next step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prints all CloudWatch Log groups\n",
    "client = boto3.client('logs')\n",
    "response = client.describe_log_groups(limit=50)\n",
    "print(json.dumps(response, indent=1, default=str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set your CloudTrail log group name as a variable for future use, note the default log group is already entered:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cloudtrail_log_group = 'CloudTrail/DefaultLogGroup'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Access Key Investigation <a name=\"access_key\"></a>\n",
    "\n",
    "This section performs searches based on an access key. Modify the example access key variable e.g. `AKIAIOSFODNN7EXAMPLE` and the number of days to search back from today."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "access_key_to_investigate = 'AKIAIOSFODNN7EXAMPLE'\n",
    "previous_days_to_search = 31"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Access Key Last Use & Owner\n",
    "\n",
    "Search for the user who owns an access key, and the last time it was used. If the ServiceName and Region return as 'N/A' the access key has not been used in last 365 days."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "client = boto3.client('iam')\n",
    "response = client.get_access_key_last_used(AccessKeyId=access_key_to_investigate)\n",
    "print (json.dumps(response, indent=1, default=str))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Access Key Created By\n",
    "\n",
    "Search for who created the access key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'filter responseElements.accessKey.accessKeyId =\"' + access_key_to_investigate + '\" | fields eventTime, userIdentity.arn, eventSource, eventName, sourceIPAddress, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Actions Performed By Access Key\n",
    "\n",
    "Search for actions performed by the access key."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'filter userIdentity.accessKeyId =\"' + access_key_to_investigate + '\" | fields eventTime, awsRegion, eventSource, eventName, sourceIPAddress, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. User Investigation <a name=\"user\"></a>\n",
    "\n",
    "This section performs searches based on a user. Modify the example username variable e.g. `test` and the number of days to search back from today."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "username = 'test'\n",
    "previous_days_to_search = 31"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 Actions Performed By Username\n",
    "\n",
    "Search for actions performed by the `username`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'filter userIdentity.userName like \"' + username + '\" | fields eventTime, awsRegion, eventSource, eventName, errorCode, errorMessage, sourceIPAddress, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Users Recently Created\n",
    "\n",
    "Searches for users recently created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'filter eventName=\"CreateUser\" | fields eventTime, requestParameters.userName, responseElements.user.arn, userIdentity.arn, responseElements.role.arn, sourceIPAddress, errorCode, userAgent, errorMessage'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Role Investigation <a name=\"role\"></a>\n",
    "\n",
    "This section performs searches based on a role. Modify the example rolename variable e.g. `test-role` and the number of days to search back from today."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rolename = 'test-role'\n",
    "previous_days_to_search = 31"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 Actions Performed By Role\n",
    "\n",
    "Search for actions performed by the `rolename`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'filter userIdentity.sessionContext.sessionIssuer.userName = \"' + rolename + '\" | fields eventTime, awsRegion, eventSource, eventName, errorCode, errorMessage, sourceIPAddress, userAgent'  \n",
    "#Alternate based on arn: query = 'filter userIdentity.sessionContext.sessionIssuer.arn = \"' + rolearn + '\" | fields eventTime, awsRegion, eventName, eventSource, errorCode, errorMessage, sourceIPAddress, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Roles Recently Created\n",
    "\n",
    "Searches for roles recently created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'filter eventName=\"CreateRole\" | fields eventTime, requestParameters.userName, responseElements.user.arn, userIdentity.arn, responseElements.role.arn, sourceIPAddress, errorCode, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Containment <a name=\"containment\"></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.1 Access Key Deactivate\n",
    "\n",
    "Modify the example access key variable e.g. `AKIAIOSFODNN7EXAMPLE` and username variable e.g. `test` to disable an access key associated with a user from being used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "access_key_to_deactivate='AKIAIOSFODNN7EXAMPLE'\n",
    "username='test'\n",
    "\n",
    "iam = boto3.resource('iam')\n",
    "access_key = iam.AccessKey(username,access_key_to_deactivate)\n",
    "response_status = access_key.deactivate()\n",
    "status_code = response_status['ResponseMetadata']['HTTPStatusCode']\n",
    "if status_code == 200:\n",
    "    print(\"Key Disabled Successfully\")\n",
    "else:\n",
    "    print(\"Key deactivation failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2 Block User Access\n",
    "\n",
    "Modify the username variable e.g. `test` to apply a policy that denies user actions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "username='test'\n",
    "\n",
    "iam = boto3.client('iam')\n",
    "response = iam.put_user_policy(UserName=username,PolicyName='Block',PolicyDocument='{\"Version\":\"2012-10-17\",\"Statement\":{\"Effect\":\"Deny\",\"Action\":\"*\",\"Resource\":\"*\"}}')\n",
    "status_code = response['ResponseMetadata']['HTTPStatusCode']\n",
    "if status_code == 200:\n",
    "    print(\"Policy attached successfully\")\n",
    "else:\n",
    "    print(\"Policy attachment failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.3 Block Role Access\n",
    "\n",
    "Modify the rolename variable e.g. `test-role` to apply a policy that denies role actions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rolename='test-role'\n",
    "\n",
    "iam = boto3.client('iam')\n",
    "response = iam.put_role_policy(RoleName=rolename,PolicyName='Block',PolicyDocument='{\"Version\":\"2012-10-17\",\"Statement\":{\"Effect\":\"Deny\",\"Action\":\"*\",\"Resource\":\"*\"}}')\n",
    "status_code = response['ResponseMetadata']['HTTPStatusCode']\n",
    "if status_code == 200:\n",
    "    print(\"Policy attached successfully\")\n",
    "else:\n",
    "    print(\"Policy attachment failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6. Other Investigations <a name=\"other_investigations\"></a>\n",
    "\n",
    "Some API requests can reveal reconnaissance being performed.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.1 Who Has Listed S3 Buckets\n",
    "\n",
    "The listing of all S3 buckets can indicate someone performing reconnaissance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "previous_days_to_search = 1\n",
    "query = 'filter eventName =\"ListBuckets\" | fields awsRegion, eventSource, eventName, sourceIPAddress, userAgent, eventTime'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.2 Actions From Specific IP"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "previous_days_to_search = 31\n",
    "ip = '10.10.10.10'\n",
    "\n",
    "query = 'filter sourceIPAddress = \"' + ip + '\" | fields awsRegion, userIdentity.arn, eventSource, eventName, sourceIPAddress, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.3 Recent Access Denied Attempts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "previous_days_to_search = 1\n",
    "\n",
    "query = 'filter errorCode like /Unauthorized|Denied|Forbidden/ | fields awsRegion, userIdentity.arn, eventSource, eventName, sourceIPAddress, userAgent'  \n",
    "response = helpers.execute_log_query(cloudtrail_log_group, query, previous_days_to_search)\n",
    "formatted_results = [helpers.convert_dictionary_to_object(r) for r in response['results']]\n",
    "pd.DataFrame(formatted_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test Cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#test cell\n",
    "# Retrieve the list of existing buckets\n",
    "s3 = boto3.client('s3')\n",
    "response = s3.list_buckets()\n",
    "\n",
    "# Output the bucket names\n",
    "print('Existing buckets:')\n",
    "for bucket in response['Buckets']:\n",
    "    print(f'  {bucket[\"Name\"]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## License\n",
    "\n",
    "Licensed under the Apache 2.0 and MITnoAttr License.\n",
    "\n",
    "Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.\n",
    "\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n",
    "\n",
    "    https://aws.amazon.com/apache2.0/\n",
    "\n",
    "or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}