Skip to content

Instantly share code, notes, and snippets.

@rkreddyp

rkreddyp/security_review_runbook.ipynb

Created February 12, 2019 23:47
Show Gist options
  • Save rkreddyp/55f6faf513909854f6800cfc2ee826a9 to your computer and use it in GitHub Desktop.
Save rkreddyp/55f6faf513909854f6800cfc2ee826a9 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
security_review_runbook.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='top'></a>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='top'></a>"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"<script>\n",
"code_show=true; \n",
"function code_toggle() {\n",
" if (code_show){\n",
" $('div.input').hide();\n",
" } else {\n",
" $('div.input').show();\n",
" }\n",
" code_show = !code_show\n",
"} \n",
"$( document ).ready(code_toggle);\n",
"</script>\n",
"It is intended that users change the runbook in anyway they want including the code (API calls) and input parameters into the code.\n",
"The raw code for this IPython notebook is by default hidden for easier reading.\n",
"To toggle on/off the raw code, click <a href=\"javascript:code_toggle()\">here</a>."
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import HTML\n",
"\n",
"HTML('''<script>\n",
"code_show=true; \n",
"function code_toggle() {\n",
" if (code_show){\n",
" $('div.input').hide();\n",
" } else {\n",
" $('div.input').show();\n",
" }\n",
" code_show = !code_show\n",
"} \n",
"$( document ).ready(code_toggle);\n",
"</script>\n",
"It is intended that users change the runbook in anyway they want including the code (API calls) and input parameters into the code.\n",
"The raw code for this IPython notebook is by default hidden for easier reading.\n",
"To toggle on/off the raw code, click <a href=\"javascript:code_toggle()\">here</a>.''')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='toc'></a>\n",
"\n",
"# Security Notebook\n",
"----\n",
"----\n",
"\n",
"## Introduction\n",
"\n",
"Security practioners have quite a few number of workflows that require manual work, such as: \n",
"\n",
" * Ensuring cloud services are configured per best practices\n",
" * Reviewing findings from various AWS services \n",
" * Investiage, probe around suspicious activity\n",
" * Enable (or disable) various security services in an account\n",
" \n",
"**The goal of this Runbook is to help a cloud security practionaer complete above workflows**. The runbook goes through all security services and gets the findings and helps security practioner . For each service, it helps a security practioner to do the following things : Ensure AWS security services are setup in a good way, Analyze findings, Setup (enable, disable, teardown), Probe and investigate various services.\n",
"\n",
"The following are the security services supported by this runbook : CloudTrail, GuardDuty, Inspector, Security Hub.\n",
"\n",
"Accordingly, the notebook has three sections - Best Practice Checks, Analysis and Findings, and Services Enablement\n",
"\n",
"\n",
"## How to Use\n",
" * **Use Case: Review Security Posture and Findings**: Hit \"Kernel\" -> \"Restart and Runall\" \n",
" * **Use Case: Investigation** : Hit the raw code button (the first cell), go to the section you want to dig deep into (such as service or user, change the parameters and Hit \"Kernel\" -> \"Restart and Runall\" \n",
"\n",
" ---\n",
" ---\n",
"\n",
" ## Table of Contents\n",
"\n",
"* [Best Practice Checks](#bpchecks)\n",
" * [AWS CloudTrail](#cloudtrail-checks) : Checks for Multi Region enablement\n",
" * [Guard Duty](#guardduty-checks) : Checks whether Guard Duty is enabled in all regions \n",
" * [Inspector Checks](#inspector-checks): Checks whether Inspector (network reachability) is enabled in all regions\n",
" * [Security Hub Checks](#securityhub-checks) : Checks whether Security hub is enabled in all regions\n",
" ---\n",
"* [Analysis and Findings](#analysis)\n",
" * [AWS CloudTrail](#cloudtrail-checks)\n",
" * [API Call Summary](#cloudtrail-summary) \n",
" * [Cloudtrail Service and EventName Analysis](#cloudtrail-analysis)\n",
" * [Cloudtrail User Agent Analysis](#cloudtrail-useragents)\n",
" * [Console Logins](#console-logins) : Summary of console logins\n",
" * [Root Activity](#root-activity) : Summary of root activity\n",
" * [Generic Service Activity](#service-activity) : Deeper analysis of a particular service \n",
" * [Generic API action](#user-activity) : Deeper analysis of a particular action / EventName\n",
" * [Generic User Activity](#user-activity) : Deeper analysis of a particular user\n",
" \t\n",
" * [Inspector Findings Analysis](#inspector-findings) : Findings from AWS Inspector\n",
" * [GuardDuty Findings](#guardduty-findings) : Findings from AWS Guard Duty\n",
" * [FIndings by AMI](#findings-by-ami) : Consolidates findings from Inspector and Guard Duty\n",
" \n",
" * [Security Hub](#securityhub-findings)\n",
" * [Security Hub Findings Analysis](#sh-get-insights)\n",
" * [Security Hub Insights](#sh-get-insights)\n",
" * [Macie](#macie)\n",
"---\n",
"* [Services Enablement](#enable-services)\n",
" * [AWS CloudTrail](#cloudtrail-enable) : Enables CloudTrail in all regions\n",
" * [Guard Duty](#guardduty-enable) : Enables GuardDuty in all regions\n",
" * [Inspector](#inspector-enable) : Enables Inspector in all regions\n",
" * [Security Hub](#securityhub-enable) : Enables Security Hub in all regions\n",
" \n",
"---\n",
"---\n",
"This security notebook can be run and results can be expored to HTML in Slack"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Admin Activity\n",
"(module installs and python imports that are required for the runbook)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Imports (import modules)\n",
"\n",
"This section includes python modules that need to be imported for the runbook"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:botocore.credentials:Found credentials in shared credentials file: ~/.aws/credentials\n"
]
}
],
"source": [
"import sys\n",
"sys.path.append('/usr/local/lib/python3.6/site-packages')\n",
"import boto3\n",
"import datetime\n",
"import pandas as pd\n",
"import logging\n",
"import datetime\n",
"import json\n",
"import numpy as np\n",
"import time\n",
"import matplotlib.pyplot as pyplot\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.style as style\n",
"from cloudgovernor.helpers import spend_helpers\n",
"from cloudgovernor.helpers import lib_helpers\n",
"from cloudgovernor.helpers import aws_helpers\n",
"from cloudgovernor.helpers import cloudtrail_helpers\n",
"#from cloudgovernor.vuln import inspector_deploy\n",
"\n",
"from importlib import reload\n",
"import pathlib\n",
"from geolite2 import geolite2\n",
"from IPython.display import Markdown as md\n",
"import os\n",
"\n",
"from IPython.display import display, HTML\n",
"\n",
"pd.set_option('display.max_colwidth', -1)\n",
"reader = geolite2.reader()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='bpchecks'></a>\n",
"\n",
"# Section 1. Best Practice Checks\n",
"This run book checks whether AWS security service are enabled and are configured per best practices.\n",
"The following ar ethe security services covered :\n",
" 1. AWS Cloudtrail\n",
" 1. AWS GuradDuty\n",
" 1. AWS Inspector (Network Reachability Scans)\n",
" 1. Security Hub"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='cloudtrail-checks'></a>\n",
" \n",
"### Check whether CloudTrail is Enabled \n",
"Every new AWS account has Cloudtrail enabled by default, the runbook checks that the trail has the following best practice configurations: \n",
" * Whether cloudtrail is enabled (this is done automatically now for every account created)\n",
" * Whether the events are stored in a bucket\n",
" * Whether the cloudtrail is enabled for all regions"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running checks for account : 221470125883\n"
]
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>Name</th>\n",
" <th>S3BucketName</th>\n",
" <th>IsMultiRegionTrail</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>2018trailvirclop</td>\n",
" <td>apr12blah</td>\n",
" <td>True</td>\n",
" </tr>\n",
" <tr>\n",
" <td>avarmiaconsolidation</td>\n",
" <td>ectrail-bucket</td>\n",
" <td>False</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
" #### All Trails are Not Multi Regional:![](https://s3.amazonaws.com/cloudgovernorimages/redcross.png )\n",
" "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"print (\"Running checks for account : \", boto3.client('sts').get_caller_identity()['Account'])\n",
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"for region in region_arr:\n",
" cclient=boto3.client('cloudtrail',region_name=region)\n",
" cdf = pd.DataFrame(cclient.describe_trails()['trailList'])\n",
" cdf = cdf [[ 'Name', 'S3BucketName', 'IsMultiRegionTrail']]\n",
" df_arr.append(cdf)\n",
"ctedf = pd.concat(df_arr)\n",
"display(HTML(ctedf.drop_duplicates().to_html(index=False)))\n",
"\n",
"if False in ctedf.IsMultiRegionTrail.tolist() :\n",
" display (md(\"\"\" #### All Trails are Not Multi Regional:![](https://s3.amazonaws.com/cloudgovernorimages/redcross.png )\n",
" \"\"\"))\n",
"else : \n",
" display (md(\"\"\" #### All Trails are Multi Regional:![](https://s3.amazonaws.com/cloudgovernorimages/greencheck.png )\n",
" \"\"\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='guardduty-checks'></a>\n",
"### Check whether GuardDuty is Enabled\n",
"Guard Duty checks for threat across various AWS log data, including CloudTrail (AWS user and API activity in your accounts), Amazon VPC Flow Logs (network traffic data), and DNS Logs (name query patterns). Though a bit noisy, best practice is to have it enabled, and look at teh findings every week or so (findings are organized in a consumable way in the [GuardDuty Findings](#guardduty-findings) section below).\n",
"\n",
"This run book checks for whether Guard Duty is enabled in all regions."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
" #### Guard Duty is enabled in regions: ['ap-south-1', 'eu-west-3', 'eu-west-2', 'eu-west-1', 'ap-northeast-2', 'ap-northeast-1', 'sa-east-1', 'ca-central-1', 'ap-southeast-1', 'ap-southeast-2', 'eu-central-1', 'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2'] ![](https://s3.amazonaws.com/cloudgovernorimages/greencheck.png )\n",
" "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
" #### Guard Duty is enabled in regions: ['eu-north-1']![](https://s3.amazonaws.com/cloudgovernorimages/redcross.png )\n",
" "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
" #### Guard Duty Detector IDs and Regions in which the Service is Enabled "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>DetectorIds</th>\n",
" <th>region</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>80b4368a1fd44954122a2bfd5dbe4581</td>\n",
" <td>ap-south-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>9eb4368a208aade7b15e86a103eb6eea</td>\n",
" <td>eu-west-3</td>\n",
" </tr>\n",
" <tr>\n",
" <td>50b4368a35ee003e492ad871a728cd70</td>\n",
" <td>eu-west-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>42b4368a36a1636a863196fe77d227f8</td>\n",
" <td>eu-west-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>f0b4368a373fff9706f0e47d1d5d5d3d</td>\n",
" <td>ap-northeast-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>1eb4368a37dbf269cd64d6bf164a612b</td>\n",
" <td>ap-northeast-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>0cb4368a3942acdfa9d812f7fda0a9df</td>\n",
" <td>sa-east-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2eb4368a39ead0a34ca2c76f8a64ac22</td>\n",
" <td>ca-central-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>f6b4368a3a656eb9c0a6967f2b3a2ae4</td>\n",
" <td>ap-southeast-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>48b4368a3b849cd8e7000634961f1403</td>\n",
" <td>ap-southeast-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>42b4368a3c6ac0c92476cc7dd326588d</td>\n",
" <td>eu-central-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>34b394a464d53c739be9c884b1ce96a8</td>\n",
" <td>us-east-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6cb4368a3d1a5c7784167d16c32dd0e3</td>\n",
" <td>us-east-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>42b4368a3d45b9d1dc2f24d6c43480a0</td>\n",
" <td>us-west-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>60b4368a3ddca4c58f3f264a8ba5e159</td>\n",
" <td>us-west-2</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"for region in region_arr:\n",
" gdf = pd.DataFrame()\n",
" \n",
" try :\n",
" gdclient = boto3.client('guardduty',region_name=region)\n",
" if len (gdclient.list_detectors()['DetectorIds']) > 0 :\n",
" gdf = pd.DataFrame(gdclient.list_detectors()['DetectorIds'], columns=['DetectorIds'])\n",
" gdf['region']=region\n",
" else :\n",
" \n",
" gdf['DetectorIds'] = \"NA\"\n",
" gdf['region']=region\n",
" \n",
" df_arr.append(gdf)\n",
" except :\n",
" \n",
" pass\n",
" \n",
"gdconcatdf = pd.concat(df_arr)\n",
"\n",
"enableregionlist = gdconcatdf.region.tolist()\n",
"disableregionlist = list (set(region_arr)-set(enableregionlist))\n",
"display (md(\"\"\" #### Guard Duty is enabled in regions: {enableregionlist} ![](https://s3.amazonaws.com/cloudgovernorimages/greencheck.png )\n",
" \"\"\".format(enableregionlist=enableregionlist)))\n",
"\n",
"display (md(\"\"\" #### Guard Duty is enabled in regions: {disableregionlist}![](https://s3.amazonaws.com/cloudgovernorimages/redcross.png )\n",
" \"\"\".format(disableregionlist=disableregionlist)))\n",
"\n",
"display (md(\"\"\" #### Guard Duty Detector IDs and Regions in which the Service is Enabled \"\"\"))\n",
"display(HTML(gdconcatdf.to_html(index=False)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='inspector-checks'></a>\n",
"\n",
"### Inspector (Network Reachability Rules) Enablement Checks\n",
"Check whether network reachability rules are enabled"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
" #### Evaluating for Network Reachability Rules in all regions "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"ec2_client = boto3.client('ec2')\n",
"display (md(\"\"\" #### Evaluating for Network Reachability Rules in all regions \"\"\" )) \n",
"\n",
"check_inspector = False\n",
"\n",
"if check_inspector :\n",
" for region in pd.DataFrame (ec2_client.describe_regions()['Regions']).RegionName.tolist():\n",
" try :\n",
"\n",
" ins_client = boto3.client('inspector', region_name=region)\n",
"\n",
" template_arns = ins_client.list_assessment_templates()['assessmentTemplateArns']\n",
"\n",
" df_arr= []\n",
" for template_arn in template_arns:\n",
"\n",
" tdf = pd.DataFrame (ins_client.describe_assessment_templates(assessmentTemplateArns=[template_arn])['assessmentTemplates'])\n",
" df_arr.append(tdf)\n",
"\n",
"\n",
" if len (df_arr) > 0 and not pd.concat(df_arr).empty :\n",
" rule_arns = [ item for sublist in pd.concat(df_arr)['rulesPackageArns'].tolist() for item in sublist ]\n",
"\n",
" if len (rule_arns) > 0 :\n",
" df_arr = []\n",
"\n",
" for rule_arn in rule_arns:\n",
" rdf = pd.DataFrame ( ins_client.describe_rules_packages(rulesPackageArns=[rule_arn])['rulesPackages'])\n",
" df_arr.append(rdf)\n",
"\n",
" if len (pd.concat(df_arr).index) > 0 :\n",
"\n",
" if 'Network Reachability' in pd.concat(df_arr).name.tolist():\n",
" display (md(\"\"\" ##### The following network reachability rules exist in region : {region} ![](https://s3.amazonaws.com/cloudgovernorimages/greencheck.png) \"\"\".format (region=region) )) \n",
" ndf = pd.concat(df_arr)\n",
" ndf = ndf [ ndf.name.str.contains(\"Network\")]\n",
" display(HTML(ndf[['name','arn']].to_html(index=False)))\n",
"\n",
" except Exception as e: print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='securityhub-checks'></a>\n",
"\n",
"### Security Hub\n",
"AWS Security Hub collects and aggregates findings from the AWS security services and has pre configured compliance checks for security standards (currently for CIS bench marks). For security hub, we check the number of standards enabled in security hub service for each region.\n",
"\n",
"(go to [top](#top))"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>Number of standards enabled in Security Hub Service</th>\n",
" <th>region</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>eu-north-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>ap-south-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>eu-west-3</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>eu-west-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>eu-west-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>ap-northeast-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>ap-northeast-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>sa-east-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>ca-central-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>ap-southeast-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>ap-southeast-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>eu-central-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>us-east-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>us-east-2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>us-west-1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>2</td>\n",
" <td>us-west-2</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Since security hub is in latest boto3, make sure its installed\n",
"#!pip uninstall -y \n",
"#!pip install 'botocore==1.12.61'\n",
"#!pip uninstall -y boto3\n",
"#!pip install boto3 \n",
"\n",
"client = boto3.client('securityhub')\n",
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"for region in region_arr:\n",
" secdf = pd.DataFrame ( { \"region\": region, \"Number of standards enabled in Security Hub Service\" : len (pd.DataFrame (client.get_enabled_standards()['StandardsSubscriptions']).shape) }, index=[0] )\n",
" df_arr.append(secdf)\n",
"secdfarr = pd.concat(df_arr)\n",
"display(HTML(secdfarr.to_html(index=False)))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='analysis'></a>\n",
"\n",
"\n",
"# Section 2. AWS Security Services Findings and Analysis\n",
"This run book gathers results and findings from all AWS security services and presents it in such a way that a security team can take a look at the results, assess the state of security and take remediative/corrective actions based on the findings.\n",
"\n",
"This section gathers API write Call data for the past 64 hours (change it as your workflows require) is organized the following way -\n",
"\n",
" * API call summary (which services are involved with API calls, what actions have been conducted and by what users)\n",
" * API call user agent summary (which user agents have been used for various actions on services)\n",
" * Deep dive analysis into a service (for a given service, what actions by which users and at what time). By default, the runbook takes the first service, and users of the runbook can change it to any serivce they want to investigate)\n",
" * Deep dive analysis into an user (for a given user what actions did the user take and at what time). By default, the runbook takes the first user, and users of the runbook can change it to any serivce they want to investigate)\n",
" * Deep dive analysis into various actions (by default, the run book brings up Console Logins, the users of the run book can change it to any action)\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='cloudtrail-summary'></a>\n",
" \n",
" \n",
" (go to [top](#top))"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/markdown": [
" ## API Calls (CloudTrail) Event Summary\n",
" * Number of Event Sources (services): 16\n",
" * Number of Event Names (actions): 51\n",
" "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"### API Calls : List of Services and Actions "
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/notebooks/eventsources-actions.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventSource</th>\n",
" <th>EventNames</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>CreateDeployment,UpdateStage,UpdateRestApi,PutIntegration,PutMethod,CreateResource,CreateRestApi</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudformation.amazonaws.com</td>\n",
" <td>UpdateStack,CreateStack</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>CreateTrail,StartLogging,PutEventSelectors,DeleteTrail</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>StopConfigurationRecorder,StartConfigurationRecorder,BatchGetResourceConfig,PutEvaluations,PutConfigRule</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>PutTargets,PutRule,RemoveTargets,DeleteRule,TestEventPattern</td>\n",
" </tr>\n",
" <tr>\n",
" <td>guardduty.amazonaws.com</td>\n",
" <td>CreateDetector,GetFindings,ListFindings</td>\n",
" </tr>\n",
" <tr>\n",
" <td>iam.amazonaws.com</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" </tr>\n",
" <tr>\n",
" <td>inspector.amazonaws.com</td>\n",
" <td>StartAssessmentRun</td>\n",
" </tr>\n",
" <tr>\n",
" <td>kms.amazonaws.com</td>\n",
" <td>CreateGrant</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2,PutFunctionConcurrency20171031,RemovePermission20150331v2,UpdateFunctionCode20150331v2,UpdateFunctionConfiguration20150331v2,CreateFunction20150331,DeleteFunction20150331</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogStream,CreateLogGroup</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>CreateBucket,PutBucketPolicy,PutBucketNotification</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>CreatePresignedNotebookInstanceUrl,DeleteNotebookInstance,StopNotebookInstance,CreateNotebookInstance,CreateNotebookInstanceLifecycleConfig</td>\n",
" </tr>\n",
" <tr>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>GetEnabledStandards</td>\n",
" </tr>\n",
" <tr>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>ConsoleLogin</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sns.amazonaws.com</td>\n",
" <td>CreateTopic,Subscribe,Unsubscribe</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"### List of Actions and Users"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/notebooks/eventactions-users.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventSource</th>\n",
" <th>EventName</th>\n",
" <th>Users</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>CreateDeployment</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>CreateResource</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>CreateRestApi</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>PutIntegration</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>PutMethod</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>UpdateRestApi</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>UpdateStage</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudformation.amazonaws.com</td>\n",
" <td>CreateStack</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudformation.amazonaws.com</td>\n",
" <td>UpdateStack</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>CreateTrail</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>DeleteTrail</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>PutEventSelectors</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>StartLogging</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>BatchGetResourceConfig</td>\n",
" <td>securityhub</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>PutConfigRule</td>\n",
" <td>securityhub</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>PutEvaluations</td>\n",
" <td>configLambdaExecution</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>StartConfigurationRecorder</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>StopConfigurationRecorder</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>DeleteRule</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>PutRule</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>PutTargets</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>RemoveTargets</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>TestEventPattern</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>guardduty.amazonaws.com</td>\n",
" <td>CreateDetector</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>guardduty.amazonaws.com</td>\n",
" <td>GetFindings</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>guardduty.amazonaws.com</td>\n",
" <td>ListFindings</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>iam.amazonaws.com</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" </tr>\n",
" <tr>\n",
" <td>inspector.amazonaws.com</td>\n",
" <td>StartAssessmentRun</td>\n",
" <td>bdda6e2710a23c92ae0706de4d29a3cb</td>\n",
" </tr>\n",
" <tr>\n",
" <td>kms.amazonaws.com</td>\n",
" <td>CreateGrant</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>CreateFunction20150331</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>DeleteFunction20150331</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>PutFunctionConcurrency20171031</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>RemovePermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>UpdateFunctionCode20150331v2</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>UpdateFunctionConfiguration20150331v2</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogGroup</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogStream</td>\n",
" <td>watcher-uxrouter</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>CreateBucket</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>PutBucketNotification</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>PutBucketPolicy</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>CreateNotebookInstance</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>CreateNotebookInstanceLifecycleConfig</td>\n",
" <td>cg-221470125883</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>DeleteNotebookInstance</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>StopNotebookInstance</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>GetEnabledStandards</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>ConsoleLogin</td>\n",
" <td>root</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sns.amazonaws.com</td>\n",
" <td>CreateTopic</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sns.amazonaws.com</td>\n",
" <td>Subscribe</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sns.amazonaws.com</td>\n",
" <td>Unsubscribe</td>\n",
" <td>venkatdev</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"reload(cloudtrail_helpers)\n",
"endtime = datetime.datetime.now() # Create start and end time for CloudTrail lookup\n",
"interval = datetime.timedelta(hours=48) \n",
"starttime = endtime - interval\n",
"reload(cloudtrail_helpers)\n",
"eventdf = cloudtrail_helpers.get_events_all_df(starttime, endtime)\n",
"\n",
"eventdf['Resources'] = eventdf['Resources'].astype(str)\n",
"\n",
"oeventdf = eventdf.copy()\n",
"\n",
"eventdf = eventdf.drop_duplicates([\"EventSource\",\"EventName\"])\n",
"\n",
"\n",
"cdf_eventnames = eventdf.groupby( [\"EventSource\"] )['EventName'].agg(','.join).reset_index(name='EventNames')\n",
"cdf_eventnames = cdf_eventnames [[\"EventSource\",'EventNames' ]]\n",
"\n",
"\n",
"display (md(\"\"\" ## API Calls (CloudTrail) Event Summary\n",
" * Number of Event Sources (services): {noservices}\n",
" * Number of Event Names (actions): {noactions}\n",
" \"\"\".format(noservices=len(eventdf.EventSource.unique().tolist()),noactions=len(eventdf.EventName.unique().tolist()) )\n",
" ))\n",
"display (md(\"### API Calls : List of Services and Actions \"))\n",
"\n",
"cdf_eventnames.to_csv(\"/tmp/eventsources-actions.csv\")\n",
"link = lib_helpers.take_uploadfilename_return_link(\"eventsources-actions.csv\", \"notebooks/eventsources-actions.csv\")\n",
"display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"display(HTML(cdf_eventnames.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"eventdf = oeventdf.copy()\n",
"eventdf = eventdf.drop_duplicates([\"EventSource\",\"EventName\"])\n",
"eventdf[\"Username\"] = eventdf[\"Username\"].astype(str)\n",
"cdf_users = eventdf.groupby( [\"EventSource\" , \"EventName\"] )['Username'].agg(','.join).reset_index(name='Users')\n",
"cdf_users = cdf_users [[\"EventSource\",\"EventName\",'Users' ]]\n",
"\n",
"display (md(\"### List of Actions and Users\"))\n",
"\n",
"cdf_eventnames.to_csv(\"/tmp/eventactions-users.csv\")\n",
"link = lib_helpers.take_uploadfilename_return_link(\"eventactions-users.csv\", \"notebooks/eventactions-users.csv\")\n",
"display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"display(HTML(cdf_users.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"eventdf = oeventdf.copy()\n",
"eventdf = eventdf.drop_duplicates([\"EventSource\",\"EventName\"])\n",
"eventdf[\"awsRegion\"] = eventdf[\"awsRegion\"].astype(str)\n",
"cdf_users = eventdf.groupby( [\"EventSource\", \"EventName\" ] )['awsRegion'].agg(','.join).reset_index(name='Regions')\n",
"cdf_users = cdf_users [[\"EventSource\",\"EventName\", 'Regions' ]]\n",
"\n",
"#display (md(\"### Event Sources by Users\"))\n",
"#display(HTML(cdf_users.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='cloudtrail-useragents'></a>\n",
"\n",
"### User Actions and User Agents\n",
"User agents are finger prints of user actions. This section gives a view of which user agents are being used for various action and corresponding services.\n",
"\n",
"Events with cosole login user agents (non programmatic) user agents should be reviewed every day and followup to see why console logins are being used for actions.\n",
"\n",
"(go to [top](#top))"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/notebooks/users-agents-ips.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"### Events with non console login User Agents"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventSource</th>\n",
" <th>Eventnames</th>\n",
" <th>Username</th>\n",
" <th>UserAgents</th>\n",
" <th>sourceIPAddress</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>CreateDeployment,UpdateRestApi,UpdateStage</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.7.68 Python/3.6.5 Linux/4.4.0-1074-aws Botocore/1.10.68</td>\n",
" <td>52.203.192.208</td>\n",
" </tr>\n",
" <tr>\n",
" <td>apigateway.amazonaws.com</td>\n",
" <td>CreateResource,CreateRestApi,PutIntegration,PutMethod</td>\n",
" <td>venkatdev</td>\n",
" <td>cloudformation.amazonaws.com</td>\n",
" <td>cloudformation.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>cloudformation.amazonaws.com</td>\n",
" <td>CreateStack,UpdateStack</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.7.68 Python/3.6.5 Linux/4.4.0-1074-aws Botocore/1.10.68</td>\n",
" <td>52.203.192.208</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>StartConfigurationRecorder,StopConfigurationRecorder</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>52.200.140.113</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>StartConfigurationRecorder,StopConfigurationRecorder</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>3.94.170.191</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>PutEvaluations</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>config.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>config.amazonaws.com</td>\n",
" <td>PutConfigRule</td>\n",
" <td>securityhub</td>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>securityhub.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>PutRule,PutTargets,TestEventPattern</td>\n",
" <td>root</td>\n",
" <td>AWS CloudWatch Console</td>\n",
" <td>173.76.14.202</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>DeleteRule,PutRule,PutTargets,RemoveTargets</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.7.68 Python/3.6.5 Linux/4.4.0-1074-aws Botocore/1.10.68</td>\n",
" <td>52.203.192.208</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>PutRule,PutTargets</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>3.82.209.115</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>PutRule,PutTargets</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>52.90.42.78</td>\n",
" </tr>\n",
" <tr>\n",
" <td>events.amazonaws.com</td>\n",
" <td>DeleteRule,RemoveTargets</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.74 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>54.81.185.224</td>\n",
" </tr>\n",
" <tr>\n",
" <td>guardduty.amazonaws.com</td>\n",
" <td>CreateDetector,GetFindings,ListFindings</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.9.84 Python/3.6.5 Linux/4.14.77-70.82.amzn1.x86_64 Botocore/1.12.84</td>\n",
" <td>34.236.55.223</td>\n",
" </tr>\n",
" <tr>\n",
" <td>guardduty.amazonaws.com</td>\n",
" <td>CreateDetector,GetFindings,ListFindings</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>im.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>iam.amazonaws.com</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>im.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>inspector.amazonaws.com</td>\n",
" <td>StartAssessmentRun</td>\n",
" <td>bdda6e2710a23c92ae0706de4d29a3cb</td>\n",
" <td>events.amazonaws.com</td>\n",
" <td>events.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>kms.amazonaws.com</td>\n",
" <td>CreateGrant</td>\n",
" <td>venkatdev</td>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>lambda.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2</td>\n",
" <td>root</td>\n",
" <td>AWS CloudWatch Console</td>\n",
" <td>173.76.14.202</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2,CreateFunction20150331,DeleteFunction20150331,RemovePermission20150331v2...</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.7.68 Python/3.6.5 Linux/4.4.0-1074-aws Botocore/1.10.68</td>\n",
" <td>52.203.192.208</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>54.175.96.43</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2,PutFunctionConcurrency20171031,RemovePermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>54.159.232.122</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>RemovePermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.74 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>184.72.187.64</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>AddPermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.74 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>3.82.114.180</td>\n",
" </tr>\n",
" <tr>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>RemovePermission20150331v2</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.74 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>34.203.188.174</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogStream</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>18.212.19.51</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogGroup</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>3.88.160.153</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogGroup,CreateLogStream</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>18.209.65.218</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogStream</td>\n",
" <td>watcher-uxrouter</td>\n",
" <td>awslambda-worker</td>\n",
" <td>3.94.81.5</td>\n",
" </tr>\n",
" <tr>\n",
" <td>logs.amazonaws.com</td>\n",
" <td>CreateLogGroup,CreateLogStream</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>im.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>PutBucketPolicy</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_...</td>\n",
" <td>10.246.174.111</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>CreateBucket</td>\n",
" <td>cg-221470125883</td>\n",
" <td>[Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botoco...</td>\n",
" <td>52.200.140.113</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>PutBucketNotification</td>\n",
" <td>cg-221470125883</td>\n",
" <td>[Boto3/1.7.68 Python/3.6.8 Linux/4.14.88-90.76.amzn2.x86_64 exec-env/AWS_Lambda_python3.6 Botoco...</td>\n",
" <td>54.175.96.43</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>CreateBucket</td>\n",
" <td>cg-221470125883</td>\n",
" <td>[Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botoco...</td>\n",
" <td>3.94.170.191</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>PutBucketNotification</td>\n",
" <td>cg-221470125883</td>\n",
" <td>[Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botoco...</td>\n",
" <td>52.55.253.179</td>\n",
" </tr>\n",
" <tr>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>PutBucketNotification</td>\n",
" <td>cg-221470125883</td>\n",
" <td>[Boto3/1.7.74 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botoco...</td>\n",
" <td>54.209.236.58</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>CreateNotebookInstance,CreateNotebookInstanceLifecycleConfig</td>\n",
" <td>cg-221470125883</td>\n",
" <td>Boto3/1.7.68 Python/3.6.8 Linux/4.14.94-73.73.amzn1.x86_64 exec-env/AWS_Lambda_python3.6 Botocor...</td>\n",
" <td>3.92.181.249</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>CreatePresignedNotebookInstanceUrl,DeleteNotebookInstance,StopNotebookInstance</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bi...</td>\n",
" <td>173.76.14.202</td>\n",
" </tr>\n",
" <tr>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>GetEnabledStandards</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.9.84 Python/3.6.5 Linux/4.14.77-70.82.amzn1.x86_64 Botocore/1.12.84</td>\n",
" <td>34.236.55.223</td>\n",
" </tr>\n",
" <tr>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>GetEnabledStandards</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>im.amazonaws.com</td>\n",
" </tr>\n",
" <tr>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>ConsoleLogin</td>\n",
" <td>root</td>\n",
" <td>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version...</td>\n",
" <td>173.76.14.202</td>\n",
" </tr>\n",
" <tr>\n",
" <td>sns.amazonaws.com</td>\n",
" <td>CreateTopic,Subscribe,Unsubscribe</td>\n",
" <td>venkatdev</td>\n",
" <td>Boto3/1.7.68 Python/3.6.5 Linux/4.4.0-1074-aws Botocore/1.10.68</td>\n",
" <td>52.203.192.208</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"eventdf = oeventdf.copy()\n",
"\n",
"eventdf = eventdf.drop_duplicates([\"EventName\",\"userAgent\"])\n",
"eventdf.userAgent = eventdf.userAgent.astype(str)\n",
"eventdf = eventdf.groupby( [\"EventSource\", \"EventName\",\"Username\", 'sourceIPAddress'] )['userAgent'].agg(','.join).reset_index(name='UserAgents')\n",
"eventdf = eventdf.groupby( [\"EventSource\",'UserAgents',\"Username\", 'sourceIPAddress'] )['EventName'].agg(','.join).reset_index(name='Eventnames')\n",
"cdf_eventnames = eventdf [[\"EventSource\", \"Eventnames\",\"Username\", 'UserAgents' ,'sourceIPAddress' ]]\n",
"cdf_eventnames.style.set_table_styles([dict(selector=\"th\",props=[('max-width', '25px')])])\n",
"pd.set_option('display.max_colwidth', 100)\n",
"cdf_eventnames.to_csv(\"/tmp/users-agents-ips.csv\")\n",
"link = lib_helpers.take_uploadfilename_return_link(\"users-agents-ips.csv\", \"notebooks/users-agents-ips.csv\")\n",
"display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"display (md(\"### Events with non console login User Agents\"))\n",
"display(HTML(cdf_eventnames[~cdf_eventnames.UserAgents.str.contains(\"console\")].drop_duplicates().to_html(index=False, justify=\"left\" , border = 1)))\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='console-logins'></a>\n",
"\n",
"\n",
"### Console Login Activity"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"scrolled": false
},
"outputs": [
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/consolelogins.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventTime</th>\n",
" <th>userAgent</th>\n",
" <th>sourceIPAddress</th>\n",
" <th>Usernames</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>02-11-2019</td>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>securityhub</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019</td>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>securityhub.amazonaws.com</td>\n",
" <td>securityhub,securityhub</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"reload(cloudtrail_helpers)\n",
"endtime = datetime.datetime.now() # Create start and end time for CloudTrail lookup\n",
"interval = datetime.timedelta(hours=48)\n",
"starttime = endtime - interval\n",
"\n",
"readonly = \"false\" # change it to false if you want write events as well\n",
"\n",
"eventdf = cloudtrail_helpers.get_events_df(\"EventName\", \"PutConfigRule\", starttime, endtime, readonly)\n",
"\n",
"eventdf.EventTime = pd.to_datetime(eventdf.EventTime, format='%m-%d-%Y:%H').apply(lambda x:x.strftime('%m-%d-%Y'))\n",
"sdf = eventdf[[\"EventTime\", 'Username', 'userAgent', 'sourceIPAddress']]\n",
"sdf = sdf.groupby( [\"EventTime\", \"userAgent\", 'sourceIPAddress'] )['Username'].agg(','.join).reset_index(name='Usernames')\n",
"\n",
"\n",
"\n",
"sdf.to_csv(\"/tmp/consolelogins.csv\")\n",
"link = lib_helpers.take_uploadfilename_return_link(\"consolelogins.csv\", \"consolelogins.csv\")\n",
"display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"#sdf['country'] = sdf['sourceIPAddress'].apply (lambda x: reader.get(x)['country']['names']['en'] )\n",
"\n",
"display(HTML(sdf.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"#eventdf['country'] = eventdf['sourceIPAddress'].apply (lambda x: reader.get(x)['country']['names']['en'] )\n",
"\n",
"\n",
"sdf = eventdf.groupby( [ \"EventTime\", \"EventName\", 'Username', 'userAgent'] )[\"EventName\"].count().unstack()\n",
"\n",
"\n",
"#sdf.plot(kind='bar', figsize=(15,15), legend=True, colormap=\"gist_rainbow\", stacked=True).legend(bbox_to_anchor=(1,1))\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='root-activity'></a>\n",
"\n",
"\n",
"### Root Activity\n",
"This section brings up the root activity (`userIdentitype.type = Root`)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/rootactivity.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventTime</th>\n",
" <th>EventSource</th>\n",
" <th>Username</th>\n",
" <th>sourceIPAddress</th>\n",
" <th>Eventnames</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>02-11-2019:00</td>\n",
" <td>events.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>PutRule,PutTargets,TestEventPattern</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:00</td>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>AddPermission20150331v2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:01</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:02</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:12</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:12</td>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>ConsoleLogin</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:21</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:00</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>ConsoleLogin</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>StopNotebookInstance</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl,DeleteNotebookInstance</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>ConsoleLogin</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:16</td>\n",
" <td>sagemaker.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>DeleteTrail</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>cloudtrail.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>173.76.14.202</td>\n",
" <td>CreateTrail,StartLogging,PutEventSelectors</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>s3.amazonaws.com</td>\n",
" <td>root</td>\n",
" <td>10.246.174.111</td>\n",
" <td>PutBucketPolicy</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"reporting into Slack /tmp/root-activity.html\n"
]
}
],
"source": [
"report_slack = True\n",
"section = 'root-activity'\n",
"eventdf = oeventdf.copy()\n",
"\n",
"eventdf.userIdentity= eventdf.userIdentity.astype(str)\n",
"eventdf = eventdf [eventdf.userIdentity.str.contains(\"Root\")]\n",
"eventdf = eventdf [['EventSource', 'EventName', 'Username', 'EventTime', 'sourceIPAddress']]\n",
"eventdf = eventdf.groupby( ['EventTime', \"EventSource\",\"Username\", 'sourceIPAddress'] )['EventName'].agg(','.join).reset_index(name='Eventnames')\n",
"\n",
"sdf.to_csv(\"/tmp/rootactivity.csv\")\n",
"link = lib_helpers.take_uploadfilename_return_link(\"rootactivity.csv\", \"rootactivity.csv\")\n",
"display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"\n",
"\n",
"display(HTML(eventdf.to_html(index=False, justify=\"left\")))\n",
"\n",
"if report_slack and not eventdf.empty :\n",
" uploadfile = '/tmp/'+section+'.html'\n",
" eventdf.to_html(uploadfile)\n",
" print (\"reporting into Slack \", uploadfile)\n",
" os.popen( 'python3 /home/ec2-user/SageMaker/slacksender.py ' + uploadfile + \" \" + section+'.html' + \" \" + section )\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## API Investigation\n",
"The API investigation section lets the user dig deep into a given\n",
" * Service\n",
" * Action\n",
" * User\n",
" \n",
" The section has a default service, action and user and the user can edit each of the sections for desired input for analysis."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='service-activity'></a>\n",
"\n",
"\n",
"### API Call Analysis of Specific Service\n",
"\n",
"This section does deeper analysis of a given service. By default, the run book analyzes `s3.amazonaws.com` service, please edit the `service` variable in the code for analysis on any other service. \n",
"\n",
"( To edit the code go to [top](#top) section and click the option to toggle the code )"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/markdown": [
"#### API Calls have been made on the below services:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"['s3.amazonaws.com']\n"
]
},
{
"data": {
"text/markdown": [
"#### The following are the keys available for display (by default, we are displaying EventTime, Username, UserAgent and requestParameters:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"['AccessKeyId', 'CloudTrailEvent', 'EventId', 'EventName', 'EventSource', 'EventTime', 'ReadOnly', 'Resources', 'Username', 'additionalEventData', 'awsRegion', 'errorCode', 'errorMessage', 'eventID', 'eventName', 'eventSource', 'eventTime', 'eventType', 'eventVersion', 'recipientAccountId', 'requestID', 'requestParameters', 'responseElements', 'sourceIPAddress', 'userAgent', 'userIdentity', 'vpcEndpointId']\n"
]
},
{
"data": {
"text/markdown": [
"#### Service selected to analyze: iam.amazonaws.com"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/api-service.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventTime</th>\n",
" <th>EventName</th>\n",
" <th>Username</th>\n",
" <th>userAgent</th>\n",
" <th>requestParameters</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>02-12-2019:17</td>\n",
" <td>GenerateCredentialReport</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:17</td>\n",
" <td>GetCredentialReport</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:17</td>\n",
" <td>GetAccountPasswordPolicy</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:17</td>\n",
" <td>ListVirtualMFADevices</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:17</td>\n",
" <td>GetAccountSummary</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25....</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListUsers</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListGroups</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListRoles</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:08</td>\n",
" <td>ListRoles</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:08</td>\n",
" <td>ListUsers</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:07</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:07</td>\n",
" <td>ListGroups</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.483 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25....</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:22</td>\n",
" <td>ListGroups</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:22</td>\n",
" <td>ListRoles</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:22</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:21</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.483 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25....</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:21</td>\n",
" <td>ListUsers</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:17</td>\n",
" <td>GenerateCredentialReport</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:17</td>\n",
" <td>GetCredentialReport</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:17</td>\n",
" <td>GetAccountPasswordPolicy</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:17</td>\n",
" <td>GetAccountSummary</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:17</td>\n",
" <td>ListVirtualMFADevices</td>\n",
" <td>configLambdaExecution</td>\n",
" <td>config.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:16</td>\n",
" <td>ListRoles</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:16</td>\n",
" <td>ListUsers</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:16</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:15</td>\n",
" <td>ListGroups</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:12</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.483 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25....</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:09</td>\n",
" <td>ListGroups</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:09</td>\n",
" <td>ListRoles</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:09</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:09</td>\n",
" <td>ListUsers</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:02</td>\n",
" <td>ListUsers</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:02</td>\n",
" <td>ListGroups</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:02</td>\n",
" <td>GenerateServiceLastAccessedDetails</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:02</td>\n",
" <td>ListRoles</td>\n",
" <td>SageMaker</td>\n",
" <td>im.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-11-2019:00</td>\n",
" <td>ListRoles</td>\n",
" <td>root</td>\n",
" <td>AWS CloudWatch Console</td>\n",
" <td>{'maxItems': 1000}</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"service_investigation = True\n",
"if service_investigation :\n",
" reload(cloudtrail_helpers)\n",
" endtime = datetime.datetime.now() # Create start and end time for CloudTrail lookup\n",
" interval = datetime.timedelta(hours=48)\n",
" starttime = endtime - interval\n",
" service = 'iam.amazonaws.com' # Edit the service below \n",
" readonly = \"false\" # change to false if you want to get write events as well\n",
" #eventdf = oeventdf.copy()\n",
" eventdf = cloudtrail_helpers.get_events_df(\"EventSource\", \"s3.amazonaws.com\", starttime, endtime, readonly)\n",
" display (md(\"#### API Calls have been made on the below services:\"))\n",
" print (eventdf.EventSource.unique().tolist())\n",
" display (md(\"#### The following are the keys available for display (by default, we are displaying EventTime, Username, UserAgent and requestParameters:\"))\n",
"\n",
" print (eventdf.columns.tolist())\n",
" display (md(\"#### Service selected to analyze: {service}\".format(service=service)))\n",
" eventdf = cloudtrail_helpers.get_events_df(\"EventSource\", service, starttime, endtime, readonly )\n",
" eventdf = eventdf [eventdf.EventSource.str.contains(service)]\n",
" eventdf.requestParameters = eventdf.requestParameters.astype(str)\n",
"\n",
" sdf = eventdf[[\"EventTime\", 'EventName','Username', 'userAgent', 'requestParameters']]\n",
"\n",
" sdf.to_csv(\"/tmp/api-service.csv\")\n",
" link = lib_helpers.take_uploadfilename_return_link(\"api-service.csv\", \"api-service.csv\")\n",
" display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"\n",
"\n",
" display(HTML(sdf.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"\n",
" sdf = eventdf.groupby( [ \"EventTime\", 'Username', 'userAgent'] )[\"EventName\"].count().unstack()\n",
"\n",
"\n",
" #sdf.plot(kind='bar', figsize=(15,15), legend=True, colormap=\"gist_rainbow\", stacked=True).legend(bbox_to_anchor=(1,1))\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='action-activity'></a>\n",
"\n",
"## API Call Analysis of Specific Action\n",
"\n",
"This section does deeper analysis of a given action (EventName). By default, the run book analyzes the first eventname service, please edit the `EventName` variable in the code for analysis on any other service. \n",
"\n",
"( To edit the code go to [top](#top) section and click the option to toggle the code )"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/markdown": [
"#### The following actions have happened API Calls have been made on the below services:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"[]\n"
]
},
{
"data": {
"text/markdown": [
"#### Service selected to analyze: CreateTrail"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/api-actions.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventTime</th>\n",
" <th>EventName</th>\n",
" <th>Username</th>\n",
" <th>userAgent</th>\n",
" <th>requestParameters</th>\n",
" <th>sourceIPAddress</th>\n",
" <th>awsRegion</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"EventName = 'CreateTrail' # change user to any user above.\n",
"\n",
"event_investigation=True\n",
"if event_investigation :\n",
" reload(cloudtrail_helpers)\n",
" endtime = datetime.datetime.now() # Create start and end time for CloudTrail lookup\n",
" interval = datetime.timedelta(hours=48)\n",
" starttime = endtime - interval\n",
" readonly = \"false\"\n",
" #eventdf = oeventdf.copy()\n",
" eventdf = cloudtrail_helpers.get_events_df(\"EventName\", \"RunInstances\", starttime, endtime, readonly)\n",
" display (md(\"#### The following actions have happened API Calls have been made on the below services:\"))\n",
" print (eventdf.EventName.unique().tolist())\n",
"\n",
" eventdf = cloudtrail_helpers.get_events_df(\"EventName\", \"GetUser\", starttime, endtime, readonly)\n",
" display (md(\"#### Service selected to analyze: {EventName}\".format(EventName=EventName)))\n",
" eventdf = eventdf [eventdf.EventName.str.contains(EventName)]\n",
" eventdf.requestParameters = eventdf.requestParameters.astype(str)\n",
"\n",
" sdf = eventdf[[\"EventTime\", 'EventName','Username', 'userAgent', 'requestParameters', 'sourceIPAddress', 'awsRegion']]\n",
"\n",
" sdf.to_csv(\"/tmp/api-actions.csv\")\n",
" link = lib_helpers.take_uploadfilename_return_link(\"api-actions.csv\", \"api-actions.csv\")\n",
" display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
"\n",
" display(HTML(sdf.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"\n",
" sdf = eventdf.groupby( [ \"EventTime\", 'Username', 'userAgent'] )[\"EventName\"].count().unstack()\n",
"\n",
"\n",
" #sdf.plot(kind='bar', figsize=(15,15), legend=True, colormap=\"gist_rainbow\", stacked=True).legend(bbox_to_anchor=(1,1))\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='user-activity'></a>\n",
"## Analysis of Specific User Actions\n",
"\n",
"This section does deeper analysis of a given user (Username). By default, the run book analyzes the user `root`, please edit the `user` variable in the code for analysis on any other service. \n",
"\n",
"( To edit the code go to [top](#top) section and click the option to toggle the code )"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"#### The following are the users involved :"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"['watcher-uxrouter', 'bdda6e2710a23c92ae0706de4d29a3cb', 'venkatdev', 'cg-221470125883', 'root', 'leadchimptweepy-tweepy', 'leadchimp-lc', 'stockrow', 'securityhub', 'watcher-telemetry', 'configLambdaExecution', 'AWSConfig-ConfigRuleValidation', 'SageMaker']\n"
]
},
{
"data": {
"text/markdown": [
"#### The following are the users available for display (by default, we are displaying EventTime, Username, UserAgent and requestParameters:"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"['AccessKeyId', 'CloudTrailEvent', 'EventId', 'EventName', 'EventSource', 'EventTime', 'ReadOnly', 'Resources', 'Username', 'additionalEventData', 'apiVersion', 'awsRegion', 'errorCode', 'errorMessage', 'eventID', 'eventName', 'eventSource', 'eventTime', 'eventType', 'eventVersion', 'managementEvent', 'readOnly', 'recipientAccountId', 'requestID', 'requestParameters', 'resources', 'responseElements', 'sourceIPAddress', 'userAgent', 'userIdentity', 'vpcEndpointId']\n"
]
},
{
"data": {
"text/markdown": [
"#### Service selected to analyze: root"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>EventTime</th>\n",
" <th>EventName</th>\n",
" <th>Username</th>\n",
" <th>userAgent</th>\n",
" <th>requestParameters</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>02-12-2019:23</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>DescribeOrganization</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>GetTrailStatus</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'name': 'arn:aws:cloudtrail:us-east-1:221470125883:trail/2018trailvirclop'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>ListBuckets</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202]</td>\n",
" <td>{'host': ['s3-external-1.amazonaws.com']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>DescribeTrails</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'trailNameList': [], 'includeShadowTrails': True}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>CreateTrail</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'name': '2018trailvirclop', 's3BucketName': 'apr12blah', 'includeGlobalServiceEvents': True, 'isMultiRegionTrail': True, 'enableLogFileValidation': True, 'kmsKeyId': '', 'isOrganizationTrail': False}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>StartLogging</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'name': '2018trailvirclop'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>PutEventSelectors</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'trailName': '2018trailvirclop', 'eventSelectors': [{'readWriteType': 'All', 'includeManagementEvents': True, 'dataResources': []}]}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>GetBucketLocation</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202]</td>\n",
" <td>{'host': ['s3-external-1.amazonaws.com'], 'bucketName': 'apr12blah', 'location': ['']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>PutBucketPolicy</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202]</td>\n",
" <td>{'bucketName': 'apr12blah', 'bucketPolicy': {'Version': '2012-10-17', 'Statement': [{'Sid': 'AWSCloudTrailAclCheck20150319', 'Effect': 'Allow', 'Principal': {'Service': 'cloudtrail.amazonaws.com'}, 'Action': 's3:GetBucketAcl', 'Resource': 'arn:aws:s3:::apr12blah'}, {'Sid': 'AWSCloudTrailWrite20150319', 'Effect': 'Allow', 'Principal': {'Service': 'cloudtrail.amazonaws.com'}, 'Action': 's3:PutObject', 'Resource': 'arn:aws:s3:::apr12blah/AWSLogs/221470125883/*', 'Condition': {'StringEquals': {'s3:x-amz-acl': 'bucket-owner-full-control'}}}]}, 'host': ['s3-external-1.amazonaws.com'], 'policy': ['']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>GetBucketPolicy</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202]</td>\n",
" <td>{'host': ['s3-external-1.amazonaws.com'], 'bucketName': 'apr12blah', 'policy': ['']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>ListAliases</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>ListKeys</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>GetBucketLocation</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202]</td>\n",
" <td>{'host': ['s3-external-1.amazonaws.com'], 'bucketName': 'alternate-trail', 'location': ['']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>Decrypt</td>\n",
" <td>root</td>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>{'encryptionContext': {'aws:lambda:FunctionArn': 'arn:aws:lambda:us-east-1:221470125883:function:aws-cost-n-usage-S3-lambda-fn-B-2'}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>ListFunctions20150331</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'marker': 'lslTXFcbLQKkb0vP9Kgh5r+jpA0aeyLAwLuDYMsxHUCk3eiDRMkct7D8EmptWfHSXssPdS7Bo66iQPTMpVOHZgANewpgGgFGGr4pVjd6VgLUO6qPe2EMAuNDBjUTxm8z6N28yhlUwEmKbrAV/m0k5qVzizwoxFwvyruMbuMx9kADFACSslcabxXl3/jDI4rfFnIsUVdzTLBgPF1hzwrE1f3lcdkBvUp+QgY+Pn3w5QuJmwsp/di8COzFemY89GgOHbLNqsrBsgR/ee2eXoJp0ZkKM4EcBK3HokqBzefLfgR02PnfNOdXwqTlhkSPW0TKiKGIYu3Bw7lSNrLd+q3+wBGNLTnq7RWa21Xjxe5me9Q0a6jPTs31CyqFyOLTeH0v1WJ+NUgUd81xrK7Dl0fwBoREx24cCyYbzuzbfXA5BIQnSdwSan39bvldu0XUfmusIil0lvcs8U/D9m9DP0IeCFXZ8OK53ECv3aGK4gHjna7NcIzBCZx1qvZQGhMoqaMWHFa/8+fsmguhsMXYOOafV9byAGmDH+iGcchbrmuN0mItC7PiAZOFb6UdoqO09HQsRwK91gKNw9nZtMvDrGzOfQ=='}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>ListTopics</td>\n",
" <td>root</td>\n",
" <td>AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>DescribeConfigurationRecorders</td>\n",
" <td>root</td>\n",
" <td>AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>DescribeConfigurationRecorderStatus</td>\n",
" <td>root</td>\n",
" <td>AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>LookupEvents</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'lookupAttributes': [{'attributeKey': 'ReadOnly', 'attributeValue': 'False'}], 'maxResults': 5.0}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:21</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>Decrypt</td>\n",
" <td>root</td>\n",
" <td>lambda.amazonaws.com</td>\n",
" <td>{'encryptionContext': {'aws:lambda:FunctionArn': 'arn:aws:lambda:us-east-1:221470125883:function:aws-cost-n-usage-Athena-lambda-fn-C'}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>ListKeys</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>DescribeOrganization</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>DescribeTrails</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'trailNameList': [], 'includeShadowTrails': True}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>DescribeConfigurationRecorders</td>\n",
" <td>root</td>\n",
" <td>AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>LookupEvents</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'lookupAttributes': [{'attributeKey': 'ReadOnly', 'attributeValue': 'False'}], 'maxResults': 5.0}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>DescribeConfigurationRecorderStatus</td>\n",
" <td>root</td>\n",
" <td>AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>ListBuckets</td>\n",
" <td>root</td>\n",
" <td>[AWSCloudTrail, aws-internal/3 aws-sdk-java/1.11.488 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.202-b08 java/1.8.0_202]</td>\n",
" <td>{'host': ['s3-external-1.amazonaws.com']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:20</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:19</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:17</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:16</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:16</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook2-221470125883', 'sessionExpirationDurationInSeconds': 43200}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:16</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:15</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:15</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeSecurityGroups</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'securityGroupSet': {}, 'securityGroupIdSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListStacks</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'stackStatusFilter': ['UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS', 'UPDATE_COMPLETE_CLEANUP_IN_PROGRESS', 'UPDATE_IN_PROGRESS', 'DELETE_IN_PROGRESS', 'ROLLBACK_IN_PROGRESS', 'REVIEW_IN_PROGRESS', 'ROLLBACK_FAILED', 'UPDATE_ROLLBACK_IN_PROGRESS', 'UPDATE_ROLLBACK_FAILED', 'ROLLBACK_COMPLETE', 'DELETE_FAILED', 'CREATE_COMPLETE', 'UPDATE_COMPLETE', 'CREATE_FAILED', 'CREATE_IN_PROGRESS', 'UPDATE_ROLLBACK_COMPLETE']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeTags</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 10000, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeSnapshots</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'snapshotSet': {}, 'ownersSet': {}, 'sharedUsersSet': {'items': [{'user': 'self'}]}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeLoadBalancers</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'pageSize': 300}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1e'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1c'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumesModifications</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeVolumesModificationsRequest': {'MaxResults': 1000}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeAddresses</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'publicIpsSet': {}, 'filterSet': {}, 'allocationIdsSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeInstanceStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}, 'includeAllInstances': False}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook2-221470125883', 'sessionExpirationDurationInSeconds': 43200}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DeleteNotebookInstance</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook-221470125883-virclop'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 51, 'sortBy': 'Status', 'statusEquals': 'InService'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListModels</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'sortBy': 'CreationTime', 'maxResults': 100, 'creationTimeAfter': 'Feb 5, 2019 2:10:41 PM'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListHyperParameterTuningJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 100, 'sortBy': 'CreationTime', 'creationTimeAfter': 'Feb 5, 2019 2:10:41 PM'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListEndpoints</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'sortBy': 'CreationTime', 'maxResults': 51, 'creationTimeAfter': 'Feb 5, 2019 2:10:41 PM'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListLabelingJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 51, 'sortBy': 'Status', 'statusEquals': 'InProgress'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListTrainingJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 100, 'creationTimeAfter': 'Feb 5, 2019 2:10:41 PM', 'sortBy': 'CreationTime'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListTransformJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'creationTimeAfter': 'Feb 5, 2019 2:10:41 PM', 'sortBy': 'CreationTime', 'maxResults': 100}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {'items': [{'volumeId': 'vol-006d466066314092f'}]}, 'filterSet': {'items': [{'name': 'volume-status.status', 'valueSet': {'items': [{'value': 'impaired'}]}}, {'name': 'volume-status.details-name', 'valueSet': {'items': [{'value': 'io-enabled'}]}}, {'name': 'volume-status.details-status', 'valueSet': {'items': [{'value': 'failed'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {'items': [{'volumeId': 'vol-006d466066314092f'}]}, 'filterSet': {'items': [{'name': 'attachment.status', 'valueSet': {'items': [{'value': 'attaching'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeInstanceStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {'items': [{'instanceId': 'i-0caf5fdb6bd813c98'}]}, 'filterSet': {}, 'includeAllInstances': True}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {'items': [{'instanceId': 'i-0caf5fdb6bd813c98'}]}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeInstanceAttribute</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instanceId': 'i-0caf5fdb6bd813c98', 'attribute': 'disableApiTermination'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeAlarms</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'maxRecords': 100}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeInstanceCreditSpecifications</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeInstanceCreditSpecificationsRequest': {'InstanceId': {'tag': 1, 'content': 'i-0caf5fdb6bd813c98'}}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeImages</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'executableBySet': {}, 'imagesSet': {'items': [{'imageId': 'ami-43a15f3e'}]}, 'ownersSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeTags</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 10000, 'filterSet': {'items': [{'name': 'resource-id', 'valueSet': {'items': [{'value': 'i-0caf5fdb6bd813c98'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeLaunchTemplates</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeLaunchTemplatesRequest': {'MaxResults': 200}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeIdFormat</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1e'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1a'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeClassicLinkInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>ConsoleLogin</td>\n",
" <td>root</td>\n",
" <td>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.2 Safari/605.1.15</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeAvailabilityZones</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'availabilityZoneSet': {}, 'availabilityZoneIdSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>GetSessionToken</td>\n",
" <td>root</td>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>{'durationSeconds': 43200}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:14</td>\n",
" <td>DescribeLoadBalancers</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeStacks</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'stackName': 'arn:aws:cloudformation:us-east-1:221470125883:stack/cg-cwchecks-221470125883ct/711571b0-d2af-11e7-a35a-500c524294f2'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListStacks</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'stackStatusFilter': ['CREATE_IN_PROGRESS', 'UPDATE_COMPLETE', 'DELETE_FAILED', 'REVIEW_IN_PROGRESS', 'ROLLBACK_IN_PROGRESS', 'UPDATE_ROLLBACK_IN_PROGRESS', 'CREATE_COMPLETE', 'UPDATE_ROLLBACK_COMPLETE', 'UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS', 'ROLLBACK_COMPLETE', 'ROLLBACK_FAILED', 'CREATE_FAILED', 'UPDATE_ROLLBACK_FAILED', 'UPDATE_COMPLETE_CLEANUP_IN_PROGRESS', 'UPDATE_IN_PROGRESS', 'DELETE_IN_PROGRESS']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListStacks</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'stackStatusFilter': ['UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS', 'UPDATE_COMPLETE_CLEANUP_IN_PROGRESS', 'UPDATE_IN_PROGRESS', 'DELETE_IN_PROGRESS', 'ROLLBACK_IN_PROGRESS', 'REVIEW_IN_PROGRESS', 'ROLLBACK_FAILED', 'UPDATE_ROLLBACK_IN_PROGRESS', 'UPDATE_ROLLBACK_FAILED', 'ROLLBACK_COMPLETE', 'DELETE_FAILED', 'CREATE_COMPLETE', 'UPDATE_COMPLETE', 'CREATE_FAILED', 'CREATE_IN_PROGRESS', 'UPDATE_ROLLBACK_COMPLETE']}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeSecurityGroups</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'securityGroupSet': {}, 'securityGroupIdSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeTags</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 10000, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1d'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1e'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumesModifications</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeVolumesModificationsRequest': {'MaxResults': 1000}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeSnapshots</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'snapshotSet': {}, 'ownersSet': {}, 'sharedUsersSet': {'items': [{'user': 'self'}]}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeAddresses</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'publicIpsSet': {}, 'filterSet': {}, 'allocationIdsSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeInstanceStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}, 'includeAllInstances': False}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeLoadBalancers</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'pageSize': 300}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook2-221470125883', 'sessionExpirationDurationInSeconds': 43200}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>StopNotebookInstance</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook-221470125883-virclop'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListTransformJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'statusEquals': 'InProgress', 'sortBy': 'Status', 'maxResults': 51}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListTransformJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'creationTimeAfter': 'Feb 5, 2019 1:15:34 PM', 'sortBy': 'CreationTime', 'maxResults': 100}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListEndpoints</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 51, 'statusEquals': 'InService'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListHyperParameterTuningJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 100, 'sortBy': 'CreationTime', 'creationTimeAfter': 'Feb 5, 2019 1:15:34 PM'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListModels</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'sortBy': 'CreationTime', 'maxResults': 100, 'creationTimeAfter': 'Feb 5, 2019 1:15:34 PM'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListLabelingJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'creationTimeAfter': 'Feb 5, 2019 1:15:34 PM', 'maxResults': 100, 'sortBy': 'CreationTime'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListTrainingJobs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 100, 'creationTimeAfter': 'Feb 5, 2019 1:15:34 PM', 'sortBy': 'CreationTime'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 100, 'sortBy': 'CreationTime', 'creationTimeAfter': 'Feb 5, 2019 1:15:34 PM'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {'items': [{'volumeId': 'vol-006d466066314092f'}]}, 'filterSet': {'items': [{'name': 'attachment.status', 'valueSet': {'items': [{'value': 'attaching'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeInstanceStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {'items': [{'instanceId': 'i-0caf5fdb6bd813c98'}]}, 'filterSet': {}, 'includeAllInstances': True}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {'items': [{'volumeId': 'vol-006d466066314092f'}]}, 'filterSet': {'items': [{'name': 'volume-status.status', 'valueSet': {'items': [{'value': 'impaired'}]}}, {'name': 'volume-status.details-name', 'valueSet': {'items': [{'value': 'io-enabled'}]}}, {'name': 'volume-status.details-status', 'valueSet': {'items': [{'value': 'failed'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeTags</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 10000, 'filterSet': {'items': [{'name': 'resource-id', 'valueSet': {'items': [{'value': 'i-0caf5fdb6bd813c98'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeLaunchTemplates</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeLaunchTemplatesRequest': {'MaxResults': 200}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeAlarms</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'maxRecords': 100}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.483 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeClassicLinkInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeInstanceAttribute</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instanceId': 'i-0caf5fdb6bd813c98', 'attribute': 'disableApiTermination'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeIdFormat</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeImages</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'executableBySet': {}, 'imagesSet': {'items': [{'imageId': 'ami-43a15f3e'}]}, 'ownersSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeInstanceCreditSpecifications</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeInstanceCreditSpecificationsRequest': {'InstanceId': {'tag': 1, 'content': 'i-0caf5fdb6bd813c98'}}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {'items': [{'instanceId': 'i-0caf5fdb6bd813c98'}]}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1c'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1b'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeLoadBalancers</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:13</td>\n",
" <td>DescribeAvailabilityZones</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'availabilityZoneSet': {}, 'availabilityZoneIdSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:12</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:12</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:04</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:04</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeSecurityGroups</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'securityGroupSet': {}, 'securityGroupIdSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeSnapshots</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'snapshotSet': {}, 'ownersSet': {}, 'sharedUsersSet': {'items': [{'user': 'self'}]}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeInstanceStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}, 'includeAllInstances': False}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeAddresses</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'publicIpsSet': {}, 'filterSet': {}, 'allocationIdsSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeTags</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 10000, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1f'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1a'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeLoadBalancers</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'pageSize': 300}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:02</td>\n",
" <td>DescribeVolumesModifications</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeVolumesModificationsRequest': {'MaxResults': 1000}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook2-221470125883', 'sessionExpirationDurationInSeconds': 43200}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>ListInstanceProfiles</td>\n",
" <td>root</td>\n",
" <td>aws-sdk-java/1.11.483 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'pathPrefix': '/'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeInstanceAttribute</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instanceId': 'i-0caf5fdb6bd813c98', 'attribute': 'disableApiTermination'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeIdFormat</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeLaunchTemplates</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'DescribeLaunchTemplatesRequest': {'MaxResults': 200}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeInstanceStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}, 'includeAllInstances': False}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 1000, 'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeClassicLinkInstances</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'instancesSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeAddresses</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'publicIpsSet': {}, 'filterSet': {}, 'allocationIdsSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeImages</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'executableBySet': {}, 'imagesSet': {'items': [{'imageId': 'ami-43a15f3e'}]}, 'ownersSet': {}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeAlarms</td>\n",
" <td>root</td>\n",
" <td>console.amazonaws.com</td>\n",
" <td>{'maxRecords': 100}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeVolumes</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1f'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeVolumeStatus</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'volumeSet': {}, 'filterSet': {'items': [{'name': 'availability-zone', 'valueSet': {'items': [{'value': 'us-east-1b'}]}}]}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeLoadBalancers</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeTags</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'maxResults': 10000, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeAvailabilityZones</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'availabilityZoneSet': {}, 'availabilityZoneIdSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>DescribeSnapshots</td>\n",
" <td>root</td>\n",
" <td>console.ec2.amazonaws.com</td>\n",
" <td>{'snapshotSet': {}, 'ownersSet': {}, 'sharedUsersSet': {'items': [{'user': 'self'}]}, 'filterSet': {}}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>GetSessionToken</td>\n",
" <td>root</td>\n",
" <td>signin.amazonaws.com</td>\n",
" <td>{'durationSeconds': 43200}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:01</td>\n",
" <td>ConsoleLogin</td>\n",
" <td>root</td>\n",
" <td>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.2 Safari/605.1.15</td>\n",
" <td>null</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:00</td>\n",
" <td>ListNotebookInstances</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:00</td>\n",
" <td>ListNotebookInstanceLifecycleConfigs</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'maxResults': 10, 'sortBy': 'CreationTime', 'sortOrder': 'Descending'}</td>\n",
" </tr>\n",
" <tr>\n",
" <td>02-12-2019:00</td>\n",
" <td>CreatePresignedNotebookInstanceUrl</td>\n",
" <td>root</td>\n",
" <td>aws-internal/3 aws-sdk-java/1.11.481 Linux/4.9.137-0.1.ac.218.74.329.metal1.x86_64 OpenJDK_64-Bit_Server_VM/25.192-b12 java/1.8.0_192</td>\n",
" <td>{'notebookInstanceName': 'cg-notebook-221470125883-virclop', 'sessionExpirationDurationInSeconds': 43200}</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"user_investigation = True\n",
"if user_investigation :\n",
" reload(cloudtrail_helpers)\n",
" user = 'root' # change user to any user above.\n",
" endtime = datetime.datetime.now() # Create start and end time for CloudTrail lookup\n",
" interval = datetime.timedelta(hours=24)\n",
" starttime = endtime - interval\n",
" readonly = \"true\"\n",
" #events = cloudtrail_helpers.get_events_df(\"EventName\", \"ConsoleLogin\", starttime, endtime)\n",
" #eventdf = pd.DataFrame(events)\n",
"\n",
" #eventdf = cloudtrail_helpers.get_events_all_df(starttime, endtime)\n",
"\n",
" eventdf = oeventdf.copy()\n",
" display (md(\"#### The following are the users involved :\"))\n",
" print (eventdf.Username.unique().tolist())\n",
"\n",
" display (md(\"#### The following are the users available for display (by default, we are displaying EventTime, Username, UserAgent and requestParameters:\"))\n",
"\n",
" print (eventdf.columns.tolist())\n",
" eventdf = cloudtrail_helpers.get_events_df(\"Username\", user, starttime, endtime, readonly)\n",
" display (md(\"#### Service selected to analyze: {user}\".format(user=user)))\n",
" eventdf = eventdf [eventdf.Username.str.contains(user, na = False)]\n",
" eventdf.requestParameters = eventdf.requestParameters.astype(str)\n",
" eventdf = eventdf[~eventdf.requestParameters.str.contains(\"Threat\")]\n",
" sdf = eventdf[[\"EventTime\", 'EventName','Username', 'userAgent', 'requestParameters']]\n",
" display(HTML(sdf.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n",
"\n",
" sdf = eventdf.groupby( [ \"EventTime\", 'Username', 'userAgent'] )[\"EventName\"].count().unstack()\n",
"\n",
"\n",
" #sdf.plot(kind='bar', figsize=(15,15), legend=True, colormap=\"gist_rainbow\", stacked=True).legend(bbox_to_anchor=(1,1))\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='inspector-findings'></a>\n",
"## AWS Inspector Findings \n",
"The runbook tries to get the findings from Amazon Inspector network reachability scans, which analyzes for instances open to Internet. Network reachability scans do not an agent to be installed. If you have not enabled AWS inspector network scans, go to the services enablement section below to enable. \n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"An error occurred (500) when calling the ListAssessmentRuns operation (reached max retries: 4): Internal Server Error\n",
"Could not connect to the endpoint URL: \"https://inspector.eu-west-3.amazonaws.com/\"\n",
"Could not connect to the endpoint URL: \"https://inspector.eu-west-2.amazonaws.com/\"\n",
"Could not connect to the endpoint URL: \"https://inspector.sa-east-1.amazonaws.com/\"\n",
"Could not connect to the endpoint URL: \"https://inspector.ca-central-1.amazonaws.com/\"\n",
"Could not connect to the endpoint URL: \"https://inspector.ap-southeast-1.amazonaws.com/\"\n",
"Index(['arn', 'assetAttributes', 'assetType', 'attributes', 'confidence',\n",
" 'createdAt', 'description', 'id', 'indicatorOfCompromise',\n",
" 'numericSeverity', 'recommendation', 'schemaVersion', 'service',\n",
" 'serviceAttributes', 'severity', 'title', 'updatedAt',\n",
" 'userAttributes'],\n",
" dtype='object')\n"
]
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>ami_id</th>\n",
" <th>description</th>\n",
" <th>inspector_network_ami_count</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>'ami-43a15f3e'</td>\n",
" <td>On instance i-0caf5fdb6bd813c98, ENI eni-06db8b783d0ae3c50 and security group sg-0b7584f79309536...</td>\n",
" <td>18</td>\n",
" </tr>\n",
" <tr>\n",
" <td>'ami-43a15f3e'</td>\n",
" <td>On this instance, TCP port 22, which is associated with SSH, is reachable from the internet. You...</td>\n",
" <td>18</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"get_inspector_findings = True\n",
"\n",
"if get_inspector_findings : \n",
" idf = pd.DataFrame()\n",
" df_arr = []\n",
" for region in pd.DataFrame (ec2_client.describe_regions()['Regions']).RegionName.tolist():\n",
" try :\n",
" ins_client = boto3.client('inspector', region_name=region)\n",
" run_arns = ins_client.list_assessment_runs()['assessmentRunArns']\n",
" if len (run_arns) > 0 :\n",
" for run_arn in run_arns :\n",
" ins_client.describe_assessment_runs(assessmentRunArns=[run_arn])\n",
" rundf = pd.DataFrame ( ins_client.describe_assessment_runs(assessmentRunArns=[run_arn])['assessmentRuns'])\n",
" finding_arns = ins_client.list_findings( assessmentRunArns=[run_arn ])['findingArns']\n",
"\n",
" for each_finding in finding_arns :\n",
"\n",
" rdf = pd.DataFrame ( ins_client.describe_findings(findingArns=[ each_finding ]) ['findings'] )\n",
"\n",
" df_arr.append(rdf)\n",
" #asdf = pd.DataFrame (ins_client.describe_findings(findingArns=[ each_finding ])['findings'])\n",
" #display(HTML(asdf.drop_duplicates().to_html(index=False)))\n",
"\n",
"\n",
" except Exception as e: print(e)\n",
" if len(df_arr) > 0 :\n",
" idf = pd.concat(df_arr)\n",
"\n",
" #display(HTML(idf.to_html(index=False)))\n",
"\n",
"\n",
" df = idf.copy()\n",
" if df.empty:\n",
" print (\"Inspector Findings Not Available Yet ... Please run in a few minutes \")\n",
" else :\n",
" print (df.columns)\n",
" df.assetAttributes = df.assetAttributes.astype(str)\n",
" df['ami_id'] = df.assetAttributes.str.split('amiId').str.get(1).str.split(\",\").str.get(0).str.split(\":\").str.get(1)\n",
"\n",
" df['inspector_port'] = df.description.str.split(\"port\").str.get(1).str.split(',') #.str.get(0)\n",
"\n",
" df['inspector_network_ami_count'] = df.groupby('ami_id')['ami_id'].transform('count')\n",
"\n",
" df = df [['ami_id','description','inspector_network_ami_count']]\n",
" display(HTML(df.drop_duplicates().to_html(index=False)))\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='guardduty-findings'></a>\n",
"## AWS GuardDuty Findings \n",
"The run book gets all the findings from AWS guard duty and organizes them pivoting on the `ami_id` involved.\n",
"\n",
"(go to [top](#top))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Get GuardDuty Findings\n",
"Guard Duty findings pivoted for each `ami_id` . Inputs : Look back period, `num_days`\n",
"\n",
"(go to [top](#top))"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/ec2-user/anaconda3/envs/python3/lib/python3.6/site-packages/ipykernel/__main__.py:26: UserWarning: Pandas doesn't allow columns to be created via a new attribute name - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute-access\n"
]
},
{
"data": {
"text/markdown": [
"##### Download the csv of the below table [here](https://console.aws.amazon.com/s3/object/cloudgovernor-221470125883/notebooks/guardduty.csv)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"### Guard Duty : AWS API Call Activity"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>Title</th>\n",
" <th>Service</th>\n",
" <th>Region</th>\n",
" <th>Severity</th>\n",
" <th>UpdatedAt</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 34, 'DetectorId': '80b4368a1fd44954122a2bfd5dbe4581', 'EventFirstSeen': '2019-01-28T04:04:07Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>ap-south-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:16:22.456</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '80b4368a1fd44954122a2bfd5dbe4581', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:20:59.171</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 35, 'DetectorId': '9eb4368a208aade7b15e86a103eb6eea', 'EventFirstSeen': '2019-01-28T04:04:08Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>eu-west-3</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:13:58.585</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 120, 'DetectorId': '9eb4368a208aade7b15e86a103eb6eea', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:18:58.953</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 34, 'DetectorId': '50b4368a35ee003e492ad871a728cd70', 'EventFirstSeen': '2019-01-28T04:04:09Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>eu-west-2</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:14:06.858</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '50b4368a35ee003e492ad871a728cd70', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:18:59.854</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 34, 'DetectorId': '42b4368a36a1636a863196fe77d227f8', 'EventFirstSeen': '2019-01-28T04:04:09Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>eu-west-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:16:28.491</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '42b4368a36a1636a863196fe77d227f8', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:21:31.727</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 35, 'DetectorId': 'f0b4368a373fff9706f0e47d1d5d5d3d', 'EventFirstSeen': '2019-01-28T04:04:07Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>ap-northeast-2</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:15:56.319</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': 'f0b4368a373fff9706f0e47d1d5d5d3d', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:21:10.343</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 40, 'DetectorId': '1eb4368a37dbf269cd64d6bf164a612b', 'EventFirstSeen': '2019-01-28T04:04:07Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>ap-northeast-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:17:12.894</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '1eb4368a37dbf269cd64d6bf164a612b', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:22:01.138</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 28, 'DetectorId': '0cb4368a3942acdfa9d812f7fda0a9df', 'EventFirstSeen': '2019-01-28T04:04:09Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>sa-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:16:40.709</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '0cb4368a3942acdfa9d812f7fda0a9df', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:21:19.117</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 34, 'DetectorId': '2eb4368a39ead0a34ca2c76f8a64ac22', 'EventFirstSeen': '2019-01-28T04:04:06Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>ca-central-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:13:22.730</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '2eb4368a39ead0a34ca2c76f8a64ac22', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:18:52.209</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 45, 'DetectorId': 'f6b4368a3a656eb9c0a6967f2b3a2ae4', 'EventFirstSeen': '2019-01-27T21:28:55Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>ap-southeast-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:18:00.019</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': 'f6b4368a3a656eb9c0a6967f2b3a2ae4', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:22:09.431</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 34, 'DetectorId': '48b4368a3b849cd8e7000634961f1403', 'EventFirstSeen': '2019-01-28T04:04:07Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>ap-southeast-2</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:17:39.704</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '48b4368a3b849cd8e7000634961f1403', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:22:26.969</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 34, 'DetectorId': '42b4368a3c6ac0c92476cc7dd326588d', 'EventFirstSeen': '2019-01-28T04:04:10Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>eu-central-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:15:00.301</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '42b4368a3c6ac0c92476cc7dd326588d', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:19:54.116</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListNotebookInstances was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListNotebookInstances', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'sagemaker.amazonaws.com'}}, 'Archived': False, 'Count': 9140, 'DetectorId': '34b394a464d53c739be9c884b1ce96a8', 'EventFirstSeen': '2019-01-26T12:46:10Z', 'EventLastSeen': '2019-02-12T23:29:03Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 23:38:45.327</td>\n",
" </tr>\n",
" <tr>\n",
" <td>AWS CloudTrail trail arn:aws:cloudtrail:us-east-1:221470125883:trail/ThreatStackIntegrationjun5-ThreatStackTrail-NKKCU0YKSZ8D was disabled.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'DeleteTrail', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'cloudtrail.amazonaws.com'}}, 'Archived': False, 'Count': 1, 'DetectorId': '34b394a464d53c739be9c884b1ce96a8', 'EventFirstSeen': '2019-02-12T20:42:32Z', 'EventLastSeen': '2019-02-12T20:42:32Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 20:43:51.125</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListAliases was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListAliases', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'kms.amazonaws.com'}}, 'Archived': False, 'Count': 78, 'DetectorId': '6cb4368a3d1a5c7784167d16c32dd0e3', 'EventFirstSeen': '2019-01-28T04:04:06Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-2</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:13:47.650</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 187, 'DetectorId': '6cb4368a3d1a5c7784167d16c32dd0e3', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:18:40.940</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 42, 'DetectorId': '42b4368a3d45b9d1dc2f24d6c43480a0', 'EventFirstSeen': '2019-01-28T04:04:07Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-west-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:14:50.381</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '42b4368a3d45b9d1dc2f24d6c43480a0', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:20:01.487</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListFunctions20150331 was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListFunctions20150331', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'lambda.amazonaws.com'}}, 'Archived': False, 'Count': 40, 'DetectorId': '60b4368a3ddca4c58f3f264a8ba5e159', 'EventFirstSeen': '2019-01-28T04:04:08Z', 'EventLastSeen': '2019-02-12T21:07:31Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-west-2</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 21:13:31.004</td>\n",
" </tr>\n",
" <tr>\n",
" <td>API ListInstanceProfiles was invoked using root credentials.</td>\n",
" <td>{'Action': {'ActionType': 'AWS_API_CALL', 'AwsApiCallAction': {'Api': 'ListInstanceProfiles', 'CallerType': 'Remote IP', 'RemoteIpDetails': {'City': {'CityName': 'Belmont'}, 'Country': {'CountryName': 'United States'}, 'GeoLocation': {'Lat': 42.3959, 'Lon': -71.1787}, 'IpAddressV4': '173.76.14.202', 'Organization': {'Asn': '701', 'AsnOrg': 'MCI Communications Services, Inc. d/b/a Verizon Business', 'Isp': 'Verizon Fios', 'Org': 'Verizon Fios'}}, 'ServiceName': 'iam.amazonaws.com'}}, 'Archived': False, 'Count': 118, 'DetectorId': '60b4368a3ddca4c58f3f264a8ba5e159', 'EventFirstSeen': '2019-01-27T14:22:59Z', 'EventLastSeen': '2019-02-12T14:10:01Z', 'ResourceRole': 'TARGET', 'ServiceName': 'guardduty'}</td>\n",
" <td>us-east-1</td>\n",
" <td>2</td>\n",
" <td>2019-02-12 14:18:19.678</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/markdown": [
"### Guard Duty : EC2 Network Call Activity (PORT PROBEs not shown)"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Index(['ami_id', 'Title', 'Service', 'Region', 'Severity'], dtype='object')\n"
]
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th>ami_id</th>\n",
" <th>Title</th>\n",
" <th>Service</th>\n",
" <th>Region</th>\n",
" <th>Severity</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"get_guardduty_findings = True\n",
"\n",
"if get_guardduty_findings: \n",
" pd.set_option('display.max_colwidth', -1)\n",
" num_days ='7' # change the look back period\n",
"\n",
" regions = aws_helpers.get_region_array()\n",
" df_arr = []\n",
"\n",
" for region in regions :\n",
" try :\n",
" client = boto3.client('guardduty', region_name = region)\n",
" DetectorIds = client.list_detectors()['DetectorIds']\n",
" for did in DetectorIds :\n",
" fis = client.list_findings(DetectorId=did)['FindingIds']\n",
" for finding in fis:\n",
" finding_df = pd.DataFrame (client.get_findings(DetectorId=did, FindingIds= [finding] )['Findings'])\n",
" df_arr.append(finding_df)\n",
" except :\n",
" pass\n",
"\n",
" if len (df_arr) > 0 :\n",
" gdf = pd.concat(df_arr)\n",
" gdf.Resource = gdf.Resource.astype(str)\n",
" gdf.Service = gdf.Service.astype(str)\n",
" gdf.region=gdf.Region\n",
" gdf.UpdatedAt = pd.to_datetime(gdf.UpdatedAt)\n",
" gdf = gdf[gdf.UpdatedAt > datetime.datetime.now() - pd.to_timedelta(num_days+\"day\")]\n",
"\n",
" tdf = gdf.copy()\n",
"\n",
" tdf.to_csv(\"/tmp/guardduty.csv\")\n",
" link = lib_helpers.take_uploadfilename_return_link(\"guardduty.csv\", \"notebooks/guardduty.csv\")\n",
" display (md(\"##### Download the csv of the below table [here]({link})\".format(link=link) ) )\n",
"\n",
" tdf = tdf[tdf.Service.str.contains('AWS')]\n",
" tdf = tdf [['Title', 'Service', 'Region', 'Severity','UpdatedAt']]\n",
"\n",
" display (md(\"### Guard Duty : AWS API Call Activity\"))\n",
" display(HTML(tdf.drop_duplicates().to_html(index=False)))\n",
"\n",
" display (md(\"### Guard Duty : EC2 Network Call Activity (PORT PROBEs not shown)\"))\n",
" tdf = gdf.copy()\n",
" tdf = tdf[~tdf.Service.str.contains('AWS')]\n",
" tdf = tdf[~tdf.Service.str.contains('PORT')]\n",
" tdf['ami_id'] = tdf.Resource.str.split('ImageId').str.get(1).str.split(\",\").str.get(0).str.split(\":\").str.get(1)\n",
" tdf.Service = tdf.Service.str.split('ActionType').str.get(1).str.split(\",\").str.get(0).str.split(\":\").str.get(1)\n",
" tdf = tdf [['ami_id', 'Title', 'Service' , 'Region', 'Severity']]\n",
" print(tdf.columns)\n",
" #print (tdf.groupby(['Resource','Region'])['Resource'].count())\n",
" display(HTML(tdf.drop_duplicates().to_html(index=False)))\n",
" else :\n",
" print (\"Guard Duty Findings Returned Empty\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='findings-by-ami'></a>\n",
"\n",
"\n",
"### Which AMIs are more Risky ? Combining GuardDuty and Inspector Findings\n",
"This section combines findings from GuardDuty and Inspector "
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" <th>ami_id</th>\n",
" <th>description</th>\n",
" <th>inspector_network_ami_count</th>\n",
" <th>Title</th>\n",
" <th>Service</th>\n",
" <th>Region</th>\n",
" <th>Severity</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#print (df.columns)\n",
"#print (tdf.columns)\n",
"try:\n",
" if df.empty or tdf.empty :\n",
" print (\"\")\n",
"except NameError:\n",
" print (\"guardduty or inspector findings not available\")\n",
"else:\n",
" cdf = pd.merge(df,tdf, left_on = 'ami_id', right_on='ami_id')\n",
" #print (cdf.columns)\n",
" display(HTML(cdf.drop_duplicates().to_html(index=False, justify=\"left\")))\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='securityhub-findings'></a>\n",
"## AWS Security Hub Finding Analysis"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get Findings from all regions"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Could not connect to the endpoint URL: \"https://securityhub.eu-north-1.amazonaws.com/findings\"\n"
]
},
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: left;\">\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"<IPython.core.display.HTML object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"shdf = pd.DataFrame ()\n",
"try :\n",
" for region in region_arr:\n",
" client = boto3.client('securityhub',region_name = region)\n",
" sdf = pd.DataFrame (client.get_findings()['Findings'])\n",
" df_arr.append(sdf)\n",
"\n",
"\n",
" shdf = pd.concat(df_arr)\n",
" shdf['Resources'] = shdf['Resources'].dropna().astype(str)\n",
" shdf['resource_arn'] = shdf['Resources'].str.split('\\'Id\\':').str.get(1).str.split(\",\").str.get(0)\n",
" shdf['Resources'] = shdf['Resources'].dropna().astype(str)\n",
" shdf['region'] = shdf.Resources.str.split('\\'Region\\':').str.get(1).str.split(\",\").str.get(0)\n",
"\n",
" shdf = shdf [['AwsAccountId', 'Title','Compliance', 'CreatedAt','Resources' ]]\n",
" shdf = shdf [['AwsAccountId', 'Title','Compliance', 'CreatedAt','region', 'resource_arn']]\n",
"except Exception as e: print(e)\n",
"\n",
"display(HTML(shdf.to_html(index=False, justify=\"left\")))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Get Insights from all regions"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Could not connect to the endpoint URL: \"https://securityhub.eu-north-1.amazonaws.com/insights/get\"\n"
]
}
],
"source": [
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"\n",
"try :\n",
" for region in region_arr:\n",
" client = boto3.client('securityhub',region_name = region)\n",
" sdf = pd.DataFrame (client.get_insights()['Insights'])\n",
" df_arr.append(sdf)\n",
"\n",
" shdf = pd.concat(df_arr)\n",
" shdf = shdf [['AwsAccountId', 'Title','Compliance', 'CreatedAt','Resources']]\n",
" display(HTML(shdf.to_html(index=False, justify=\"left\")))\n",
"except Exception as e: print(e)\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='enable-services'></a>\n",
"\n",
"\n",
"# Enable Disable Services\n",
"\n",
"(go to [top](#top))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='cloudtrail-enable'></a>\n",
"\n",
"### CloudTrail"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"enable_ctrail = False # type True if you want to create a trail\n",
"account_id = boto3.client('sts').get_caller_identity()['Account']\n",
"\n",
"\n",
"trial_name = 'cgtrail'\n",
"trail_bucket = 'cg-'+account_id+'-trail'\n",
"cclient=boto3.client('cloudtrail')\n",
"if enable_ctrail :\n",
" ccclient.create_trail(trail_name, s3_bucket_name=trail_bucket, IsMultiRegionTrail=True, include_global_service_events=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='inspector-enable'></a>\n",
"## Inspector\n",
"\n",
"(go to [top](#top))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='inspector-enable'></a>\n",
"### Enable Network Vulns"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"enable_ins = False # True enables network reachability checks, False skips the code, will not enable network reachability checks\n",
"if enable_ins : \n",
" reload(inspector_deploy)\n",
" ec2_client = boto3.client('ec2')\n",
" for region in pd.DataFrame (ec2_client.describe_regions()['Regions']).RegionName.tolist():\n",
" inspector_deploy.network_vuln_setup(region)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='inspector-disable'></a>\n",
"### Disable Network Vulns\n",
"Note this will disable only the stuff that is enabled by this runbook. Disabled , so not doing anything."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"disable_ins = False\n",
"\n",
"if disable_ins:\n",
" ec2_client = boto3.client('ec2')\n",
" for region in pd.DataFrame (ec2_client.describe_regions()['Regions']).RegionName.tolist():\n",
" ins_client = boto3.client('inspector', region_name=region)\n",
"\n",
" try :\n",
"\n",
" if len (ins_client.list_assessment_targets(filter={ 'assessmentTargetNamePattern': 'cg-target-network' })['assessmentTargetArns']) > 0:\n",
" try :\n",
" target_arn = ins_client.list_assessment_targets( filter={ 'assessmentTargetNamePattern': 'cg-target-network' } )['assessmentTargetArns'][0] \n",
" response = ins_client.delete_assessment_target(assessmentTargetArn=target_arn) \n",
" print (target_arn, response) \n",
" # remove templates\n",
" template_arn = ins_client.list_assessment_templates(assessmentTargetArns=[ target_arn ] )['assessmentTemplateArns'][0]\n",
" response = ins_client.delete_assessment_template(assessmentTemplateArn=template_arn)\n",
" print (template_arn, response)\n",
"\n",
" assessment_arn = ins_client.list_assessment_runs(assessmentTemplateArns=[template_arn ])['assessmentRunArns'][0]\n",
" response = ins_client.delete_assessment_run(assessmentRunArn=assessment_arn)\n",
" print (\"assessment_arn\", assessment_arn, response)\n",
" except Exception as e: print(e)\n",
" except Exception as e: print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='securityhub-enable'></a>\n",
"\n",
"## Guard Duty"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Enable Guard Duty"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Enabled Guard Duty in eu-north-1\n",
"Could not connect to the endpoint URL: \"https://guardduty.eu-north-1.amazonaws.com/detector\"\n",
"Enabled Guard Duty in ap-south-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in eu-west-3\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in eu-west-2\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in eu-west-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in ap-northeast-2\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in ap-northeast-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in sa-east-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in ca-central-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in ap-southeast-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in ap-southeast-2\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in eu-central-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in us-east-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in us-east-2\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in us-west-1\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n",
"Enabled Guard Duty in us-west-2\n",
"An error occurred (BadRequestException) when calling the CreateDetector operation: The request is rejected because a detector already exists for the current account.\n"
]
}
],
"source": [
"#!pip uninstall -y \n",
"#!pip install 'botocore==1.12.61'\n",
"#!pip uninstall -y boto3\n",
"#!pip install boto3 \n",
"\n",
"import boto3\n",
"enable_gd = True\n",
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"if enable_gd:\n",
" for region in region_arr:\n",
" try :\n",
" gdclient = boto3.client('guardduty', region_name = region)\n",
" print (\"Enabled Guard Duty in \", region)\n",
" response = gdclient.create_detector( Enable=True, FindingPublishingFrequency='ONE_HOUR') #enable security hub\n",
" \n",
" except Exception as e: print(e)\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='securityhub-disable'></a>\n",
"### Disable Guard Duty"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [],
"source": [
"# aws events put-rule --name Test --event-pattern \"{\\\"source\\\":[\\\"aws.guardduty\\\"],\\\"detail-type\\\":[\\\"GuardDuty Finding\\\"],\\\"detail\\\":{\\\"severity\\\":[7.0,7.1,7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,8.0,8.1,8.2,8.3,8.4,8.5,8.6,8.7,8.8,8.9,7,8]}}\""
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"disable_security_hub = False\n",
"disable_gd = False ## Change to True\n",
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"if disable_security_hub :\n",
" for region in region_arr:\n",
" gdclient = boto3.client('guardduty', region_name = region)\n",
" if disable_gd:\n",
" try :\n",
" DetectorId = gdclient.list_detectors()['DetectorIds'].tolist()[0]\n",
" response = client.delete_detector(DetectorId=DetectorId)\n",
" response = gdclient.create_detector( Enable=True, FindingPublishingFrequency='ONE_HOUR') #enable security hub\n",
" except Exception as e: print(e)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='securityhub-enable'></a>\n",
"\n",
"## Security Hub"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Enable Security Hub"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [],
"source": [
"\n",
"\n",
"enable_sh = False\n",
"client = boto3.client('securityhub')\n",
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"if enable_sh:\n",
" for region in region_arr:\n",
" response = client.enable_security_hub() #enable security hub\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id='securityhub-disable'></a>\n",
"### Disable Security Hub"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"# aws events put-rule --name Test --event-pattern \"{\\\"source\\\":[\\\"aws.guardduty\\\"],\\\"detail-type\\\":[\\\"GuardDuty Finding\\\"],\\\"detail\\\":{\\\"severity\\\":[7.0,7.1,7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,8.0,8.1,8.2,8.3,8.4,8.5,8.6,8.7,8.8,8.9,7,8]}}\""
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"client = boto3.client('securityhub')\n",
"disable_security_hub = False ## Change to True\n",
"region_arr = aws_helpers.get_region_array()\n",
"df_arr = []\n",
"if disable_security_hub :\n",
" for region in region_arr:\n",
" if disable_security_hub:\n",
" response = client.disable_security_hub()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "conda_python3",
"language": "python",
"name": "conda_python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment