Creating a VPC on AWS and enabling Flow logs

.gitignore

________  ________  ___  __    ___     
|\_____  \|\   __  \|\  \|\  \ |\  \    
 \|___/  /\ \  \|\  \ \  \/  /|\ \  \   
     /  / /\ \   __  \ \   ___  \ \  \  
    /  /_/__\ \  \ \  \ \  \\ \  \ \  \ 
   |\________\ \__\ \__\ \__\\ \__\ \__\
    \|_______|\|__|\|__|\|__| \|__|\|__|
                                                         
                                        

Other cloud platforms such as GCP, Azure.

VPCs and Subnets - basic concepts.

A Virtual Private Cloud `VPC` is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. You can launch your AWS resources, such as Amazon EC2 instances, into your VPC. You can specify an IP address range for the VPC, add subnets, associate security groups, and configure route tables.</li>

A subnet is a range of IP addresses in your VPC. You can launch AWS resources into a specified subnet. Use a public subnet for resources that must be connected to the internet, and a private subnet for resources that won't be connected to the internet. For more information about public and private subnets, see VPC and Subnet Basics.

To protect the AWS resources in each subnet, you can use multiple layers of security, including security groups and network access control lists (ACL).

README.md

Creating Virtual Private Cloud on AWS an enabling Flow logs - Hands-on

I'm currently preparing AWS Certified Solution Architect - Associate certificate.
The following gist is intended to anyone looking to create a VPC and see it's associated logs on AWS.
We're going to be using Amazon VPC as well as an Amazon Flow Logs.
I performed this setup on my Ubuntu 18.04.2 LTS.
To check your OS version, execute $ lsb_release -a in your Terminal.

Installations

None. Just log into your AWS management console, https://console.aws.amazon.com.
You'll need to perform several tasks in your CLI regarding SSH keygen, so make sure you check the following prerequisites.

Prerequisites

First, make sure Oracle jdk is installed. I recommend java 1.8.0
To uninstall effectively your current jdk, perform this:
$ sudo apt-get remove openjdk*
$ sudo apt-get remove --auto-remove openjdk*
$ sudo apt-get purge openjdk*
$ sudo apt-get purge --auto-remove openjdk*

To install java 1.8.0, open Terminal Ctrl+Alt+T and run the command:
$ sudo add-apt-repository ppa:webupd8team/java // adds PPA repository
$ sudo apt-get update // updates package list
$ sudo apt-get install openjdk-8-jdk // installs openjdk


$ javmyDataBaseServermyDataBaseServerac -version // shows your new java version

Author

• Isaac Arnault - AWS Cloud series - Related tags: #EC2 #EFS #AWSCLI #Linux

VPC_Architecture.md

I deployed for this gist:

.EC2 instances x2: one deployed on a Public Subnet, the other on a Private Subnet
.Subnets x2 : Public, Private
.NAT gateway x1
.ACL network x1
.Route Tables x2
.VPC x1

🔵 See solution architecture

VPC_AWS.md

We can skip Part 1 if we have a User and a Group already provisioned.

Otherwise, make sure to create a User and a Group via IAM, https://console.aws.amazon.com/iam/home.

Part 1 : create a User and a Group using IAM

• We log into our AWS management console using $ https://console.aws.amazon.com.
  

I'm using MFA to secure my root account access coupled with Google Authenticator on my Android smartphone.

We can bypass this step and login normally to AWS Management Console.

🔴 See output

We go to Services > IAM > Users > Add user

Your VPCs > Create

Access type : Programmatic access

🔴 See output

Next : Permissions > Create group

Group name : Developers

Administrator Access > Create group

🔴 See output

Next : Tags

Key: dev-1 | Value: name of the developer

Create user

🔴 See output

Download .csv (you're going to use these credentials later on in this tutorial)

• We write down our Access key ID and Secret access key > close the window
  

🔴 See output

• Now in Groups we should have one group named Developers which should list user-1.

🔴 See output

---

Part 2 : create a custom Virtual Private Cloud - VPC - on AWS

• We log into our AWS management console using $ https://console.aws.amazon.com.
  

I'm using MFA to secure my root account access coupled with Google Authenticator on my Android smartphone.

We can bypass this step and login normally to AWS Management Console.

🔴 See output

We go to Services > VPC > Networking & Content Delivery > VPC

Your VPCs > Create VPC > Name Tag: myVPC > IPv4 CIDR block : 10.0.0.0/16 (it's the biggest address block possible)

Select : Amazon provided IPv6 CIDR block > Tenancy : Default > Create

We go to "Your VPCs" to check if "myVPC" has been created.

🔴 See output

Now let's create a Subnet. Go to "Subnets".

VPC : choose myVPC

Name : myVPC - Subnet 1

AZ : choose us-east-1a

IPv4 CIDR block : 10.0.1.0/24

🔴 See output

🔴 See output

Let's create another subnet

Name : myVPC - Subnet 2

AZ : choose us-east-1b

IPv4 CIDR block : 10.0.2.0/24

Let's check that the two subnets (Subnet 1 and Subnet 2) have been created : go to Subnets

🔴 See output

We created two subnets to have a private and public subnet.

Let's assign a public IP address to "myVPC - Subnet 1".

Select "myVPC - Subnet 1" > Actions > Modify auto-assign IP settings > Check : Auto-assign IPv4

🔴 See output

🔴 See output

Let's add an Internet Gateway to our VPC.

Click on "Internet Gateways" > Create Internet gateway > Name tag: myIG > Create

We see that in our Internet Gateways dashboard, "myIG" State is "detached". Let's fix this.

Select "myIG" gateway > click Actions > Attach to VPC (this will attach our IG to our VPC).

Select "myVPC" > Attach.

🔴 See output

Our IG is now attached to our VPC.

🔴 See output

Important
You can have only one IG per VPC.

We now have to go to Route Tables to configure our main route.

Click "Create route table" > Name tag: myPublicRoute > VPC: select myVPC > Create

🔴 See output

From "myPublicRoute", let's create a route out to the Internet.

Select "myPublicRoute", select "Routes" > clic "Edit routes" > Add route > Destination : 0.0.0.0/0 > Target : select IGW >

Add route > Destination: ::/0 > Target

🔴 See output

Select "Subnet associations" > Edit subnet associations > Select Subnet 1 to add it to your public route table > Save

Let's check that "myPublicRoute" has a route out to the Internet. Select "myPublicRoute", click "Routes".

🔴 See output

Part 3 : create one EC2 instance in our Public Subnet and one in our Private subnet

Go to Services > EC2 > Launch Instance > let's create our 1st instance: select Amazon Linux 2 AMI

🔴 See output

Select t2.micro > Next: Configure instance Details > Network : change it to myVPC

🔴 See output

Next: Add Storage (leave as default) > Next : Add tags > Add Tag > Key: Name > Value: WebServer-1

Next: Configure Security Groups > Create a new security group > SG name : mySG > Description: My EC2 Security Group

Add rule > Select HTTP > Review and Launch > Launch >

🔴 See output

Create a new Key Pair > Name : KP > Download Key Pair > Launch instances > View instances

🔴 See output

Now let's create our second EC2 instance.

Launch instance > select Amazon Linux 2 AMI > select t2.micro > Next: Configure instance Details

Network: change it to myVPC > for the Subnet, select Subnet 2 > Next : Add Storage (leave as it is by default)

Next: Add Tags > Key: Name, Value: myDataBaseServer > Next: Configure Security Group > Select an existing security group

Select "default VPC security group" > Review and Launch > Choose an existing key pair: select KP > Launch instances

Now we should have 2 running EC2 instances, one for our Public subnet and the other for our Private subnet.

🔴 See output

Part 4 : let's create a new Security Group for our Database Server

Services > EC2 > Security groups > Create security group

Security group name: myDB-SG, Description: My Database Security Group

VPC: choose "myVPC"

Add Rule: select "ALL ICMP IPv4", Source: 10.0.1.0/24

Add Rule: select "SSH", Source: 10.0.1.0/24

Add Rule: select "HTTP", Source: 10.0.1.0/24

Add Rule: select "HTTPS", Source: 10.0.1.0/24

Add Rule: select "SSH", Source: 10.0.1.0/24

Add Rule: select "MYSQL/Aurora", Source: 10.0.1.0/24

Click Create. We now have two provisioned Security Groups, mySG and myDB-SG.

🔴 See output

Go to EC2 and assign a new Securtity Group to "myDataBaseServer":

Click on myDataBaseServer > Actions > Networking > Change Security Groups

Unselect default VPC > Select myDB-SG > Click "Assign Security Groups"

🔴 See output

Copy to the clipboard the Private IP address of "MyDatabaseServer"

🔴 See output

Part 5 : Let's perform some tests

5.1 let's connect to our EC2 instances using SSH

Ctrl + Alt +t to open a new CLI.

We enter the folder where we have placed our KP.pem file (we downloaded this file while setting up our first EC2 instance).

Use the following commands:

$ chmod 400 KP.pem - to grant access to our KP.pem

$ ssh ec2-user@WebServer-1_IPv4_Address -i KP.pem

Type "yes" when prompted by the CLI

🔴 See output

5.1 let's see if the two EC2 instances communicate with each other

Method 1 : ping Server 2 in SSH from Server 1

One is on a public subnet, the other one on a private subnet.

On your CLI, connect to WebServer-1 in SSH in order to ping myDatabaseServer

$ ssh ec2-user@WebServer-1_IPv4_Address -i KP.pem

$ ping myDatabaseServer_private_IP

🔴 See output

Do not close your CLI.

Method 2: copy the content of the KP.pem file

In a new CLI, use as follows:

Place yourself to the root of the .pem file and use $ nano KP.pem

Copy to your clipboard the content of the file.

Go back to your CLI (see Method 1) and use as follows :

$ nano PVKEY.pem - to create a new file, and paste the content of KP.pem file in there.

$ Ctrl + x then y to save and exit.

$ chmod 400 PVKEY.pem - to grant access to your file

Copy the IP address of "myDataBaseServer" and use it as follows:

$ ssh ec2-user@myDatabaseServer_private_IP -i PVKEY.pem - to connect to "myDataBaseServer" from "WebServer-1"

🔴 See output

Part 6 : Install a NAT gateway

The purpose of installing a NAT gateway is to enable and allow our EC2 instances and Private subnet to go out of the VPC and download softwares.

To do that they need to communicate to the Internet Gateway that we've installed.

Go to Services > EC2 > Launch Instance > Go to Community AMIs > Search for "nat"

Select "amzn-ami-vpc-nat-hvm-2018.03.0.20181116-x86_64-ebs"

🔴 See output

• Next : Configure Instance Details > change Network to "myVPC"
  

🔴 See output

• Next: Add Storage (leave as it is by default) > Next: Add Tags > Add Tag: Name, Value: NAT_Instance
  

• Configure Security Groups > Select an existing Security Group > Select mySG Security Group > Review and Launch
  

🔴 See output

• Boot from General Purpose (SSD): choose "Make General Purpose (SSD) the boot volume for this instance."
  

• Launch Instance
  Choose an existing Key Pair: select "KP" > Launch Instances.
  

• Check that your instance is correctly running:
  

🔴 See output

• We need to change the Source and Destination Check on our NAT instance:
  

• Click on your NAT_Instance > Actions > Networking > Change Source / Dest. Check > click "Yes, Disable"

🔴 See output

To allow our EC2 instances to talk to our NAT instance, we need to create a new route on our VPC.

Go to Services > Networking & Content Delivery > VPC > Route tables > Select 2nd Route of our VPC

🔴 See output

Click Routes > Edit routes > Add route > Destination: 0.0.0.0/0 > Target: Instance > Select NAT_Instance > Save Route

🔴 See output

Go to Services > EC2 > Copy to your clipboard IPv4 Public IP of WebServer-1

SSH to that public address

Connect to the Private IP of myDataBaseServer using SSH.

🔴 See output

Let's perform a $ yum update -y while being connected in SSH to our IP adress of myDataBaseServer.

If updates perform, it means that everything is set up correctly and that your EC2 instances use our NAT Instance to connect to the Internet.

🔴 See output

VPC_FlowLogs.md

Part 1 : create a Flow Logs

• Go to Services > Under Management & Governance click "CloudWatch" > Logs > Let's get started
  

• Create log group > Log Group name: VPC_FlowLogs
  

🔴 See output

• Go to Services > VPC > Click on VPCs > Select our Custom VPC (myVPC) > Actions > Create flow log
  

• Filter: select "All" > Destination log group: VPC_FlowLogs > to inform a IAM role click "Setup permissions"
  

• Leave everything as it is by default and click "Allow"
  

🔴 See output

Let's go back to our Flow Logs setup tools and select flowlogsRole.

🔴 See output

Go to Management & Governance, click on "CloudWatch" > Logs > VPC_FlowLogs

🔴 See output

If you have no events appearing in Log Streams, you should wait until they appear or you can try opening WebServer-1 IPv4 in your browser. This will generate logs and they should appear in your console.

🔵 See Log Streams

🔵 See Logs