Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Creating a VPC on AWS and enabling Flow logs
________ ________ ___ __ ___
|\_____ \|\ __ \|\ \|\ \ |\ \
\|___/ /\ \ \|\ \ \ \/ /|\ \ \
/ / /\ \ __ \ \ ___ \ \ \
/ /_/__\ \ \ \ \ \ \\ \ \ \ \
|\________\ \__\ \__\ \__\\ \__\ \__\
\|_______|\|__|\|__|\|__| \|__|\|__|
Other cloud platforms such as GCP, Azure.
VPCs and Subnets - basic concepts.
A Virtual Private Cloud `VPC` is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. You can launch your AWS resources, such as Amazon EC2 instances, into your VPC. You can specify an IP address range for the VPC, add subnets, associate security groups, and configure route tables.</li>
A subnet is a range of IP addresses in your VPC. You can launch AWS resources into a specified subnet. Use a public subnet for resources that must be connected to the internet, and a private subnet for resources that won't be connected to the internet. For more information about public and private subnets, see VPC and Subnet Basics.
To protect the AWS resources in each subnet, you can use multiple layers of security, including security groups and network access control lists (ACL).

Creating Virtual Private Cloud on AWS an enabling Flow logs - Hands-on

Project Status: Concept – Minimal or no implementation has been done yet, or the repository is only intended to be a limited example, demo, or proof-of-concept.

I'm currently preparing AWS Certified Solution Architect - Associate certificate.
The following gist is intended to anyone looking to create a VPC and see it's associated logs on AWS.
We're going to be using Amazon VPC as well as an Amazon Flow Logs.
I performed this setup on my Ubuntu 18.04.2 LTS.
To check your OS version, execute $ lsb_release -a in your Terminal.

Installations

None. Just log into your AWS management console, https://console.aws.amazon.com.
You'll need to perform several tasks in your CLI regarding SSH keygen, so make sure you check the following prerequisites.

Prerequisites

First, make sure Oracle jdk is installed. I recommend java 1.8.0
To uninstall effectively your current jdk, perform this:
$ sudo apt-get remove openjdk*
$ sudo apt-get remove --auto-remove openjdk*
$ sudo apt-get purge openjdk*
$ sudo apt-get purge --auto-remove openjdk*

To install java 1.8.0, open Terminal Ctrl+Alt+T and run the command:
$ sudo add-apt-repository ppa:webupd8team/java // adds PPA repository
$ sudo apt-get update // updates package list
$ sudo apt-get install openjdk-8-jdk // installs openjdk

java-8.png
$ javmyDataBaseServermyDataBaseServerac -version // shows your new java version

Author

  • Isaac Arnault - AWS Cloud series - Related tags: #EC2 #EFS #AWSCLI #Linux

I deployed for this gist:

.EC2 instances x2: one deployed on a Public Subnet, the other on a Private Subnet
.Subnets x2 : Public, Private
.NAT gateway x1
.ACL network x1
.Route Tables x2
.VPC x1

πŸ”΅ See solution architecture

isaac-arnault-AWS-solution-architecture.png

We can skip Part 1 if we have a User and a Group already provisioned.

Otherwise, make sure to create a User and a Group via IAM, https://console.aws.amazon.com/iam/home.

Part 1 : create a User and a Group using IAM

  • We log into our AWS management console using $ https://console.aws.amazon.com.

I'm using MFA to secure my root account access coupled with Google Authenticator on my Android smartphone.

We can bypass this step and login normally to AWS Management Console.

πŸ”΄ See output

isaac-arnault-AWS-1.jpg

We go to Services > IAM > Users > Add user

  • Your VPCs > Create

  • Access type : Programmatic access

  • πŸ”΄ See output

    isaac-arnault-AWS-16.png

    Next : Permissions > Create group

  • Group name : Developers

  • Administrator Access > Create group

    πŸ”΄ See output

    isaac-arnault-AWS-17.png

    Next : Tags

  • Key: dev-1 | Value: name of the developer

  • Create user

    πŸ”΄ See output

    isaac-arnault-AWS-18.png

    Download .csv (you're going to use these credentials later on in this tutorial)

    • We write down our Access key ID and Secret access key > close the window
    πŸ”΄ See output

    isaac-arnault-AWS-28.png

    • Now in Groups we should have one group named Developers which should list user-1.
    πŸ”΄ See output

    isaac-arnault-AWS-20.png


    Part 2 : create a custom Virtual Private Cloud - VPC - on AWS

    • We log into our AWS management console using $ https://console.aws.amazon.com.

    I'm using MFA to secure my root account access coupled with Google Authenticator on my Android smartphone.

    We can bypass this step and login normally to AWS Management Console.

    πŸ”΄ See output

    isaac-arnault-AWS-1.jpg

    We go to Services > VPC > Networking & Content Delivery > VPC

    Your VPCs > Create VPC > Name Tag: myVPC > IPv4 CIDR block : 10.0.0.0/16 (it's the biggest address block possible)

    Select : Amazon provided IPv6 CIDR block > Tenancy : Default > Create

    We go to "Your VPCs" to check if "myVPC" has been created.

    πŸ”΄ See output

    isaac-arnault-AWS-50.png

    Now let's create a Subnet. Go to "Subnets".

  • VPC : choose myVPC

  • Name : myVPC - Subnet 1

  • AZ : choose us-east-1a

  • IPv4 CIDR block : 10.0.1.0/24

  • πŸ”΄ See output

    isaac-arnault-AWS-51.png

    πŸ”΄ See output

    isaac-arnault-AWS-52.png

    Let's create another subnet

  • Name : myVPC - Subnet 2

  • AZ : choose us-east-1b

  • IPv4 CIDR block : 10.0.2.0/24

  • Let's check that the two subnets (Subnet 1 and Subnet 2) have been created : go to Subnets

    πŸ”΄ See output

    isaac-arnault-AWS-53.png

    We created two subnets to have a private and public subnet.

    Let's assign a public IP address to "myVPC - Subnet 1".

    Select "myVPC - Subnet 1" > Actions > Modify auto-assign IP settings > Check : Auto-assign IPv4

    πŸ”΄ See output

    isaac-arnault-AWS-54.png

    πŸ”΄ See output

    isaac-arnault-AWS-54.png

    Let's add an Internet Gateway to our VPC.

    Click on "Internet Gateways" > Create Internet gateway > Name tag: myIG > Create

    We see that in our Internet Gateways dashboard, "myIG" State is "detached". Let's fix this.

    Select "myIG" gateway > click Actions > Attach to VPC (this will attach our IG to our VPC).

    Select "myVPC" > Attach.

    πŸ”΄ See output

    isaac-arnault-AWS-55.png

    Our IG is now attached to our VPC.

    πŸ”΄ See output

    isaac-arnault-AWS-57.png

    Important
    You can have only one IG per VPC.

    We now have to go to Route Tables to configure our main route.

    Click "Create route table" > Name tag: myPublicRoute > VPC: select myVPC > Create

    πŸ”΄ See output

    isaac-arnault-AWS-58.png

    From "myPublicRoute", let's create a route out to the Internet.

    Select "myPublicRoute", select "Routes" > clic "Edit routes" > Add route > Destination : 0.0.0.0/0 > Target : select IGW >

    Add route > Destination: ::/0 > Target

    πŸ”΄ See output

    isaac-arnault-AWS-60.png

    Select "Subnet associations" > Edit subnet associations > Select Subnet 1 to add it to your public route table > Save

    Let's check that "myPublicRoute" has a route out to the Internet. Select "myPublicRoute", click "Routes".

    πŸ”΄ See output

    isaac-arnault-AWS-61.png

    Part 3 : create one EC2 instance in our Public Subnet and one in our Private subnet

    Go to Services > EC2 > Launch Instance > let's create our 1st instance: select Amazon Linux 2 AMI

    πŸ”΄ See output

    isaac-arnault-AWS-62.png

    Select t2.micro > Next: Configure instance Details > Network : change it to myVPC

    πŸ”΄ See output

    isaac-arnault-AWS-63.png

    Next: Add Storage (leave as default) > Next : Add tags > Add Tag > Key: Name > Value: WebServer-1

    Next: Configure Security Groups > Create a new security group > SG name : mySG > Description: My EC2 Security Group

    Add rule > Select HTTP > Review and Launch > Launch >

    πŸ”΄ See output

    isaac-arnault-AWS-64.png

    Create a new Key Pair > Name : KP > Download Key Pair > Launch instances > View instances

    πŸ”΄ See output

    isaac-arnault-AWS-65.png

    Now let's create our second EC2 instance.

    Launch instance > select Amazon Linux 2 AMI > select t2.micro > Next: Configure instance Details

    Network: change it to myVPC > for the Subnet, select Subnet 2 > Next : Add Storage (leave as it is by default)

    Next: Add Tags > Key: Name, Value: myDataBaseServer > Next: Configure Security Group > Select an existing security group

    Select "default VPC security group" > Review and Launch > Choose an existing key pair: select KP > Launch instances

    Now we should have 2 running EC2 instances, one for our Public subnet and the other for our Private subnet.

    πŸ”΄ See output

    isaac-arnault-AWS-67.png

    Part 4 : let's create a new Security Group for our Database Server

    Services > EC2 > Security groups > Create security group

    Security group name: myDB-SG, Description: My Database Security Group

    VPC: choose "myVPC"

    Add Rule: select "ALL ICMP IPv4", Source: 10.0.1.0/24

    Add Rule: select "SSH", Source: 10.0.1.0/24

    Add Rule: select "HTTP", Source: 10.0.1.0/24

    Add Rule: select "HTTPS", Source: 10.0.1.0/24

    Add Rule: select "SSH", Source: 10.0.1.0/24

    Add Rule: select "MYSQL/Aurora", Source: 10.0.1.0/24

    Click Create. We now have two provisioned Security Groups, mySG and myDB-SG.

    πŸ”΄ See output

    isaac-arnault-AWS-72.png

    Go to EC2 and assign a new Securtity Group to "myDataBaseServer":

    Click on myDataBaseServer > Actions > Networking > Change Security Groups

    Unselect default VPC > Select myDB-SG > Click "Assign Security Groups"

    πŸ”΄ See output

    isaac-arnault-AWS-73.png

    Copy to the clipboard the Private IP address of "MyDatabaseServer"

    πŸ”΄ See output

    isaac-arnault-AWS-73.png

    Part 5 : Let's perform some tests

    5.1 let's connect to our EC2 instances using SSH

    Ctrl + Alt +t to open a new CLI.

    We enter the folder where we have placed our KP.pem file (we downloaded this file while setting up our first EC2 instance).

    Use the following commands:

    $ chmod 400 KP.pem - to grant access to our KP.pem

    $ ssh ec2-user@WebServer-1_IPv4_Address -i KP.pem

    Type "yes" when prompted by the CLI

    πŸ”΄ See output

    isaac-arnault-AWS-67.png

    5.1 let's see if the two EC2 instances communicate with each other

    Method 1 : ping Server 2 in SSH from Server 1

    One is on a public subnet, the other one on a private subnet.

    On your CLI, connect to WebServer-1 in SSH in order to ping myDatabaseServer

    $ ssh ec2-user@WebServer-1_IPv4_Address -i KP.pem

    $ ping myDatabaseServer_private_IP

    πŸ”΄ See output

    isaac-arnault-AWS-77.png

    Do not close your CLI.

    Method 2: copy the content of the KP.pem file

    In a new CLI, use as follows:

    Place yourself to the root of the .pem file and use $ nano KP.pem

    Copy to your clipboard the content of the file.

    Go back to your CLI (see Method 1) and use as follows :

    $ nano PVKEY.pem - to create a new file, and paste the content of KP.pem file in there.

    $ Ctrl + x then y to save and exit.

    $ chmod 400 PVKEY.pem - to grant access to your file

    Copy the IP address of "myDataBaseServer" and use it as follows:

    $ ssh ec2-user@myDatabaseServer_private_IP -i PVKEY.pem - to connect to "myDataBaseServer" from "WebServer-1"

    πŸ”΄ See output

    isaac-arnault-AWS-78.png

    Part 6 : Install a NAT gateway

    The purpose of installing a NAT gateway is to enable and allow our EC2 instances and Private subnet to go out of the VPC and download softwares.

    To do that they need to communicate to the Internet Gateway that we've installed.

    Go to Services > EC2 > Launch Instance > Go to Community AMIs > Search for "nat"

    Select "amzn-ami-vpc-nat-hvm-2018.03.0.20181116-x86_64-ebs"

    πŸ”΄ See output

    isaac-arnault-AWS-80.png

    • Next : Configure Instance Details > change Network to "myVPC"
    πŸ”΄ See output

    isaac-arnault-AWS-81.png

    • Next: Add Storage (leave as it is by default) > Next: Add Tags > Add Tag: Name, Value: NAT_Instance

    • Configure Security Groups > Select an existing Security Group > Select mySG Security Group > Review and Launch

    πŸ”΄ See output

    isaac-arnault-AWS-82.png

    • Boot from General Purpose (SSD): choose "Make General Purpose (SSD) the boot volume for this instance."

    • Launch Instance
      Choose an existing Key Pair: select "KP" > Launch Instances.

    • Check that your instance is correctly running:

    πŸ”΄ See output

    isaac-arnault-AWS-84.png

    • We need to change the Source and Destination Check on our NAT instance:

    • Click on your NAT_Instance > Actions > Networking > Change Source / Dest. Check > click "Yes, Disable"

    πŸ”΄ See output

    isaac-arnault-AWS-85.png

    To allow our EC2 instances to talk to our NAT instance, we need to create a new route on our VPC.

    Go to Services > Networking & Content Delivery > VPC > Route tables > Select 2nd Route of our VPC

    πŸ”΄ See output

    isaac-arnault-AWS-85.png

    Click Routes > Edit routes > Add route > Destination: 0.0.0.0/0 > Target: Instance > Select NAT_Instance > Save Route

    πŸ”΄ See output

    isaac-arnault-AWS-87.png

    Go to Services > EC2 > Copy to your clipboard IPv4 Public IP of WebServer-1

    SSH to that public address

    Connect to the Private IP of myDataBaseServer using SSH.

    πŸ”΄ See output

    isaac-arnault-AWS-88.png

    Let's perform a $ yum update -y while being connected in SSH to our IP adress of myDataBaseServer.

    If updates perform, it means that everything is set up correctly and that your EC2 instances use our NAT Instance to connect to the Internet.

    πŸ”΄ See output

    isaac-arnault-AWS-89.png

    Part 1 : create a Flow Logs

    • Go to Services > Under Management & Governance click "CloudWatch" > Logs > Let's get started

    • Create log group > Log Group name: VPC_FlowLogs

    πŸ”΄ See output

    isaac-arnault-AWS-90.png

    • Go to Services > VPC > Click on VPCs > Select our Custom VPC (myVPC) > Actions > Create flow log

    • Filter: select "All" > Destination log group: VPC_FlowLogs > to inform a IAM role click "Setup permissions"

    • Leave everything as it is by default and click "Allow"

    πŸ”΄ See output

    isaac-arnault-AWS-91.png

    Let's go back to our Flow Logs setup tools and select flowlogsRole.

    πŸ”΄ See output

    isaac-arnault-AWS-92.png

    Go to Management & Governance, click on "CloudWatch" > Logs > VPC_FlowLogs

    πŸ”΄ See output

    isaac-arnault-AWS-93.png

    If you have no events appearing in Log Streams, you should wait until they appear or you can try opening WebServer-1 IPv4 in your browser. This will generate logs and they should appear in your console.

    πŸ”΅ See Log Streams

    isaac-arnault-AWS-94.png

    πŸ”΅ See Logs

    isaac-arnault-AWS-95.png

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    You can’t perform that action at this time.