People use aws as a virtual datacenter, so they deploy various services there like EC2 instances, RDS instance etc.
Now these instances/machines need to live somewhere in the datacenter and typically need to connect to internet. This is where VPC comes into the picture. I will use the term instance interchangebly with machine throughout this post. By instance, I mean a deployment of a service.
Before checking out what all facilities does a VPC provide us, lets understand the terms region, avalability zones
AWS has different regions, where it has it data centeres. In those regions we have availability zones. So if North Virginia is a region with id us-east-1, we have availability zones (AZs) us-east-1a, us-east-1b, us-east-1c. They provide multiple AZs for resilience, so if one AZ is having some issues, deployment in other AZs won't be affected. So you deploy your instances in a particular AZ which in turn belongs to a particular region.
As you can see in the image above, VPC is an isolated network in a region, it spans all the AZs in that region.
When you create an aws account, it gives you a default vpc. I will explain what all things a default VPC offers, then I will show you how you can create one yourself.
So, a default VPC basically gives you 4 things -
- IP address ranges/CIDR block
- Subnets in availabity zones
- Routers to route out to the internet
- Security groups and Network access control lists (NACLs) for security
So, I will be talking about each of these 4 topics - I will be talking about -
- IP addressing
- Creating Subnets
- Routing
- Security
In some places, I will also be explaining some common networking terms like CIDR blocks etc. If you are familiar with them, feel free to skip that part.
In the default VPC, aws gives you some range of IP addresses so that anything that will be deployed in that VPC will get an IP address from that IP address range. We denote IP address ranges with a CIDR block. For example aws gives you 172.31.0.0/16 range of addresses for the default VPC.
Lets understand this CIDR block -
Open any IP calculator like http://jodies.de/ipcalc and calculate the IP ranges that belong to this CIDR block.
As you can see in the image, here 16 is the netmask, saying first 16 bits in the addressses are constant (here equal to 172.31), so that gives us option to change rest 16 bits. By changing the last 16 bits i.e. last two blocks of 8 bits (172.31.x.x) we can generate 65536 (255*255+1) IP addresses.
Thats how you represent IP address range with CIDR blocks.
So, In the address range 172.31.0.0/16 (172.31 describes the network because of the 16 netmask, and 0.0 part will describe the host i.e. the actual system on the network)
Also 172.31.0.0/16 falls within RFC 1918 ranges, RFC 1918 are reserved IP address ranges for private networks, so they don't exist publically on the internet. So if you create something in your VPC, it is not going to conflict with anything on the internet. If you want to talk to something on the internet you can't have an IP address conflict. Even when you have two service talking to each other on premise, both of those services' VPCs should have different CIDR ranges so that IP addresses of instances inside them don't overlap, because overlapping addresses can't talk to each other easily.
So, when you create a VPC, you specify a CIDR block (saying all the machines that will be deployed in this VPC will receive IP address from this range).
Subnets are sub networks inside a VPC. As we know VPC is specific to a region, subnets are specific to availability zones.
For example if a VPC belongs to a region us-east-1, this region will have AZs us-east-1a, us-east-1b, us-east-1c etc.
Now you can assign a subnet to each of these AZs, so that machines in these AZs can talk to each other across AZs. Subnets receive IP address ranges that are subdivisions of the IP adrress/cidr range VPC has been assigned. For example - vpc - 172.31.0.0/16 subnets - 172.31.0.0/24, 172.31.1.0/24, 172.31.2.0/24 so here again looking at the network mask in each subnet (24) first three numbers denote the network (172.31.x) and last number denotes the machine, so here we can have about 250 (2^8-1 = 255, but 5 of them are reserved by aws for its internal networking) machines in each subnet.
Machines are launched in a specific Subnet. So, machines receive IP addresses from the address range of Subnets and Subnets receive address range from the address range of the VPC. Needless to say, each Subnet belongs to a VPC.
We are talking about IPV4 addresses here, You can also configure the machines to receive IPV6 addresses.
To talk between the addresses, we need to be able to route and routing table tells the system how to move the packets around, where to put the next packet.
Basically, a routing table contains routes. A route contains destination - target mapping.
Destination—The destination CIDR where you want traffic from your subnet to go. For example, an external corporate network with a 172.16.0.0/12 CIDR.
Target—The target through which to send the destination traffic; for example, an internet gateway.
AWS gives you a default Main route table for every VPC. You can create custom route tables and also associate them with a subnet (for example when you don't want any data to be allowed to leave a subnet). AWS also creates a default route for communication within the VPC.
In this image, the first route is the default route 172.31.0.0/16 - local , It says any traffic that is destined to an IP address belonging to this CIDR block should be routed internally/locally, or that is destined to somewhere inside this VPC.
Second route 0.0.0.0/0 - igw-5a67df33 says that traffic destined to any other IP address should be routed through Internet gateway (igw-5a67df33 is its id). 0.0.0.0/0 denotes all IPV4 addresses, so if any other entry in your route table doesn't handle the traffic, this route entry will. So you can say routing table is used in order of the most specific to the least specific route rule.
Every VPC has a default route table that says everything thats inside my CIDR range, (in any of the AZs) is inside my VPC, they can talk to one another. So considering the CIDR ranges we assigned to a subnet earlier, say 172.31.0.0/24, Now if a request comes for 172.31.0.1 it will be routed internally on local network.
Thats about communication within a VPC. How do you connect machines in that VPC to the internet?
To connect to the internet, you need -
- some form of connection to the internet
- some route to the internet
- some public address (172.31.x.x we are using in our example is a private address, not available on the internet)
For some machine to connect to internet, It has to be present in a public subnet.
what is a public subnet?
AWS doesn't let you create private or public subnet as such. A subnet is public or private based on its routing table.
If a subnet allows outgoing requests (to internet) by connecting to the internet gateway, it is a public subnet.
If a subnet blocks outgoing requests to internet or redirect them to a NAT gateway, it is a private subnet.
But why do you need to connect the private subnet to a NAT gateway?
For use cases where your machine wants to remain private but needs to fetch something from the internet (say to download updates). For that you create a subnet, you don't create a route to connect it to internet gateway, but if needed you rather connect it to a NAT gateway. NAT gateway is a one way valve, it only allows responses to the requests that come from inside, rejects outside (from internet) requests. NAT gateway is created in a public subnet and connected to Internet gateway (igw). So it acts as a proxy between your private subnet and the internet gateway.
Now to deal with the 3 points mentioned above.
- public IP address - create the instance in a public subnet, Also while creating the instance check the option to assign it a public (elastic) IP address (say 198.51.100.3). It will still have a private address as well.
- need a connection to the internet - create an Internet gateway and connect it to the VPC.
- route - create a route
0.0.0.0/0 - igw_idin the public subnet's route table. When you connect the subnet to igw (Internet gateway) this machine having address 198.51.100.3 can talk to anything on the internet and anything on the intenet can talk to it as well, like a typical web server.
Here we will discuss 3 things -
- Security groups
- Network access control lists (NACLs)
- Flow logs (for monitoring, analyzing network traffic, dubugging network calls)
Let's first see, why do we need security? (specially Security groups here)
Consider this case, where you have one logical group for web servers and one for backend. Now we have a obnvious constraint, things on the internet should be allowed to talk to the web server and only web server group should be allowed to talk to the backend group.
- web server will have allow 0.0.0.0/0 traffic rule.
- backend will have only allow webservers group traffic rule.
So here we can assign one security group to webservers intances and another security group to backend instances.
As is evident from the example above, security groups are distributed firewalls and they are specific to an instance/machine.
Also, Security groups are stateful.
Stateful means if a request comes from one direction, it automatically sets permission for the response to the request that comes from the other direction.
But that doesn't mean if you created an inbound access rules then you don't need outbound access rules.
what are inbound and outbound rules?
The inbound rules govern how externally initated connections are handled, such as serving HTTP requests for instance. The outbound rules govern how internally initiated connections are handled, such as fetching server updates with yum or apt.
Stateful nature of security groups can be compared to a router (Not exactly though). The router protects your computer from the evils of the Internet. Traffic can't come through your router to your computer. However, if you make a request to go to a website, that request goes out of your router and the response is allowed back in because it is stateful. That is, the router remembers that you made the request to that website and it permits the response to come back to your computer.
NACLs gives you option to set ALLOW/DENY traffic rules. But they are specific to a subnet. NACLs are stateless, So you will have to explicity allow traffic rules in both directions. Please refer to aws documentation for when to use security groups and when to use NACL.
For example, you can use NACLs when you want a rule to allow/deny traffic from a particular network or range of IP addresses. NACLs are relatively short compared to security groups. So, If you are writing complex rules around ports and IP addresses, you should probably use security groups.
Security Group act as a firewall for associated Amazon EC2 instances, controlling both inbound and outbound traffic at the instance level, while ACLs act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level.
Flow logs work at VPC, subnet, instance level. You can dump the network calls to s3 or to cloudwatch and analyze the traffic flow. configure in you vpc (you will find a flow logs tab) Flow logs basically contain network interface, source IP address and port, destination IP address and port, bytes, accept/reject
Now you can debug you traffic rules, like an inbound ssh request is getting rejected so you should check if you have setup that rule.
You also have dns options in VPC to map those IP addresses to some names. Check out Route53 for details.
You can check AWS direct connect, transit gateway for building inter-vpn connections.