how-to-setup-lambda-to-talk-to-internet-and-vpc.md

How to setup AWS lambda function to talk to the internet and VPC

I'm going to walk you through the steps for setting up a AWS Lambda to talk to the internet and a VPC. Let's dive in.

So it might be really unintuitive at first but lambda functions have three states.

1. No VPC, where it can talk openly to the web, but can't talk to any of your AWS services.
2. VPC, the default setting where the lambda function can talk to your AWS services but can't talk to the web.
3. VPC with NAT, The best of both worlds, AWS services and web.

I'm gonna walk you through the steps to set up number 3.

Note: This tutorial isn't exactly in order of steps, you may need to create one thing (subnet, nat, route table) then go back into the settings for something previously created and edit it to use a newly thing.

Creating Subnets

VPC Dashboard > Subnets

This is what I had to start with, my existing vpc that I wanted to connect to already had 4 subnets. Here I noticed I had a couple of subnets already set up. Below is a totally fake ip I pulled from the internet. But the patten of increments of 16 is recreated here.

Note: DO NOT use 131.179.0.0/16 it's just an example.

VPC	CIDR
vpc-████████ (131.179.0.0/16)	131.179.0.0/20
vpc-████████ (131.179.0.0/16)	131.179.16.0/20
vpc-████████ (131.179.0.0/16)	131.179.32.0/20
vpc-████████ (131.179.0.0/16)	131.179.48.0/20

Here I created three four new subnets.

VPC	CIDR	name
vpc-████████ (131.179.0.0/16)	131.179.64.0/20	lambda-subnet-point-to-nat-1
vpc-████████ (131.179.0.0/16)	131.179.80.0/20	lambda-subnet-point-to-nat-2
vpc-████████ (131.179.0.0/16)	131.179.96.0/20	lambda-subnet-point-to-nat-3
vpc-████████ (131.179.0.0/16)	131.179.112.0/20	lambda-subnet-point-to-igw

Note: Here igw stands for Internet Gateway and nat stands for network address translation gateway (NAT Gateway).

Three of them will point to the nat and one points to the igw.

Let's create the Route Tables now.

Creating Route Tables

VPC Dashboard > Route Tables

Your going to want to set up two Route Tables.

One that points to your nat let's call this lambda-rt-to-nat:

Destination	Target
131.179.0.0/16	local
0.0.0.0/0	nat-█████████████████

One that points to your igw let's call this lambda-rt-to-igw:

Destination	Target
131.179.0.0/16	local
0.0.0.0/0	igw-████████

Your gonna want to go into each of the subnet and assign them to their corresponding route table.

subnet name	route table name
lambda-subnet-point-to-nat-1	lambda-rt-to-nat
lambda-subnet-point-to-nat-2	lambda-rt-to-nat
lambda-subnet-point-to-nat-3	lambda-rt-to-nat
lambda-subnet-point-to-igw	lambda-rt-to-igw

Set your lambda up

Lambda > Functions > my-function > Configuration > Advanced Settings

Now you want to set up your lambda function to use the subnets you created.

Setup your lambda to use your VPC.

VPC

vpc-████████ (131.179.0.0/16)

Here you setup lambda to use the subnets that point directly to your nat.

Subnets*

subnet name
lambda-subnet-point-to-nat-1
lambda-subnet-point-to-nat-2
lambda-subnet-point-to-nat-3

Create a NAT

VPC Dashboard > NAT Gateways > Create NAT Gateway

Your going to want click Create NAT Gateway and set the Subnet* to lambda-subnet-point-to-igw, and Create New EIP.

Fin

That should be it! Your lambda should be able to talk to both the VPS and the web through a NAT! Comment below if you need help or want to clarify anything here!

Links

• Essentials: Introducing VPC Support for AWS Lambda
• AWS Lambda: How to setup a NAT gateway for a lambda function with VPC access
• New – Access Resources in a VPC from Your Lambda Functions
• Configuring a Lambda Function to Access Resources in an Amazon VPC
• February 2016 Webinar Series - Introducing VPC Support for AWS Lambda

Shameless SEO terms

• amazon lambda nat
• aws lambda vpc web
• aws lambda rds and web
• aws lambda rds and http request
• lambda timeout
• AWS lambda timeout random vpc

@caitlinosh @bgarciaentornos Did either of you manage to resolve the issue of connecting to your RDS? I'm having the same issue.

[EDIT]
@caitlinosh @bgarciaentornos I've solved our problem. To allow lambdas to have internet access, RDS access and to allow the RDS to be publicly available (hopefully within a specific ip range - i.e your static IP), your RDS's subnet group must contain only public subnets; that is subnets which have a route table that points 0.0.0.0/0 to an IGW which is attached to your VPC. You must create a subnet group in Amazon RDS > Subnet Groups which contains the public subnets and then launch an RDS instance with this new subnet group (or configure your current system to these specifications).

In my case I have 6 subnets, 3 are private pointing to a nat gateway, the other three public pointing to an IGW as described above. It's important to note that as long as the Lambda function and the RDS instance are in the same VPC, then the Lambda function will have access to the RDS instance.

Guys, I've created a CloudFormation that creates the appropriate network resources - have a look at https://gist.github.com/efi-mk/d6586669a472be8ea16b6cf8e9c6ba7f

This is a wonderful guide, thank you.
One question I am pondering - I have the majority of my Lambda's NOT using the NAT, as I deploy them to a separate subnet from the NAT. That is all fine, so Lambda's that are needing S3/DynamoDB access are going out through the service endpoints. (no extra cost). I guess the Lambda's I have pointed at my NAT subnet, if they need S3 access will be incurring the extra overhead of going out over that instead of the service endpoint.

I do "love" how Amazon builds a restriction then charges you more to get around it. Running a Lambda inside a secure VPC shouldn't have to have this many hoops to jump through.

This is awesome. I have been trying to setup Lambda inside VPC with internet access for a while now. Follow this guide and everything will work flawlessly.

Thanks! 👍

Thanks for the great tutorial.

Just as a note, please be aware that VPC is free, but NAT GWs do have an hourly cost

No lucky with this.
Please if somebody have experience on this, contact me, I am willing to pay just to learn the best way to achieve this.

beautiful! thank you for putting this together.

It works! Thank you 👍

Sweet! worked for me!

For me i just added the Gateway EndPoint and Selected the S3 Service and Lambda was able to access the S3 without any issues.

@NirmalkumarV that's because it's S3, it's not an external service outside of AWS, so yeah, it's easier to setup.

But if you need access to external API, you'll need this. Otherwise the lambda within a VPC can't talk to the outside world.

@chefren Thanks! I was surprised to see it costs money. That's really not a good thing for me because it would increase the bill of my staging env a lot (NAT Gateway will cost me about 0.048/h => $30/month just for being active. This is gonna raise my AWS bill from $2 to $32 for my staging env. I hadn't see that coming and totally relates to https://gist.github.com/reggi/dc5f2620b7b4f515e68e46255ac042a7#gistcomment-2674658

I'll try it out to see how things work out, I wish it had been an on-demand pricing based.

oh wow, thanks a lot! very detailed and worked like a charm!

thank you so much!

Very helpful! Thank you

Thanks! saved my day!

Thanks a lot. This is the only method that worked.

But there is one issue that I am facing now. I am not able to access the RDS from my local machine.
I had an Aurora RDS (MySql 5.6 Compatible) running on this same VPC that I configured for lambda after following this guide. But after configuration, I couldn't connect to MySql anymore. My IP address is in the security group. I tried allowing public access in the security group but still no success. Restarted my machine, tried again, still couldn't connect. I think there is some issue with subnets.

One thing that (I think) I did differently from the mentioned steps was, the VPC was originally attached to a routing table that was attached to an IGW. I tried attaching a new routing table with VPC but couldn't do so, so I modified that Routing table, removed the existing IGW from that table and linked that table with NAT as suggested in this guide above.

I tried tweaking around but couldn't connect to MySql from my Machine. The lambda function seems to work fine though.

Do let me know if anyone can help.

Thank You.

Update:
Ok so I figured it out myself. My Lambda function already had 3 subnets attached to it. I followed the above guide and attached three more subnets as told in the guide above. But I didn't remove the already existing ones. These were the main culprit.

So I removed those three and everything started working beautifully.

Very helpful! This was better than the Amazon docs for boiling down the essential steps. Use Amazon docs for the details, this for the solution!

Ive set this all up but my function is timing out. Im trying to access KMS to decrypt my key. Any ideas? Thanks!!

If you are running small services, consider using NAT Instance instead of using NAT Gateway. You can solve the issue with cheaper price.

I have been using NAT Gateway for a year, and always thought that this is too expansive since the usage of NAT gateway isn't that high in my service :( I wish I knew about this earlier.

Thanks a lot!!!! Works for me!

This article made me understand why this works besides following the step by step. You might also find it useful.
https://medium.com/@shontauro/how-can-i-turn-my-restful-api-into-a-serverless-application-and-deploy-it-to-aws-lambda-step-by-8ff6cc97780f

Beware of NAT pricing because it's not free.

is there something on how to setup this using an vpc endpoint instead of a NAT gateway?

rel: https://stackoverflow.com/a/52994841/6323666

This saved me quite a lot of struggling - eternally grateful for the article, 10/10!

Thank you! Works for me!!!

Thanks! This works for me too 👍

And how the fuck I must to solve the issue with boto3 hangs up in lambda without this....
Nice!

Advice: Create new VPC for Lambda! Don't use your current production one, if you don't understand what is 'route tables', igw, nat or you under the risk to stop the show.

Holy crap what a mess, but it works. Just be very careful to follow the pattern for the CIDR values in the new subnets you will create. I had to inspect my existing subnets and pick up where they left off, incrementing the CIDR by 16 for each. So I wound up creating subnets with CIDR values of:

131.179.96.0/20
131.179.112.0/20
131.179.128.0/20
131.179.144.0/20

AND please be careful to note that the 131.179 portion is dummy data - you have to look at your existing subnets and get your values there.

Lastly... I tested connectivity to my RDS instance as I went through the steps. It was when I applied the new subnets to the Lambdas' VPC that they lost connectivity to RDS. So I added my previous subnets back in, and then each Lambda has 5 subnets in it's VPC assignment - three that go through to the nat and two that go to RDS. This allowed my Lambdas to hit both networks.

Thanks this helped a lot. I set it up and it worked, but then after a while I am back as before... Lambda timing out. Are there any additional configurations to be made to the NAT or subnets so that they don't sleep or something? Also, why 3 subnets pointing to NAT?

Check out a new feature on Lambda called SnapStart. That might fix the timeouts.
I haven't tried it though.