| title | date | draft |
|---|---|---|
Build and deploy a scalable website preview service on AWS Lambda in under 15 mins. |
2017-12-08 21:37:03 +0530 |
false |
Build and deploy a scalable website preview service using Node.js, Express.js, memory-cache and Up within 15 mins.
#### What is it that we'll be making? • A RESTful API service (a microservice) that will take in a website URL and reply with its title, description, a thumbnail preview of the first image found on the website along with the site name. Scrapping will be done using [nunkisoftware/link-preview](https://github.com/nunkisoftware/link-preview). It will be serverless and will run on AWS Lambda as a FaaS. Since there is no server or other hardware considerations it can scale to mammoth proportions as Amazon will automatically deploy copies of our exported functions depending on the load.
(Perhaps read up on [AWS Lambda](http://docs.aws.amazon.com/lambda/latest/dg/welcome.html) and [FaaS](https://en.wikipedia.org/wiki/Serverless_computing) to see how it works?)
• Ever came across a situation where you typed a url in a post and saw its preview below right away or when a post carried a link along with a website preview below it?
Aha! That's the one. Let's build a service that makes this possible in under 15 mins!
Gather the tools we need.
• First, install Up globally.
$ npm i -g up
• Then, initialise our project by creating these files.
$ touch package.json up.json app.js
• Then, add a few local packages.
$ npm i express memory-cache @nunkisoftware/link-preview --save
• Add a scripts section to your package.json so Up knows how to start your express server.
https://gist.github.com/8271a18e54b2407253c2c97c644190ba
• Write an express server inside app.js with a single GET endpoint at / which would take a query param url. This would be our website url whose preview we require.
https://gist.github.com/abf78e1be2af1f4e53e3cf917a33f492
Please note that we also employ an in memory cache to store recent website previews so we don't query link-preview on every frequent homogenous request (as that's a time/resource expensive thing to do) and thus serve the cached result to our client.
The following two steps can also be accomplished using environment variables but making a separate file is much cleaner and will make our deployment super easy by writing a single command, Up.
• Add a single entry to our up.json so Up knows where and how to find our AWS credentials.
https://gist.github.com/8be4417094a90a124c381819ee128848
This is a one time step and won't be required for subsequent Up deployments.
• Finally, create the aws credentials file at `~/.aws/` and fill in your IAM credentials.
$ mkdir -p ~/.aws && touch ~/.aws/credentials
$ gedit ~/.aws/credentials or $ nano ~/.aws/credentials and paste the following in.
Replace $YOUR_ACCESS_ID and $YOUR_ACCESS_KEY with your own. Find them from here. It could be beneficial to create a new IAM user just for this purpose.
https://gist.github.com/da965561ce633ba7cc3d5e69e18f1a51
Save the file and that's it.
To verify our installation, key in npm start from the directory that houses our app.js.
From another terminal window, request our service like so.
$ curl localhost:3000?url=https://www.youtube.com/watch?v=NUWViXhvW3k
You should see a familiar JSON and this verifies our installation.
https://gist.github.com/f627db6611dd057e7d9554558eaa271a
Inside the same directory, deploy the service with a single command.
## `$ up`
After the deployment is complete, get the service's URL like so.
$ up url
The url with the query param could look similar to this.
https://hfnuua77fd.execute-api.us-west-2.amazonaws.com/development?url=https://www.youtube.com/watch?v=NUWViXhvW3k
And there you have it, your own serverless and scalable website preview service built and deployed in under 15 mins.
Query with your favourite http client inside any application.
Documentation for Up can be found here.
I've also added a repository as a reference to this article and is located here https://github.com/MustansirZia/serverless-link-preview.