erwindev/scalable-web.md

## scalable-web.md

      
    Raw
  

              scalable-web.md
            
          
    Strategies for a Scalable Web Application


Scale Up vs Scale Out

Scale Up (Vertical Scaling)


Add more CPU’s or add faster CPU’s to a single server

Scale Out (Horizontal Scaling)


Add more servers so we can distribute the load to these servers

Advice: Because a single server capacity is finite, choose a scale out strategy over scale up.  In this scenario, the load is distributed across the different servers.
To take advantage of these solution, make applications stateless because it is much easier to spin up applications without any state in it.  Additionally, consider a micro services architecture in order to scale out only resources that need to get scaled.
Stateless over Stateful

Stateless applications


The application do not retain session information or status about the client

Stateful applications


Session information is tied to an instance of the application.  As such, succeeding requests are tied to the application instance where the session is created and stored.

Advice: Stateless applications are preferred over stateful applications because stateless applications can take advantage of the Scale Up strategy with simple round-robin load balancing.  A stateful application will work with Scale Up strategy.  However, the load balancer will have to use sticky session load balancing strategy.  Furthermore, if the system do not have the ability to replicate sessions and a server goes down, the sessions in the affected server is gone, affecting our users.
Scaling the Database

There are number of strategies that we can apply depending on the type of database that the application uses.
RDBMS (MariaDB, PostgreSQL)


Master/Read Replica

The database can be scaled by having a master/read replica setup with read/writes on the master database and most reads happen to the read replica database.


Database Splitting

In addition to a master/read replica setup, a database splitting strategy can also be applied.  The database can be split vertically (partitioning) or horizontally (sharding)
With partitioning, the database can be split into multiple databases base on domain.  For example: Student Database, College Database, Major Database, Scholarship Database, etc.
With sharing, the database can be split into multiple database based on some attribute like demographics.  For example, California Student Database, Florida Student Database, etc.


NoSQL (MongoDB, DynamoDB)


If the application requires a super low-latency, then consider NoSQL database.
If there is no relational data, use a NoSQL database.
If there is a need to work on large amounts of semi-structured or unstructured data.
With MongoDB, scaling out via sharing is easier.

Cache (Redis)


Any data that is worthy of storing in a cache, can be stored in Redis.  This in-memory key-value store allows faster recovery of data.

Microservice Architecture

Microservice architecture is aligned with teams’ Agile and DevOps goals.  It facilitates building of software in a much faster cycle because it allows the teams to independently focus on a certain part of the system.  It also facilitates continuous delivery and deployment of large, complex applications.  It allows teams to isolate part of the system thus allowing redeploy of parts of the application without affecting other parts of the application.  Because a micro service architecture is able to isolate parts of the system, it minimizes the risk of affecting the entire application during a micro service issue.
However, not of all these benefits come for free.  Distributed system is complex.  Each service in a micro services architecture is an independent service where communication between these services is now happen through the network calls.  Because networks are not reliable, engineers need to factor in patterns that will make their applications resilient (Circuit Breaker, Retry, Bulkheads, etc).  Additionally, it is more difficult to debug and trace issues in a micro services architecture.  Engineers have to implement a different logging mechanism where logs with unique transaction ids’ are sent to a central system like logstash.
Event Driven Architecture

A micro services architecture fosters a loosely coupled complex system.  An event driven architecture is well suited for these types of systems.  Micro services can remain autonomous.  They talk to each other by sending events (messages) through a broker (queue).  Services can subscribe to these events from the queue.  Because services talk to each other via events sent to the queue, a request can be processed asynchronously.  This type of processing improves the scalability of the system.
Single Page Applications

Single Page Applications create a better user experience for users.    But from a architecture standpoint, a single page application creates a true separation of concern.  The SPA focuses more on the user interface logic thus alleviating the services from these type of processing.  Additionally, it also allows teams to work on the system in parallel.  As long as the API contract between the client and the service have been defined, the SPA engineers can move forward in developing the front piece of the system.  Once the service is ready, the integration between the frontend and the service would be seamless.
Things to consider:

For requests that will be long running, allow the ability to asynchronously process this request.  The backend service (service A) will process this request asynchronously by submitting an event/message to a queue and returning a callback URL to the client.  Another service (service B) will pickup these event and perform the long running work.  Once the work is done, service B will post the result (typically a message payload) to Redis.  The client will poll the callback URL for the results that are stored in Redis.
Polling is not very efficient.  WebSockets are more efficient.  Therefore, use WebSockets if allowed.

“Function as a Service”

Setting up micro services are difficult.  Typically, application running in a micro service need to be built into a Docker image and pushed out to a Docker repo.  To deploy a micro service, a container platform like Kubernetes or Rancher is needed.  Installing and maintaining a container platform is difficult and requires specific skillsets.  So, to avoid these complexities, a new technology called “Serverless” or “Function as a Service” is gaining some ground.  With “Function as a Service”, the engineers just deploys code to these “Serverless” platform and the platform manages and runs the application for the engineers.  In Amazon Web Services, we call this Lambda Functions.