Skip to content

Instantly share code, notes, and snippets.

@1UC1F3R616
Last active May 5, 2024 08:03
Show Gist options
  • Save 1UC1F3R616/3ff1061f93c33307d1ba74bd75a51e4c to your computer and use it in GitHub Desktop.
Save 1UC1F3R616/3ff1061f93c33307d1ba74bd75a51e4c to your computer and use it in GitHub Desktop.
Component Functionality Where Used Examples
API Gateway Manages all client requests, provides a single point of entry, handles routing, authentication, rate-limiting, and API metering. The first point of entry between users and backend services. Kong, Amazon API Gateway, Apigee
Load Balancer Distributes incoming network traffic across multiple servers to ensure no single server bears too much demand. In front of server clusters to distribute client requests. NGINX, HAProxy, AWS Elastic Load Balancing
Web Server Serves web pages to clients in response to HTTP requests. Handles the basic delivery of web content. Hosting and serving static and dynamic web pages. Apache HTTP Server, NGINX, Microsoft IIS
Database Stores and manages data. Provides data querying capabilities. Backend, handling data storage, and management. MySQL, PostgreSQL, MongoDB, Oracle Database
Caching System Temporarily stores frequently accessed data to speed up data retrieval and reduce database load. In front of databases or within services for quick access. Redis, Memcached, Varnish
Message Broker Facilitates message-driven communication between different components or services, often used for decoupling services in a microservice architecture. Between services for asynchronous communication. RabbitMQ, Apache Kafka, Amazon SQS
Microservices Small, independently deployable services that work together. Each service performs a specific function. Throughout the system, handling various business capabilities. Spring Boot, Flask, Django
Container Orchestration Manages the lifecycle of containers in large, dynamic environments. Provides tools for deploying, scaling, and monitoring containers. In environments running containerized applications. Kubernetes, Docker Swarm, Apache Mesos
CDN (Content Delivery Network) Distributes content to multiple, geographically dispersed servers to improve access speed for downloading content. Used for delivering static content closer to users. Cloudflare, Akamai, Amazon CloudFront
Monitoring and Logging Collects, processes, aggregates, and displays real-time quantitative data about a system's performance and health. Used across all parts of the infrastructure. Prometheus, ELK Stack, Datadog, Grafana
Authentication Service Manages user authentication and authorization. Ensures that only legitimate users can access certain functionalities or data. At entry points requiring secure user access. Auth0, Amazon Cognito, OAuth2 servers
Firewall Monitors and controls incoming and outgoing network traffic based on predetermined security rules. At the network boundary to protect internal networks. Palo Alto Networks, Fortinet, Cisco ASA

Component Functionality Where Used Examples
Service Mesh Provides a dedicated infrastructure layer for handling service-to-service communication in complex architectures, handling load balancing, service discovery, etc. In microservices architectures to manage network traffic and policies. Istio, Linkerd, Consul Connect
Serverless Functions Allows developers to write and deploy code that runs in response to events without managing server infrastructures. In cloud environments to handle specific tasks like processing events. AWS Lambda, Azure Functions, Google Cloud Functions
Distributed File System Manages storage across multiple networked servers to facilitate data sharing and redundancy. In systems requiring large data storage spread across geographical locations. HDFS, GlusterFS, Ceph
Configuration Management Automates the provisioning and management of software and its dependencies across the infrastructure. In operations to maintain consistency in system settings and software on multiple machines. Ansible, Chef, Puppet
Orchestration Software Automates the management, coordination, and organization of complex computer systems, middleware, and services. In managing groups of microservices and containers at scale. Kubernetes, Docker Swarm, Apache Mesos
Data Pipeline Tools and processes used to automate the transport and transformation of data from one system to another. In data processing architectures, for ETL processes and real-time data streaming. Apache NiFi, Apache Kafka, AWS Data Pipeline
Reverse Proxy Retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client as if they originated from the proxy server. In front of web servers to balance load, ensure security, and enhance performance. NGINX, HAProxy, Apache Traffic Server
Identity and Access Management (IAM) Manages digital identities and specifies what users can and can't do within a system based on recognized and verified identities. Across all services requiring secure user access and action control. Okta, Keycloak, AWS IAM
Disaster Recovery (DR) Tools Plans and implements recovery processes in the event of a disaster to restore data and critical system functionality. Used across IT environments to ensure data and application availability post-disaster. Zerto, Veeam, Azure Site Recovery
Blockchain Infrastructure Provides a decentralized ledger for secure and transparent transactions. In systems requiring secure, immutable records or transactions. Ethereum, Hyperledger, R3 Corda
Business Intelligence (BI) Tools Analyzes data and presents actionable information to help executives, managers, and other corporate end users make informed business decisions. In data-rich environments to drive decision-making and strategy. PowerBI, Tableau, Looker

Database Type Functionality Typical Usage Examples
PostgreSQL Relational ACID-compliant, supports complex queries, indexing, and relational data integrity. Web applications, analytics applications, any system requiring complex queries. EnterpriseDB, TimescaleDB (extension for time-series)
MongoDB Document-based NoSQL Stores data in JSON-like documents with dynamic schemas, making data integration in certain types of applications easier and faster. Content management, IoT applications, real-time analytics. MongoDB Atlas, Stitch (MongoDB service)
Cassandra Wide-column NoSQL Highly scalable and designed to handle large amounts of data across many commodity servers without a single point of failure. Large scale data logging, time-series data, real-time sensor data. DataStax, Scylla (Cassandra compatible)
Redis Key-value NoSQL In-memory data structure store, used as a database, cache, and message broker. Caching, session management, pub/sub systems, real-time applications. Redis Labs, AWS ElastiCache
DynamoDB Key-value NoSQL Managed NoSQL database service that supports key-value and document data structures. Serverless applications, microservices, mobile backends. Amazon DynamoDB (Managed Service)
MySQL Relational Popular open-source relational database management system known for its reliability and ease of use. Web applications, e-commerce, logging applications. Oracle MySQL, Percona (enhanced MySQL variant)
Oracle Database Relational Comprehensive RDBMS solution known for its performance, scalability, and multi-model support. Enterprise applications, high transaction systems, banking systems. Oracle Exadata (engineered system for Oracle DB)
Microsoft SQL Server Relational Provides robust transactional support, strong data integrity, and performance for complex queries. Enterprise environments, business intelligence, complex transaction processing. SQL Server on Azure, SQL Server Enterprise
Neo4j Graph-based NoSQL Stores and processes data as graphs, making it ideal for systems where data relationships are key. Social networks, recommendation engines, fraud detection. Neo4j Aura (cloud service), Neo4j Enterprise
Elasticsearch Search Engine Optimized for search operations on large volumes of data, typically used for log and event data analysis. Full-text search, real-time analytics, security intelligence analysis. Elastic Cloud, Open Distro for Elasticsearch

AWS Service Category Functionality Typical Usage
EC2 (Elastic Compute Cloud) Compute Provides scalable computing capacity in the cloud. Allows users to run and manage server instances. Hosting applications, running backend services.
S3 (Simple Storage Service) Storage Object storage service offering industry-leading scalability, data availability, security, and performance. Storing and protecting any amount of data for a range of use cases.
RDS (Relational Database Service) Database Simplifies setup, operation, and scaling of a relational database in the cloud. Provides cost-efficient and resizable capacity. Managing common database engines like MySQL, PostgreSQL, and Oracle.
DynamoDB Database A NoSQL database service that supports key-value and document data structures. Offers built-in security, backup and restore, and in-memory caching. Building flexible, scalable applications without worrying about hardware provisioning.
Lambda Compute A serverless compute service that runs code in response to events and automatically manages the underlying compute resources. Running code in response to HTTP requests via Amazon API Gateway or event triggers.
Elastic Load Balancing (ELB) Networking Automatically distributes incoming application traffic across multiple targets, such as EC2 instances. Improving fault tolerance of applications by distributing traffic.
VPC (Virtual Private Cloud) Networking Provides a logically isolated section of the AWS cloud where you can launch AWS resources in a virtual network that you define. Hosting secure and scalable web applications.
IAM (Identity and Access Management) Security Manages access to AWS services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources. Controlling access to AWS resources for users and systems.
CloudFront Content Delivery A fast content delivery network (CDN) service that securely delivers data, videos, applications, and APIs to customers globally with low latency and high transfer speeds. Delivering website content and video streaming.
SNS (Simple Notification Service) Messaging Provides a managed service for sending notifications and messages to a large number of subscribers. Coordinating the delivery of messages to subscribing endpoints or clients.
SQS (Simple Queue Service) Messaging Offers a secure, durable, and available hosted queue that lets you integrate and decouple distributed software systems and components. Managing message queues for microservices, distributed systems, and serverless applications.
CloudWatch Monitoring Provides monitoring and observability of your AWS resources and applications on AWS and on-premises, offering you actionable insights and system-wide visibility. Monitoring application performance and operational health.

Concept Definition How to Achieve Tools/Technologies
Fault Tolerance The ability of a system to continue operating properly in the event of the failure of some of its components. Implement redundancy, failovers, and robust error handling. AWS EC2 Auto Scaling, VMware HA, MySQL Replication
Scalability The capability of a system to handle a growing amount of work by adding resources to the system. Design systems that can scale out or scale up efficiently. Use load balancers and distributed architectures. Kubernetes, Apache Kafka, AWS Elastic Load Balancer
Observability The ability to measure the internal states of a system by examining its outputs. Implement comprehensive logging, monitoring, and tracing. Prometheus, Grafana, Elasticsearch, Jaeger
Security Protecting systems, networks and data from digital attacks. Implement secure coding practices, encryption, access controls, and regular security assessments. OAuth2, JWT, AWS IAM, Let's Encrypt SSL
Reliability The ability of a system to perform and maintain its functions in routine circumstances, as well as hostile or unexpected circumstances. Use rigorous testing, update management practices, and disaster recovery planning. Selenium, JUnit, Terraform, Veeam Backup
Manageability The ease with which a system can be managed in terms of monitoring, maintenance, and upgrades. Use automation tools for deployment, management, and real-time updates. Ansible, Chef, Puppet, AWS CloudFormation
Performance The speed with which a system performs its tasks. This can be measured in terms of latency and throughput. Optimize code and infrastructure, utilize caching and content delivery networks. Redis, CDN services, New Relic, Apache Benchmark
Availability The degree to which a system is operational and accessible when required for use. Minimize downtime using high availability setups and redundant systems. Amazon RDS multi-AZ, Microsoft Azure Site Recovery
Disaster Recovery The strategy and processes involved in preparing for and recovering from a disaster, whether natural or man-made. Implement backup solutions and data replication across geographically diverse locations. AWS Disaster Recovery, DRaaS providers, Azure Site Recovery
Cost Efficiency Managing a system's economic efficiency regarding operational and capital costs. Optimize resource usage and cloud expenses with scaling and efficient architectures. AWS Cost Explorer, Google Cloud Cost Management

Non-Functional Requirement Description How to Achieve Tools/Technologies
High Availability The system should be operational and accessible at all times, minimizing downtime even during failures or maintenance. Implement redundant systems, failover mechanisms, and regular health checks. AWS EC2 Auto Scaling, Kubernetes, Azure Availability Sets
Low Latency The system should process requests and deliver responses within a short amount of time. Optimize application performance, use content delivery networks, and improve network configurations. CDN services like Cloudflare, AWS CloudFront
High Scalability The system should be able to handle increased loads smoothly by scaling resources up or out as needed. Use scalable architectures like microservices, employ load balancers, and leverage elastic cloud resources. Docker, Kubernetes, AWS Elastic Load Balancing
High Consistency The system should ensure that all users see the same data at the same time, especially in distributed systems. Implement strong consistency models in database transactions, use synchronous replication techniques. Apache Kafka, Google Spanner, MongoDB Transactions
High Security The system should protect data and resources from unauthorized access and attacks. Use encryption, secure access protocols, regular security audits, and compliance with security standards. SSL/TLS, OAuth2, AWS IAM, HashiCorp Vault
Disaster Recovery The system should be capable of recovering from critical failures or disasters without significant data loss. Develop and regularly test backup and restore procedures, use geographically redundant data storage. AWS Backup, Azure Site Recovery, Veeam
Data Integrity The system should maintain accuracy and consistency of data over its entire lifecycle. Use checksums, RAID storage, and transactional databases with rollbacks and error handling. PostgreSQL, Oracle Database, ZFS file system
Usability The system should be easy to use and navigate by its intended users, requiring minimal training. Follow best UI/UX practices, conduct user testing, and implement feedback loops for improvements. Adobe XD, Sketch, UserTesting.com
Performance The system should meet performance criteria specified in terms of transaction times, throughput, and responsiveness. Profile and optimize software, choose efficient algorithms, and select appropriate hardware and infrastructure. New Relic, Apache JMeter, Grafana
Maintainability The system should be easy to maintain, allowing updates, bug fixes, and improvements without significant downtime. Adhere to coding standards, use modular architectures, and ensure comprehensive documentation. Git, JIRA, Docker
Cost Efficiency The system should operate within a budget, with optimizations to minimize unnecessary expenditures. Monitor resource utilization, choose cost-effective technologies, and implement auto-scaling solutions. AWS Cost Explorer, Google Cloud Billing, Azure Cost Management
Compliance The system should adhere to relevant laws and regulations governing data handling and operations. Regularly review and align system operations with legal requirements, possibly using compliance tools. AWS Compliance Programs, Microsoft Compliance Manager

Concept Description Application
CAP Theorem States that a distributed system can only simultaneously provide two of the following three guarantees: Consistency, Availability, and Partition Tolerance. Used in the design and understanding of distributed databases and systems.
ACID Properties Set of properties of database transactions intended to guarantee validity even in the event of errors, power failures, etc. These properties are Atomicity, Consistency, Isolation, and Durability. Fundamental in transactional systems and relational databases to ensure reliable processing of transactions.
BASE Properties An acronym for Basically Available, Soft state, and Eventual consistency. Offers a more flexible approach compared to ACID in distributed systems where strict consistency is less feasible. Often applied in NoSQL and distributed databases where availability and fault tolerance are more critical than strict consistency.
Eventual Consistency A consistency model used in distributed computing to achieve high availability that informally guarantees that, if no new updates are made to a given data item, eventually all accesses will return the last updated value. Common in distributed network systems and databases like Cassandra and DynamoDB.
Two-Phase Commit (2PC) A distributed algorithm that lets all nodes in a distributed system agree to commit a transaction. The protocol results in either all nodes committing the transaction or none of them. Used in distributed databases and systems to ensure data integrity across multiple nodes.
Quorum In distributed computing, a quorum is the minimum number of votes that a distributed transaction has to obtain to be allowed to perform an operation in a distributed system. Used to achieve consistency in distributed systems like clustered configurations of databases or in systems using consensus algorithms.
Isolation Levels Define the degree to which a transaction must be isolated from data modifications made by any other transaction in a database system. Levels include Read Uncommitted, Read Committed, Repeatable Read, and Serializable. Critical in database management systems to control the effects of concurrently executing transactions.
Data Replication The process of storing data in more than one site or node. It is used to improve reliability, fault-tolerance, or accessibility. Widely used in distributed databases to ensure data availability and durability.

@1UC1F3R616
Copy link
Author

SSE: Server Side Events -> real-time updates such as live scores, stock ticker updates, or any real-time feed without requiring the client to make an HTTP request for each update. Any backend technology that can hold a connection open and send data in a stream.
Long Polling

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment