Echosight Architecture

architecture.md

Echosight Application Deployment Architecture

This document outlines a detailed deployment architecture for the Echosight Rails application, utilizing AWS for hosting and Cloudflare for DNS and CDN functionalities. The deployment is automated through GitHub Actions, triggered by merges to the master branch, ensuring an efficient CI/CD pipeline.

Architecture Overview

graph LR
    subgraph Cloudflare
    DNS(DNS Management for app.echosight.io)
    CDN(CDN for content delivery)
    end

    subgraph AWS
    ECR(ECR - Docker Image Repository)
    ECS(ECS with Fargate - Rails & Sidekiq)
    RDS(RDS - PostgreSQL)
    Elasticache(Elasticache - Redis)
    S3(S3 - Storage)
    ALB(ALB - Application Load Balancer)
    IAM(IAM - Roles and Policies)
    AutoScaling(Auto Scaling)
    end

    Internet --> DNS
    DNS --> CDN
    CDN --> ALB
    ALB --> ECS
    ECS --> ECR
    ECS --> RDS
    ECS --> Elasticache
    ECS --> S3
    ECS --> IAM
    ECS -.-> AutoScaling

Loading

AWS Elastic Container Service (ECS) with Fargate

Components

• ECS: Hosts the Echosight application and Sidekiq worker in containerized environments, facilitating scaling and management without manual orchestration.
• Fargate: Provides a serverless compute engine, eliminating the need to provision or manage underlying servers.

Task Definitions

• Rails Application: Configured with the necessary environment variables for database and Redis connections, it runs the web server process.
• Sidekiq Worker: Configured similarly to the Rails application but specifically for background job processing.

AWS Relational Database Service (RDS)

• PostgreSQL: Offers a managed database solution, eliminating administrative tasks like manual backups, patching, and scaling.

AWS Elasticache

• Redis: Provides an in-memory data store for Sidekiq's background job processing, enhancing performance and scalability.

AWS Simple Storage Service (S3)

• Storage: Serves as a repository for static assets or file uploads, integrating seamlessly with the Rails application.

Cloudflare Integration

• DNS Management: Directs app.echosight.io traffic to the AWS-hosted application, ensuring efficient DNS resolution.
• CDN: Enhances global content delivery, reducing latency and improving user experience by caching content at edge locations.

CI/CD Pipeline with GitHub Actions

sequenceDiagram
    participant GH as GitHub
    participant GA as GitHub Actions
    participant ECR as ECR
    participant ECS as ECS

    GH->>GA: Merge to master
    GA->>ECR: Build & push Docker image
    ECR->>ECS: Update ECS service
    ECS->>ECS: Deploy new image

Loading

Steps

1. Build & Push: On a merge to master, GitHub Actions builds the Docker image and pushes it to ECR.
2. Deployment: ECS updates the running services with the new image, applying rolling updates to minimize downtime.

Security and Compliance

• IAM Roles: Ensures ECS tasks have appropriate permissions with minimal scope.
• Security Groups: Defines network access rules for ECS, RDS, and Elasticache to ensure secure communication.
• Secrets Management: Utilizes AWS Secrets Manager for secure handling of sensitive information, referenced by ECS tasks.

Monitoring and Logging

• Integration: Uses Axiom for comprehensive monitoring and logging, aggregating data across services for a holistic view.
• Performance Metrics: Tracks application performance, enabling proactive issue resolution and optimization.

AWS Infrastructure Components with Terraform Considerations

1. ECS Cluster:

   • Variable: Cluster name (e.g., echosight-cluster).

2. ECS Task Definitions:

   • Rails Application:
     • Variables: Docker image, CPU, memory, environment variables (e.g., database URL, Redis URL).
   • Sidekiq Worker:
     • Variables: Docker image, CPU, memory, environment variables.

3. ECS Services:

   • Variables: Desired number of instances, service names.

4. ECS Task Execution Role:

   • Consideration: Ensure the role has policies allowing access to ECR for image retrieval and Axiom/CloudWatch for logging.

5. ECR Repository:

   • Variable: Repository name (e.g., echosight-app).

6. RDS Instance for PostgreSQL:

   • Variables: Instance class, storage size, engine version, database name, username, password.
   • Consideration: Ensure that the database is accessible from ECS tasks, which may involve security group configurations.

7. Elasticache Redis:

   • Variables: Node type, engine version, number of nodes.
   • Consideration: Configure security groups to allow connectivity from ECS tasks.

8. S3 Buckets:

   • Variables: Bucket names, access policies.
   • Consideration: Define bucket policies for appropriate access levels, possibly integrating with the application for asset or file uploads.

9. VPC, Subnets, and Security Groups:

   • Variables: CIDR blocks, public/private subnet distinctions, NAT gateways (if using private subnets), security group rules.
   • Consideration: Ensure the security groups are correctly configured to allow traffic for the application, Sidekiq, database, and Redis cache.

10. Elastic Load Balancer (ELB):

    • Variables: Listener configurations, health check configurations.
    • Consideration: Set up listener rules to route traffic correctly to the ECS service.

11. Auto Scaling:

    • Variables: Target values for scaling, scaling policies.
    • Consideration: Define appropriate metrics and thresholds for scaling to ensure cost-effectiveness and performance.

12. IAM Roles and Policies:

    • Consideration: Define least-privilege policies specifically for accessing necessary AWS resources.

Cloudflare Configuration

• Consideration: Since Terraform is not managing Cloudflare directly in this context, ensure that DNS records are correctly pointed to the AWS resources, such as the ELB.

Logging and Monitoring

• Consideration: Configure the ECS task definition to integrate with Axiom for logging. This may involve setting up log configuration in the task definition to route logs to Axiom.

Conclusion

The proposed architecture offers a robust, scalable, and efficient deployment solution for the Echosight Rails application, leveraging AWS's managed services and Cloudflare's network capabilities to ensure a seamless and resilient production environment.