hrms_setup_python.playbook.md

Playbook: Setting Up System

Overview

This playbook will guide you through the process of setting up an environment and generating Docker files for deployment.

What’s Needed From User

• AWS Aurora PostgreSQL instance details (username, password,database name,cluster name, storage and instance class).

Procedure

1. Set up Python:

   • Install the latest version of Python:

     sudo apt-get update
     sudo apt-get install python3

2. Set up Apache Maven:

   • Download and install Apache Maven:

     sudo apt-get update
     sudo apt-get install maven

   • Set the M2_HOME environment variable and add it to the system path:

     export M2_HOME=/opt/apache-maven
     export PATH=$PATH:$M2_HOME/bin

3. Set up Node.js:

   • Install Node.js and npm if not already installed:

     sudo apt-get update
     sudo apt-get install nodejs

4. Set up Kafka and Zookeeper:

   • Install Kafka and Zookeeper using the following commands:

      wget https://downloads.apache.org/kafka/<latest-version-no.>/kafka_<latest-version-no.>.tgz
      tar -xzf kafka_<latest-version-no.>.tgz
      pip install kafka-python

   • Update Kafka server configuration file (server.properties) to configure listeners:

      listeners=PLAINTEXT://:9092
      advertised.listeners=PLAINTEXT://localhost:9092

   • Start Kafka and Zookeeper :

      bin/zookeeper-server-start.sh config/zookeeper.properties
      bin/kafka-server-start.sh config/server.properties

5. Set up AWS Aurora PostgreSQL:

   • Install the AWS CLI and postgresql-client on the system :

     sudo apt-get install awscli
     sudo apt-get install postgresql-client

   • Create an AWS Aurora PostgreSQL instance using AWS CLI using present credentials in the system ,use user specified storage and instance class if provided otherwise use default values as in the command below:

     aws rds create-db-instance --db-instance-identifier <instance-name> --db-instance-class db.t3.medium --engine aurora-postgresql --master-username <username> --master-user-password <password> --allocated-storage 20 --db-cluster-identifier <cluster-name> --availability-zone us-west-2a

   • Note down the connection details (endpoint, username, password) for later use.

6. Configure the system:

   • Create a database for the system.

   • Connect to the AWS Aurora PostgreSQL instance and Create a user with the necessary permissions on the database from the system.psql --host=<endpoint> --port=5432 --username=<master-username> --password --dbname=postgres

   • Create a new user with the necessary permissions on the database:

       CREATE USER <new-username> WITH PASSWORD '<new-password>';
       CREATE DATABASE <database-name>;
       GRANT ALL PRIVILEGES ON DATABASE <database-name> TO <new-username>;

   • Update the system configuration files with the database connection details.

7. Test the environment:

   • Verify that the database is accessible from the system where the system will be deployed.
   • Verify that the user can access the database and perform CRUD operations on the tables.
   • Verify that all the installed components are running and healthy.
   • Connect to the AWS Aurora PostgreSQL instance and ensure that the database is accessible.

8. Dockerize the environment:

   • Create Dockerfile for each service (Python, Node.js, Kafka, Zookeeper, PostgreSQL).
   • Define the required dependencies and configurations in the Dockerfiles.
   • Create a docker-compose.yml file to define and configure the services required for the system.

9. Build and run the Docker containers:

   • Build the Docker images for each service using the docker build command.
   • Run the Docker containers using the docker-compose up command.
   • Verify that all the Docker containers are running and healthy.
   • Test the system to ensure that it is working as expected.

10. Create a README file:

    • Document the entire setup process, including prerequisites, installation steps, and any additional configurations required.
    • Provide instructions on how to build and run the Docker containers.
    • Include any relevant troubleshooting tips or known issues.

11. Share the files with the user:

    • Create a compressed archive (e.g., ZIP or TAR) containing the project files, Dockerfiles, and the README file.
    • Share the archive as an attachment using the messaging interface in the browser to the user.

Specifications

• Setup an environment for the following:

  • Latest Python version
  • Latest version of Apache Maven
  • Node.js for React
  • Kafka and Zookeeper
  • AWS Aurora PostgreSQL

• Test the environment:

  • Verify that the environment is up, running, healthy, and working.

• Dockerize the entire environment.

• Create a README file with detailed instructions for setting up the environment.

• Share all the files with the user.

Advice and Pointers

• Do not create tables and schemas until unless specified by the user.
• Ensure that all the dependencies are installed correctly and the environment is set up properly before proceeding with the Dockerization.
• Test each component individually to ensure that it is working as expected.
• Document the setup process and configurations for future reference.
• Follow best practices for setting up the environment and Dockerizing the services.