- AWS
- Consul
- Terraform
- Ansible
- Nginx
In this project I configured Consul as Service Discovery tool that enables my frontend of web application to communicate with backend servers regardless of their IP addresses.
So In this project My frontend application connects with a load-balancer to obtain data that fetches the current IP addresses of backend servers from the consul-server with the help of consul-template tool And IP addresses in Consul-server will be kept updating by backend servers themselves with the help of consul-agent (Self Registering Property) running on them.
-
Deployed frontend of my web application on a AWS ec2 instance (Ubuntu 20.04 OS) and started at port 80 with the help of Nginx.
-
Started 5 other ec2 instances in my AWS VPC using Terraform, Where three to work as backend servers and one for load-balancer and other one for consul-server.
-
Then, I started setting up all the servers using a ansible playbook by following steps:
-
Installed nginx on backend and loadbalancer servers:
- name: Installs nginx on backend and loadbalancer servers hosts: - backends - loadbalancer tasks: - name: updating package manager ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo apt-get update -y - name: installing nginx ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo apt install nginx -y
-
Deployed my backend of application to all (three) backend servers and set nginx on them to listen every request:
-
Then, Installed consul on backend and consul-server servers:
- name: Installs consul on backend and consul-server servers hosts: - backends - consulserver tasks: - name: Updating package manager ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo apt-get update -y - name: Fetching gpg key for consul ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg - name: Signing key ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list - name: Updating & Installing ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo apt update && sudo apt install consul
-
Set up consul on consulserver:
- name: Sets up consul on consulserver hosts: - consulserver tasks: - name: Swaping /etc/consul.d/consul.hcl & creating new consul.hcl with configuration ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: cat > /home/vagrant/consul.hcl && sudo rm -f /etc/consul.d/consul.hcl && sudo mv /home/vagrant/consul.hcl /etc/consul.d/ stdin: "\"bind_addr\" = \"CONSUL_SERVER_IP\"\n \"client_addr\" = \"CONSUL_SERVER_IP\"\n \"data_dir\" = \"/var/consul\"\n \"encrypt\" = \"ZENZNrsXU336Uma+S4XUj9sxvICj32N7XdEzrbYbRpY=\"\n \"datacenter\" = \"dc1\"\n \"ui\" = true\n \"server\" = true\n \"log_level\" = \"INFO\""
-
Set up Consul on each backend server:
- name: Sets up consul on each backend hosts: - backend1 - backend2 - backend3 tasks: - name: Swaping /etc/consul.d/consul.hcl & creating new consul.hcl with configuration ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: cat > /home/vagrant/consul.hcl && sudo rm -f /etc/consul.d/consul.hcl && sudo mv /home/vagrant/consul.hcl /etc/consul.d/ stdin: "\"bind_addr\" = \"SERVER_IP\"\n \"server\" = false\n \"datacenter\" = \"dc1\"\n \"data_dir\" = \"/var/consul\"\n \"encrypt\" = \"ZENZNrsXU336Uma+S4XUj9sxvICj32N7XdEzrbYbRpY=\"\n \"log_level\" = \"INFO\"\n \"enable_script_checks\" = true\n \"enable_syslog\" = true\n \"leave_on_terminate\" = true\n \"start_join\" = [\"CONSUL_SERVER_IP\"]\n \"node_name\" = \"backend-$BACKEND_NUM\"" - name: Creates service for consul each backend server ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo cat > backend.hcl && sudo mv backend.hcl /etc/consul.d/backend.hcl stdin: "\"service\" = {\n \t\"Name\" = \"backend\"\n \t\"Port\" = 80\n \t\"check\" = {\n \t\t\"args\" = [\"curl\", \"localhost\"]\n \t\t\"interval\" = \"3s\"\n \t}\n }"
-
Then, Installed consul-template on loadbalancer & configured:
- name: Installs consul-template on loadbalancer hosts: - loadbalancer tasks: - name: installing unzip tool ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo apt-get install unzip -y - name: installing consul-template ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo curl -L https://releases.hashicorp.com/consul-template/0.30.0/consul-template_0.30.0_linux_amd64.zip -o /opt/consul-template.zip && sudo unzip /opt/consul-template.zip -d /usr/local/bin/ - name: Setting up load-balancer.conf.ctmpl ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo cat > /etc/nginx/conf.d/load-balancer.conf.ctmpl stdin: "upstream backend {\n # \t {{- range service \"backend\" }} \n # \t\t server {{ .Address }}:{{ .Port }}; \n # \t {{- end }} \n # }\n # \n # server {\n # \tlisten 80;\n # \n # \t location / {\n # \t\t proxy_pass http://backend;\n # \t}\n # }" - name: Sets up consul-template ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: cat > /home/vagrant/consul-template.hcl && sudo mv /home/vagrant/consul-template.hcl /etc/nginx/conf.d/ stdin: "consul {\n \t address = \"CONSUL_SERVER_IP:8500\"\n \n \t retry {\n \t\t enabled = true\n \t\t attempts = 12\n \t\t backoff = \"250ms\"\n \t }\n }\n template {\n \t source = \"/etc/nginx/conf.d/load-balancer.conf.ctmpl\"\n \t destination = \"/etc/nginx/conf.d/load-balancer.conf\"\n \t perms = 0600\n \t command = \"service nginx reload\"\n }" - name: Removing default nginx configuration on loadbalancer ansible.builtin.shell: args: chdir: /home/vagrant/ cmd: sudo rm /etc/nginx/sites-enabled/default && sudo systemctl restart nginx
-
-
Then, started Consul agent on all the servers one by one as follows:
- First, started on consul-server agent:
sudo consul agent -config-dir /etc/consul.d/ & - Then Started consul-agent on all the backend servers:
sudo consul agent -config-dir /etc/consul.d/ & - Then Started on Load-balancer consul-agent:
sudo consul-template -config=/etc/nginx/conf.d/consul-template.hcl &
- First, started on consul-server agent:
That's it, After all the configuration my frontend application was able to communicate with backend servers to obtain data regardless of their IP addresses and Consul Server was working as a meadium that was helping frontend to connecting with backend.