Deploy Kafka Producer and Consumer to Ubuntu 18.04

deploy_kafka_producer_and_consumer.md

Setup Gemfile

after follow this to deploy the project first

move listen gem out of development block in Gemfile

gem 'listen', '>= 3.0.5', '< 3.2'

install delivery_boy, racecar and foreman

gem 'delivery_boy'
gem 'racecar'
gem 'foreman'

Foreman

create Procfile in Project directory

racecar-{{name_in_lower_case}}: bundle exec racecar {{name}}Consumer

Setup Racecar

edit racecar.yml for localhost

# These config values will be shared by all environments but can be overridden.
common: &common
  group_id:           'development_group_id'
  client_id:          'development_client_id'
  log_level:          'INFO'

development:
  <<: *common
  brokers:
    - localhost:9092

test:
  <<: *common
  brokers:
    - localhost:9092

edit racecar.yml for production

cd {{project_dir}}/shared/config
vi racecar.yml

# These config values will be shared by all environments but can be overridden.
common: &common
  group_id:           'production_group_id'
  client_id:          'production_client_id'
  log_level:          'INFO'

production:
  <<: *common
  logfile:            'log/racecar.log'
  brokers:
    - kafka1.myapp.com:9092
    - kafka2.myapp.com:9092
    - kafka3.myapp.com:9092

link racecar.yml in deploy.rb

append :linked_files, "config/database.yml", "config/application.yml", "config/secrets.yml", "config/racecar.yml"

Deploy

cap production deploy

Setup Server systemd

export to systemd

go to server

cd {{project_dir}}/current
rvmsudo bundle exec foreman export --app {{project_name_in_lower_case}} --user deploy systemd /etc/systemd/system/

this will generate project.target, project-consumer.target, project-consumer.target.wants and project-consumer@.service

rename project-consumer@.service

sudo mv project-consumer@.service project-consumer.service

tweak service file

sudo vi project-consumer.service

set the following

WorkingDirectory=/home/deploy/{{project_dir}}/current
ExecStart=/bin/bash -lc 'RAILS_ENV=production bundle exec racecar {{consumer_name}}Consumer'

start service

sudo systemctl start project-consumer.service

useful command

sudo systemctl stop project-consumer.service
sudo systemctl restart project-consumer.service
sudo systemctl status project-consumer.service

racecar log should be available here

cd {{project_dir}}/current/log
tail -f racecar.log

Happy Coding ^_^