Skip to content

Instantly share code, notes, and snippets.

@maelfosso
Created July 13, 2020 21:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save maelfosso/48e9595d923e942496066da418de9b12 to your computer and use it in GitHub Desktop.
Save maelfosso/48e9595d923e942496066da418de9b12 to your computer and use it in GitHub Desktop.
I tried to set Karafka on a Ruby on rails application to be able to consumer and produce kafka message but it didn't work here are my configuration file
# app/consumers/business_consumer.rb
class BusinessConsumer < ApplicationConsumer
def consume
Karafka.logger.info "New [Business] event: #{params}"
end
end
# Load the Rails application.
require_relative 'application'
require Rails.root.join(Karafka.boot_file)
# Initialize the Rails application.
Rails.application.initialize!
# frozen_string_literal: true
# require_relative './app/consumers/business_consumer'
ENV['RAILS_ENV'] ||= 'development'
ENV['KARAFKA_ENV'] = ENV['RAILS_ENV']
require ::File.expand_path('../config/environment', __FILE__)
Rails.application.eager_load!
# This lines will make Karafka print to stdout like puma or unicorn
if Rails.env.development?
Rails.logger.extend(
ActiveSupport::Logger.broadcast(
ActiveSupport::Logger.new($stdout)
)
)
end
class KarafkaApp < Karafka::App
setup do |config|
config.kafka.seed_brokers = %w[kafka://192.168.8.102:9092]
config.client_id = 'nkapsi_accounts_development'
config.logger = Rails.logger
end
# Comment out this part if you are not using instrumentation and/or you are not
# interested in logging events for certain environments. Since instrumentation
# notifications add extra boilerplate, if you want to achieve max performance,
# listen to only what you really need for given environment.
# Karafka.monitor.subscribe(WaterDrop::Instrumentation::StdoutListener.new)
# Karafka.monitor.subscribe(Karafka::Instrumentation::StdoutListener.new)
# Karafka.monitor.subscribe(Karafka::Instrumentation::ProctitleListener.new)
# Uncomment that in order to achieve code reload in development mode
# Be aware, that this might have some side-effects. Please refer to the wiki
# for more details on benefits and downsides of the code reload in the
# development mode
#
# Karafka.monitor.subscribe(
# Karafka::CodeReloader.new(
# *Rails.application.reloaders
# )
# )
consumer_groups.draw do
topic :business do
# consumer ExampleConsumer
# Karafka.logger.info "New [Business] event: #{params}"
consumer BusinessConsumer
end
# consumer_group :bigger_group do
# topic :test do
# consumer TestConsumer
# end
#
# topic :test2 do
# consumer Test2Consumer
# end
# end
end
end
Karafka.monitor.subscribe('app.initialized') do
# Put here all the things you want to do after the Karafka framework
# initialization
end
KarafkaApp.boot!
# config/initializers/karafka_init.rb
#
# I then
class KarafkaApp < Karafka::App
setup do |config|
config.kafka.seed_brokers = %w[kafka://192.168.8.100:9092]
config.client_id = 'nkapsi_accounts_development'
# config.logger = Rails.logger
end
# Comment out this part if you are not using instrumentation and/or you are not
# interested in logging events for certain environments. Since instrumentation
# notifications add extra boilerplate, if you want to achieve max performance,
# listen to only what you really need for given environment.
# Karafka.monitor.subscribe(WaterDrop::Instrumentation::StdoutListener.new)
# Karafka.monitor.subscribe(Karafka::Instrumentation::StdoutListener.new)
# Karafka.monitor.subscribe(Karafka::Instrumentation::ProctitleListener.new)
# Uncomment that in order to achieve code reload in development mode
# Be aware, that this might have some side-effects. Please refer to the wiki
# for more details on benefits and downsides of the code reload in the
# development mode
#
# Karafka.monitor.subscribe(
# Karafka::CodeReloader.new(
# *Rails.application.reloaders
# )
# )
consumer_groups.draw do
topic :business do
# consumer ExampleConsumer
# Karafka.logger.info "New [Business] event: #{params}"
consumer BusinessConsumer
end
# consumer_group :bigger_group do
# topic :test do
# consumer TestConsumer
# end
#
# topic :test2 do
# consumer Test2Consumer
# end
# end
end
end
Karafka.monitor.subscribe('app.initialized') do
# Put here all the things you want to do after the Karafka framework
# initialization
end
KarafkaApp.boot!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment