Skip to content

Instantly share code, notes, and snippets.

@mfelsche
Created September 9, 2021 12:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mfelsche/6217a0e76bac75c63a6784c1f3ac3850 to your computer and use it in GitHub Desktop.
Save mfelsche/6217a0e76bac75c63a6784c1f3ac3850 to your computer and use it in GitHub Desktop.
Tremor Config to consume json documents from udp and forward them to kafka while applying backpressure in case of error or timeout after > 1s
onramp:
- id: udp_in
type: udp
codec: json
config:
host: localhost
port: 65535
offramp:
- id: kafka_out
type: kafka
codec: json
config:
brokers:
- my_kafka:9092
- my_other_kafka:9092
topic: my_topic
binding:
- id: my_binding
links:
"/onramp/udp_in/{instance}/out": ["/pipeline/main/{instance}/in"]
"/pipeline/main/{instance}/out": ["/offramp/kafka_out/{instance}/in"]
"/pipeline/main/{instance}/err": ["/offramp/system::stderr/system/in"]
mapping:
/binding/my_binding/01:
instance: 01
define qos::backpressure operator bp
with
timeout = 1000
end;
create operator bp;
select event from in into bp;
select event from bp into out;
select event from bp/err into err;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment