This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import org.apache.kafka.common.serialization.Serdes; | |
import org.apache.kafka.streams.KafkaStreams; | |
import org.apache.kafka.streams.StreamsBuilder; | |
import org.apache.kafka.streams.StreamsConfig; | |
import org.apache.kafka.streams.kstream.KStream; | |
import com.github.megachucky.kafka.streams.machinelearning.TensorflowObjectRecogniser; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Licensed to the Apache Software Foundation (ASF) under one or more | |
* contributor license agreements. See the NOTICE file distributed with | |
* this work for additional information regarding copyright ownership. | |
* The ASF licenses this file to You under the Apache License, Version 2.0 | |
* (the "License"); you may not use this file except in compliance with | |
* the License. You may obtain a copy of the License at | |
* | |
* http://www.apache.org/licenses/LICENSE-2.0 | |
* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// imports and license left out for clarity | |
public class OptimizedStreams { | |
public static void main(String[] args) { | |
final Properties properties = new Properties(); | |
properties.setProperty(StreamsConfig.APPLICATION_ID_CONFIG, "test-application"); | |
properties.setProperty(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092 "); | |
properties.setProperty(StreamsConfig.TOPOLOGY_OPTIMIZATION, StreamsConfig.OPTIMIZE); | |
properties.setProperty(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName()); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Licensed to the Apache Software Foundation (ASF) under one or more | |
* contributor license agreements. See the NOTICE file distributed with | |
* this work for additional information regarding copyright ownership. | |
* The ASF licenses this file to You under the Apache License, Version 2.0 | |
* (the "License"); you may not use this file except in compliance with | |
* the License. You may obtain a copy of the License at | |
* | |
* http://www.apache.org/licenses/LICENSE-2.0 | |
* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Licensed to the Apache Software Foundation (ASF) under one or more | |
* contributor license agreements. See the NOTICE file distributed with | |
* this work for additional information regarding copyright ownership. | |
* The ASF licenses this file to You under the Apache License, Version 2.0 | |
* (the "License"); you may not use this file except in compliance with | |
* the License. You may obtain a copy of the License at | |
* | |
* http://www.apache.org/licenses/LICENSE-2.0 | |
* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
events | |
.groupByKey() | |
.windowedBy(TimeWindows.of(Duration.ofMinutes(2))) | |
.count(Materialized.as("count-metric")) | |
// graph servlet queries "count-metric" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
events | |
.groupByKey() | |
.windowedBy(TimeWindows.of(Duration.ofMinutes(2))) | |
.count(Materialized.as("count-metric")) | |
.filter( _ < 4 ) | |
.toStream() | |
.foreach( /* Send that email! */) | |
// graph servlet queries "count-metric" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
events | |
.groupByKey() | |
.windowedBy( | |
TimeWindows.of(Duration.ofMinutes(2).withGrace(Duration.ofMinutes(2)) | |
) | |
.count(Materialized.as("count-metric")) | |
.suppress(Suppressed.untilWindowClose(BufferConfig.unbounded())) | |
.filter( _ < 4 ) | |
.toStream() | |
.foreach( /* Send that email! */) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
builder | |
.table("users") | |
.suppress(Suppressed.untilTimeLimit( | |
BufferConfig.maxBytes(myConfig.getUsersBufferSize()) | |
)) | |
... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
driver.pipeInput(recordFactory.create( | |
/* topic */ "input", | |
/* key */ "A", | |
/* value */ "v1", | |
/* timestamp */ 10L | |
)); | |
// Stream time is now 10L | |
driver.pipeInput(recordFactory.create("input", "A", "v2", 11L)); | |
// Stream time is now 11L |