Last active
January 23, 2023 07:51
-
-
Save merlin-quix/639fafd0ba72277bcfb4cabbdcadb664 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Add a switch to let us easily switch between methods | |
compmethod = 'topic' | |
# Initialize a simple counter for the message key | |
message_key = 0 | |
# Iterate through file line-by-line | |
with open(file, "r") as file_handle: | |
for line in file_handle: | |
print(f"Sending: {line}") | |
# Select the relevant compression method based on the switch | |
if compmethod == 'topic': | |
producer.send(topic="nginx-log-topic-compression", value=line, key=message_key) | |
else: | |
producer_compression.send(topic="nginx-log-producer-compression", value=line, key=message_key) | |
message_key = message_key+1 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment