Skip to content

Instantly share code, notes, and snippets.

@heiko-braun
Created December 4, 2023 10:45
Show Gist options
  • Save heiko-braun/a5629b7ecbfffece320f9579d1764605 to your computer and use it in GitHub Desktop.
Save heiko-braun/a5629b7ecbfffece320f9579d1764605 to your computer and use it in GitHub Desktop.

Keywords: Camel | Kinesis | ResumeAdapter | KinesisDefaultResumeAdapter

Messages

user: A customer has a use case where they want the Camel's route to continue consuming from Kinesis at the last consumed message after restarting the POD. To achieve this, Kinesis has a "checkpoint" feature. Unfortunately, Camel Kinesis component lacks the checkpoint support. However, it supports Resume API (I had to glean this information from GitHub because there is hardly any documentation, testing or examples available neither upstream). Then I would suggest implementing a ResumeAdapter that supports checkpoint starting from KinesisDefaultResumeAdapter. But I don't know where to start from.  Any Advice?

assistant: It looks like there isn't specific documentation available for Camel Kinesis component with respect to the Resume API or implementing a ResumeAdapter. However, based on the information you provided, it seems like you want to implement a custom ResumeAdapter for the Camel Kinesis component that supports the "checkpoint" feature for resuming from the last consumed message.

To go about implementing a ResumeAdapter, here are the general steps you can follow:

  1. Understand the Resume API: Gather more information about the Resume API and how it is expected to work with the Camel Kinesis component. Ensure that you have a clear understanding of how the Resume API is expected to interact with the Kinesis stream and handle the "checkpoint" feature.

  2. Extend KinesisDefaultResumeAdapter: It seems like you want to build your custom ResumeAdapter based on the KinesisDefaultResumeAdapter. You can extend this class and add the necessary logic to support the checkpoint feature.

  3. Implement Checkpointing Logic: Within your custom ResumeAdapter, you'll need to implement the checkpointing logic that allows the route to continue consuming from the last consumed message after a restart. This may involve storing the checkpoint information and using it to resume consumption.

  4. Testing and Integration: Once you've implemented the custom ResumeAdapter, you can test it with a Camel route that consumes from the Kinesis stream. Ensure that the checkpointing behavior works as expected.

  5. Documentation and Examples: As you mentioned, there is a lack of documentation, testing, and examples for this feature. It would be beneficial to create documentation and examples for the custom ResumeAdapter to help other developers understand how to use it.

Unfortunately, I don't have access to specific information about the Resume API or the internal workings of the Camel Kinesis component. Therefore, you may need to refer to the source code for the Camel Kinesis component on GitHub to understand how the Resume API is implemented and how you can create a custom ResumeAdapter.

If you have any specific questions or need further assistance with any of the steps mentioned above, please feel free to ask!

@orpiske
Copy link

orpiske commented Dec 4, 2023

Wrong answer: already discussed via email.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment