- I have a paginated REST API which I would like to poll
- Since the source data is growing, I would like to set this up as a scheduled sync job
and rerun it from the point where the previous execution left off (even if it was
COMPLETED
) basically advancing on the new data periodically AbstractPaginatedDataItemReader
seems to be suitable, it is able to:- Track the read item count (actually it is
AbstractItemCountingItemStreamItemReader
) - Maintain and persist its state in the
ExecutionContext
(read.count
)
- Track the read item count (actually it is
- What I'm missing is creating the new
JobParameters
based on the previousExecutionContext
- It seems
DefaultJobParametersExtractor
does the trick I need - And it is used from the
JobStep
class in a way I need - Unfortunately this is only available to
JobStep
and I have aTaskletStep
- So I ended up reimplementing the parts I need from
JobStep
andDefaultJobParametersExtractor
Is there a better way to do this?
The code in Example.java
is what I would like to get rid off by providing the RunIdIncrementer
and a JobParametersExtractor
.
Thanks for the clarifications. I thought the increment was a fixed value, but now I understand it is rather dynamic. In that case, we can always use a custom incrementer that gets the value from the execution context of the last run. Here is an updated version of the incrementer with a complete example:
You can ignore my reader, I created it to simulate things. Your reader already puts the index in the execution context of the step, you would need to promote it to the job execution context because this is where the inrementer would get it.
The example uses a random index (and might fail if the random value is the same), but if you run it you would see that each run will start from the previous index: