A sample job for Spring Batch repository metadata - copies execution history from one repository to another.
Created
December 23, 2010 13:56
-
-
Save dsyer/753006 to your computer and use it in GitHub Desktop.
A sample job for Spring Batch repository metadata - copies execution history from one repository to another.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="UTF-8"?> | |
<classpath> | |
<classpathentry kind="src" path="src/main/resources"/> | |
<classpathentry kind="src" path="src/main/java"/> | |
<classpathentry kind="src" output="target/test-classes" path="src/test/java"/> | |
<classpathentry kind="src" path="src/test/resources"/> | |
<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/> | |
<classpathentry kind="con" path="org.maven.ide.eclipse.MAVEN2_CLASSPATH_CONTAINER"> | |
<attributes> | |
<attribute name="org.eclipse.jst.component.nondependency" value=""/> | |
</attributes> | |
</classpathentry> | |
<classpathentry kind="con" path="org.eclipse.jst.j2ee.internal.module.container"/> | |
<classpathentry kind="output" path="target/classes"/> | |
</classpath> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
target | |
bin | |
.settings |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="UTF-8"?> | |
<projectDescription> | |
<name>metadata-job</name> | |
<comment>This plugin is an archetype that creates a command line batch sample from Spring Batch. Once installed you can create the archetype project using<br/><pre>$ mvn archetype:create -DgroupId=com.mycompany -DartifactId=batch \ -DarchetypeGroupId=org.springframework.batch \ -DarchetypeArtifactId=spring-batch-cli -DarchetypeVersion=1.0-m3-SNAPSHOT</pre><br/> Then you should be able to "cd batch; mvn package exec:exec", and see the app run. | |
</comment> | |
<projects> | |
</projects> | |
<buildSpec> | |
<buildCommand> | |
<name>org.eclipse.wst.common.project.facet.core.builder</name> | |
<arguments> | |
</arguments> | |
</buildCommand> | |
<buildCommand> | |
<name>org.eclipse.jdt.core.javabuilder</name> | |
<arguments> | |
</arguments> | |
</buildCommand> | |
<buildCommand> | |
<name>org.springframework.ide.eclipse.core.springbuilder</name> | |
<arguments> | |
</arguments> | |
</buildCommand> | |
<buildCommand> | |
<name>org.maven.ide.eclipse.maven2Builder</name> | |
<arguments> | |
</arguments> | |
</buildCommand> | |
<buildCommand> | |
<name>org.eclipse.wst.validation.validationbuilder</name> | |
<arguments> | |
</arguments> | |
</buildCommand> | |
</buildSpec> | |
<natures> | |
<nature>org.eclipse.jem.workbench.JavaEMFNature</nature> | |
<nature>org.eclipse.wst.common.modulecore.ModuleCoreNature</nature> | |
<nature>org.springframework.ide.eclipse.core.springnature</nature> | |
<nature>org.eclipse.jdt.core.javanature</nature> | |
<nature>org.maven.ide.eclipse.maven2Nature</nature> | |
<nature>org.eclipse.wst.common.project.facet.core.nature</nature> | |
</natures> | |
</projectDescription> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="UTF-8"?> | |
<beansProjectDescription> | |
<version>1</version> | |
<pluginVersion><![CDATA[2.3.3.201005160334-M1]]></pluginVersion> | |
<configSuffixes> | |
<configSuffix><![CDATA[xml]]></configSuffix> | |
</configSuffixes> | |
<enableImports><![CDATA[true]]></enableImports> | |
<configs> | |
<config>src/main/resources/launch-context.xml</config> | |
<config>src/main/resources/META-INF/spring/batch/jobs/module-context.xml</config> | |
</configs> | |
<configSets> | |
<configSet> | |
<name><![CDATA[test]]></name> | |
<allowBeanDefinitionOverriding>true</allowBeanDefinitionOverriding> | |
<incomplete>false</incomplete> | |
<configs> | |
<config>src/main/resources/launch-context.xml</config> | |
</configs> | |
</configSet> | |
</configSets> | |
</beansProjectDescription> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Placeholders batch.* | |
# for HSQLDB: | |
batch.jdbc.driver=org.hsqldb.jdbcDriver | |
batch.jdbc.url=jdbc:hsqldb:mem:testdb;sql.enforce_strict_size=true | |
# use this one for a separate server process so you can inspect the results | |
# (or add it to system properties with -D to override at run time). | |
# batch.jdbc.url=jdbc:hsqldb:hsql://localhost:9005/samples | |
batch.jdbc.user=sa | |
batch.jdbc.password= | |
batch.schema= | |
batch.schema.script=org/springframework/batch/core/schema-hsqldb.sql |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Copyright 2006-2007 the original author or authors. | |
* | |
* Licensed under the Apache License, Version 2.0 (the "License"); | |
* you may not use this file except in compliance with the License. | |
* You may obtain a copy of the License at | |
* | |
* http://www.apache.org/licenses/LICENSE-2.0 | |
* | |
* Unless required by applicable law or agreed to in writing, software | |
* distributed under the License is distributed on an "AS IS" BASIS, | |
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
* See the License for the specific language governing permissions and | |
* limitations under the License. | |
*/ | |
package org.springframework.sample.batch.example; | |
import static org.junit.Assert.assertEquals; | |
import javax.sql.DataSource; | |
import org.junit.Before; | |
import org.junit.Test; | |
import org.junit.runner.RunWith; | |
import org.springframework.batch.core.BatchStatus; | |
import org.springframework.batch.core.Job; | |
import org.springframework.batch.core.JobExecution; | |
import org.springframework.batch.core.JobParameters; | |
import org.springframework.batch.core.JobParametersBuilder; | |
import org.springframework.batch.core.JobParametersIncrementer; | |
import org.springframework.batch.core.StepExecution; | |
import org.springframework.batch.core.launch.JobLauncher; | |
import org.springframework.batch.core.repository.JobRepository; | |
import org.springframework.batch.sample.metadata.JobInstanceItemWriter; | |
import org.springframework.batch.test.JobRepositoryTestUtils; | |
import org.springframework.beans.factory.annotation.Autowired; | |
import org.springframework.jdbc.core.simple.SimpleJdbcTemplate; | |
import org.springframework.test.context.ContextConfiguration; | |
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner; | |
import org.springframework.test.jdbc.SimpleJdbcTestUtils; | |
/** | |
* @author Dave Syer | |
* | |
*/ | |
@ContextConfiguration(locations = { "/launch-context.xml" }) | |
@RunWith(SpringJUnit4ClassRunner.class) | |
public class ExampleJobConfigurationTests { | |
@Autowired | |
private JobLauncher jobLauncher; | |
@Autowired | |
private JobParametersIncrementer jobParametersIncrementer; | |
@Autowired | |
private JobRepository jobRepository; | |
@Autowired | |
private Job job; | |
private SimpleJdbcTemplate jdbcTemplate; | |
private DataSource dataSource; | |
@Autowired | |
public void setDataSource(DataSource dataSource) { | |
this.dataSource = dataSource; | |
this.jdbcTemplate = new SimpleJdbcTemplate(dataSource); | |
} | |
@Before | |
public void clearJobRepository() throws Exception { | |
new JobRepositoryTestUtils(jobRepository, dataSource).removeJobExecutions(); | |
} | |
@Test | |
public void testLaunchJob() throws Exception { | |
JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); | |
// The only job instance to copy was the current execution, and it | |
// is incomplete, so it gets filtered out... | |
assertEquals(1, stepExecution.getFilterCount()); | |
assertEquals(1, SimpleJdbcTestUtils.countRowsInTable(jdbcTemplate, "BATCH_JOB_INSTANCE")); | |
} | |
@Test | |
public void testLaunchJobTwiceWithDuplicateModeForce() throws Exception { | |
JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
long maxJobInstanceId = jdbcTemplate.queryForLong("SELECT MAX(JOB_INSTANCE_ID) FROM BATCH_JOB_INSTANCE"); | |
// Since we are copying into the same repository as a test, we want to | |
// force insert | |
jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString(JobInstanceItemWriter.DUPLICATE_MODE, | |
"FORCE").toJobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
// The previous instance was complete, so start.from is that value | |
assertEquals(maxJobInstanceId, jobExecution.getExecutionContext().getLong("start.from")); | |
StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); | |
// The current job instance is skipped because it has no complete | |
// executions... | |
assertEquals(1, stepExecution.getFilterCount()); | |
// The previous, completed instance is (force) copied, adding one | |
// instance to the existing 2 | |
assertEquals(3, SimpleJdbcTestUtils.countRowsInTable(jdbcTemplate, "BATCH_JOB_INSTANCE")); | |
jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString(JobInstanceItemWriter.DUPLICATE_MODE, | |
"FORCE").addString("run.id","1").toJobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
// The previous 3 completed instances are added | |
assertEquals(7, SimpleJdbcTestUtils.countRowsInTable(jdbcTemplate, "BATCH_JOB_INSTANCE")); | |
jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString(JobInstanceItemWriter.DUPLICATE_MODE, | |
"FORCE").addString("run.id","2").toJobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
// The previous 7 completed instances are added | |
assertEquals(15, SimpleJdbcTestUtils.countRowsInTable(jdbcTemplate, "BATCH_JOB_INSTANCE")); | |
} | |
@Test | |
public void testLaunchJobTwiceWithDuplicateModeIgnore() throws Exception { | |
JobExecution jobExecution = jobLauncher.run(job, new JobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
// Since we are copying into the same repository as a test, we want to | |
// ignore duplicates | |
jobExecution = jobLauncher.run(job, new JobParametersBuilder().addString(JobInstanceItemWriter.DUPLICATE_MODE, | |
"IGNORE").toJobParameters()); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
StepExecution stepExecution = jobExecution.getStepExecutions().iterator().next(); | |
// The current job instance is skipped because it has no complete | |
// executions... | |
assertEquals(1, stepExecution.getFilterCount()); | |
// Duplicates are ignored, so only the two | |
assertEquals(2, SimpleJdbcTestUtils.countRowsInTable(jdbcTemplate, "BATCH_JOB_INSTANCE")); | |
} | |
@Test | |
public void testLaunchJobTwiceWithDuplicateModeFail() throws Exception { | |
JobExecution jobExecution = jobLauncher.run(job, jobParametersIncrementer.getNext(new JobParameters())); | |
assertEquals(BatchStatus.COMPLETED, jobExecution.getStatus()); | |
jobExecution = jobLauncher.run(job, jobParametersIncrementer.getNext(new JobParametersBuilder().addString( | |
JobInstanceItemWriter.DUPLICATE_MODE, "FAIL").toJobParameters())); | |
assertEquals(BatchStatus.FAILED, jobExecution.getStatus()); | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package org.springframework.batch.sample.metadata; | |
import org.springframework.batch.core.JobExecution; | |
import org.springframework.batch.core.JobInstance; | |
import org.springframework.batch.core.explore.JobExplorer; | |
import org.springframework.batch.item.ExecutionContext; | |
import org.springframework.batch.item.ItemProcessor; | |
/** | |
* Transform a Long (primary key) into a {@link JobInstance}. | |
*/ | |
public class JobInstanceItemProcessor implements ItemProcessor<Long, JobInstance> { | |
public static final String START_FROM = "start.from"; | |
private JobExplorer jobExplorer; | |
private ExecutionContext executionContext; | |
/** | |
* @param executionContext the executionContext to set | |
*/ | |
public void setExecutionContext(ExecutionContext executionContext) { | |
this.executionContext = executionContext; | |
} | |
/** | |
* @param jobExplorer the jobExplorer to set | |
*/ | |
public void setJobExplorer(JobExplorer jobExplorer) { | |
this.jobExplorer = jobExplorer; | |
} | |
public JobInstance process(Long item) throws Exception { | |
JobInstance jobInstance = jobExplorer.getJobInstance(item); | |
boolean complete = true; | |
for (JobExecution jobExecution : jobExplorer.getJobExecutions(jobInstance)) { | |
if (jobExecution.isRunning()) { | |
// There is at least one incomplete execution | |
complete = false; | |
} | |
} | |
if (complete && executionContext!=null) { | |
// If the step completes successfully then this will be saved in | |
// the repository and the next execution can look back and find out | |
// the last successfully processed instance id... | |
executionContext.putLong(START_FROM, item); | |
} | |
return complete ? jobInstance : null; | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package org.springframework.batch.sample.metadata; | |
import java.text.DecimalFormat; | |
import java.text.NumberFormat; | |
import java.util.ArrayList; | |
import java.util.Arrays; | |
import java.util.List; | |
import org.apache.commons.logging.Log; | |
import org.apache.commons.logging.LogFactory; | |
import org.springframework.batch.core.JobExecution; | |
import org.springframework.batch.core.JobInstance; | |
import org.springframework.batch.core.JobParameters; | |
import org.springframework.batch.core.JobParametersBuilder; | |
import org.springframework.batch.core.StepExecution; | |
import org.springframework.batch.core.explore.JobExplorer; | |
import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException; | |
import org.springframework.batch.core.repository.JobRepository; | |
import org.springframework.batch.item.ItemWriter; | |
import org.springframework.util.StringUtils; | |
/** | |
* An {@link ItemWriter} which copies {@link JobInstance job instances} to a | |
* {@link JobRepository}. | |
*/ | |
public class JobInstanceItemWriter implements ItemWriter<JobInstance> { | |
private static final String JOB_REPOSITORY_COPY_INSTANCE_ID = "instance.id"; | |
private static final Log logger = LogFactory.getLog(JobInstanceItemWriter.class); | |
private static final String JOB_REPOSITORY_SOURCE = "source"; | |
public static final String DUPLICATE_MODE = "duplicate.mode"; | |
private JobRepository jobRepository; | |
private JobExplorer jobExplorer; | |
private String jobNamePrefix = ""; | |
private DuplicateMode duplicateMode = DuplicateMode.FAIL; | |
private Long jobInstanceId; | |
private String sourceName; | |
private NumberFormat numberFormat; | |
public JobInstanceItemWriter() { | |
DecimalFormat numberFormat = (DecimalFormat) NumberFormat.getIntegerInstance(); | |
numberFormat.applyPattern("#"); | |
this.numberFormat = numberFormat; | |
} | |
/** | |
* @param sourceName the sourceName to set | |
*/ | |
public void setSourceName(String sourceName) { | |
this.sourceName = sourceName; | |
} | |
/** | |
* @param jobInstanceId the job instance id to set | |
*/ | |
public void setJobInstanceId(Long jobInstanceId) { | |
this.jobInstanceId = jobInstanceId; | |
} | |
/** | |
* Flag that determines the action to take when duplicate job instances are | |
* detected. Defaults to {@link DuplicateMode#FAIL FAIL} so duplicates will | |
* cause an exception to be thrown. Legal values are | |
* {@link DuplicateMode#FAIL FAIL}, {@link DuplicateMode#FAIL IGNORE}, | |
* {@link DuplicateMode#FAIL FORCE}. | |
* | |
* @param duplicateMode the duplicate mode to set | |
*/ | |
public void setDuplicateMode(DuplicateMode duplicateMode) { | |
this.duplicateMode = duplicateMode; | |
} | |
/** | |
* @param jobNamePrefix the jobNamePrefix to set | |
*/ | |
public void setJobNamePrefix(String jobNamePrefix) { | |
this.jobNamePrefix = jobNamePrefix; | |
} | |
/** | |
* @param jobExplorer the jobExplorer to set | |
*/ | |
public void setJobExplorer(JobExplorer jobExplorer) { | |
this.jobExplorer = jobExplorer; | |
} | |
/** | |
* @param jobRepository the jobRepository to set | |
*/ | |
public void setJobRepository(JobRepository jobRepository) { | |
this.jobRepository = jobRepository; | |
} | |
/** | |
* @see ItemWriter#write(Object) | |
*/ | |
public void write(List<? extends JobInstance> data) throws Exception { | |
if (data == null) { | |
return; | |
} | |
for (JobInstance jobInstance : data) { | |
logger.debug("Processing job instance id=" + jobInstance.getId()); | |
for (JobExecution jobExecution : jobExplorer.getJobExecutions(jobInstance)) { | |
if (!jobExecution.isRunning()) { | |
saveJobExecution(jobExecution); | |
} | |
} | |
} | |
} | |
/** | |
* @param jobExecution | |
*/ | |
private void saveJobExecution(JobExecution input) { | |
JobInstance jobInstance = input.getJobInstance(); | |
JobParameters jobParameters = jobInstance.getJobParameters(); | |
JobParametersBuilder builder = new JobParametersBuilder(jobParameters); | |
if (sourceName != null) { | |
builder.addString(JOB_REPOSITORY_SOURCE, sourceName); | |
} | |
JobExecution output; | |
try { | |
output = jobRepository.createJobExecution(jobNamePrefix + jobInstance.getJobName(), builder | |
.toJobParameters()); | |
} | |
catch (JobInstanceAlreadyCompleteException e) { | |
if (duplicateMode == DuplicateMode.IGNORE) { | |
return; | |
} | |
else if (duplicateMode == DuplicateMode.FORCE) { | |
List<String> instances = extractInstances(jobParameters | |
.getString(JOB_REPOSITORY_COPY_INSTANCE_ID, "[]")); | |
instances.add(numberFormat.format(jobInstanceId)); | |
builder.addString(JOB_REPOSITORY_COPY_INSTANCE_ID, instances.toString()); | |
try { | |
output = jobRepository.createJobExecution(jobNamePrefix + jobInstance.getJobName(), builder | |
.toJobParameters()); | |
} | |
catch (Exception ex) { | |
throw new IllegalStateException( | |
"Could not create JobExecution in local repository in duplicate mode " + duplicateMode, ex); | |
} | |
} | |
else { | |
throw new IllegalStateException( | |
"Could not create JobExecution in local repository and duplicates are not allowed", e); | |
} | |
} | |
catch (Exception e) { | |
throw new IllegalStateException("Could not create JobExecution in local repository", e); | |
} | |
output.setEndTime(input.getEndTime()); | |
output.setExecutionContext(input.getExecutionContext()); | |
output.setExitStatus(input.getExitStatus()); | |
output.setStartTime(input.getStartTime()); | |
output.setStatus(input.getStatus()); | |
jobRepository.update(output); | |
jobRepository.updateExecutionContext(output); | |
logger.debug("Saved JobExecution: " + output); | |
for (StepExecution stepExecution : input.getStepExecutions()) { | |
saveStepExecution(output, stepExecution); | |
} | |
} | |
/** | |
* @param instances | |
* @return | |
*/ | |
private ArrayList<String> extractInstances(String instances) { | |
return new ArrayList<String>(Arrays.asList(StringUtils.trimArrayElements(StringUtils | |
.commaDelimitedListToStringArray(instances.substring(1, instances.length() - 1))))); | |
} | |
/** | |
* @param jobExecution | |
* @param stepExecution | |
*/ | |
private void saveStepExecution(JobExecution jobExecution, StepExecution input) { | |
StepExecution output = jobExecution.createStepExecution(input.getStepName()); | |
output.setCommitCount(input.getCommitCount()); | |
output.setEndTime(input.getEndTime()); | |
output.setExecutionContext(input.getExecutionContext()); | |
output.setExitStatus(input.getExitStatus()); | |
output.setFilterCount(input.getFilterCount()); | |
output.setProcessSkipCount(input.getProcessSkipCount()); | |
output.setReadCount(input.getReadCount()); | |
output.setReadSkipCount(input.getReadSkipCount()); | |
output.setRollbackCount(input.getRollbackCount()); | |
output.setStartTime(input.getStartTime()); | |
output.setStatus(input.getStatus()); | |
output.setWriteCount(input.getWriteCount()); | |
output.setWriteSkipCount(input.getWriteSkipCount()); | |
jobRepository.add(output); | |
jobRepository.updateExecutionContext(output); | |
logger.debug("Saved StepExecution: " + output); | |
} | |
public static enum DuplicateMode { | |
/** | |
* Swallow exceptions caused by duplicate job instances | |
*/ | |
IGNORE, | |
/** | |
* Force insertion of duplicates by making the job parameters unique | |
*/ | |
FORCE, | |
/** | |
* Fail if a duplicate is detected | |
*/ | |
FAIL; | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package org.springframework.sample.batch.example; | |
import org.springframework.batch.sample.metadata.JobInstanceItemWriter; | |
import junit.framework.TestCase; | |
public class JobInstanceItemWriterTests extends TestCase { | |
private JobInstanceItemWriter writer = new JobInstanceItemWriter(); | |
public void testWrite() throws Exception { | |
writer.write(null); // nothing bad happens | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Copyright 2009-2010 the original author or authors. | |
* | |
* Licensed under the Apache License, Version 2.0 (the "License"); | |
* you may not use this file except in compliance with the License. | |
* You may obtain a copy of the License at | |
* | |
* http://www.apache.org/licenses/LICENSE-2.0 | |
* | |
* Unless required by applicable law or agreed to in writing, software | |
* distributed under the License is distributed on an "AS IS" BASIS, | |
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
* See the License for the specific language governing permissions and | |
* limitations under the License. | |
*/ | |
package org.springframework.batch.sample.metadata; | |
import org.springframework.batch.core.JobExecution; | |
import org.springframework.batch.core.JobParameters; | |
import org.springframework.batch.core.JobParametersBuilder; | |
import org.springframework.batch.core.JobParametersIncrementer; | |
import org.springframework.batch.core.repository.JobRepository; | |
/** | |
* @author Dave Syer | |
* | |
*/ | |
public class JobInstanceJobParametersIncrementer implements JobParametersIncrementer { | |
private String jobName; | |
private JobRepository jobRepository; | |
/** | |
* @param jobName the jobName to set | |
*/ | |
public void setJobName(String jobName) { | |
this.jobName = jobName; | |
} | |
/** | |
* @param jobRepository the jobRepository to set | |
*/ | |
public void setJobRepository(JobRepository jobRepository) { | |
this.jobRepository = jobRepository; | |
} | |
public JobParameters getNext(JobParameters parameters) { | |
if (parameters==null) { | |
parameters = new JobParameters(); | |
} | |
JobParametersBuilder builder = new JobParametersBuilder(parameters); | |
long startFrom = parameters.getLong(JobInstanceItemProcessor.START_FROM, -1L); | |
String ignoreMode = parameters.getString(JobInstanceItemWriter.DUPLICATE_MODE, "FAIL"); | |
if (ignoreMode!=null && ignoreMode.equals("FORCE")) { | |
// If we are forcing inserts then always start from the beginning | |
startFrom = -1; | |
} | |
JobExecution jobExecution = jobRepository.getLastJobExecution(jobName, parameters); | |
if (jobExecution!=null) { | |
startFrom = jobExecution.getExecutionContext().getLong(JobInstanceItemProcessor.START_FROM, startFrom); | |
} | |
builder.addLong(JobInstanceItemProcessor.START_FROM, startFrom); | |
return builder.toJobParameters(); | |
} | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="UTF-8"?> | |
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | |
xmlns:batch="http://www.springframework.org/schema/batch" xmlns:p="http://www.springframework.org/schema/p" | |
xmlns:jdbc="http://www.springframework.org/schema/jdbc" | |
xsi:schemaLocation="http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.0.xsd | |
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd | |
http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.0.xsd"> | |
<import resource="classpath:/META-INF/spring/batch/jobs/module-context.xml" /> | |
<bean id="jobLauncher" | |
class="org.springframework.batch.core.launch.support.SimpleJobLauncher"> | |
<property name="jobRepository" ref="jobRepository" /> | |
</bean> | |
<batch:job-repository id="jobRepository" /> | |
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"> | |
<property name="driverClassName" value="${batch.jdbc.driver}" /> | |
<property name="url" value="${batch.jdbc.url}" /> | |
<property name="username" value="${batch.jdbc.user}" /> | |
<property name="password" value="${batch.jdbc.password}" /> | |
</bean> | |
<bean id="transactionManager" | |
class="org.springframework.jdbc.datasource.DataSourceTransactionManager" | |
lazy-init="true"> | |
<property name="dataSource" ref="dataSource" /> | |
</bean> | |
<bean id="placeholderProperties" | |
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> | |
<property name="location" value="classpath:batch.properties" /> | |
<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" /> | |
<property name="ignoreUnresolvablePlaceholders" value="true" /> | |
<property name="order" value="1" /> | |
</bean> | |
<!-- | |
Initialize the datasource - remove this bean definition for production | |
use! | |
--> | |
<jdbc:initialize-database data-source="dataSource" enabled="true"> | |
<jdbc:script location="${batch.schema.script}"/> | |
</jdbc:initialize-database> | |
</beans> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
log4j.rootCategory=INFO, stdout | |
log4j.appender.stdout=org.apache.log4j.ConsoleAppender | |
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout | |
log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - <%m>%n | |
log4j.category.org.apache.activemq=ERROR | |
log4j.category.org.springframework.batch=DEBUG | |
log4j.category.org.springframework.transaction=INFO | |
log4j.category.org.hibernate.SQL=DEBUG | |
# for debugging datasource initialization | |
# log4j.category.test.jdbc=DEBUG |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="UTF-8"?> | |
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | |
xsi:schemaLocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.1.xsd | |
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd"> | |
<description>Job that copies JobRepository data from another repository. The source data comes from a DataSource that | |
can be specified through job parameters as a bean definition id (job parameter key = "source.data.source", defaults to | |
"dataSource" so that the current repository is copied to itself by default). If the job encounters duplicate job | |
instances in the source and target repositories (which it will if the source data source is not explicitly specified) | |
the default behaviour is to throw an exception, and the job will terminate. This behaviour can be changed through a | |
job parameter (key = "duplicate.mode", possible values FAIL, IGNORE, FORCE, where in the latter case | |
the duplicate job instances are forcefully inserted with an additional unique job parameter). | |
</description> | |
<job id="repository-copy" xmlns="http://www.springframework.org/schema/batch" incrementer="incrementer"> | |
<step id="repository-copy-step"> | |
<tasklet> | |
<chunk reader="reader" processor="processor" writer="writer" commit-interval="100" /> | |
<listeners> | |
<listener> | |
<!-- This is used by the incrementer to look at the previous execution and start processing where we left off --> | |
<bean class="org.springframework.batch.core.listener.ExecutionContextPromotionListener" xmlns="http://www.springframework.org/schema/beans"> | |
<property name="keys" value="start.from" /> | |
</bean> | |
</listener> | |
</listeners> | |
</tasklet> | |
</step> | |
</job> | |
<bean id="incrementer" class="org.springframework.batch.sample.metadata.JobInstanceJobParametersIncrementer"> | |
<property name="jobName" value="repository-copy" /> | |
<property name="jobRepository" ref="jobRepository" /> | |
</bean> | |
<bean id="reader" class="org.springframework.batch.item.database.JdbcPagingItemReader" scope="step"> | |
<property name="dataSource" ref="#{jobParameters['source.data.source']?:'dataSource'}" /> | |
<property name="queryProvider"> | |
<bean class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean"> | |
<property name="dataSource" ref="#{jobParameters['source.data.source']?:'dataSource'}" /> | |
<property name="fromClause" value="BATCH_JOB_INSTANCE" /> | |
<property name="selectClause" value="JOB_INSTANCE_ID" /> | |
<property name="sortKey" value="JOB_INSTANCE_ID" /> | |
<property name="whereClause" value="JOB_INSTANCE_ID>:startFrom" /> | |
</bean> | |
</property> | |
<property name="parameterValues"> | |
<map> | |
<entry key="startFrom" value="#{jobParameters['start.from']?:-1}" /> | |
</map> | |
</property> | |
<property name="rowMapper"> | |
<bean class="org.springframework.jdbc.core.SingleColumnRowMapper" /> | |
</property> | |
</bean> | |
<bean id="processor" class="org.springframework.batch.sample.metadata.JobInstanceItemProcessor" scope="step"> | |
<property name="jobExplorer"> | |
<!-- This is a JobExplorer for the source data (not the target) --> | |
<bean class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean"> | |
<!-- This dataSource is not usually the same one as the JobRepository --> | |
<property name="dataSource" ref="#{jobParameters['source.data.source']?:'dataSource'}" /> | |
</bean> | |
</property> | |
<property name="executionContext" value="#{stepExecution.executionContext}" /> | |
</bean> | |
<bean id="writer" class="org.springframework.batch.sample.metadata.JobInstanceItemWriter" scope="step"> | |
<property name="jobInstanceId" value="#{stepExecution.jobExecution.jobId}" /> | |
<property name="duplicateMode" value="#{jobParameters['duplicate.mode']?:'FAIL'}" /> | |
<property name="jobExplorer"> | |
<!-- | |
This is a JobExplorer for the target data (injected into the writer). The same data source as for the JobRepository | |
--> | |
<bean class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean"> | |
<property name="dataSource" ref="dataSource" /> | |
</bean> | |
</property> | |
<property name="jobRepository" ref="bareJobRepository" /> | |
</bean> | |
<bean id="bareJobRepository" parent="jobRepository"> | |
<property name="validateTransactionState" value="false"/> | |
</bean> | |
</beans> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="UTF-8"?> | |
<project xmlns="http://maven.apache.org/POM/4.0.0" | |
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd" | |
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> | |
<modelVersion>4.0.0</modelVersion> | |
<groupId>org.springframework.batch</groupId> | |
<artifactId>spring-batch-metadata-job</artifactId> | |
<version>2.0.0.CI-SNAPSHOT</version> | |
<packaging>jar</packaging> | |
<name>Meta Data Job</name> | |
<properties> | |
<maven.test.failure.ignore>true</maven.test.failure.ignore> | |
<spring.framework.version>3.0.5.RELEASE</spring.framework.version> | |
<spring.batch.version>2.1.6.RELEASE</spring.batch.version> | |
</properties> | |
<profiles> | |
<profile> | |
<id>strict</id> | |
<properties> | |
<maven.test.failure.ignore>false</maven.test.failure.ignore> | |
</properties> | |
</profile> | |
<profile> | |
<id>tiger</id> | |
<activation> | |
<jdk>1.5</jdk> | |
</activation> | |
<dependencies> | |
<dependency> | |
<groupId>javax.xml.stream</groupId> | |
<artifactId>com.springsource.javax.xml.stream</artifactId> | |
<version>1.0.1</version> | |
</dependency> | |
</dependencies> | |
</profile> | |
<profile> | |
<id>staging</id> | |
<distributionManagement> | |
<repository> | |
<id>staging</id> | |
<url>file:///${user.dir}/target/staging</url> | |
</repository> | |
<snapshotRepository> | |
<id>staging</id> | |
<url>file:///${user.dir}/target/staging</url> | |
</snapshotRepository> | |
</distributionManagement> | |
</profile> | |
</profiles> | |
<dependencies> | |
<dependency> | |
<groupId>junit</groupId> | |
<artifactId>junit</artifactId> | |
<version>4.8.2</version> | |
<scope>test</scope> | |
</dependency> | |
<dependency> | |
<groupId>org.springframework</groupId> | |
<artifactId>spring-test</artifactId> | |
<version>${spring.framework.version}</version> | |
<scope>test</scope> | |
</dependency> | |
<dependency> | |
<groupId>org.springframework</groupId> | |
<artifactId>spring-aop</artifactId> | |
<version>${spring.framework.version}</version> | |
</dependency> | |
<dependency> | |
<groupId>org.springframework</groupId> | |
<artifactId>spring-jdbc</artifactId> | |
<version>${spring.framework.version}</version> | |
</dependency> | |
<dependency> | |
<groupId>org.springframework</groupId> | |
<artifactId>spring-context</artifactId> | |
<version>${spring.framework.version}</version> | |
</dependency> | |
<dependency> | |
<groupId>org.springframework.batch</groupId> | |
<artifactId>spring-batch-core</artifactId> | |
<version>${spring.batch.version}</version> | |
</dependency> | |
<dependency> | |
<groupId>org.springframework.batch</groupId> | |
<artifactId>spring-batch-test</artifactId> | |
<version>${spring.batch.version}</version> | |
<scope>test</scope> | |
</dependency> | |
<dependency> | |
<groupId>commons-dbcp</groupId> | |
<artifactId>commons-dbcp</artifactId> | |
<version>1.2.2</version> | |
</dependency> | |
<dependency> | |
<groupId>commons-io</groupId> | |
<artifactId>commons-io</artifactId> | |
<version>1.4</version> | |
</dependency> | |
<dependency> | |
<groupId>hsqldb</groupId> | |
<artifactId>hsqldb</artifactId> | |
<version>1.8.0.7</version> | |
</dependency> | |
<dependency> | |
<groupId>org.aspectj</groupId> | |
<artifactId>aspectjrt</artifactId> | |
<version>1.6.8</version> | |
</dependency> | |
<dependency> | |
<groupId>org.aspectj</groupId> | |
<artifactId>aspectjweaver</artifactId> | |
<version>1.6.8</version> | |
</dependency> | |
<dependency> | |
<groupId>log4j</groupId> | |
<artifactId>log4j</artifactId> | |
<version>1.2.14</version> | |
</dependency> | |
</dependencies> | |
<build> | |
<pluginManagement> | |
<plugins> | |
<plugin> | |
<artifactId>maven-assembly-plugin</artifactId> | |
<inherited>false</inherited> | |
<configuration> | |
<descriptorRefs> | |
<descriptorRef>project</descriptorRef> | |
</descriptorRefs> | |
</configuration> | |
</plugin> | |
</plugins> | |
</pluginManagement> | |
<plugins> | |
<plugin> | |
<groupId>org.apache.maven.plugins</groupId> | |
<artifactId>maven-compiler-plugin</artifactId> | |
<configuration> | |
<source>1.5</source> | |
<target>1.5</target> | |
</configuration> | |
</plugin> | |
<plugin> | |
<groupId>org.apache.maven.plugins</groupId> | |
<artifactId>maven-surefire-plugin</artifactId> | |
<configuration> | |
<!--forkMode>pertest</forkMode--> | |
<includes> | |
<include>**/*Tests.java</include> | |
</includes> | |
<excludes> | |
<exclude>**/Abstract*.java</exclude> | |
</excludes> | |
</configuration> | |
</plugin> | |
<plugin> | |
<groupId>org.codehaus.mojo</groupId> | |
<artifactId>exec-maven-plugin</artifactId> | |
<version>1.1</version> | |
<configuration> | |
<mainClass>org.springframework.batch.core.launch.support.CommandLineJobRunner</mainClass> | |
<arguments> | |
<!-- job configuration file --> | |
<argument>classpath:/launch-context.xml</argument> | |
<!-- job name --> | |
<argument>job1</argument> | |
</arguments> | |
</configuration> | |
</plugin> | |
<plugin> | |
<groupId>org.apache.maven.plugins</groupId> | |
<artifactId>maven-dependency-plugin</artifactId> | |
<executions> | |
<execution> | |
<id>copy-dependencies</id> | |
<phase>package</phase> | |
<goals> | |
<goal>copy-dependencies</goal> | |
</goals> | |
<configuration> | |
<outputDirectory>${project.build.directory}/lib</outputDirectory> | |
</configuration> | |
</execution> | |
</executions> | |
</plugin> | |
<plugin> | |
<groupId>org.apache.maven.plugins</groupId> | |
<artifactId>maven-jar-plugin</artifactId> | |
<configuration> | |
<archive> | |
<index>true</index> | |
<manifest> | |
<mainClass>org.springframework.batch.core.launch.support.CommandLineJobRunner</mainClass> | |
<addClasspath>true</addClasspath> | |
<classpathPrefix>lib/</classpathPrefix> | |
</manifest> | |
</archive> | |
</configuration> | |
</plugin> | |
</plugins> | |
</build> | |
<pluginRepositories> | |
<pluginRepository> | |
<id>Codehaus</id> | |
<url>http://repository.codehaus.org/</url> | |
<snapshots> | |
<enabled>false</enabled> | |
</snapshots> | |
</pluginRepository> | |
</pluginRepositories> | |
<distributionManagement> | |
<downloadUrl>http://www.springframework.org/download</downloadUrl> | |
<site> | |
<id>staging</id> | |
<url>file:///${user.dir}/target/staging/org.springframework.batch.archetype/${pom.artifactId}</url> | |
</site> | |
<repository> | |
<id>spring-release</id> | |
<name>Spring Release Repository</name> | |
<url>file:///${user.dir}/target/staging/release</url> | |
</repository> | |
<snapshotRepository> | |
<id>spring-snapshot</id> | |
<name>Spring Snapshot Repository</name> | |
<url>file:///${user.dir}/target/staging/snapshot</url> | |
</snapshotRepository> | |
</distributionManagement> | |
</project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<?xml version="1.0" encoding="ISO-8859-1"?> | |
<project name="Spring Batch: ${project.name}"> | |
<bannerLeft> | |
<name>Spring Batch: ${project.name}</name> | |
<href>index.html</href> | |
</bannerLeft> | |
<skin> | |
<groupId>org.springframework.maven.skins</groupId> | |
<artifactId>maven-spring-skin</artifactId> | |
<version>1.0.5</version> | |
</skin> | |
<body> | |
<links> | |
<item name="${project.name}" href="index.html"/> | |
</links> | |
<menu ref="reports"/> | |
</body> | |
</project> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment