Spring batch 实例
Spring Batch是一个轻量级的，完全面向Spring的批处理框架，可以应用于企业级大量的数据处理系统。
Spring Batch可以提供大量的，可重复的数据处理功能，包括日志记录/跟踪，事务管理，作业处理统计工作重新启动、跳过，和资源管理等重要功能。
业务方案：
1、批处理定期提交。
2、并行批处理：并行处理工作。
3、企业消息驱动处理
4、大规模的并行处理
5、手动或是有计划的重启
6、局部处理：跳过记录（如：回滚）
每个Batch都会包含一个Job。Job就像一个容器，这个容器里装了若干Step，Batch中实际工作的也就是这些Step
step->ItemReader用来读取数据，ItemProcessor用来处理数据，ItemWriter用来写数据
JobLauncher用来启动Job，JobRepository是上述处理提供的一种持久化机制，它为JobLauncher，Job，和Step实例提供CRUD操作。
下面是一个实例：
还是先说需要的jar包
pom
spring jar包：
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>4.1.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>4.1.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>servlet-api-2.5</artifactId>
<version>6.0.1</version>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.1.2</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.14</version>
</dependency>然后是batch的
<!-- Spring Batch dependencies -->
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
<version>2.2.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-infrastructure</artifactId>
<version>2.2.0.RELEASE</version>
</dependency>
web.xml
<servlet>
<servlet-name>Spring</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<init-param>
<param-name>contextConfigLocation</param-name>
<param-value>/WEB-INF/applicationContext.xml</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
<servlet-mapping>
<servlet-name>Spring</servlet-name>
<url-pattern>/</url-pattern>
</servlet-mapping>spring-batch.xml
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:batch="http://www.springframework.org/schema/batch"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch.xsd">
<!-- spring batch -jobRepository -->
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean" />
<!-- spring - transactionManager -->
<bean id="transactionManager" class="org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
<!-- batch luncher -->
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<batch:job id="helloWorldJob">
<batch:step id="step_hello" next="step_world">
<batch:tasklet ref="hello" transaction-manager="transactionManager"></batch:tasklet>
</batch:step>
<batch:step id="step_world">
<batch:tasklet>
<batch:chunk reader="xxcReader" processor="xxcProcessor" writer="xxcWriter" commit-interval="10" />
</batch:tasklet>
</batch:step>
</batch:job>
<bean id="hello" class="com.batch.tasklet.WriteTasklet">
<property name="message" value="Hello" />
</bean>
<bean id="xxcReader" class="com.batch.reader.XxcReader">
<property name="lineMapper" ref="lineMapper"/>
<property name="resource" value="/com/batch/common/user.txt"/>
</bean>
<bean id="lineMapper" class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer" ref="lineTokenizer"/>
<property name="fieldSetMapper" ref="fieldSetMapper"/>
</bean>
<bean id="fieldSetMapper" class="com.batch.map.XxcMapper"/>
<bean id="lineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer"/>
<bean id="xxcProcessor" class="com.batch.processor.XxcProcessor"/>
<bean id="xxcWriter" class="com.batch.writer.XxcWriter"/>
</beans>然后下面都是Java代码了。。。
先看一下bean文件
/*
* Creation : 6 Jan 2016
*/
package com.batch.bean;
/**
* The Class Message.
*/
public class Message {
/** The code. */
private String code;
/** The value. */
private String value;
/**
* Gets the code.
*
* @return the code
*/
public String getCode() {
return code;
}
/**
* Sets the code.
*
* @param code the new code
*/
public void setCode(String code) {
this.code = code;
}
/**
* Gets the value.
*
* @return the value
*/
public String getValue() {
return value;
}
/**
* Sets the value.
*
* @param value the new value
*/
public void setValue(String value) {
this.value = value;
}
}
/*
* Creation : 6 Jan 2016
*/
package com.batch.bean;
/**
* The Class Users.
*/
public class Users {
/** The name. */
private String name;
/** The value. */
private String value;
/**
* Gets the name.
*
* @return the name
*/
public String getName() {
return name;
}
/**
* Sets the name.
*
* @param name the new name
*/
public void setName(String name) {
this.name = name;
}
/**
* Gets the value.
*
* @return the value
*/
public String getValue() {
return value;
}
/**
* Sets the value.
*
* @param value the new value
*/
public void setValue(String value) {
this.value = value;
}
}
然后根据batch文件中的依次：
/*
* Creation : 6 Jan 2016
*/
package com.batch.tasklet;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
public class WriteTasklet implements Tasklet {
/** Message */
private String message;
/**
* @param message the message to set
*/
public void setMessage(String message) {
this.message = message;
}
@Override
public RepeatStatus execute(StepContribution arg0, ChunkContext arg1) throws Exception {
System.out.println(message);
return RepeatStatus.FINISHED;
}
}
</pre><p></p><p>Map把读到的数据放到对象中</p><p></p><pre name="code" class="java">/*
* Creation : 6 Jan 2016
*/
package com.batch.map;
import org.springframework.batch.item.file.mapping.FieldSetMapper;
import org.springframework.batch.item.file.transform.FieldSet;
import org.springframework.validation.BindException;
import com.batch.bean.Users;
public class XxcMapper implements FieldSetMapper<Users> {
@Override
public Users mapFieldSet(FieldSet fieldSet) throws BindException {
Users user = new Users();
user.setName(fieldSet.readString(0));
user.setValue(fieldSet.readString(1));
return user;
}
}
因为Read的操作都在配置文件中做完了，我这里就是个空，你也可以直接在配置文件中class的路径直接写成FlatFileItemReader,这样XXCReader文件就不要创建了
/*
* Creation : 6 Jan 2016
*/
package com.batch.reader;
import org.springframework.batch.item.file.FlatFileItemReader;
import com.batch.bean.Users;
public class XxcReader extends FlatFileItemReader<Users> {
}
/*
* Creation : 6 Jan 2016
*/
package com.batch.processor;
import org.springframework.batch.item.ItemProcessor;
import com.batch.bean.Message;
import com.batch.bean.Users;
public class XxcProcessor implements ItemProcessor<Users, Message> {
@Override
public Message process(Users item) throws Exception {
Message message = new Message();
message.setCode(item.getName() + "---name");
message.setValue("000---" + item.getValue());
return message;
}
}
/*
* Creation : 6 Jan 2016
*/
package com.batch.writer;
import java.util.List;
import org.springframework.batch.item.ItemWriter;
import com.batch.bean.Message;
public class XxcWriter implements ItemWriter<Message> {
@Override
public void write(List<? extends Message> items) throws Exception {
for (Message item : items) {
System.out.println(item.getCode());
System.out.println(item.getValue());
}
}
}
这个是user.txt
jim,aaa tom,bbb mkl,ccc
好了，现在就全部写完了
运行一下
/*
* Creation : 6 Jan 2016
*/
package com.batch;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
public class JobLaunch {
/**
* @param args
*/
public static void main(String[] args) {
ApplicationContext context = new ClassPathXmlApplicationContext(new String[] { "classpath:spring-batch.xml" });
JobLauncher launcher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("helloWorldJob");
try {
JobExecution result = launcher.run(job, new JobParameters());
System.out.println(result.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
}
直接 run Main function
看一下控制台的结果：
