数据处理---Spring Batch之实践

上面介绍了Spring Batch的基本概念和简单的demo项目，显然这些还是不够实际使用的。下面我们来更多的代码实践。

        在上面的基础项目上面，我们来更多的修改：

        不用项目默认的hsql DB，用mysql,让ItemReader,ItemWriter 支持mysql；

        支持处理结果自定义保存到数据库，我们用项目里面的JPA；

        让Quartz来定时调用spring batch的Job

        可以读取文件，而不只是数据库；

下面开始动手。

          修改pom.xml，节选部分，实在太长了。

          3.2.0.RELEASE
          2.1.7.RELEASE  

mysql
mysql-connector-java
5.1.23


org.springframework.data
spring-data-jpa
1.3.2.RELEASE.rebuild


org.hibernate
hibernate-core
3.6.10.Final


org.hibernate
hibernate-entitymanager
3.6.10.Final


org.quartz-scheduler
quartz
2.1.7


org.springframework
spring-context-support
3.2.0.RELEASE

launch-context.xml定义需要用到的bean,

class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">



   
 

class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver" />


class="org.springframework.jdbc.datasource.DriverManagerDataSource">

value="jdbc:mysql://localhost:3306/testbatch?useUnicode=true&characterEncoding=UTF-8&autoReconnect=true" />









       





  





   
   
   
       
   
   
   








data-source="dataSource" transaction-manager="transactionManager"
isolation-level-for-create="SERIALIZABLE" table-prefix="BATCH_" />
       
       






   
   
        */10 * * * * ?
   










        class="org.springframework.batch.core.launch.support.SimpleJobLauncher">

 

  class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">

       

       

有几个定制的东西，需要说下。

我们用到的domain对象是这样的，很多不必要的都省略了

@Entity
@Table(name = "customer")
public class CustomerCredit {
public static final String TABLE_NAME = "customer";

@Column(name = "name", nullable = true, length = 12)
private String name;

@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "id", nullable = false)
private Integer id;

@Column(name = "credit", nullable = true, precision = 15, scale = 2)
private BigDecimal credit;

FlatFileItemReader 使用默认的DefaultLineMapper，里面指定了lineTokenizer和fieldSetMapper，就是说要怎么读一行行的数据，读了之后怎么和Bean的字段Mapper，这样后面的步骤好继续处理。

public class CustomerMapper implements FieldSetMapper {
@Override
public CustomerCredit mapFieldSet(FieldSet fieldSet) throws BindException {
CustomerCredit lv = new CustomerCredit();
lv.setId(Integer.parseInt(fieldSet.readString(0)));
lv.setName(fieldSet.readString(1));
lv.setCredit(fieldSet.readBigDecimal(2));
return lv;
}
}

如果是读数据库呢？就是下面那个JdbcCursorItemReader，里面指定了dataSource，sql，rowMapper，这些都类似文件

public class CustomerCreditRowMapper implements RowMapper {
    public static final String ID_COLUMN = "id";
    public static final String NAME_COLUMN = "name";
    public static final String CREDIT_COLUMN = "credit";

    public Object mapRow(ResultSet rs, int rowNum) throws SQLException {
        CustomerCredit customerCredit = new CustomerCredit();
        customerCredit.setId(rs.getInt(ID_COLUMN));
        customerCredit.setName(rs.getString(NAME_COLUMN));
        customerCredit.setCredit(rs.getBigDecimal(CREDIT_COLUMN));
        return customerCredit;
    }
}

还可以和hibernate结合来读取数据，可以读取存储过程的数据，等等，具体可以参考spring batch文档

说好的自定义处理数据呢，比如我们把每个人的Credit加10.

@Component("customProcessor")
public class CustomProcessor implements
ItemProcessor {
@PersistenceContext
private EntityManager em;
@Override
public CustomerCredit process(CustomerCredit item) throws Exception {
System.out.println(new Date().toString()+"start to process");
if (item == null) {
return null;
}
try {
item = em.find(CustomerCredit.class, item.getId());
item.setCredit(item.getCredit().add(new BigDecimal(10)));
//find by id才可以persist (否则出现detached entity passed to persist)
em.persist(item);
} catch (Exception e) {
e.printStackTrace();
}
return item;
}
}

每一步的处理，我们想看看结果，可以定制一个ItemReadListener

public class CustomStepListener implements ItemReadListener   具体代码省略

修改module-context.xml

start-limit="100">
commit-interval="3" />

在META-INF下面加个persistence.xml

xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd">


org.hibernate.ejb.HibernatePersistence

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'exampleConfiguration': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private org.springframework.batch.core.repository.JobRepository com.test.batch.ExampleConfiguration.jobRepository; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jobRepository': Cannot resolve reference to bean 'transactionManager' while setting bean property 'transactionManager'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager' defined in class path resource [launch-context.xml]: Cannot resolve reference to bean 'entityManagerFactory' while setting bean property 'entityManagerFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [launch-context.xml]: Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: No persistence unit with name 'spring' found

运行的时候遇到上面的错误，就是没有上面的配置的原因

org.springframework.transaction.InvalidIsolationLevelException: Standard JPA does not support custom isolation levels - use a special JpaDialect for your JPA implementation
at org.springframework.orm.jpa.DefaultJpaDialect.beginTransaction(DefaultJpaDialect.java:66)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.beginTransaction(HibernateJpaDialect.java:59)

遇到这样的错误，是因为默认的JPA不支持自定义的事物隔离级别。可以自定义一个CustomHibernateJpaDialect extends HibernateJpaDialect，具体代码没有列出，可以找下。

在用Quartz的时候遇到

Caused by: java.lang.IncompatibleClassChangeError: class org.springframework.scheduling.quartz.CronTriggerBean has interface org.quartz.CronTrigger as super class  

这个是由于网上的很多例子都是quartz版本稍旧的原因，我用的是quartz 2.1.7

错误：Jobs added with no trigger must be durable
 

坑真的是不少，需要一个个解决。最后测试一下，是不是定时执行我们的job：

public class App {
public static void main(String[] args) {
String springConfig = "launch-context.xml";
ApplicationContext context = new ClassPathXmlApplicationContext(
springConfig);
}
}

祝你好运，能够成功，。