在使用hibernate进行批处理的时候,需要考虑这两个问题:OutOfMemoryError、处理速度
解决方案:
<property name="hibernate.jdbc.batch_size">20</property> <--使用JDBC batching--> <property name="hibernate.cache.use_second_level_cache">false</property> <--关闭二级缓存-->使用session的flush()和clear()方法阶段性清除一级缓存
1.方法一:使用服务器游标
1.1.批量插入
<span style="font-size:12px;">Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); for ( int i=0; i<100000; i++ ) { Customer customer = new Customer(.....); session.save(customer); if ( i % 20 == 0 ) { //此处的值应和配置文件中的一致 //强制内存到数据库同步,释放内存 session.flush(); session.clear(); } } tx.commit(); session.close();</span>
Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); ScrollableResults customers = session.getNamedQuery("GetCustomers") //使用了外置命令查询 .setCacheMode(CacheMode.IGNORE) .scroll(ScrollMode.FORWARD_ONLY); int count=0; while ( customers.next() ) { Customer customer = (Customer) customers.get(0); customer.updateStuff(...); if ( ++count % 20 == 0 ) { //<span style="font-size:12px;">强制内存到数据库同步,释放内存</span> session.flush(); session.clear(); } } tx.commit(); session.close();
2.方法二:使用StatelessSession接口
StatelessSession 不实现第一级 cache,也不和第二级缓存,或者查询缓存交互。它不实现事务化写,也不实现脏数据检查。没有第一级缓存。无状态 session 是低层的抽象,和低层 JDBC 相当接近。
StatelessSession session = sessionFactory.openStatelessSession(); Transaction tx = session.beginTransaction(); ScrollableResults customers = session.getNamedQuery("GetCustomers") .scroll(ScrollMode.FORWARD_ONLY); while ( customers.next() ) { Customer customer = (Customer) customers.get(0); customer.updateStuff(...); session.update(customer); } tx.commit(); session.close();
3.1批量插入
Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); String hqlInsert = "insert into DelinquentAccount (id, name) select c.id, c.name from Customer c where ..."; int createdEntities = s.createQuery( hqlInsert ) .executeUpdate(); tx.commit(); session.close();
3.2批量更新
Session session = sessionFactory.openSession(); Transaction tx = session.beginTransaction(); String hqlVersionedUpdate = "update versioned Customer set name = :newName where name = :oldName"; int updatedEntities = s.createQuery( hqlUpdate ) .setString( "newName", newName ) .setString( "oldName", oldName ) .executeUpdate(); tx.commit(); session.close();