java 如何使用会话工厂提高批量插入的性能
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/13772953/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
how to increase the performance of bulk inserts using session factory
提问by nag
I am using jpa with hibernate using seam. Ineed to insert 200,000 records at a time.
我使用接缝将 jpa 与 hibernate 结合使用。需要一次插入 200,000 条记录。
Here is my code:
这是我的代码：
hibernate.cfg.xml
休眠文件.cfg.xml
<hibernate-configuration>
<session-factory name="java:/mobeeSessionFactory">
<property name="hibernate.connection.pool_size">10</property>
<property name="hibernate.connection.driver_class">oracle.jdbc.OracleDriver</property>
<property name="hibernate.connection.url">jdbc:oracle:thin:@localhost:1521:mobee</property>
<property name="hibernate.connection.username">mobeemigrate</property>
<property name="hibernate.connection.password">mobeemigrate</property>
<property name="hibernate.show_sql">false</property>
<property name="hibernate.dialect">org.hibernate.dialect.OracleDialect</property>
<property name="hibernate.format_sql">false</property>
<property name="hibernate.hbm2ddl.auto">update</property>
<property name="hibernate.session_factory_name">java:/mobeeSessionFactory</property>
<property name="hibernate.connection.datasource">mobeeadminDataSource</property>
<property name="hibernate.jdbc.batch_size">10000</property>
<property name="hibernate.cache.use_first_level_cache">true</property>
<property name="hibernate.cache.use_second_level_cache">false</property>
<property name="hibernate.transaction.manager_lookup_class">org.hibernate.transaction.JBossTransactionManagerLookup</property>
<property name="hibernate.transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</property>
<property name="hibernate.transaction.auto_close_session">false</property>
<property name="hibernate.cache.provider_class">org.hibernate.cache.HashtableCacheProvider</property>
<property name="hibernate.transaction.flush_before_completion">true</property>
<!-- Here are the mappings -->
<mapping package="com.manam.mobee.persist.entity"/>
<mapping class="com.manam.mobee.persist.entity.TempCustomers"/>
<mapping class="com.manam.mobee.persist.entity.TempAccounts"/>
</session-factory>
component.xml
组件.xml
<persistence:hibernate-session-factory name="hibernateSessionFactory" cfg-resource-name="hibernate.cfg.xml"/>
<persistence:managed-hibernate-session name="session"
auto-create="true"
session-factory-jndi-name="java:/mobeeSessionFactory"/>
Sample Code:
示例代码：
Session session =hibernateSessionFactory.openSession();
Transaction tx = session.beginTransaction();
for(int i=0;i<doTempAccounts.size();i++){
try {
TempAccounts temp=new TempAccounts();
BeanUtils.copyProperties(temp, doTempAccounts.get(i));
session.save(temp);
if ( i % 10000 == 0 ) { //10000, same as the JDBC batch size
//flush a batch of inserts and release memory:
log.info("********** Batch Updates**********");
session.flush();
session.clear();
}
}
}
tx.commit();
session.close();
The above code is working but it takes around 10 minutes to insert 200,000 records. Is there any configuration in my code to increase the performance of bulk inserts?
上面的代码可以工作，但插入 200,000 条记录需要大约 10 分钟。我的代码中是否有任何配置可以提高批量插入的性能？
In above code I do a session.flush() every 10000 records but its not insert Database every time. Can you explain how to perform bulk inserts to db.?
在上面的代码中，我每 10000 条记录执行一次 session.flush()，但不是每次都插入数据库。你能解释一下如何对数据库执行批量插入吗？
采纳答案by sixbitproxywax
Look into StatelessSession and how to use it to insert items into your DB. This appears to be the go to answer when this question is asked and I personally just used it to make a 15 minute job complete in about 15 seconds. It doesn't fit every scenario, but if it does, it works great.
查看 StatelessSession 以及如何使用它来将项目插入到您的数据库中。当提出这个问题时，这似乎是答案，我个人只是用它在大约 15 秒内完成了 15 分钟的工作。它并不适用于所有场景，但如果适用，则效果很好。
回答by Stephen C
In above code I do a session.flush() every 10000 records but its not insert Database every time.
在上面的代码中，我每 10000 条记录执行一次 session.flush()，但不是每次都插入数据库。
I presume that you mean that the records don't appear to be there until you reach the end; i.e. they don't show up in a query made in a different transaction.
我想你的意思是，直到你到达终点时，这些记录才会出现；即它们不会出现在不同事务中的查询中。
If that is what you mean, then the explanation is simple. You are doing the insert in a single transaction, and the records only become visible when the transaction is committed ... right at the end. Try splitting the inserts into multiple transactionsif you need them to be visible sooner.
如果这就是你的意思，那么解释很简单。您正在单个事务中进行插入，并且只有在提交事务时记录才可见……就在最后。如果您需要更快地看到它们，请尝试将插入内容拆分为多个事务。
You should also read the Hibernate documentation on batch operations - http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/batch.html. I note that it says that a reasonablebatch size is 20 to 50 ... not 10000. It also mentions the approaches of using SQL directly for batch data operations and StatelessSession.
您还应该阅读有关批处理操作的 Hibernate 文档 - http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/batch.html。我注意到它说合理的批处理大小是 20 到 50 ...而不是 10000。它还提到了直接使用 SQL 进行批处理数据操作的方法和StatelessSession.
Here's another resource which is specific to insertion and includes insertion with StatelessSession: http://javainnovations.blogspot.com.au/2008/07/batch-insertion-in-hibernate.html
这是另一个特定于插入的资源，包括插入StatelessSession：http: //javainnovations.blogspot.com.au/2008/07/batch-insertion-in-hibernate.html
回答by otavio.leitao
You can do it almost the same way you do in SQL. Look at the following HQL example:
您几乎可以像在 SQL 中那样进行操作。请看以下 HQL 示例：
insert into DelinquentAccount (id, name) select c.id, c.name from Customer c where ...
Reference: http://docs.jboss.org/hibernate/orm/4.0/devguide/en-US/html/ch04.html#d0e2184
参考：http: //docs.jboss.org/hibernate/orm/4.0/devguide/en-US/html/ch04.html#d0e2184
