java 批量保存的 Spring Data JPA 性能问题

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/29899662/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-11-02 16:06:30  来源:igfitidea点击:

Spring Data JPA performnace issue with batch save

javaspringhibernatejpa

提问by Sharath Karnati

I have the following Entity

我有以下实体

@Entity
@Table(name = "APP_ITEM")
public class AppItem implements Serializable {
private static final long serialVersionUID = 1L;

@EmbeddedId
private AppItemPK AppItemPK;

 public AppItemPK getAppItemPK() {
    return appItemPK;
}

public void setAppItemPK(
        AppItemPK appItemPK) {
    this.appItemPK = appItemPK;
}
} 


@Embeddable
public class AppItemPK implements Serializable {

private static final long serialVersionUID = 1L;
@Column(name = "app_id")
private Long appId;
@Column(name = "item_id")
private Long itemId;

public Long getAppId() {
    return appId;
}

public void setAppId(Long appId) {
    this.appId = appId;
}

public Long getItemId() {
    return itemId;
}

public void setItemId(Long itemId) {
    this.itemId = itemId;
}

public boolean equals(Object obj) {
    if (obj instanceof AppItemPK) {
        AppItemPK appItemPK = (AppItemPK) obj;
        if (appItemPK.getItemId().equals(this.itemId)
                && appItemPK.getAppId().equals(
                        this.appId)) {
            return true;
        }
    }
    return false;
}

public int hashCode() {
    return this.itemId.hashCode() + this.applicationId.hashCode();
}

}

Using below code to insert record into app_item table

使用下面的代码将记录插入到 app_item 表中

@Transactional(readOnly = false)
public boolean saveItemSelection(PageHeaderViewData pageHeaderViewData, Map<Long, Boolean> selectedItems,String savedItems){
    long millisSaveStart = Calendar.getInstance().getTimeInMillis();
    log.debug("Inside saveItemSelection appId:"+pageHeaderData.getAppId());
    boolean saveStatus = false;     
    List<AppItem> appItemInsList = new ArrayList<SavedApplicationItem>();

    if (pageHeaderData.getAppId() != null) {

         for (Entry<Long, Boolean> idEntry : selectedItems.entrySet() ) {
             if (idEntry!= null){                    

                     if (idEntry.getValue() && !savedItems.contains(idEntry.getKey().toString())){                           
                         //log.debug("Inside saveItemSelection SAVED itemId:"+idEntry.getKey()+" , Value:"+idEntry.getValue());          
                         AppItem appItem = new AppItem();
                         AppItemPK appItemPK = new AppItemPK();
                         appItemPK.setAppId(pageHeaderData.getAppId());
                         appItemPK.setItemId(idEntry.getKey());
                         appItem.setAppItemPK(appItemPK);
                         appItem.setUpdateInd(ToggleEnum.Y);                        
                         appItemInsList.add(appItem);
                         //appItemRepository.saveAndFlush(appItem);                          
                     }
                 }
             }


         } 


         if (appItemInsList.size() != 0){
             long millisJPASaveStart = Calendar.getInstance().getTimeInMillis();
             appItemRepository.save(appItemInsList);    
             long millisJPASaveEnd = Calendar.getInstance().getTimeInMillis();
             log.debug("JPA save time:"+(millisJPASaveEnd-millisJPASaveStart));
         }

         saveStatus = true;

         long millisSaveEnd = Calendar.getInstance().getTimeInMillis();
         log.debug("Total save time:"+(millisSaveEnd-millisSaveStart));
    }

    return saveStatus;

}//end of saveItemSelection

For inserting 5000 records it is taking 13826 milliseconds.

插入 5000 条记录需要 13826 毫秒。

Can someone please let me know, how to improve the performance in above JPA code. We are using hibernate for jpa implementation.

有人可以让我知道如何提高上述 JPA 代码的性能。我们正在使用 hibernate 进行 jpa 实现。

回答by Nitin Arora

To improve the performance your inserts, you should implement batch insert using custom code. The method below will make sure that batches are flushed to the database. Tweak the batch size based on your performance tests. Generally 50 is a good number to start.

为了提高插入的性能,您应该使用自定义代码实现批量插入。下面的方法将确保批次刷新到数据库。根据您的性能测试调整批量大小。通常从 50 开始是一个不错的数字。

@Transactional
public void bulkPersist(List<Entity> entities) {
  int i = 0;
  for (Entity entity : entities) {
    em.persist(entity);
    i++;

    if (i % batchSize == 0) {
      flush();
      clear();
    }
  }
}

Above change will create multiple insert statements. You can further optimize the insert query by setting up the hibernate configuration.

以上更改将创建多个插入语句。您可以通过设置休眠配置进一步优化插入查询。

<prop key="hibernate.order_inserts">true</prop>
<prop key="hibernate.order_updates">true</prop> 

Tip: Enable Debug log to see 1 insert query is being generated per batch.

提示:启用调试日志以查看每批生成 1 个插入查询。