java Apache Camel ftp 消费者一次又一次地加载相同的文件
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/5708425/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Apache Camel ftp consumer loads the same files again and again
提问by endryha
I have following spring configuration
我有以下弹簧配置
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean id="downloadLogger" class="com.thomsonreuters.oa.sdi.camel.DownloadLogger" />
<bean id="fileFilter" class="com.thomsonreuters.oa.sdi.camel.IgnoreReadyFilesFilter" />
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="ftp://url_to_ftp?password=*******&noop=true&stepwise=false&binary=true&consumer.delay=10s&recursive=true&filter=#fileFilter" />
<process ref="downloadLogger" />
<to uri="file:data/outbox" />
</route>
</camelContext>
</beans>
At the ftp side I have 3 folders with files which I want to download. I want to achieve following scenario:
在 ftp 端，我有 3 个文件夹，其中包含要下载的文件。我想实现以下场景：
- On ftp is fixed amount of files (for isntance 5) at the first data pull consumer loads these files to the destination folder
- At the second attempt to load files, ftp state still the same (5 files) and camel ftp consumer just does nothing (except check for new files)
- To ftp arrives new 2 files, and at this data pull consumer downloads only these new two files
- 在 ftp 上是固定数量的文件（例如 5）在第一个数据拉取消费者将这些文件加载到目标文件夹
- 在第二次尝试加载文件时，ftp 状态仍然相同（5 个文件）并且骆驼 ftp 消费者什么都不做（除了检查新文件）
- 到ftp到达新的2个文件，在这个数据拉消费者只下载这两个新的文件
At the moment my current solutions downloads all files each time when I run dataload process, how I can manage information about downloaded files to prevent downloads of duplicates (I mean already copied files from ftp), I can write my own filter which will filter out already downloaded files but I belive there should be built in feature which will give me controle of this (maybe idempotentRepository, actually I am not sure)...
目前，我当前的解决方案每次运行数据加载过程时都会下载所有文件，我如何管理有关下载文件的信息以防止重复下载（我的意思是已经从 ftp 复制的文件），我可以编写自己的过滤器来过滤掉已经下载的文件，但我相信应该有内置功能可以让我控制这个（也许idempotentRepository，实际上我不确定）......
回答by Claus Ibsen
You need to use a persistent idempotent repository if you want Camel to be able to remember which files it previously have downloaded, between restarts.
如果您希望 Camel 能够记住它之前在重新启动之间下载了哪些文件，则需要使用持久的幂等存储库。
You need to set this option on the ftp endpoint: idempotentRepository
您需要在 ftp 端点上设置此选项：idempotentRepository
See more details here: http://camel.apache.org/file2(Note: The FTP component inheritsthe options from the file component.)
在此处查看更多详细信息：http: //camel.apache.org/file2（注意：FTP 组件从文件组件继承选项。）
There are some examples on the wiki page how to use different stores. And you can also build you custom store.
wiki 页面上有一些如何使用不同商店的示例。你也可以建立你的自定义商店。
回答by endryha
Finally I end up with following solution:
最后我得到以下解决方案：
public class SdiRouteBuilder extends RouteBuilder {
@Override
public void configure() throws Exception {
from("ftp://login@url_to_ftp/RootFolder?" +
"password=******&noop=true&stepwise=false&binary=true&consumer.delay=10s&recursive=true&filter=#fileFilter")
.idempotentConsumer(header("CamelFileName"), FileIdempotentRepository.fileIdempotentRepository(new File("data", "repo.dat")))
.process(new DownloadLogger())
.to("file:data/outbox");
}
}
回答by user1686407
Maybe @endryha answer work well in 2011, but not with Camel 2.20.1
也许@endryha 的回答在 2011 年效果很好，但不适用于 Camel 2.20.1
In Camel 2.20.1, these code will create two idempotentRepository
在 Camel 2.20.1 中，这些代码将创建两个 idempotentRepository
- ftp default memory idempotentRepository
- idempotentConsumer custom idempotentRepository(file based in this case)
- ftp 默认内存 idempotentRepository
- idempotentConsumer 自定义 idempotentRepository（在这种情况下基于文件）
So the correct way to use idempotentRepository is (I remove most parameter for readability)
所以使用 idempotentRepository 的正确方法是（为了可读性，我删除了大部分参数）
"ftp://login@url_to_ftp/RootFolder?&idempotent=true&idempotentRepository=#myIdempotentRepo"
and a Bean
和一个豆子
@Bean
private IdempotentRepository<String> myIdempotentRepo() {
return FileIdempotentRepository.fileIdempotentRepository(new File("data", "repo.dat");
}
