java - 该进程无法访问该文件，因为它正被另一个进程使用

java - The process cannot access the file because it is being used by another process

提问by Furquan Ahmed

I have a piece of code that monitors a directory for addition of files. Whenever a new file is added to the directory, the contents of the file are picked and published on kafka and then the file is deleted.

我有一段代码可以监视目录以添加文件。每当一个新文件被添加到目录中时，该文件的内容被挑选并发布到 kafka 上，然后该文件被删除。

This works when I make a single request but as soon as I subject my code to 5 or 10 user request from jMeter, the contents are published on kafka successfully but the code isn't able to delete the file. I get a FileSystemExceptionwith a message that The process cannot access the file because it is being used by another process..

这在我发出单个请求时有效，但是一旦我将代码置于来自 jMeter 的 5 或 10 个用户请求中，内容就会成功发布到 kafka 上，但代码无法删除文件。我收到一条FileSystemException消息，说The process cannot access the file because it is being used by another process..

I guess there is some concurrency issue which I am unable to see.

我想有一些我看不到的并发问题。

public void monitor() throws IOException, InterruptedException {
    Path faxFolder = Paths.get(TEMP_FILE_LOCATION);
    WatchService watchService = FileSystems.getDefault().newWatchService();
    faxFolder.register(watchService, StandardWatchEventKinds.ENTRY_CREATE);
    boolean valid = true;
    do {
        WatchKey watchKey = watchService.take();
        for (WatchEvent<?> event : watchKey.pollEvents()) {
            if (StandardWatchEventKinds.ENTRY_CREATE.equals(event.kind())) {
                String fileName = event.context().toString();
                publishToKafka(new File(TEMP_FILE_LOCATION + fileName).toPath(), "topic");
            }
        }
        valid = watchKey.reset();
    } while (valid);
}

private void publishToKafka(Path path, String topic) {
    try (BufferedReader reader = Files.newBufferedReader(path)) {
        String input = null;
        while ((input = reader.readLine()) != null) {
            kafkaProducer.publishMessageOnTopic(input, topic);
        }
    } catch (IOException e) {
        LOG.error("Could not read buffered file to send message on kafka.", e);
    } finally {
        try {
            Files.deleteIfExists(path); // This is where I get the exception
        } catch (IOException e) {
            LOG.error("Problem in deleting the buffered file {}.", path.getFileName(), e);
        }
    }
}

Exception Log :

异常日志：

java.nio.file.FileSystemException: D:\upload\notif-1479974962595.csv: The process cannot access the file because it is being used by another process.

    at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
    at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(Unknown Source)
    at java.nio.file.Files.deleteIfExists(Unknown Source)
    at com.panasonic.mdw.core.utils.MonitorDirectory$FileContentPublisher.publishToKafka(MonitorDirectory.java:193)
    at com.panasonic.mdw.core.utils.MonitorDirectory$FileContentPublisher.sendData(MonitorDirectory.java:125)
    at com.panasonic.mdw.core.utils.MonitorDirectory$FileContentPublisher.run(MonitorDirectory.java:113)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)

回答by Arun Kasyakar

You have to close all the connection which is accessing that file before deleting it.

在删除该文件之前，您必须关闭所有访问该文件的连接。

回答by Goro

Looking at your code it seems when one file is picked by thread for publishing again another thread is picking it up for publishing. That's why no one is able to delete it. It must be concurrency issue only. You should redesign code based up on criterion : steps which can be run concurrently and those which cannot be. So steps in the entire process are :

查看您的代码，似乎当一个文件被线程选择再次发布时，另一个线程正在选择它进行发布。这就是为什么没有人能够删除它。它必须只是并发问题。您应该根据标准重新设计代码：可以同时运行的步骤和不能同时运行的步骤。所以整个过程的步骤是：

1. pick up a file (main thread should do it)
2. publish a file (call other thread to do it )
3. delete the file (called thread should delete it)
4. check if any file present (again main thread can do it)

1. 拿起一个文件（主线程应该这样做）
2. 发布一个文件（调用其他线程来做）
3. 删除文件（被调用的线程应该删除它）
4. 检查是否存在任何文件（再次主线程可以做到）

Also the moment a file is selected, you can read it into buffer , delete it and then continue with publish. This will make sure that main thread does not assign this file to some other thread.

此外，在选择文件的那一刻，您可以将其读入缓冲区，将其删除，然后继续发布。这将确保主线程不会将此文件分配给其他线程。

回答by I Made Putrama

I had similar problem as well as in the following thread: Multithreading on Queuewhen trying to upload a dynamically created files to a Queue service and it was taking 2 days for me to resolve. Thanks to Holgerwho gave answer as above whereby the locking occurs might be due to the creation had not fully done upon reading by another thread, it has saved me a lot of time.

我在以下线程中也遇到了类似的问题： 尝试将动态创建的文件上传到队列服务时队列上的多线程问题，我花了 2 天的时间来解决。感谢Holger给出了上面的答案，锁定发生可能是由于在另一个线程读取时没有完全完成创建，它为我节省了很多时间。

My initial solution as found a lot from the internet, was:

从互联网上找到了很多，我最初的解决方案是：

WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
//queueUploadFile(child);
if (kind == ENTRY_CREATE) {
    uploadToQueue(this.queueId, child);
}

I changed it to:

我把它改成：

WatchEvent<Path> ev = cast(event);
Path name = ev.context();
Path child = dir.resolve(name);
//queueUploadFile(child);
if (kind == ENTRY_MODIFY) {
    uploadToQueue(this.queueId, child);
}

And everything works perfectly. To handle the "somehow" multiple ENTRY_MODIFY events firing (duplicated files uploaded), I perform deletion on the file inside the uploadToQueue() method once it's uploaded.

一切正常。为了处理“以某种方式”触发的多个 ENTRY_MODIFY 事件（上传的重复文件），我在上传后对 uploadToQueue() 方法中的文件执行删除。

I hope my approach taken based on the above contribution will also help others with similar problem.

我希望我基于上述贡献采取的方法也能帮助其他有类似问题的人。

回答by Krishna Kothoor

It is always a better idea to add sleep time in WatchService events:

在 WatchService 事件中添加睡眠时间总是一个更好的主意：

if (StandardWatchEventKinds.ENTRY_CREATE.equals(event.kind())) {

       try {
            Thread.sleep(3000);
       } catch (InterruptedException e) {
            e.printStackTrace();
       }

// now do your intended jobs ... 

I needed to add sleep time otherwise it wouldn't work for multiple requests and used to get the error:

我需要添加睡眠时间，否则它不适用于多个请求并用于获取错误：

The process cannot access the file because it is being used by another process