bash 如何从文件中删除重复项并写入同一个文件？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/11375426/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to remove duplicates from a file and write to the same file?
提问by ronnie
I know my title is not much self-explanatory but let me try to explain it here.
我知道我的标题不是很容易解释，但让我在这里尝试解释一下。
I have a file name test.txtwhich has some duplicate lines. Now, what I want to do is remove those duplicate lines and at the same time update test.txtwith the new content.
我有一个文件名test.txt，其中有一些重复的行。现在，我想要做的是删除那些重复的行，同时删除update test.txt新内容。
test.txt
测试.txt
AAAA
BBBB
AAAA
CCCC
I know I can use sort -u test.txtto remove the duplicates but to update the file with new content how do I redirect it's output to the same file. The below command doesn't work.
我知道我可以sort -u test.txt用来删除重复项，但要使用新内容更新文件，我该如何将其输出重定向到同一个文件。下面的命令不起作用。
sort -u test.txt > test.txt
So, why the above command is not working and whats the correct way?
那么，为什么上面的命令不起作用，正确的方法是什么？
Also is there any other way like
还有没有其他方式像
sort_and_update_file test.txt
which sorts and automatically updates my file without any need of redirection.
它可以排序并自动更新我的文件，无需任何重定向。
采纳答案by potong
This might work for you:
这可能对你有用：
sort -u -o test.txt test.txt
回答by nhed
Redirection in the shell will not work as you are trying to read and write from the same file at the same time. Actually the file is opened for writing (> file.txt) before the sort is even executed
当您尝试同时从同一文件读取和写入时，shell 中的重定向将不起作用。实际上，在> file.txt执行排序之前，文件已打开用于写入 ( )
@potong's answer works because the sort program itself probably stores all lines in memory, I would not rely on it because it does not explicitly specifies in the manpage that it CAN be the same as the input file (though it will likely work). Unless documented to work "in place" I would not do it (@perreal's answer would work, or you can store intermediate results in shell memory)
@potong 的答案有效，因为排序程序本身可能将所有行都存储在内存中，我不会依赖它，因为它没有在联机帮助页中明确指定它可以与输入文件相同（尽管它可能会起作用）。除非记录在“就地”工作，否则我不会这样做（@perreal 的答案会起作用，或者您可以将中间结果存储在 shell 内存中）
回答by Todd A. Jacobs
Use Sponge for Reading/Writing to Same File
使用 Sponge 读取/写入同一文件
You can use the spongeutility from moreutilsto soak up standard output before writing the file. This prevents you from having to shuffle files around, and approximates an in-place edit. For example:
您可以使用moreutils的海绵实用程序在写入文件之前吸收标准输出。这可以防止您不得不四处乱放文件，并近似于就地编辑。例如：
sort -u test.txt | sponge test.txt
Sample Output
样本输出
Using your corpus, this results in the expected output.
使用您的语料库，这会产生预期的输出。
$ cat test.txt
AAAA
BBBB
CCCC
回答by perreal
this is not as inefficient as it looks:
这并不像看起来那么低效：
sort -u test.txt > test.txt.tmp && mv test.txt.tmp test.txt
回答by kenorb
You can use vim for editing file in-place:
您可以使用 vim 就地编辑文件：
$ ex -s +'%!sort' -cxa test.txt
Multiple files:
多个文件：
$ ex -s +'bufdo!%!sort' -cxa *.*
