bash 合并 CSV 文件：追加而不是合并
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/24641948/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Merging CSV files : Appending instead of merging
提问by user2233834
So basically i want to merge a couple of CSV files. Im using the following script to do that :
所以基本上我想合并几个 CSV 文件。我使用以下脚本来做到这一点：
paste -d , *.csv > final.txt
However this has worked for me in the past but this time it doesn't work. It appends the data next to each other as opposed to below each other. For instance two files that contain records in the following format
然而，这在过去对我有用，但这次不起作用。它将数据彼此相邻而不是彼此下方附加。例如，包含以下格式记录的两个文件
CreatedAt ID
Mon Jul 07 20:43:47 +0000 2014 4.86249E+17
Mon Jul 07 19:58:29 +0000 2014 4.86238E+17
Mon Jul 07 19:42:33 +0000 2014 4.86234E+17
When merged give
当合并给
CreatedAt ID CreatedAt ID
Mon Jul 07 20:43:47 +0000 2014 4.86249E+17 Mon Jul 07 18:25:53 +0000 2014 4.86215E+17
Mon Jul 07 19:58:29 +0000 2014 4.86238E+17 Mon Jul 07 17:19:18 +0000 2014 4.86198E+17
Mon Jul 07 19:42:33 +0000 2014 4.86234E+17 Mon Jul 07 15:45:13 +0000 2014 4.86174E+17
Mon Jul 07 15:34:13 +0000 2014 4.86176E+17
Would anyone know what the reason behind this is? Or what i can do to force merge below records?
有谁知道这背后的原因是什么？或者我可以做些什么来强制合并以下记录？
回答by Hastur
Assuming that all the csv files have the same format and all start with the same header, you can write a little script as the following to append all files in only oneand to take only one time the header.
假设所有的 csv 文件都具有相同的格式并且都以相同的标题开头，您可以编写如下的小脚本来将所有文件附加到一个中，并且只占用一次 header。
#!/bin/bash
OutFileName="X.csv" # Fix the output name
i=0 # Reset a counter
for filename in ./*.csv; do
if [ "$filename" != "$OutFileName" ] ; # Avoid recursion
then
if [[ $i -eq 0 ]] ; then
head -1 "$filename" > "$OutFileName" # Copy header if it is the first file
fi
tail -n +2 "$filename" >> "$OutFileName" # Append from the 2nd line each file
i=$(( $i + 1 )) # Increase the counter
fi
done
Notes:
笔记：
- The
head -1orhead -n 1command print the first line of a file (the head). - The
tail -n +2prints the tail of a file starting from the lines number 2 (+2) - Test
[ ... ]is used to exclude the output file from the input list. - The output file is rewritteneach time.
- The command
cat a.csv b.csv > X.csvcan be simply used to append a.csv and b csv in a single file (but you copy 2 times the header).
- 的
head -1或head -n 1命令打印文件（头）的第一行。 - 在
tail -n +2打印从线数2开始的文件的尾部（+2） - 测试
[ ... ]用于从输入列表中排除输出文件。 - 在输出文件被改写各一次。
- 该命令
cat a.csv b.csv > X.csv可以简单地用于将 a.csv 和 b csv 附加到单个文件中（但您复制标题的 2 倍）。
The pastecommand pastes the files one on a side of the other. If a file has white spaces as lines you can obtain the output that you reported above.
The use of -d ,asks to paste commandto define fields separated by a comma ,, but this is not the case for the format of the files you reported above.
该paste命令将文件一个粘贴到另一侧。如果文件有空格作为行，您可以获得上面报告的输出。
使用-d ,asks topaste command定义由逗号分隔的字段,，但对于您上面报告的文件格式，情况并非如此。
The catcommand instead concatenates files and prints on the standard output, that means it writes one file after the other.
该cat命令改为连接文件并在标准输出上打印，这意味着它一个接一个地写入文件。
Refer to man heador man tailfor the syntax of the single options (some version allows head -1other instead head -n 1)...
参考man head或man tail了解单个选项的语法（某些版本允许head -1其他选项head -n 1）...
回答by Andrea
Thank you so much @wahwahwah. I used your script to make nautilus-action, but it work correctly only with this changes:
非常感谢@wahwahwah。我使用你的脚本来制作nautilus-action，但它只有在进行以下更改时才能正常工作：
#!/bin/bash
for last; do true; done
OutFileName=$last/RESULT_`date +"%d-%m-%Y"`.csv # Fix the output name
i=0 # Reset a counter
for filename in "$last/"*".csv"; do
if [ "$filename" != "$OutFileName" ] ; # Avoid recursion
then
if [[ $i -eq 0 ]] ; then
head -1 "$filename" > "$OutFileName" # Copy header if it is the first file
fi
tail -n +2 "$filename" >> "$OutFileName" # Append from the 2nd line each file
i=$(( $i + 1 )) # Increase the counter
fi
done
