bash 如何按日期对多个日志文件的输出进行排序
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/3302900/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to Sort Output from several Log Files by date
提问by Martin
I have got output from several different log files:
我从几个不同的日志文件中得到了输出：
logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
I would like to sort this output by date, but keep the name of the logfile above the log lines, so it should look like:
我想按日期对此输出进行排序，但将日志文件的名称保留在日志行上方，因此它应该如下所示：
logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
Do you have any idea how to sort output like this with bash commands, sed or awk? Thanks a lot!
您知道如何使用 bash 命令、sed 或 awk 对这样的输出进行排序吗？非常感谢！
UPDATE: This is the source of the output
更新：这是输出的来源
for i in $( find log/ -iname *debug*.log -size +0 );do
if [ `grep -c 'ERROR' $i` -gt 0 ];then
echo -e "\n$i"
grep 'ERROR' --color=auto -A 5 -B 5 $i
fi
done
Martin
马丁
采纳答案by Martin
Thank you all.
谢谢你们。
I improved script from Dennis Williamson to sort errors by date. Each log file with error inside is saved in file named by the timestamp of last error occured. These files are later sorted and put together. There may be cleaner solutions for that than to use of temp files.
我改进了 Dennis Williamson 的脚本以按日期对错误进行排序。每个包含错误的日志文件都保存在以上次发生错误的时间戳命名的文件中。这些文件稍后会被排序并放在一起。可能有比使用临时文件更清洁的解决方案。
find log/ -iname "*debug*.log" -size +0 | while read -r file
do
if grep -qsm 1 'ERROR' "$file"
then
echo -e "$i \t$file"
errors=$(grep 'ERROR' --color=auto -C 5 "$file")
#get the timestamp of last error occured
time=$(echo $errors | head -n 1 | awk '{print " "}')
timestamp=$(date -d "$time" +%s)
#save it to temp file
echo -e "\n$file\n$errors" > tmp/logs/$timestamp.$i
fi
let i++
done
#put files together
rm -f output.txt
for i in `ls tmp/logs/*|sort`;do cat $i >> output.txt ; rm $i; done
Opinions and suggestions for improvement appreciated!
改进意见和建议表示赞赏！
回答by Paused until further notice.
You may be able to get satisfactory results from this (as long as none of your filenames contain colons):
您可以从中获得满意的结果（只要您的文件名都不包含冒号）：
grep -C 5 --recursive 'ERROR' log/* | sort --field-separator=: --key=2
Each line will be prepended by the filename. Your output will look something like this:
每一行都将以文件名开头。您的输出将如下所示：
logfile2:2010/07/21 13:28:52 INFO xxx
logfile2:2010/07/21 13:31:25 INFO xxx
logfile2:2010/07/21 13:31:25 DEBUG xxx
logfile3:2010/07/21 15:28:52 INFO xxx
logfile3:2010/07/21 15:31:25 INFO xxx
logfile3:2010/07/21 15:31:25 DEBUG xxx
etc.
You can use AWK to reformat that into the format that you show in your example:
您可以使用 AWK 将其重新格式化为您在示例中显示的格式：
grep -C 5 --recursive 'ERROR' log/* | sort --field-separator=: --key=2 |
awk '{colon = match(find log/ -iname "*debug*.log" -size +0 | while read -r file
do
if grep -qsm 1 'ERROR' "$file"
then
echo -e "\n$file"
grep 'ERROR' --color=auto -C 5 "$file"
fi
done
,":"); file = substr($/=undef;
$t=<>;
@t=split(/\s*\n*(logfile.*)$/m,$t);
foreach $f (@t) {
next unless $f;
if($f =~ /^logfile/) {
print $f;
} else {
print join("\n",sort (split(/\n/,$f))) . "\n\n";
}
}
,1,colon - 1);
if (file != prevfile) {print "\n" file; prevfile = file};
print substr(@lines = ();
while($t=<>) {
if($t!~ /^2\d\d\d/) {
print sort @lines if(scalar(@lines));
@lines = ();
print $t;
}
else {
push @lines,$t;
}
}
print sort @lines if(scalar(@lines));
,colon+1)}'
Here are several improvements to your script, in case you still use it:
以下是对您的脚本的一些改进，以防您仍然使用它：
$ awk 'FNR==1{$NF=$NF" "FILENAME;}1' logfile*|sort -t" " -k1 -k2|awk 'NF==5{ h=$NF;$NF="";Nicholas-Knights-MacBook-Pro:~/logtest$ ls
logfile1 logfile2 logfile3
Nicholas-Knights-MacBook-Pro:~/logtest$ cat logfile*
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
Nicholas-Knights-MacBook-Pro:~/logtest$ for i in `ls logfile*` ; do printf "$i"; sort -n $i; printf '\n'; done
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 19:31:25 INFO xxx
logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 13:31:25 INFO xxx
logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
2010/07/21 15:31:25 INFO xxx
Nicholas-Knights-MacBook-Pro:~/logtest$
=h"\n"##代码## }1'
logfile2
2010/07/21 13:28:52 INFO xxx
2010/07/21 13:31:25 DEBUG xxx
2010/07/21 13:31:25 INFO xxx
logfile3
2010/07/21 15:28:52 INFO xxx
2010/07/21 15:31:25 DEBUG xxx
2010/07/21 15:31:25 INFO xxx
logfile1
2010/07/21 19:28:52 INFO xxx
2010/07/21 19:31:25 DEBUG xxx
2010/07/21 19:31:25 INFO xxx
回答by leonbloy
If you have you output already in a file (or script output) I'd go Perl:
如果你已经在一个文件（或脚本输出）中输出了我会去 Perl：
##代码##Or, a little more clean:
或者，更干净一点：
##代码##