bash 从日志文件中提取最后 10 分钟
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/20649387/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
extract last 10 minutes from logfile
提问by user1204671
Trying to find a simple way for watching for recent events (from less than 10 minutes), I've tried this:
试图找到一种简单的方法来观看最近的事件（从不到 10 分钟开始），我试过这个：
awk "/^$(date --date="-10 min" "+%b %_d %H:%M")/{p++} p" /root/test.txt
but it doesn't work as expected...
但它没有按预期工作......
Log files are in form :
日志文件的格式为：
Dec 18 09:48:54 Blah
Dec 18 09:54:47 blah bla
Dec 18 09:55:33 sds
Dec 18 09:55:38 sds
Dec 18 09:57:58 sa
Dec 18 09:58:10 And so on...
采纳答案by janos
You can match the date range using simple string comparison, for example:
您可以使用简单的字符串比较来匹配日期范围，例如：
d1=$(date --date="-10 min" "+%b %_d %H:%M")
d2=$(date "+%b %_d %H:%M")
while read line; do
[[ $line > $d1 && $line < $d2 || $line =~ $d2 ]] && echo $line
done
For example if d1='Dec 18 10:19'and d2='Dec 18 10:27'then the output will be:
例如，如果d1='Dec 18 10:19'和d2='Dec 18 10:27'然后输出将是：
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32
Or using awkif you wish:
或者，awk如果您愿意，可以使用：
awk -v d1="$d1" -v d2="$d2" 'sed -n "/^$(date --date='10 minutes ago' '+%b %_d %H:%M')/,$p" /var/log/blaaaa
> d1 && perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time()>time-600' /path/log
< d2 || sudo cat /var/log/syslog |
perl -MDate::Parse -ne '
print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time() > time-600'
~ d2'
回答by Zeus
Here is nice tool range is any you wish from -10 till now
这是不错的工具范围，从 -10 到现在
#!/usr/bin/perl -wn
use strict;
use Date::Parse;
print if /^(\S+\s+\d+\s+\d+:\d+:\d+)\s/ && str2time() > time-600
回答by F. Hauri
That's a (common) job for perl!:
这是perl的（常见）工作！：
Simple and efficient:
简单高效：
date -r logfile +%s
tail -n1 logfile | perl -MDate::Parse -nE 'say str2time() if /^(.{15})/'
This version print last 10 minutes event, upto now, by using timefunction.
此版本使用函数打印最近 10 分钟的事件，到现在为止time。
You could test this with:
你可以用以下方法测试：
perl -MDate::Parse -ne 'print if/^(.{15})\s/&&str2time()>'$(
date -r logfile +%s)
Note that first representation use only firsts 15 chars from each lines, while second construct use more detailed regexp.
请注意，第一个表示仅使用每行中的前 15 个字符，而第二个构造使用更详细的正则表达式。
As a perl script: last10m.pl
作为 perl 脚本： last10m.pl
perl -MDate::Parse -E 'open IN,"<".$ARGV[0];seek IN,-200,2;while (<IN>) {
$ref=str2time() if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;};seek IN,0,0;
while (<IN>) {print if /^(.{15})\s/&&str2time()>$ref-600}' logfile
Strictly: extract last 10 minutes from logfile
严格地：从日志文件中提取最后 10 分钟
Meaning not relative to current time, but to last entry in logfile:
意思不是相对于当前时间，而是相对于日志文件中的最后一个条目：
There is two way for retrieving end of period:
有两种方法可以检索期末：
#!/usr/bin/perl -w
use strict;
use Date::Parse;
my $ref; # The only variable I will use in this.
open IN,"<".$ARGV[0]; # Open (READ) file submited as 1st argument
seek IN,-200,2; # Jump to 200 character before end of logfile. (This
# could not suffice if log file hold very log lines! )
while (<IN>) { # Until end of logfile...
$ref=str2time() if /^(\S+\s+\d+\s+\d+:\d+:\d+)/;
}; # store time into $ref variable.
seek IN,0,0; # Jump back to the begin of file
while (<IN>) {
print if /^(.{15})\s/&&str2time()>$ref-600;
}
Where logically, last modification time of the logfile must be the time of the last entry.
在逻辑上，日志文件的最后修改时间必须是最后一个条目的时间。
So the command could become:
所以命令可以变成：
#!/bin/bash
declare -A month
for i in {1..12};do
LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400))
month[$var]=$i
done
printf -v now "%(%s)T" -1
printf -v ref "%(%m%d%H%M%S)T" $((now-600))
while read line;do
printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \
$((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \
$((10#${line:13:2}))
# echo " $crt < $ref ??" # Uncomment this line to print each test
[ $crt -gt $ref ] && break
done
cat
or you could take thelast entry as reference:
或者您可以将最后一个条目作为参考：
cat >last10min.sh
chmod +x last10min.sh
sudo cat /var/log/syslog | ./last10min.sh
Second version seem stronger, but access to file only once.
第二个版本似乎更强大，但只能访问一次文件。
As a perl script, this could look like:
作为 perl 脚本，这可能如下所示：
#!/bin/bash
declare -A month
for i in {1..12};do
LANG=C printf -v var "%(%b)T" $(((i-1)*31*86400))
month[$var]=$i
done
read now < <(date -d "$(tail -n1 |head -c 15)" +%s)
printf -v ref "%(%m%d%H%M%S)T" $((now-600))
export -A month
{
while read line;do
printf -v crt "%02d%02d%02d%02d%02d" ${month[${line:0:3}]} \
$((10#${line:4:2})) $((10#${line:7:2})) $((10#${line:10:2})) \
$((10#${line:13:2}))
[ $crt -gt $ref ] && break
done
cat
} <
But if you really wanna use bash
但是如果你真的想使用bash
There is a very quick pure bashscript:
有一个非常快速的纯 bash脚本：
Warning: This use recent bashisms, require $BASH_VERSION4.2 or higher.
警告：这使用最近的bashisms，需要$BASH_VERSION4.2 或更高版本。
now10=$(($(date +%s) - (10 * 60)))
while read line; do
[ $(date -d "${line:0:15}" +%s) -gt $now10 ] && printf "$line\n"
done < logfile
Store this script and run:
存储此脚本并运行：
$ lastline=$(tail -n1 logfile)
$ last10=$(($(date -d "$lastline" +%s) - (10 * 60)))
$ while read line; do
> [ $(date -d "${line:0:15}" +%s) -gt $last10 ] && printf "$line\n"
> done < logfile
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32
$
Strictly: extract last 10 minutes from logfile
严格地：从日志文件中提取最后 10 分钟
Simply replace line 10, but you have to place filename inthe script and not use it as a filter:
只需替换第 10 行，但您必须在脚本中放置文件名，而不是将其用作过滤器：
$ { while read line; do
> [ $(date -d "${line:0:15}" +%s) -gt $last10 ] && printf "$line\n" && break
> done ; cat ; } < logfile
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32
$
回答by Digital Trauma
In bash, you can use the datecommand to parse the timestamps. The "%s" format specifier converts the given date to the number of seconds since 1970-01-01 00:00:00 UTC. This simple integer is easy and accurate to do basic arithmetic on.
在bash 中，您可以使用该date命令来解析时间戳。在“％S”格式说明转换自1970-01-01 00:00:00 UTC指定日期的秒数。对这个简单的整数进行基本算术运算既简单又准确。
If you want the log messages from the last 10 minutes of actual time:
如果您想要实际时间的最后 10 分钟的日志消息：
from datetime import datetime
astack=[]
with open("x.txt") as f:
for aline in f:
astack.append(aline.strip())
lasttime=datetime.strptime(astack[-1], '%b %d %I:%M:%S')
for i in astack:
if (lasttime - datetime.strptime(i, '%b %d %I:%M:%S')).seconds <= 600:
print i
Note the ${line:0:15}expression is a bash parameter expansionwhich gives the first 15 characters of the line, i.e. the timestamp itself.
请注意，${line:0:15}表达式是一个bash 参数扩展，它给出了该行的前 15 个字符，即时间戳本身。
If you want the log messages from the last 10 minutes relative to the end of the log:
如果您想要相对于日志结尾的过去 10 分钟的日志消息：
Dec 18 10:19:16
Dec 18 10:19:23
Dec 18 10:21:03
Dec 18 10:22:54
Dec 18 10:27:32
Here's a mild performance enhancement over the above:
这是对上述内容的轻微性能增强：
#! /usr/bin/env ruby
require 'date'
require 'pathname'
if ARGV.length != 4
$stderr.print "usage: #{##代码##} -d|-h|-m|-s time expression log_file\n"
exit 1
end
begin
total_amount = Integer ARGV[1]
rescue ArgumentError
$stderr.print "error: parameter 'time' must be an Integer\n"
$stderr.print "usage: #{##代码##} -d|-h|-m|-s time expression log_file\n"
end
if ARGV[0] == "-m"
gap = Rational(60, 86400)
time_str = "%b %d %H:%M"
elsif ARGV[0] == "-s"
gap = Rational(1, 86400)
time_str = "%b %d %H:%M:%S"
elsif ARGV[0] == "-h"
gap = Rational(3600, 86400)
time_str = "%b %d %H"
elsif ARGV[0] == "-d"
time_str = "%b %d"
gap = 1
else
$stderr.print "usage: #{##代码##} -d|-h|-m|-s time expression log_file\n"
exit 1
end
pn = Pathname.new(ARGV[3])
if pn.exist?
log = (pn.directory?) ? ARGV[3] + "/*" : ARGV[3]
else
$stderr.print "error: file '" << ARGV[3] << "' does not exist\n"
$stderr.print "usage: #{##代码##} -d|-h|-m|-s time expression log_file\n"
end
search_str = ARGV[2]
now = DateTime.now
total_amount.times do
now -= gap
system "cat " << log << " | grep '" << now.strftime(time_str) << ".*" << search_str << "'"
end
This assumes the log entries are in strict chronological order. Once we match the timestamp in question, we exit the for loop, and then just use catto dump the remaining entries.
这假设日志条目严格按时间顺序排列。一旦我们匹配了有问题的时间戳，我们就退出 for 循环，然后只是cat用来转储剩余的条目。
回答by mkc
In python, you could do as follows:
在python中，您可以执行以下操作：
##代码##Put the lines from the file into a stack (a python list). pop the last item and get difference between the successive date items until you get the difference as less than 600 seconds.
将文件中的行放入堆栈（python 列表）。弹出最后一个项目并获取连续日期项目之间的差异，直到差异小于 600 秒。
Running on your input, I get the following:
根据您的输入运行，我得到以下信息：
##代码##回答by simi
A Ruby solution (tested on ruby 1.9.3)
Ruby 解决方案（在 ruby 1.9.3 上测试）
You can pass days, hours, minutes or seconds as a parameter and it will search for the expression and on the file specified (or directory, in which case it will append '/*' to the name):
您可以将天、小时、分钟或秒作为参数传递，它将搜索表达式和指定的文件（或目录，在这种情况下，它将在名称后附加“/*”）：
In your case just call the script like so: $0 -m 10 "expression" log_file
在您的情况下，只需像这样调用脚本： $0 -m 10 "expression" log_file
Note: Also if you know the location of 'ruby' change the shebang (first line of the script), for security reasons.
注意：此外，如果您知道“ruby”的位置，出于安全原因，请更改shebang（脚本的第一行）。
##代码##