将 JSON 与 LogStash 结合使用
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/22941739/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Using JSON with LogStash
提问by user70192
I'm going out of my mind here. I have an app that writes logs to a file. Each log entry is a JSON object. An example of my .json file looks like the following:
我要疯了。我有一个将日志写入文件的应用程序。每个日志条目都是一个 JSON 对象。我的 .json 文件示例如下所示：
{"Property 1":"value A","Property 2":"value B"}
{"Property 1":"value x","Property 2":"value y"}
I'm trying desperately to get the log entries into LogStash. In an attempt to do this, I've created the following LogStash configuration file:
我正在拼命地将日志条目放入 LogStash。为此，我创建了以下 LogStash 配置文件：
input {
file {
type => "json"
path => "/logs/mylogs.log"
codec => "json"
}
}
output {
file {
path => "/logs/out.log"
}
}
Right now, I'm manually adding records to mylogs.log to try and get it working. However, they appear oddly in the stdout. When I look open out.log, I see something like the following:
现在，我正在手动将记录添加到 mylogs.log 以尝试使其正常工作。但是，它们在标准输出中的出现很奇怪。当我查看 open out.log 时，我看到如下内容：
{"message":"\"Property 1\":\"value A\", \"Property 2\":\"value B\"}","@version":"1","@timestamp":"2014-04-08T15:33:07.519Z","type":"json","host":"ip-[myAddress]","path":"/logs/mylogs.log"}
Because of this, if I send the message to ElasticSearch, I don't get the fields. Instead I get a jumbled mess. I need my properties to still be properties. I do not want them crammed into the message portion or the output. I have a hunch this has something to do with Codecs. Yet, I'm not sure. I'm not sure if I should change the codec on the logstash input configuration. Or, if I should change the input on the output configuration. I would sincerely appreciate any help as I'm getting desperate at this point.
因此，如果我将消息发送到 ElasticSearch，我将无法获取字段。相反，我得到了一个混乱的烂摊子。我需要我的属性仍然是属性。我不希望它们塞进消息部分或输出中。我有一种预感，这与编解码器有关。然而，我不确定。我不确定是否应该更改 logstash 输入配置上的编解码器。或者，如果我应该更改输出配置上的输入。我会真诚地感谢任何帮助，因为我在这一点上变得绝望。
THanks.
谢谢。
回答by vzamanillo
Try removing the json codecand adding a json filter
input {
file {
type => "json"
path => "/logs/mylogs.log"
}
}
filter{
json{
source => "message"
}
}
output {
file {
path => "/logs/out.log"
}
}
you do not need the json codec because you do not want decode the source JSON but you want filter the input to get the JSON data in the @message field only.
您不需要 json 编解码器，因为您不想解码源 JSON，但您只想过滤输入以获取 @message 字段中的 JSON 数据。
Hope this helps.
希望这可以帮助。
回答by Tinkal Gogoi
By default tcp put everything to message field if json codec not specified.
默认情况下，如果未指定 json 编解码器，tcp 会将所有内容放入消息字段。
An workaround to _jsonparsefailureof the message field after we specify the json codec also can be rectified by doing the following:
在我们指定 json 编解码器后消息字段的_jsonparsefailure的解决方法也可以通过执行以下操作来纠正：
input {
tcp {
port => '9563'
}
}
filter{
json{
source => "message"
target => "myroot"
}
json{
source => "myroot"
}
}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}
It will parse message field to proper json string to field myroot and then myroot is parsed to yield the json.
它会将消息字段解析为正确的 json 字符串以输入 myroot，然后解析 myroot 以生成 json。
We can remove the redundant field like messageas
我们可以删除冗余字段类似消息作为
filter {
json {
source => "message"
remove_field => ["message"]
}
}
