在 shell 脚本中读取 JSON 数据

Read JSON data in a shell script

提问by user1305398

In shell I have a requirement wherein I have to read the JSON response which is in the following format:

在 shell 中，我有一个要求，我必须读取以下格式的 JSON 响应：

 { "Messages": [ { "Body": "172.16.1.42|/home/480/1234/5-12-2013/1234.toSort", "ReceiptHandle": "uUk89DYFzt1VAHtMW2iz0VSiDcGHY+H6WtTgcTSgBiFbpFUg5lythf+wQdWluzCoBziie8BiS2GFQVoRjQQfOx3R5jUASxDz7SmoCI5bNPJkWqU8ola+OYBIYNuCP1fYweKl1BOFUF+o2g7xLSIEkrdvLDAhYvHzfPb4QNgOSuN1JGG1GcZehvW3Q/9jq3vjYVIFz3Ho7blCUuWYhGFrpsBn5HWoRYE5VF5Bxc/zO6dPT0n4wRAd3hUEqF3WWeTMlWyTJp1KoMyX7Z8IXH4hKURGjdBQ0PwlSDF2cBYkBUA=", "MD5OfBody": "53e90dc3fa8afa3452c671080569642e", "MessageId": "e93e9238-f9f8-4bf4-bf5b-9a0cae8a0ebc" } ] }

Here I am only concerned with the "Body" property value. I made some unsuccessful attempts like:

这里我只关心“Body”属性值。我做了一些不成功的尝试，例如：

 jsawk -a 'return this.Body' 

or

或者

 awk -v k="Body" '{n=split(
 jq '.Body'
,a,","); for (i=1; i<=n; i++) print a[i]} 

But that did not suffice. Can anyone help me with this?

但这还不够。谁能帮我这个？

回答by user1305398

There is jqfor parsing json on the command line:

有jq用于在命令行上解析 json：

$ cat /tmp/so.json | underscore select '.Messages .Body' 
["172.16.1.42|/home/480/1234/5-12-2013/1234.toSort"]

Visit this for jq: https://stedolan.github.io/jq/

访问 jq：https://stedolan.github.io/jq/

回答by édouard Lopez

tl;dr

tl;博士

underscore select ".addons > .name"

Javascript CLI tools

Javascript CLI 工具

You can use Javascript CLI tools like

您可以使用 Javascript CLI 工具，例如

• underscore-cli:
• json:select(): CSS-like selectors for JSON.

• 下划线 cli：
• json:select()：用于 JSON 的类 CSS 选择器。

Example

例子

Select all namechildren of a addons:

选择 a 的所有name子项addons：

key="Body"
re="\"($key)\": \"([^\"]*)\""

while read -r l; do
    if [[ $l =~ $re ]]; then
        name="${BASH_REMATCH[1]}"
        value="${BASH_REMATCH[2]}"
        echo "$name=$value"
    else
        echo "No match"
    fi
done

The underscore-cliprovide others real world examplesas well as the json:select() doc.

在underscore-cli为他人提供现实世界的例子还有JSON：select（）的文档。

回答by Mindaugas Kubilius

Similarly using Bash regexp. Shall be able to snatch any key/value pair.

同样使用 Bash 正则表达式。应该能够抢夺任何键/值对。

json2bash() {
perl -MJSON -0777 -n -E 'sub J {
my ($p,$v) = @_; my $r = ref $v;
if ($r eq "HASH") { J("${p}_$_", $v->{$_}) for keys %$v; }
elsif ($r eq "ARRAY") { $n = 0; J("$p"."[".$n++."]", $_) foreach @$v; }
else { $v =~ '"s/'/'\\''/g"'; $p =~ s/^([^[]*)\[([0-9]*)\](.+)$/\[\]/;
$p =~ tr/-/_/; $p =~ tr/A-Za-z0-9_[]//cd; say "$p='\''$v'\'';"; }
}; J("json", decode_json($_));'
}

Regular expression can be tuned to match multiple spaces/tabs or newline(s). Wouldn't work if value has embedded ". This is an illustration. Better to use some "industrial" parser :)

可以调整正则表达式以匹配多个空格/制表符或换行符。如果 value 已嵌入". 这是一个插图。最好使用一些“工业”解析器:)

回答by Tino

Here is a crude way to do it: Transform JSON into bashvariables to evalthem.

这是一个粗略的方法：将 JSON 转换为bash变量eval。

This only works for:

这仅适用于：

• JSON which does not contain nested arrays, and
• JSON from trustworthy sources (else it may confuse your shell script, perhaps it may even be able to harm your system, You have been warned)

• 不包含嵌套数组的 JSON，以及
• 来自可靠来源的 JSON（否则它可能会混淆您的 shell 脚本，甚至可能会损害您的系统，您已被警告）

Well, yes, it uses PERL to do this job, thanks to CPAN, but is small enough for inclusion directly into a script and hence is quick and easy to debug:

嗯，是的，它使用 PERL 来完成这项工作，这要归功于 CPAN，但它足够小，可以直接包含到脚本中，因此调试起来又快又容易：

bson2json()
{
printf '[';
{ bsondump ""; echo "\"END$?\""; } | sed '/^{/s/$/,/';
echo ']';
};

bsons2json()
{
printf '{';
c='';
for a;
do
  printf '%s"%q":' "$c" "$a";
  c=',';
  bson2json "$a";
done;
echo '}';
};

bsons2json */*.bson | json2sh | ..

use it like eval "$(json2bash <<<'{"a":["b","c"]}')"

像使用它一样 eval "$(json2bash <<<'{"a":["b","c"]}')"

Not heavily tested, though. Updates, warnings and more examples see my GIST.

不过没有经过严格的测试。更新、警告和更多示例见我的GIST。

Update

更新

(Unfortunately, following is a link-only-solution, as the C code is far too long to duplicate here.)

（不幸的是，以下是仅链接解决方案，因为 C 代码太长而无法在此处复制。）

For all those, who do not like the above solution, there now is a C program json2shwhich (hopefully safely) converts JSON into shell variables. In contrast to the perlsnippet, it is able to process any JSON, as long as it is well formed.

对于所有不喜欢上述解决方案的人，现在有一个C 程序 json2sh（希望安全地）将 JSON 转换为 shell 变量。与perl代码段相反，它能够处理任何 JSON，只要它格式正确。

Caveats:

注意事项：

• json2shwas not tested much.
• json2shmay create variables, which start with the shellshock pattern () {

• json2sh没有经过太多测试。
• json2sh可能会创建以 shellshock 模式开头的变量 () {

I wrote json2shto be able to post-process .bsonwith Shell:

我写信是json2sh为了能够.bson使用 Shell进行后处理：

##代码##

Explained:

解释：

• bson2jsondumps a .bsonfile such, that the records become a JSON array
  • If everything works OK, an END0-Marker is applied, else you will see something like END1.
  • The END-Marker is needed, else empty .bsonfiles would not show up.
• bsons2jsondumps a bunch of .bsonfiles as an object, where the output of bson2jsonis indexed by the filename.

• bson2json转储.bson文件，使记录成为 JSON 数组
  • 如果一切正常，END0则应用 -Marker，否则您将看到类似END1.
  • 在END需要-Marker，否则空的.bson文件将不会显示出来。
• bsons2json将一堆.bson文件转储为一个对象，其中的输出bson2json由文件名索引。

This then is postprocessed by json2sh, such that you can use grep/source/eval/etc. what you need, to bring the values into the shell.

然后，这是后处理json2sh，这样你可以使用grep/ source/ eval/等。您需要什么，将值带入外壳。

This way you can quickly process the contents of a MongoDB dump on shell level, without need to import it into MongoDB first.

通过这种方式，您可以在 shell 级别快速处理 MongoDB 转储的内容，而无需先将其导入 MongoDB。