bash - sed 查询以编辑 yaml 文件

bash - sed query to edit yaml file

提问by jaywalker

I have a config.yaml file which contains among other values the following list of kafka brokers which I want to remove from the config using a bash script.

我有一个 config.yaml 文件，其中包含以下我想使用 bash 脚本从配置中删除的 kafka 代理列表。

kafka.brokers:
    - "node003"
    - "node004"

I am doing this currently by invoking vifrom inside the script using the command:

我目前通过使用以下命令从脚本内部调用vi来执行此操作：

vi $CONF_BENCHMARK/config.yaml -c ":%s/kafka.brokers:\(\n\s*-\s".*"\)*/kafka.brokers:/g" -c ':wq!'

I understand that sedis a more appropriate tool to accomplish the same task but when I try to translate the above regex to sed, it does not work.

我知道sed是完成相同任务的更合适的工具，但是当我尝试将上述正则表达式转换为sed 时，它不起作用。

sed -i -e "s/kafka.brokers:\(\n\s*-\s".*"\)*/kafka.brokers:/g" $CONF_BENCHMARK/config.yaml

I am doing something wrong ?

我做错了什么？

采纳答案by andlrc

As other have pointed out, you will need to be explicit to get sed working with multiply lines.

正如其他人指出的那样，您需要明确地让 sed 使用乘法行。

The real answer is to use AWK a beautiful answer is provided by karakfa. But for the educational purpose I will provide an sed answer:

真正的答案是使用AWK karakfa 提供了一个漂亮的答案。但出于教育目的，我将提供一个 sed 答案：

sed  '
  /kafka.brokers/ {
    :a
    $be
    N
    /\n[[:space:]]*-[[:space:]]"[^\n]*"[^\n]*$/ba
    s/\n.*\(\n\)//
    P;D
    :e
    s/\n.*//
  }
' input

Basically sed will keep append lines to the pattern space from when kafka.brokersup until \n[[:space:]]*-[[:space:]]"[^\n]*"[^\n]*$is not matches.

基本上 sed 会从kafka.brokers直到\n[[:space:]]*-[[:space:]]"[^\n]*"[^\n]*$不匹配时一直将行附加到模式空间。

This will leave the pattern space with one trailing line it in, i.e:

这将在模式空间中留下一个尾随行，即：

kafka.brokers:\n    - "node003"\n    - "node004"\nother stuff$

Replacing everything \n.*\(\n\)with a newline leaves the following pattern space:

\n.*\(\n\)用换行符替换所有内容会留下以下模式空间：

kafka.brokers:\nother stuff$

P;Dwill print the first line from the pattern space and then restart the cycle with the remaning pattern space. Making the input support:

P;D将从模式空间打印第一行，然后用剩余的模式空间重新开始循环。使输入支持：

kafka.brokers:
    - "node003"
    - "node004"
kafka.brokers:
    - "node005"
more_input

回答by karakfa

awkto the rescue!

awk来救援！

sedis line based, this should work...

sed是基于行的，这应该有效......

$ awk 's{if(/\s*-\s*"[^"]*"/) next; else s=0} /kafka.brokers:/{s=1}1' file

Explanation

解释

if(/\s*-\s*"[^"]*"/) nextif pattern matches skip to next line
s{if(/\s...check pattern if only s is set
/kafka.brokers:/{s=1}when header seen set s
1shorthand for print lines (if not skipped)
s{... else s=0}if s was set but pattern not found, reset s

if(/\s*-\s*"[^"]*"/) next如果模式匹配，则跳到下一行
s{if(/\s...检查模式，如果仅设置了 s，则
/kafka.brokers:/{s=1}在看到标题时设置 s
1打印行的简写（如果未跳过）
s{... else s=0}如果设置了 s 但未找到模式，则重置 s

回答by Ben

Your Vim pattern matches across multiple lines, but sed works line-by-line. (That is, it first tries to match your pattern against kafka.brokers:and fails, then it tries to match - "node003", and so on.) Your instinct to use something other than Vim was right, but sed probably isn't the best tool for the job here.

您的 Vim 模式匹配多行，但 sed 逐行工作。（也就是说，它首先尝试匹配您的模式kafka.brokers:并失败，然后尝试匹配- "node003"，依此类推。）您使用 Vim 以外的东西的直觉是正确的，但 sed 可能不是这里工作的最佳工具.

This answeraddresses the problem of matching multi-line patterns with sed in more detail.

此答案更详细地解决了将多行模式与 sed 匹配的问题。

My personal recommendation would be to use a scripting language like Python or Perl to deal with complicated pattern-matching. You can run a Python command with python -c <command>, for instance, just like you did with Vim, or you could write a small Python script that you call from your Bash script. It's a little more complicated than a sed one-liner, but it will probably save you a lot of debugging and make your script easier to maintain and modify.

我个人的建议是使用像 Python 或 Perl 这样的脚本语言来处理复杂的模式匹配。python -c <command>例如，您可以使用 运行 Python 命令，就像使用 Vim 一样，或者您可以编写一个从 Bash 脚本调用的小型 Python 脚本。它比单行 sed 稍微复杂一些，但它可能会为您节省大量调试时间并使您的脚本更易于维护和修改。