bash 气流参数传递
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/42512305/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Airflow parameter passing
提问by intra
I have a simple job that I'd like to move under an Airflow process, if possible. As it stands, I have a string of bash scripts that access a server and download the latest version of a file and then perform a variety of downstream manipulations to that file.
我有一份简单的工作，如果可能的话，我想在 Airflow 流程下进行。就目前而言，我有一系列 bash 脚本，它们访问服务器并下载文件的最新版本，然后对该文件执行各种下游操作。
exec ./somescript.sh somefileurl
What I'd like to know is: how can I pass in the URL to this file every time I need to run this process?
我想知道的是：每次需要运行此过程时，如何将 URL 传递给此文件？
It seems that if I try to run the bash script as a bash command like so:
看来，如果我尝试将 bash 脚本作为 bash 命令运行，如下所示：
download = BashOperator(
task_id='download_release',
bash_command='somescript.sh',
# params={'URL': 'somefileurl'},
dag=dag)
I have no way of passing in the one parameter that the bash script requires. Otherwise, if I try to send the bash script in as a bash command like so:
我无法传递 bash 脚本所需的一个参数。否则，如果我尝试将 bash 脚本作为 bash 命令发送，如下所示：
download = BashOperator(
task_id='download_release',
bash_command='./somescript.sh {{ URL }}',
params={'URL': 'somefileurl'},
dag=dag)
I receive an execution error as the program tries to execute the script in the context of a temporary directory. This breaks the script as it requires access to some credentials files that sit in the same directory and I'd like to keep the relative file locations intact...
当程序尝试在临时目录的上下文中执行脚本时，我收到执行错误。这会破坏脚本，因为它需要访问位于同一目录中的一些凭据文件，我想保持相对文件位置不变......
Thoughts?
想法？
Update: What worked for me
更新：什么对我有用
download = BashOperator(
task_id='download_release',
bash_command='cd {{ params.dir }} && ./somescript.sh {{ params.url }}',
params={'url': 'somefileurl',
'dir': 'somedir'},
dag=dag)
I did not implement any parameter passing yet, though.
不过，我还没有实现任何参数传递。
采纳答案by Jeremy Farrell
Here is an example of passing a parameter to your BashOperator:
这是将参数传递给 BashOperator 的示例：
templated_command = """
cd /working_directory
somescript.sh {{ dag_run.conf['URL'] }}
"""
download = BashOperator(
task_id='download_release',
bash_command=templated_command,
dag=dag)
For a discussion about this see passing parameters to externally trigged dag. Airflow has two example DAG's that demonstrate this: example_trigger_controller_dagand example_trigger_target_dag. Also, see the Airflow api reference on macros.
有关此问题的讨论，请参阅将参数传递给外部触发的 dag。Airflow 有两个示例 DAG 来证明这一点：example_trigger_controller_dag和example_trigger_target_dag。另外，请参阅宏上的Airflow api 参考。
