Python 如何在 Airflow 中运行 bash 脚本文件
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/40146934/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to run bash script file in Airflow
提问by DougKruger
I have a bash script that creates a file (if it does not exist) that I want to run in Airflow, but when I try it fails. How do I do this?
我有一个 bash 脚本，它创建了一个我想在 Airflow 中运行的文件（如果它不存在），但是当我尝试时它失败了。我该怎么做呢？
#!/bin/bash
#create_file.sh
file=filename.txt
if [ ! -e "$file" ] ; then
touch "$file"
fi
if [ ! -w "$file" ] ; then
echo cannot write to $file
exit 1
fi
and here's how I'm calling it in Airflow:
这是我在 Airflow 中的调用方式：
create_command = """
./scripts/create_file.sh
"""
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)
lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 83, in execute
raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed
采纳答案by Jean-Fran?ois Fabre
From the tutorial this is OK:
从教程这是可以的：
t2 = BashOperator(
task_id='sleep',
bash_command='sleep 5',
retries=3,
dag=dag)
But you're passing a multi-line command to it
但是你向它传递了一个多行命令
create_command = """
./scripts/create_file.sh
"""
should be
应该
create_command = "./scripts/create_file.sh "
Moreover, you also have to make sure that you are in the correct directory to avoid cryptic errors. Do it like this for example:
此外，您还必须确保您位于正确的目录中，以避免出现神秘错误。例如这样做：
create_command = "./scripts/create_file.sh"
if os.path.exists(create_command):
t1 = BashOperator(
task_id= 'create_file',
bash_command=create_command,
dag=dag
)
else:
raise Exception("Cannot locate {}".format(create_command))
