bash 如何设置 Airflow 的电子邮件配置以发送有关错误的电子邮件？
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/44573923/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How do I setup Airflow's email configuration to send an email on errors?
提问by simplycoding
I'm trying to make an Airflow task intentionally fail and error out by passing in a Bash line (thisshouldnotrun) that doesn't work. Airflow is outputting the following:
我试图通过传入thisshouldnotrun不起作用的 Bash 行 ( )故意使 Airflow 任务失败并出错。气流输出以下内容：
[2017-06-15 17:44:17,869] {bash_operator.py:94} INFO - /tmp/airflowtmpLFTMX7/run_bashm2MEsS: line 7: thisshouldnotrun: command not found
[2017-06-15 17:44:17,869] {bash_operator.py:97} INFO - Command exited with return code 127
[2017-06-15 17:44:17,869] {models.py:1417} ERROR - Bash command failed
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
result = task_copy.execute(context=context)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute
raise AirflowException("Bash command failed")
AirflowException: Bash command failed
[2017-06-15 17:44:17,871] {models.py:1433} INFO - Marking task as UP_FOR_RETRY
[2017-06-15 17:44:17,878] {models.py:1462} ERROR - Bash command failed
Traceback (most recent call last):
File "/home/ubuntu/.local/bin/airflow", line 28, in <module>
args.func(args)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/bin/cli.py", line 585, in test
ti.run(ignore_task_deps=True, ignore_ti_state=True, test_mode=True)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/utils/db.py", line 53, in wrapper
result = func(*args, **kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models.py", line 1374, in run
result = task_copy.execute(context=context)
File "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/operators/bash_operator.py", line 100, in execute
raise AirflowException("Bash command failed")
airflow.exceptions.AirflowException: Bash command failed
Will Airflow send an email for these kind of errors? If not, what would be the best way to send an email for these errors?
Airflow 会针对此类错误发送电子邮件吗？如果没有，针对这些错误发送电子邮件的最佳方式是什么？
I'm not even sure if airflow.cfgis setup properly... Since the ultimate goal is to test the email alerting notification, I want to make sure airflow.cfg is setup properly. Here's the setup:
我什至不确定airflow.cfg设置是否正确......由于最终目标是测试电子邮件警报通知，我想确保airflow.cfg 设置正确。这是设置：
[email]
email_backend = airflow.utils.email.send_email_smtp
[smtp]
# If you want airflow to send emails on retries, failure, and you want to use
# the airflow.utils.email.send_email_smtp function, you have to configure an
# smtp server here
smtp_host = emailsmtpserver.region.amazonaws.com
smtp_starttls = True
smtp_ssl = False
# Uncomment and set the user/pass settings if you want to use SMTP AUTH
# smtp_user = airflow_data_user
# smtp_password = password
smtp_port = 587
smtp_mail_from = [email protected]
What is smtp_starttls? I can't find any info for it in the documentation or online. If we have 2-factor authentication needed to view emails, will that be an issue here for Airflow?
什么是smtp_starttls？我在文档或在线找不到它的任何信息。如果我们需要 2 因素身份验证来查看电子邮件，这对 Airflow 来说会是一个问题吗？
Here's my Bash command:
这是我的 Bash 命令：
task1_bash_command = """
export PATH=/home/ubuntu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
export rundate=`TZ='America/Los_Angeles' date +%F -d "yesterday"`
export AWS_CONFIG_FILE="/home/ubuntu/.aws/config"
/home/ubuntu/bin/snowsql -f //home/ubuntu/sql/script.sql 1> /home/ubuntu/logs/"$rundate"_dev.log 2> /home/ubuntu/logs/"$rundate"_error_dev.log
if [ -e /home/ubuntu/logs/"$rundate"_error_dev.log ]
then
exit 64
fi
And my task:
而我的任务：
task1 = BashOperator(
task_id = 'run_bash',
bash_command = task1_bash_command,
dag = dag,
retries = 2,
email_on_failure = True,
email = '[email protected]')
回答by jhnclvr
smtp_starttlsbasically means Use TLS
smtp_starttls基本上意味着使用TLS
Set this to Falseand set smtp_sslto Trueif you want to use SSL instead. You probably need smtp_userand smtp_passwordfor either.
如果您想改用 SSL，请将此设置为False并设置smtp_ssl为True。你可能需要smtp_user和smtp_password。
Airflow will not handle 2 step authentication. However, is you are using AWS you likely don't need it as your SMTP (SES) credentials are different from your AWS credentials.
Airflow 不会处理两步验证。但是，如果您使用的是 AWS，您可能不需要它，因为您的 SMTP (SES) 凭证与您的 AWS 凭证不同。
See here.
见这里。
EDIT:
For airflow to send an email on failure, there are a couple things that need to be set on your task, email_on_failureand email.
编辑：为了让气流在失败时发送电子邮件，需要在您的任务中设置几件事情，email_on_failure并且email.
See here for example:
例如，请参见此处：
def throw_error(**context):
raise ValueError('Intentionally throwing an error to send an email.')
t1 = PythonOperator(task_id='throw_error_and_email',
python_callable=throw_error,
provide_context=True,
email_on_failure=True,
email='[email protected]',
dag=dag)
回答by Ganesh
Use below link for creating airflow dag.
How to trigger daily DAG run at midnight local time instead of midnight UTC time
使用以下链接创建气流 dag。
如何触发每日在当地时间午夜而不是 UTC 时间午夜运行的 DAG
Approach 1 : You can setup SMTP locally and make it send email on jobs failure.
方法1：您可以在本地设置SMTP并使其在作业失败时发送电子邮件。
[email]
email_backend = airflow.utils.email.send_email_smtp
[smtp]
smtp_host = localhost
smtp_starttls = False
smtp_ssl = False
smtp_port = 25
smtp_mail_from = [email protected]
Approach 2 : You can use Gmail to send email. I have written an article to do this. https://helptechcommunity.wordpress.com/2020/04/04/airflow-email-configuration/
方法二：您可以使用 Gmail 发送电子邮件。我写了一篇文章来做到这一点。 https://helptechcommunity.wordpress.com/2020/04/04/airflow-email-configuration/
