Python 如何从 UI 停止/终止 Airflow 任务

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/43631693/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-19 23:15:11  来源:igfitidea点击:

How to stop/kill Airflow tasks from the UI

pythonapachehadoopairflowapache-airflow

提问by Chetan J

How can I stop/kill a running task on Airflow UI? I am using LocalExecutor. Even if I use CeleryExecutor, how do can I kill/stop the running task?

如何在 Airflow UI 上停止/终止正在运行的任务?我正在使用LocalExecutor. 即使我使用CeleryExecutor,我该如何终止/停止正在运行的任务?

回答by Jorge Almeida

In the DAGs screen you can see the running tasks:

在 DAG 屏幕中,您可以看到正在运行的任务:

enter image description here

在此处输入图片说明

Example

例子

On 'Recent Tasks' press the running icon and Airflow will automatically run the search query with the filters for the Dag Id and State equal to 'running' and show the results on the Task Instances screen (you can find it manually on the tab Browse > Task Instances).

在“最近的任务”上按下正在运行的图标,Airflow 将自动运行搜索查询,其中 Dag Id 和状态的过滤器等于“正在运行”,并在“任务实例”屏幕上显示结果(您可以在“浏览”选项卡上手动找到它> 任务实例)。

There you can select the presented tasks and set them to another state or delete them.

在那里您可以选择呈现的任务并将它们设置为另一种状态或删除它们。

Please notice that if the DAG is currently running, the Airflow scheduler will start again the tasks you delete. So either you stop the DAG first by changing its state or stop the scheduler (if you are running on a test environment).

请注意,如果 DAG 当前正在运行,Airflow 调度程序将重新启动您删除的任务。因此,要么首先通过更改其状态来停止 DAG,要么停止调度程序(如果您在测试环境中运行)。

回答by Pablo

from airflow gitter (@villasv)

来自气流 gitter (@villasv)

" Not gracefully, no. You can stop a dag (unmark as running) and clear the tasks states or even delete them in the UI. The actual running tasks in the executor won't stop, but might be killed if the executor realizes that it's not in the database anymore. "

" 不优雅,不。您可以停止 dag(取消标记为正在运行)并清除任务状态,甚至在 UI 中删除它们。执行程序中实际运行的任务不会停止,但如果执行程序意识到这一点可能会被杀死它不再在数据库中了。”

回答by Luis

As menioned by Pablo and Jorge pausing the Dag will not stop the task from being executed if the execution already started. However there is a way to stop a running task from the UI but it's a bit hacky.

正如 Pablo 和 Jorge 所提到的,如果执行已经开始,暂停 Dag 不会停止执行任务。然而,有一种方法可以从 UI 停止正在运行的任务,但它有点笨拙。

When the task is on runningstate you can click on CLEARthis will call job.kill()the task will be set to shut_downand moved to up_for_retryimmediately hence it is stopped.

当任务处于开启running状态时,您可以单击CLEAR这将调用job.kill()任务将被设置为shut_downup_for_retry立即移动到因此它停止。

Clearly Airflow did not meant for you to clear tasks in Runningstate however since Airflow did not disable it either you can use it as I suggested. Airflow meant CLEARto be used with failed, up_for_retryetc... Maybe in the future the community will use this bug(?) and implement this as a functionality with "shut down task" button.

显然 Airflow 并不意味着您要清除Running状态中的任务,但是由于 Airflow 没有禁用它,您也可以按照我的建议使用它。气流意味着CLEAR与使用failedup_for_retry等等......也许在未来的社会将利用这个bug(?),并实施这一与“关机任务”按钮的功能。

回答by kk17

Simply set the task to failed state will stop the running task.

只需将任务设置为失败状态即可停止正在运行的任务。

[2019-09-17 23:53:28,040] {logging_mixin.py:82} INFO - [2019-09-17 23:53:28,039] {jobs.py:2695} WARNING - State of this instance has been externally set to failed. Taking the poison pill.
[2019-09-17 23:53:28,041] {helpers.py:240} INFO - Sending Signals.SIGTERM to GPID 20977