Python 如何从 UI 停止/终止 Airflow 任务
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/43631693/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to stop/kill Airflow tasks from the UI
提问by Chetan J
How can I stop/kill a running task on Airflow UI? I am using LocalExecutor
.
Even if I use CeleryExecutor
, how do can I kill/stop the running task?
如何在 Airflow UI 上停止/终止正在运行的任务?我正在使用LocalExecutor
. 即使我使用CeleryExecutor
,我该如何终止/停止正在运行的任务?
回答by Jorge Almeida
In the DAGs screen you can see the running tasks:
在 DAG 屏幕中,您可以看到正在运行的任务:
On 'Recent Tasks' press the running icon and Airflow will automatically run the search query with the filters for the Dag Id and State equal to 'running' and show the results on the Task Instances screen (you can find it manually on the tab Browse > Task Instances).
在“最近的任务”上按下正在运行的图标,Airflow 将自动运行搜索查询,其中 Dag Id 和状态的过滤器等于“正在运行”,并在“任务实例”屏幕上显示结果(您可以在“浏览”选项卡上手动找到它> 任务实例)。
There you can select the presented tasks and set them to another state or delete them.
在那里您可以选择呈现的任务并将它们设置为另一种状态或删除它们。
Please notice that if the DAG is currently running, the Airflow scheduler will start again the tasks you delete. So either you stop the DAG first by changing its state or stop the scheduler (if you are running on a test environment).
请注意,如果 DAG 当前正在运行,Airflow 调度程序将重新启动您删除的任务。因此,要么首先通过更改其状态来停止 DAG,要么停止调度程序(如果您在测试环境中运行)。
回答by Pablo
from airflow gitter (@villasv)
来自气流 gitter (@villasv)
" Not gracefully, no. You can stop a dag (unmark as running) and clear the tasks states or even delete them in the UI. The actual running tasks in the executor won't stop, but might be killed if the executor realizes that it's not in the database anymore. "
" 不优雅,不。您可以停止 dag(取消标记为正在运行)并清除任务状态,甚至在 UI 中删除它们。执行程序中实际运行的任务不会停止,但如果执行程序意识到这一点,可能会被杀死它不再在数据库中了。”
回答by Luis
As menioned by Pablo and Jorge pausing the Dag will not stop the task from being executed if the execution already started. However there is a way to stop a running task from the UI but it's a bit hacky.
正如 Pablo 和 Jorge 所提到的,如果执行已经开始,暂停 Dag 不会停止执行任务。然而,有一种方法可以从 UI 停止正在运行的任务,但它有点笨拙。
When the task is on running
state you can click on CLEAR
this will call job.kill()
the task will be set to shut_down
and moved to up_for_retry
immediately hence it is stopped.
当任务处于开启running
状态时,您可以单击CLEAR
这将调用job.kill()
任务将被设置为shut_down
并up_for_retry
立即移动到因此它停止。
Clearly Airflow did not meant for you to clear tasks in Running
state however since Airflow did not disable it either you can use it as I suggested. Airflow meant CLEAR
to be used with failed
, up_for_retry
etc... Maybe in the future the community will use this bug(?) and implement this as a functionality with "shut down task" button.
显然 Airflow 并不意味着您要清除Running
状态中的任务,但是由于 Airflow 没有禁用它,您也可以按照我的建议使用它。气流意味着CLEAR
与使用failed
,up_for_retry
等等......也许在未来的社会将利用这个bug(?),并实施这一与“关机任务”按钮的功能。
回答by kk17
Simply set the task to failed state will stop the running task.
只需将任务设置为失败状态即可停止正在运行的任务。
[2019-09-17 23:53:28,040] {logging_mixin.py:82} INFO - [2019-09-17 23:53:28,039] {jobs.py:2695} WARNING - State of this instance has been externally set to failed. Taking the poison pill.
[2019-09-17 23:53:28,041] {helpers.py:240} INFO - Sending Signals.SIGTERM to GPID 20977