Python+Celery:链接工作?

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/3901101/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-18 13:16:51  来源:igfitidea点击:

Python+Celery: Chaining jobs?

pythoncelery

提问by David Wolever

The Celery documentationsuggests that it's a bad idea to have tasks wait on the results of other tasks… But the suggested solution (see “good” heading) leaves a something to be desired. Specifically, there's no clear way of getting the subtask's result back to the caller (also, it's kind of ugly).

芹菜文件表明,这是一个坏主意,有任务等待的其他任务的结果。但建议的解决方案(见“好”的标题)离开是可喜爱的东西。具体来说,没有明确的方法将子任务的结果返回给调用者(而且,这有点难看)。

So, is there any way of “chaining” jobs, so the caller gets the result of the final job? Eg, to use the addexample:

那么,有没有办法“链接”作业,让调用者得到最终作业的结果?例如,使用add示例:

>>> add3 = add.subtask(args=(3, ))
>>> add.delay(1, 2, callback=add3).get()
6

Alternately, is it OK to return instances of Result? For example:

或者,是否可以返回 Result 的实例?例如:

@task
def add(x, y, callback=None):
    result = x + y
    if callback:
        return subtask(callback).delay(result)
    return result

This would let the result of the “final” job in the chain could be retrived with a simple:

这将使链中“最终”作业的结果可以通过一个简单的方式检索:

result = add(1, 2, callback=add3).delay()
while isinstance(result, Result):
    result = result.get()
print "result:", result

采纳答案by ax003d

You can do it with a celery chain. See https://celery.readthedocs.org/en/latest/userguide/canvas.html#chains

你可以用芹菜链来做到这一点。见https://celery.readthedocs.org/en/latest/userguide/canvas.html#chains

@task()
def add(a, b):
    time.sleep(5) # simulate long time processing
    return a + b

Chaining job:

链式作业:

# import chain from celery import chain
# the result of the first add job will be 
# the first argument of the second add job
ret = chain(add.s(1, 2), add.s(3)).apply_async()

# another way to express a chain using pipes
ret2 = (add.s(1, 2) | add.s(3)).apply_async()

...

# check ret status to get result
if ret.status == u'SUCCESS':
    print "result:", ret.get()