Python 如何在远程系统上配置和运行 celery worker
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/26986746/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
how to configure and run celery worker on remote system
提问by krishna
i am working on celery and using rabbitmq server and created a project in django project in a server(where message queue,database exists) and it is working fine, i have created multiple workers also
我正在使用 celery 并使用 rabbitmq 服务器并在服务器中的 django 项目中创建了一个项目（其中存在消息队列、数据库）并且它工作正常，我还创建了多个工作人员
from kombu import Exchange, Queue
CELERY_CONCURRENCY = 8
CELERY_ACCEPT_CONTENT = ['pickle', 'json', 'msgpack', 'yaml']
CELERY_RESULT_BACKEND = 'amqp'
CELERYD_HIHyman_ROOT_LOGGER = True
CELERY_HIHyman_ROOT_LOGGER = True
BROKER_URL = 'amqp://guest:guest@localhost:5672//'
CELERY_QUEUES = (
Queue('default', Exchange('default'), routing_key='default'),
Queue('q1', Exchange('A'), routing_key='routingKey1'),
Queue('q2', Exchange('B'), routing_key='routingKey2'),
)
CELERY_ROUTES = {
'my_taskA': {'queue': 'q1', 'routing_key': 'routingKey1'},
'my_taskB': {'queue': 'q2', 'routing_key': 'routingKey2'},
}
AMQP_SERVER = "127.0.0.1"
AMQP_PORT = 5672
AMQP_USER = "guest"
AMQP_PASSWORD = "guest"
AMQP_VHOST = "/"`
CELERY_INCLUDE = ('functions')
`
`
but i want to run workers from another server.so i need some information regarding how to run a worker in another system when i referred few sites it is saying that we need to run the django project on the remote system also is it necessary?
但我想从另一台服务器运行工作程序。所以我需要一些关于如何在另一个系统中运行工作程序的信息，当我提到几个站点时，它说我们需要在远程系统上运行 django 项目也有必要吗？
回答by ChillarAnand
Here is the gist of the idea:
这是这个想法的要点：
On Machine A:
在机器 A 上：
- Install Celery & RabbitMQ.
- Configure rabbitmq so that Machine B can connect to it.
- Create my_tasks.py with some tasks and put some tasks in queue.
- 安装 Celery 和 RabbitMQ。
- 配置 rabbitmq 以便机器 B 可以连接到它。
- 使用一些任务创建 my_tasks.py 并将一些任务放入队列中。
On Machine B:
在机器 B 上：
- Install Celery.
- Copy my_tasks.py file from machine A to this machine.
- Run a worker to consume the tasks
- 安装芹菜。
- 将 my_tasks.py 文件从机器 A 复制到这台机器。
- 运行一个工人来消费任务
I had the same requirement and experimented with celery. It is a lot easier to do that. I wrote a detailed blog post on that few days back. Check out how to send tasks to remote machine?
我有同样的要求并尝试了芹菜。这样做要容易得多。几天前我写了一篇详细的博客文章。查看如何将任务发送到远程机器？
回答by lajarre
You can make use of app.send_task()with something like the following in your django project:
您可以app.send_task()在 django 项目中使用以下内容：
from celery import Celery
import my_client_config_module
app = Celery()
app.config_from_object(my_client_config_module)
app.send_task('dotted.path.to.function.on.remote.server.relative.to.worker',
args=(1, 2))
回答by bluebird_lboro
basically I will take ChillarAnand's answer. I would like to add comment on his answer, but I can't cause I don't have 50 reputation.
基本上我会接受 ChillarAnand 的回答。我想对他的回答发表评论，但我不能因为我没有 50 声望。
so...
所以...
the answer to your question...
你的问题的答案...
First you would like to read "how to send tasks to remote machine?", as ChillarAnandmentioned.
首先，您想阅读“如何将任务发送到远程机器？” ，正如 ChillarAnand提到的。
That is really good article, with one small flaw, such as "does not have '@app.task' on the function def add(), in the content remote.py", it caused problem and confused me as a newbie to celery.
那真是一篇好文章，有一个小缺陷，例如“在内容 remote.py 中，函数 def add() 上没有‘@app.task’”，这引起了问题并使我作为 celery 的新手感到困惑.
And the answer to "[Errno 113] No route to host."part,
以及“[Errno 113] 没有到主机的路由”的答案。部分，
I guess... I guess you have a firewall running in your rabbitmq server, you might want to have a check. Most of time, it is iptables, but it could something else. Switch it off, or change the rules. Then you can give it another try.
我猜...我猜你的rabbitmq服务器上运行了一个防火墙，你可能想检查一下。大多数时候，它是 iptables，但也可以是其他东西。关闭它，或更改规则。然后你可以再试一次。
回答by GrvTyagi
First, think about how celery really work?
首先，想想芹菜到底是如何工作的？
Celery producer adds a task to queue with name and other important headers to identify the location of your task.
Celery 生产者将一个任务添加到队列中，并使用名称和其他重要的标题来标识您的任务的位置。
Celery does not add a complete executable function to MQ.
Celery 没有向 MQ 添加完整的可执行函数。
So, When you look at worker(consumer) side.
因此，当您查看工人（消费者）方面时。
Celery gets task details from MQ and tries to run this. To run this task there should be available module/files/environment/codebase to execute this task.
Celery 从 MQ 获取任务详细信息并尝试运行它。要运行此任务，应该有可用的模块/文件/环境/代码库来执行此任务。
Now lets come to your question ...
现在让我们来回答你的问题......
You try to set worker on a separate machine so logically to execute a function pointed by the task you need complete code environment of tasks and you should connect(Otherwise how you gonna get tasks from MQ ?) with your MQ where tasks live.
您尝试在单独的机器上设置工作人员，以便在逻辑上执行任务指向的功能，您需要完整的任务代码环境，并且您应该连接（否则您将如何从 MQ 获取任务？）与任务所在的 MQ。
