Python 守护进程打包最佳实践

Python Daemon Packaging Best Practices

提问by SeanOC

I have a tool which I have written in python and generally should be run as a daemon. What are the best practices for packaging this tool for distribution, particularly how should settings files and the daemon executable/script be handled?

我有一个用 python 编写的工具，通常应该作为守护程序运行。打包此工具以进行分发的最佳实践是什么，特别是应如何处理设置文件和守护程序可执行文件/脚本？

Relatedly are there any common tools for setting up the daemon for running on boot as appropriate for the given platform (i.e. initscripts on linux, services on windows, launchdon os x)?

相关地，是否有任何通用工具可用于设置在启动时运行的守护进程，以适合给定平台（即linux 上的init脚本、windows上的服务、os x上的launchd）？

采纳答案by Van Gale

To answer one part of your question, there are no tools I know of that will do daemon setup portably even across Linux systems let alone Windows or Mac OS X.

要回答您的问题的一部分，我知道没有任何工具可以在 Linux 系统上进行可移植的守护程序设置，更不用说 Windows 或 Mac OS X。

Most Linux distributions seem to be using start-stop-daemonwithin init scripts now, but you're still going to have minor difference in filesystem layout and big differences in packaging. Using autotools/configure, or distutils/easy_install if your project is all Python, will go a long way to making it easier to build packages for different Linux/BSD distributions.

大多数 Linux 发行版现在似乎都start-stop-daemon在 init 脚本中使用，但您仍然会在文件系统布局上有细微的差异，而在打包方面会有很大的差异。如果您的项目都是 Python，则使用 autotools/configure 或 distutils/easy_install 将大大简化为不同 Linux/BSD 发行版构建包的过程。

Windows is a whole different game and will require Mark Hammond's win32extensions and maybe Tim Golden's WMIextensions.

Windows 是一个完全不同的游戏，需要Mark Hammond 的 win32扩展和Tim Golden 的 WMI扩展。

I don't know Launchd except that "none of the above" are relevant.

我不知道 Launchd，除了“以上都不是”是相关的。

For tips on daemonizing Python scripts, I would look to Python apps that are actually doing it in the real world, for example inside Twisted.

有关守护 Python 脚本的技巧，我会查看在现实世界中实际执行此操作的 Python 应用程序，例如在 Twisted 中。

回答by Ali Afshar

The best tool I found for helping with init.d scripts is "start-stop-daemon". It will run any application, monitor run/pid files, create them when necessary, provide ways to stop the daemon, set process user/group ids, and can even background your process.

我发现帮助 init.d 脚本的最佳工具是“start-stop-daemon”。它将运行任何应用程序，监视运行/pid 文件，在必要时创建它们，提供停止守护进程的方法，设置进程用户/组 ID，甚至可以将您的进程置于后台。

For example, this is a script which can start/stop a wsgi server:

例如，这是一个可以启动/停止 wsgi 服务器的脚本：

#! /bin/bash

case "" in
  start)
    echo "Starting server"

    # Activate the virtual environment
    . /home/ali/wer-gcms/g-env/bin/activate

    # Run start-stop-daemon, the $DAEMON variable contains the path to the
    # application to run
    start-stop-daemon --start --pidfile $WSGI_PIDFILE \
        --user www-data --group www-data \
        --chuid www-data \
        --exec "$DAEMON"
    ;;
  stop)
    echo "Stopping WSGI Application"

    # Start-stop daemon can also stop the application by sending sig 15
    # (configurable) to the process id contained in the run/pid file
    start-stop-daemon --stop --pidfile $WSGI_PIDFILE --verbose
    ;;
  *)
    # Refuse to do other stuff
    echo "Usage: /etc/init.d/wsgi-application.sh {start|stop}"
    exit 1
    ;;
esac

exit 0

You can also see there an example of how to use it with a virtualenv, which I would always recommend.

您还可以在那里看到一个关于如何将它与 virtualenv 一起使用的示例，我总是推荐它。

回答by fulmicoton

There are many snippets on the internet offering to write a daemon in pure python (no bash scripts)

互联网上有很多片段提供用纯 python 编写守护程序（没有 bash 脚本）

http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/looks clean...

http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/看起来很干净...

If you want to write your own,
the principle is the same as with the bash daemon function.

如果你想自己写，
原理和用bash daemon函数一样。

Basically:

基本上：

On start:

开始时：

• you fork to another process
• open a logfile to redirect your stdout and stderr
• Save the pid somewhere.

• 你分叉到另一个进程
• 打开一个日志文件来重定向你的标准输出和标准错误
• 将 pid 保存在某处。

On stop:

在停止：

• You send SIGTERM to the process with pid stored in your pidfile.
• With signal.signal(signal.SIGTERM, sigtermhandler) you can bind a stopping procedure to the SIGTERM signal.

• 您将 SIGTERM 发送到进程，pid 存储在您的 pidfile 中。
• 使用 signal.signal(signal.SIGTERM, sigtermhandler) 您可以将停止过程绑定到 SIGTERM 信号。

I don't know any widely used package doing this though.

我不知道有任何广泛使用的包这样做。

回答by juj

Check the Ben Finney's daemon module. He has started to write a PEP targeting python 3.X:

检查 Ben Finney 的守护进程模块。他已经开始编写针对 python 3.X 的 PEP：

http://www.python.org/dev/peps/pep-3143/

http://www.python.org/dev/peps/pep-3143/

But an implementation is already available here :

但是这里已经有一个实现：

http://pypi.python.org/pypi/python-daemon/

http://pypi.python.org/pypi/python-daemon/

回答by Benjamin W. Smith

Not a silver bullet for what you're asking, but check out supervisord. It handles all the fun bits of managing processes. I use it heavily in a large production environment. Also, it's written in Python!

不是您要问的问题的灵丹妙药，但请查看supervisord。它处理管理流​​程的所有有趣部分。我在大型生产环境中大量使用它。此外，它是用 Python 编写的！

回答by Jim Carroll

I can't remember where I downloaded it... but this is the best daemonizing script that I've found. It works beautifully (on Mac and Linux.) (save it as daemonize.py)

我不记得我在哪里下载它...但这是我找到的最好的守护进程脚本。它运行良好（在 Mac 和 Linux 上。）（另存为 daemonize.py）

import sys, os
def daemonize (stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'):
    # Perform first fork.
    try:
        pid = os.fork( )
        if pid > 0:
            sys.exit(0) # Exit first parent.
    except OSError, e:
        sys.stderr.write("fork #1 failed: (%d) %sn" % (e.errno, e.strerror))
        sys.exit(1)
    # Decouple from parent environment.
    os.chdir("/")
    os.umask(0)
    os.setsid( )
    # Perform second fork.
    try:
        pid = os.fork( )
        if pid > 0:
            sys.exit(0) # Exit second parent.
    except OSError, e:
        sys.stderr.write("fork #2 failed: (%d) %sn" % (e.errno, e.strerror))
        sys.exit(1)
    # The process is now daemonized, redirect standard file descriptors.
    for f in sys.stdout, sys.stderr: f.flush( )
    si = file(stdin, 'r')
    so = file(stdout, 'a+')
    se = file(stderr, 'a+', 0)
    os.dup2(si.fileno( ), sys.stdin.fileno( ))
    os.dup2(so.fileno( ), sys.stdout.fileno( ))
    os.dup2(se.fileno( ), sys.stderr.fileno( ))

In your script, you would simply:

在您的脚本中，您只需：

from daemonize import daemonize
daemonize()

And you can also specify places to redirect the stdio, err, etc...

您还可以指定重定向 stdio、err 等的位置...

回答by Rabarberski

This blog entrymade it clear for me that there are actually two common ways to have your Python program run as a deamon (I hadn't figured that out so clearly from the existing answers):

这篇博客文章让我清楚地知道，实际上有两种常见的方法可以让你的 Python 程序作为守护进程运行（我没有从现有的答案中清楚地想到这一点）：

There are two approaches to writing daemon applications like servers in Python.

• The first is to handle all the tasks of sarting and stopping daemons in Python code itself. The easiest way to do this is with the python-daemonpackage which might eventually make its way into the Python distribution.

有两种方法可以在 Python 中编写守护程序应用程序，例如服务器。

• 第一个是在 Python 代码本身中处理所有启动和停止守护进程的任务。最简单的方法是使用python-daemon可能最终进入 Python 发行版的包。

Poeljapon's answeris an example of this 1st approach, although it doesn't use the python-daemonpackage, but links to a custom but very clean python script.

Poeljapon 的回答是第一种方法的一个例子，虽然它不使用python-daemon包，但链接到一个自定义但非常干净的 python 脚本。

• The other approach is to use the tools supplied by the operating system. In the case of Debain, this means writing an init script which makes use of the start-stop-daemonprogram.

• 另一种方法是使用操作系统提供的工具。在 Debain 的情况下，这意味着编写一个使用该start-stop-daemon程序的 init 脚本。

Ali Afshar's answeris a shell script example of the 2nd approach, using the start-stop-daemon.

Ali Afshar 的回答是第二种方法的shell 脚本示例，使用start-stop-daemon.

The blog entry I quoted has a shell script example, and some additional details on things such as starting your daemon at system startup and restarting your daemon automatically when it stopped for any reason.

我引用的博客条目有一个 shell 脚本示例，以及一些其他细节，例如在系统启动时启动守护程序，以及在它因任何原因停止时自动重新启动守护程序。

回答by user7233963

correct me if wrong, but I believe the question is how to DEPLOY the daemon. Set your app to install via pip and then make the entry_point a cli(daemon()). Then create an init script that simply runs $app_name &

如果错了请纠正我，但我相信问题是如何部署守护进程。将您的应用程序设置为通过 pip 安装，然后将 entry_point 设为 a cli(daemon())。然后创建一个简单运行的初始化脚本$app_name &

回答by David Z

On Linux systems, the system's package manager (Portage for Gentoo, Aptitude for Ubuntu/Debian, yum for Fedora, etc.) usually takes care of installing the program including placing init scripts in the right places. If you want to distribute your program for Linux, you might want to look into bundling it up into the proper format for various distributions' package managers.

在 Linux 系统上，系统的包管理器（Gentoo 的 Portage、Ubuntu/Debian 的 Aptitude、Fedora 的 yum 等）通常负责安装程序，包括将 init 脚本放置在正确的位置。如果你想为 Linux 分发你的程序，你可能需要考虑将它捆绑成各种分发包管理器的正确格式。

This advice is obviously irrelevant on systems which don't have package managers (Windows, and Mac I think).

这个建议显然与没有包管理器的系统（我认为是 Windows 和 Mac）无关。

回答by S.Lott

"generally should be run as a daemon?"

“一般应该作为守护进程运行？”

Doesn't -- on surface -- make a lot of sense. "Generally" isn't sensible. It's either a a daemon or not. You might want to update your question.

没有 - 从表面上看 - 很有意义。“一般”是不合理的。它要么是一个守护进程，要么不是。您可能想更新您的问题。

For examples of daemons, read up on daemons like Apache's httpd or any database server (they're daemons) or the SMTPD mail daemon.

有关守护进程的示例，请阅读守护进程，例如 Apache 的 httpd 或任何数据库服务器（它们是守护进程）或 SMTPD 邮件守护进程。

Or, perhaps, read up on something simpler, like the FTP daemon, SSH daemon, Telnet daemon.

或者，也许，阅读一些更简单的东西，比如 FTP 守护程序、SSH 守护程序、Telnet 守护程序。

In Linux world, you'll have your application installation directory, some working directory, plus the configuration file directories.

在 Linux 世界中，您将拥有应用程序安装目录、一些工作目录以及配置文件目录。

We use /opt/ourappfor the application (it's Python, but we don't install in Python's lib/site-packages)

我们/opt/ourapp用于应用程序（它是 Python，但我们不安装在 Python 的lib/site-packages）

We use /var/ourappfor working files and our configuration files.

我们/var/ourapp用于工作文件和我们的配置文件。

We could use /etc/ourappfor configuration files -- it would be consistent -- but we don't.

我们可以使用/etc/ourapp配置文件——它会是一致的——但我们没有。

We don't -- yet -- use the init.dscripts for startup. But that's the final piece, automated startup. For now, we have sys admins start the daemons.

我们还没有——还——使用init.d脚本来启动。但这是最后一部分，自动启动。现在，我们让系统管理员启动守护进程。

This is based, partly, on http://www.pathname.com/fhs/and http://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/Linux-Filesystem-Hierarchy.html.

这部分基于http://www.pathname.com/fhs/和http://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/Linux-Filesystem-Hierarchy.html。