Linux 从线程内分叉是否安全？

Is it safe to fork from within a thread?

提问by ?lex

Let me explain: I have already been developing an application on Linux which forks and execs an external binary and waits for it to finish. Results are communicated by shm files that are unique to the fork + process. The entire code is encapsulated within a class.

让我解释一下：我已经在 Linux 上开发了一个应用程序，它派生并执行一个外部二进制文件并等待它完成。结果由 fork + 进程独有的 shm 文件传达。整个代码封装在一个类中。

Now I am considering threading the process in order to speed things up. Having many different instances of class functions fork and execute the binary concurrently (with different parameters) and communicate results with their own unique shm files.

现在我正在考虑线程化进程以加快速度。拥有许多不同的类函数实例，同时 fork 和执行二进制文件（使用不同的参数），并将结果与​​它们自己独特的 shm 文件进行通信。

Is this thread safe? If I fork within a thread, apart from being safe, is there something I have to watch for? Any advice or help is much appreciated!

这个线程安全吗？如果我在一个线程中分叉，除了安全之外，还有什么需要注意的吗？非常感谢任何建议或帮助！

采纳答案by Diego Sevilla

forking, even with threads, is safe. Once you fork, the threads are independent per process. (That is, threading is orthogonal to forking). However, if the threads in different processes use the same shared memory to comunicate, you have to devise a synchronization mechanism.

forking，即使使用线程，也是安全的。分叉后，每个进程的线程都是独立的。（也就是说，线程与分叉是正交的）。但是，如果不同进程中的线程使用相同的共享内存进行通信，则必须设计同步机制。

回答by Kevin

If you are using the unix 'fork()' system call, then you are not technically using threads- you are using processes- they will have their own memory space, and therefore cannot interfere with eachother.

如果您使用的是 unix 'fork()' 系统调用，那么您在技术上并没有使用线程——您使用的是进程——它们将拥有自己的内存空间，因此不会相互干扰。

As long as each process uses different files, there should not be any issue.

只要每个进程使用不同的文件，就不会有任何问题。

回答by sarnold

While you canuse Linux's NPTL pthreads(7)support for your program, threads are an awkward fit on Unix systems, as you've discovered with your fork(2)question.

虽然您可以pthreads(7)为您的程序使用 Linux 的 NPTL支持，但正如您在fork(2)问题中发现的那样，线程在 Unix 系统上并不合适。

Since fork(2)is a very cheapoperation on modern systems, you might do better to just fork(2)your process when you have more handling to perform. It depends upon how much data you intend to move back and forth, the share-nothing philosophy of forked processes is good for reducing shared-data bugs but does mean you either need to create pipes to move data between processesor use shared memory (shmget(2)or shm_open(3)).

由于在现代系统上这fork(2)是一个非常便宜的操作，fork(2)当您有更多的处理要执行时，您可能会更好地处理您的过程。这取决于您打算来回移动多少数据，forked 进程的无共享理念有助于减少共享数据错误，但这意味着您要么需要创建管道以在进程之间移动数据或使用共享内存（shmget(2)或shm_open(3)）。

But if you choose to use threading, you canfork(2)a new process, with the following hints from the fork(2)manpage:

但是，如果您选择使用线程，则可以使用联机帮助页中fork(2)的以下提示创建一个新进程fork(2)：

   *  The child process is created with a single thread — the
      one that called fork().  The entire virtual address space
      of the parent is replicated in the child, including the
      states of mutexes, condition variables, and other pthreads
      objects; the use of pthread_atfork(3) may be helpful for
      dealing with problems that this can cause.

   *  The child process is created with a single thread — the
      one that called fork().  The entire virtual address space
      of the parent is replicated in the child, including the
      states of mutexes, condition variables, and other pthreads
      objects; the use of pthread_atfork(3) may be helpful for
      dealing with problems that this can cause.

回答by Charlie Martin

Back at the Dawn of Time, we called threads "lightweight processes" because while they act a lot like processes, they're not identical. The biggest distinction is that threads by definition live in the same address space of one process. This has advantages: switching from thread to thread is fast, they inherently share memory so inter-thread communications are fast, and creating and disposing of threads is fast.

回到时间的黎明，我们称线程为“轻量级进程”，因为虽然它们的行为很像进程，但它们并不完全相同。最大的区别是线程根据定义位于一个进程的同一地址空间中。这样做的优点是：从线程到线程的切换速度很快，它们固有地共享内存，因此线程间通信速度很快，并且线程的创建和处理速度也很快。

The distinction here is with "heavyweight processes", which are complete address spaces. A new heavyweight process is created by fork(2). As virtual memory came into the UNIX world, that was augmented with vfork(2)and some others.

这里的区别在于“重量级进程”，它们是完整的地址空间。fork(2)创建了一个新的重量级进程。随着虚拟内存进入 UNIX 世界，它被vfork(2)和其他一些扩展。

A fork(2)copies the entire address space of the process, including all the registers, and puts that process under the control of the operating system scheduler; the next time the scheduler comes around, the instruction counter picks up at the next instruction -- the forked child process is a clone of the parent. (If you want to run another program, say because you're writing a shell, you follow the fork with an exec(2)call, which loads that new address space with a new program, replacing the one that was cloned.)

甲叉（2）复制处理的整个地址空间，包括所有的寄存器，并提出该操作系统调度程序的控制下处理; 下一次调度程序出现时，指令计数器会在下一条指令处启动——分叉的子进程是父进程的克隆。（如果你想运行另一个程序，比如说你正在编写一个 shell，你可以通过exec(2)调用跟随 fork ，它用一个新程序加载新的地址空间，替换被克隆的那个。）

Basically, your answer is buried in that explanation: when you have a process with many LWPsthreads and you fork the process, you will have two independent processes with many threads, running concurrently.

基本上，您的答案隐藏在该解释中：当您有一个具有许多LWP线程的进程并分叉该进程时，您将有两个具有许多线程的独立进程并发运行。

This trick is even useful: in many programs, you have a parent process that may have many threads, some of which fork new child processes. (For example, an HTTP server might do that: each connection to port 80 is handled by a thread, and then a child process for something like a CGI program could be forked; exec(2)would then be called to run the CGI program in place of the parent process close.)

这个技巧甚至很有用：在许多程序中，您的父进程可能有许多线程，其中一些线程会派生出新的子进程。（例如，HTTP 服务器可能会这样做：到端口 80 的每个连接都由一个线程处理，然后可以分叉 CGI 程序之类的子进程；然后将调用exec(2)来运行 CGI 程序代替父进程关闭。）

回答by Igor Nazarenko

It's safe to fork in a multithreaded program as long as you are verycareful about the code between fork and exec. You can make only re-enterant (aka asynchronous-safe) system calls in that span. In theory, you are not allowed to malloc or free there, although in practice the default Linux allocator is safe, and Linux libraries came to rely on it End result is that you mustuse the default allocator.

只要您非常注意 fork 和 exec 之间的代码，在多线程程序中 fork 是安全的。您只能在该范围内进行重新输入（也称为异步安全）系统调用。理论上，您不允许在那里 malloc 或 free，尽管在实践中默认的 Linux 分配器是安全的，并且 Linux 库开始依赖它 最终结果是您必须使用默认分配器。

回答by Kevin

The problem is that fork() only copies the calling thread, and any mutexes held in child threads will be forever locked in the forked child. The pthread solution was the pthread_atfork()handlers. The idea was you can register 3 handlers: one prefork, one parent handler, and one child handler. When fork()happens prefork is called prior to fork and is expected to obtain all application mutexes. Both parent and child must release all mutexes in parent and child processes respectively.

问题在于 fork() 只复制调用线程，并且子线程中持有的任何互斥锁将永远锁定在分叉子线程中。pthread 解决方案是pthread_atfork()处理程序。这个想法是你可以注册 3 个处理程序：一个 prefork、一个父处理程序和一个子处理程序。当fork()发生时 prefork 在 fork 之前被调用，并有望获得所有应用程序互斥锁。父进程和子进程都必须分别释放父进程和子进程中的所有互斥锁。

This isn't the end of the story though! Libraries call pthread_atforkto register handlers for library specific mutexes, for example Libc does this. This is a good thing: the application can't possibly know about the mutexes held by 3rd party libraries, so each library must call pthread_atforkto ensure it's own mutexes are cleaned up in the event of a fork().

但这还不是故事的结局！库调用pthread_atfork为库特定的互斥体注册处理程序，例如 Libc 就是这样做的。这是一件好事：应用程序不可能知道 3rd 方库持有的互斥锁，因此每个库都必须调用pthread_atfork以确保它自己的互斥锁在fork().

The problem is that the order that pthread_atforkhandlers are called for unrelated libraries is undefined (it depends on the order that the libraries are loaded by the program). So this means that technically a deadlock can happen inside of a prefork handler because of a race condition.

问题在于pthread_atfork为不相关的库调用处理程序的顺序是未定义的（这取决于程序加载库的顺序）。因此，这意味着从技术上讲，由于竞争条件，可能会在 prefork 处理程序内部发生死锁。

For example, consider this sequence:

例如，考虑这个序列：

1. Thread T1 calls fork()
2. libc prefork handlers are called in T1 (e.g. T1 now holds all libc locks)
3. Next, in Thread T2, a 3rd party library A acquires its own mutex AM, and then makes a libc call which requires a mutex. This blocks, because libc mutexes are held by T1.
4. Thread T1 runs prefork handler for library A, which blocks waiting to obtain AM, which is held by T2.

1. 线程 T1 调用 fork()
2. 在 T1 中调用 libc prefork 处理程序（例如 T1 现在持有所有 libc 锁）
3. 接下来，在线程 T2 中，第 3 方库 A 获取自己的互斥锁 AM，然后进行需要互斥锁的 libc 调用。这会阻塞，因为 libc 互斥锁由 T1 持有。
4. 线程 T1 为库 A 运行 prefork 处理程序，该处理程序阻塞等待获取由 T2 持有的 AM。

There's your deadlock and its unrelated to your own mutexes or code.

这是您的死锁，它与您自己的互斥锁或代码无关。

This actually happened on a project I once worked on. The advice I had found at that time was to choose fork or threads but not both. But for some applications that's probably not practical.

这实际上发生在我曾经参与过的一个项目中。我当时找到的建议是选择 fork 或 thread，但不能两者兼而有之。但是对于某些应用程序来说这可能是不切实际的。

回答by MarkR

Provided you quickly either call exec()or _exit()in the forked child process, you're ok in practice.

如果您快速调用exec()或_exit()在分叉子进程中，您在实践中没问题。

You might want to use posix_spawn()instead which will probably do the Right Thing.

您可能想posix_spawn()改用它，这可能会做正确的事情。

回答by Alexis Wilke

My experience of fork()'ing within threads is really bad. The software generally fails pretty quickly.

我fork()在线程中的体验真的很糟糕。该软件通常很快就会失败。

I've found several solutions to the matter, although you may not like them much, I think these are generally the best way to avoid close to undebuggableerrors.

我找到了几个解决这个问题的方法，虽然你可能不太喜欢它们，但我认为这些通常是避免接近不可调试错误的最佳方法。

1. Fork first

   Assuming you know the number of external processes you need at the start, you can create them upfront and just have them sit there waiting for an event (i.e. read from a blocking pipe, wait on a semaphore, etc.)

   Once you forked enough children you are free to use threads and communicate with those forked processes via your pipes, semaphores, etc. From the time you create a first thread, you cannot call fork anymore. Keep in mind that if you're using 3rd party libraries which may create threads, those have to be used/initialized after the fork()calls happened.

   Note that you can then start using threads within the main and fork()'ed processes.

2. Know your state

   In some circumstances, it may be possible for you to stop all of your threads to start a process and then restart your threads. This is somewhat similar to point (1) in the sense that you do not want threads running at the time you call fork(), although it requires a way for you to know about all the threads currently running in your software (something not always possible with 3rd party libraries).

   Remember that "stopping a thread" using a wait is not going to work. Such a wait require a mutex and those need to be unlocked when you call fork(). You just cannot know when the wait is going to unlock/re-lock the mutex.

3. Choose one or the other

   The other obvious possibility is to choose one or the other and not bother with whether you're going to interfere with one or the other. This is by far the simplest method if at all possible in your software.

1. 先分叉

   假设你一开始就知道你需要的外部进程的数量，你可以预先创建它们，然后让它们坐在那里等待事件（即从阻塞管道读取，等待信号量等）

   一旦你分叉了足够多的孩子，你就可以自由地使用线程并通过你的管道、信号量等与那些分叉的进程通信。从你创建第一个线程开始，你就不能再调用 fork 了。请记住，如果您使用可能会创建线程的 3rd 方库，则必须在fork()调用发生后使用/初始化这些库。

   请注意，然后您可以开始在 main 和fork()'ed 进程中使用线程。

2. 了解你的状态

   在某些情况下，您可能会停止所有线程以启动进程，然后重新启动线程。这有点类似于第 (1) 点，因为您不希望在调用时运行线程fork()，尽管它需要一种方式让您了解当前在您的软件中运行的所有线程（第 3党图书馆）。

   请记住，使用等待“停止线程”是行不通的。这样的等待需要一个互斥锁，并且当您调用fork(). 您只是不知道等待何时解锁/重新锁定互斥锁。

3. 选择其中之一

   另一种明显的可能性是选择一个或另一个，而不是打扰你是否会干扰一个或另一个。如果在您的软件中可能，这是迄今为止最简单的方法。

In my programming, I used all three solutions. I used Point (2) because the threaded version of log4cplusand I needed to use fork()for some parts of my software.

在我的编程中，我使用了所有三种解决方案。我使用了第 (2) 点，因为log4cplus我需要将 和的线程版本fork()用于我的软件的某些部分。

As mentioned by others, if you are using a fork()to then call execve()then the idea is to use as little as possible between the two calls. That is likely to work 99.999% of the time (many people use system()or popen()with fairly good successes too and these do similar things).

正如其他人所提到的，如果您使用的是fork()to then 调用，execve()那么想法是在两次调用之间尽可能少地使用。这很可能在 99.999% 的时间里都有效（许多人也使用system()或popen()取得了相当好的成功，这些人也做类似的事情）。

On the other, if like me you want to do a fork()and never call execve(), then it's not likely to work right while any thread is running.

另一方面，如果像我一样你想做 afork()并且从不调用execve()，那么在任何线程运行时它都不太可能正常工作。