C++ 使用多核使用 g++ 进行编译
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/414714/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Compiling with g++ using multiple cores
提问by bsofman
Quick question: what is the compiler flag to allow g++ to spawn multiple instances of itself in order to compile large projects quicker (for example 4 source files at a time for a multi-core CPU)?
快速问题:允许 g++ 生成自身的多个实例以更快地编译大型项目的编译器标志是什么(例如,多核 CPU 一次编译 4 个源文件)?
回答by frankodwyer
You can do this with make - with gnu make it is the -j flag (this will also help on a uniprocessor machine).
您可以使用 make - 使用 gnu make 它是 -j 标志(这也将有助于单处理器机器)。
For example if you want 4 parallel jobs from make:
例如,如果您想要来自 make 的 4 个并行作业:
make -j 4
You can also run gcc in a pipe with
您还可以在管道中运行 gcc
gcc -pipe
This will pipeline the compile stages, which will also help keep the cores busy.
这将流水线化编译阶段,这也将有助于保持内核忙碌。
If you have additional machines available too, you might check out distcc, which will farm compiles out to those as well.
如果您还有其他可用的机器,您可以查看distcc,它也会将编译输出到这些机器上。
回答by Mihai Limb??an
There is no such flag, and having one runs against the Unix philosophy of having each tool perform just one function and perform it well. Spawning compiler processes is conceptually the job of the build system. What you are probably looking for is the -j (jobs) flag to GNU make, a la
没有这样的标志,并且有一个违背 Unix 哲学的,即让每个工具只执行一个功能并很好地执行它。生成编译器进程在概念上是构建系统的工作。您可能正在寻找的是 GNU make 的 -j (jobs) 标志,a la
make -j4
Or you can use pmake or similar parallel make systems.
或者您可以使用 pmake 或类似的并行 make 系统。
回答by MattyT
People have mentioned make
but bjam
also supports a similar concept. Using bjam -jx
instructs bjam to build up to x
concurrent commands.
人们已经提到make
但bjam
也支持类似的概念。usingbjam -jx
指示 bjam 构建x
并发命令。
We use the same build scripts on Windows and Linux and using this option halves our build times on both platforms. Nice.
我们在 Windows 和 Linux 上使用相同的构建脚本,使用此选项将我们在两个平台上的构建时间减半。好的。
回答by rmeador
make
will do this for you. Investigate the -j
and -l
switches in the man page. I don't think g++
is parallelizable.
make
会为你做这件事。调查手册页中的-j
和-l
开关。我不认为g++
是可并行化的。
回答by Havok
If using make, issue with -j
. From man make
:
如果使用 make,则使用-j
. 来自man make
:
-j [jobs], --jobs[=jobs] Specifies the number of jobs (commands) to run simultaneously. If there is more than one -j option, the last one is effective. If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously.
-j [jobs], --jobs[=jobs] Specifies the number of jobs (commands) to run simultaneously. If there is more than one -j option, the last one is effective. If the -j option is given without an argument, make will not limit the number of jobs that can run simultaneously.
And most notably, if you want to script or identify the number of cores you have available (depending on your environment, and if you run in many environments, this can change a lot) you may use ubiquitous Python function cpu_count()
:
最值得注意的是,如果您想编写脚本或确定可用内核的数量(取决于您的环境,如果您在许多环境中运行,这可能会发生很大变化),您可以使用无处不在的 Python 函数cpu_count()
:
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.cpu_count
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.cpu_count
Like this:
像这样:
make -j $(python3 -c 'import multiprocessing as mp; print(int(mp.cpu_count() * 1.5))')
If you're asking why 1.5
I'll quote user artless-noise in a comment above:
如果您问1.5
我为什么要在上面的评论中引用用户 artless-noise:
The 1.5 number is because of the noted I/O bound problem. It is a rule of thumb. About 1/3 of the jobs will be waiting for I/O, so the remaining jobs will be using the available cores. A number greater than the cores is better and you could even go as high as 2x.
1.5 数字是因为提到的 I/O 绑定问题。这是一个经验法则。大约 1/3 的作业将等待 I/O,因此剩余的作业将使用可用内核。比核心数更大的数字更好,您甚至可以高达 2 倍。
回答by Jason
distcc can also be used to distribute compiles not only on the current machine, but also on other machines in a farm that have distcc installed.
distcc 还可用于分发编译,不仅在当前机器上,而且在场中安装了 distcc 的其他机器上。
回答by Andy
I'm not sure about g++, but if you're using GNU Make then "make -j N" (where N is the number of threads make can create) will allow make to run multple g++ jobs at the same time (so long as the files do not depend on each other).
我不确定 g++,但是如果您使用的是 GNU Make,那么“make -j N”(其中 N 是 make 可以创建的线程数)将允许 make 同时运行多个 g++ 作业(这么长时间)因为文件不相互依赖)。