multithreading 我的 Go 程序如何让所有 CPU 内核都忙碌?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/17868419/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How can my Go program keep all the CPU cores busy?
提问by Rick-777
Goroutines are light-weight processes that are automatically time-sliced onto one or more operating system threads by the Go runtime. (This is a really coolfeature of Go!)
Goroutines 是轻量级进程,由 Go 运行时自动将时间切片到一个或多个操作系统线程上。(这是Go 的一个非常酷的功能!)
Suppose I have a concurrent application like a webserver. There is plenty of stuff happening concurrently in my hypothetical program, without much non-concurrent (Amdahl's Law) ratio.
假设我有一个像网络服务器这样的并发应用程序。在我的假设程序中同时发生了很多事情,没有太多的非并发(阿姆达尔定律)比率。
It seems that the default number of operating system threads in use is currently 1. Does this mean that only one CPU core gets used?
似乎当前使用的操作系统线程的默认数量是 1。这是否意味着只有一个 CPU 内核被使用?
If I start my program with
如果我开始我的程序
runtime.GOMAXPROCS(runtime.NumCPU())
will that give reasonably efficient use of allthe cores on my PC?
这会合理有效地使用我 PC 上的所有内核吗?
Is there any "parallel slackness" benefit from having even moreOS threads in use, e.g. via some heuristic
使用更多操作系统线程是否有任何“并行松弛”好处,例如通过一些启发式
runtime.GOMAXPROCS(runtime.NumCPU() * 2)
?
?
回答by LinearZoetrope
From the Go FAQ:
来自 Go 常见问题解答:
Why doesn't my multi-goroutine program use multiple CPUs?
You must set the GOMAXPROCS shell environment variable or use the similarly-named function of the runtime package to allow the run-time support to utilize more than one OS thread.
Programs that perform parallel computation should benefit from an increase in GOMAXPROCS. However, be aware that concurrency is not parallelism.
为什么我的多 goroutine 程序不使用多个 CPU?
您必须设置 GOMAXPROCS shell 环境变量或使用运行时包的类似命名函数,以允许运行时支持使用多个操作系统线程。
执行并行计算的程序应该受益于 GOMAXPROCS 的增加。但是,请注意并发不是并行。
(UPDATE 8/28/2015: Go 1.5 is set to make the default value of GOMAXPROCS the same as the number of CPUs on your machine, so this shouldn't be a problem anymore)
(更新 8/28/2015:Go 1.5 设置为使 GOMAXPROCS 的默认值与您机器上的 CPU 数量相同,因此这不再是问题)
And
和
Why does using GOMAXPROCS > 1 sometimes make my program slower?
It depends on the nature of your program. Problems that are intrinsically sequential cannot be sped up by adding more goroutines. Concurrency only becomes parallelism when the problem is intrinsically parallel.
In practical terms, programs that spend more time communicating on channels than doing computation will experience performance degradation when using multiple OS threads. This is because sending data between threads involves switching contexts, which has significant cost. For instance, the prime sieve example from the Go specification has no significant parallelism although it launches many goroutines; increasing GOMAXPROCS is more likely to slow it down than to speed it up.
Go's goroutine scheduler is not as good as it needs to be. In future, it should recognize such cases and optimize its use of OS threads. For now, GOMAXPROCS should be set on a per-application basis.
为什么使用 GOMAXPROCS > 1 有时会使我的程序变慢?
这取决于您的程序的性质。添加更多 goroutine 无法加速解决本质上是连续的问题。只有当问题本质上是并行时,并发才会变成并行。
实际上,在使用多个 OS 线程时,花费更多时间在通道上进行通信而不是进行计算的程序会遇到性能下降。这是因为在线程之间发送数据涉及切换上下文,这具有显着的成本。例如,Go 规范中的主要筛选示例虽然启动了许多 goroutine,但没有显着的并行性;增加 GOMAXPROCS 更有可能减慢它而不是加速它。
Go 的 goroutine 调度器没有它需要的那么好。将来,它应该识别这种情况并优化其对操作系统线程的使用。目前,应该在每个应用程序的基础上设置 GOMAXPROCS。
In short: it is very difficult to make Go use "efficient use of all your cores". Simply spawning a billion goroutines and increasing GOMAXPROCS is just as likely to degrade your performance as speed it up because it will be switching thread contexts all the time. If you have a large program that is parallelizable, then increasing GOMAXPROCS to the number of parallel components works fine. If you have a parallel problem embedded in a largely non-parallel program, it may speed up, or you may have to make creative use of functions like runtime.LockOSThread() to ensure the runtime distributes everything correctly (generally speaking Go just dumbly spreads currently non-blocking Goroutines haphazardly and evenly among all active threads).
简而言之:让 Go 使用“有效使用所有内核”是非常困难的。简单地产生 10 亿个 goroutines 并增加 GOMAXPROCS 可能会降低你的性能,因为它会一直切换线程上下文。如果您有一个可并行化的大型程序,那么将 GOMAXPROCS 增加到并行组件的数量可以正常工作。如果你有一个嵌入在很大程度上非并行程序中的并行问题,它可能会加速,或者你可能必须创造性地使用诸如 runtime.LockOSThread() 之类的函数来确保运行时正确分发所有内容(一般来说,Go 只是愚蠢地传播当前非阻塞 Goroutine 在所有活动线程中随意且均匀)。
Also, GOMAXPROCS is the number of CPU cores to use, if it's greater than NumCPU I'm fairly sure that it simply clamps to NumCPU. GOMAXPROCS isn't strictly equal to the number of threads. I'm not 100% sure of exactly when the runtime decides to spawn new threads, but one instance is when the number of blocking goroutines using runtime.LockOSThread() is greater than or equal to GOMAXPROCs -- it will spawn more threads than cores so it can keep the rest of the program running sanely.
此外,GOMAXPROCS 是要使用的 CPU 内核数,如果它大于 NumCPU,我很确定它只是夹在 NumCPU 上。GOMAXPROCS 并不严格等于线程数。我不是 100% 确定运行时决定产生新线程的确切时间,但一个实例是当使用 runtime.LockOSThread() 的阻塞 goroutines 的数量大于或等于 GOMAXPROCs 时——它会产生比核心更多的线程所以它可以保持程序的其余部分正常运行。
Basically, it's quite simple to increase GOMAXPROCS and make go useall cores of your CPU. It's quite another thing at this point in Go's development to actually get it to smartly and efficiently useall cores of your CPU, requiring a lot of program design and finagling to get right.
基本上,增加 GOMAXPROCS 并使用CPU 的所有内核非常简单。在 Go 开发的这一点上,真正让它智能有效地使用CPU 的所有内核是另一回事,需要大量的程序设计和细化才能正确。
回答by Volker
This question cannot be answered, it is much too broad.
这个问题无法回答,太宽泛了。
Take your problem, your algorithm and your workload and measure what is best for this combination.
带着你的问题、你的算法和你的工作量,衡量什么最适合这种组合。
Nobody can answer a question like "Is there any heuristic that adding twice as much salt to my lunch will make it taste better?" as this depends on the lunch (tomatoes benefit much more from salt than strawberries) your taste and how much salt there is already. Try it.
没有人能回答这样的问题:“有没有什么启发式方法可以让我的午餐加两倍的盐会使其味道更好?” 因为这取决于午餐(西红柿比草莓更能从盐中受益)你的口味和已经有多少盐。尝试一下。
On more: runtime.GOMAXPROCS(runtime.NumCPU())
has achieved cult status but controlling the number of threads by setting the GOMAXPROCS environment variable from the outsidemight be the much better option.
更多:runtime.GOMAXPROCS(runtime.NumCPU())
已经获得了狂热的地位,但通过从外部设置 GOMAXPROCS 环境变量来控制线程数可能是更好的选择。
回答by sjakobi
runtime.GOMAXPROCS()
sets the number of (virtual) CPU cores that your program can use simultaneously. Allowing Go to use more CPU cores than you actually have won't help, as your system only has so many CPU cores.
runtime.GOMAXPROCS()
设置您的程序可以同时使用的(虚拟)CPU 内核数。允许 Go 使用比实际使用更多的 CPU 内核无济于事,因为您的系统只有这么多 CPU 内核。
In order to run in more than one thread, your program has to have several goroutines, typically function calls with go someFunc()
. If your program doesn't start any additional goroutines it will naturally run in only one thread no matter how many CPUs/cores you allow it to use.
为了在多个线程中运行,您的程序必须有多个 goroutine,通常使用go someFunc()
. 如果你的程序没有启动任何额外的 goroutines,无论你允许它使用多少 CPU/内核,它自然只会在一个线程中运行。
Check out thisand the following exercises on how to create goroutines.
查看这个和以下关于如何创建 goroutines 的练习。