Javascript Node.js 堆内存不足
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/38558989/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Node.js heap out of memory
提问by Lapsio
Today I ran my script for filesystem indexing to refresh RAID files index and after 4h it crashed with following error:
今天我运行我的文件系统索引脚本来刷新 RAID 文件索引，4 小时后它崩溃并出现以下错误：
[md5:] 241613/241627 97.5%
[md5:] 241614/241627 97.5%
[md5:] 241625/241627 98.1%
Creating missing list... (79570 files missing)
Creating new files list... (241627 new files)
<--- Last few GCs --->
11629672 ms: Mark-sweep 1174.6 (1426.5) -> 1172.4 (1418.3) MB, 659.9 / 0 ms [allocation failure] [GC in old space requested].
11630371 ms: Mark-sweep 1172.4 (1418.3) -> 1172.4 (1411.3) MB, 698.9 / 0 ms [allocation failure] [GC in old space requested].
11631105 ms: Mark-sweep 1172.4 (1411.3) -> 1172.4 (1389.3) MB, 733.5 / 0 ms [last resort gc].
11631778 ms: Mark-sweep 1172.4 (1389.3) -> 1172.4 (1368.3) MB, 673.6 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x3d1d329c9e59 <JS Object>
1: SparseJoinWithSeparatorJS(aka SparseJoinWithSeparatorJS) [native array.js:~84] [pc=0x3629ef689ad0] (this=0x3d1d32904189 <undefined>,w=0x2b690ce91071 <JS Array[241627]>,L=241627,M=0x3d1d329b4a11 <JS Function ConvertToString (SharedFunctionInfo 0x3d1d3294ef79)>,N=0x7c953bf4d49 <String[4]\: ,\n >)
2: Join(aka Join) [native array.js:143] [pc=0x3629ef616696] (this=0x3d1d32904189 <undefin...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/bin/node]
2: 0xe2c5fc [/usr/bin/node]
3: v8::Utils::ReportApiFailure(char const*, char const*) [/usr/bin/node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/bin/node]
5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/bin/node]
6: v8::internal::Runtime_SparseJoinWithSeparator(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/bin/node]
7: 0x3629ef50961b
Server is equipped with 16gb RAM and 24gb SSD swap. I highly doubt my script exceeded 36gb of memory. At least it shouldn't
服务器配备 16GB RAM 和 24GB SSD 交换。我非常怀疑我的脚本是否超过了 36GB 的内存。至少不应该
Script creates index of files stored as Array of Objects with files metadata (modification dates, permissions, etc, no big data)
脚本使用文件元数据（修改日期、权限等，无大数据）创建存储为对象数组的文件索引
Here's full script code: http://pastebin.com/mjaD76c3
这是完整的脚本代码：http: //pastebin.com/mjaD76c3
I've already experiend weird node issues in the past with this script what forced me eg. split index into multiple files as node was glitching when working on such big files as String. Is there any way to improve nodejs memory management with huge datasets?
过去我已经用这个脚本经历过奇怪的节点问题，例如什么迫使我。将索引拆分为多个文件，因为节点在处理诸如 String 之类的大文件时出现故障。有没有办法用庞大的数据集改进 nodejs 内存管理？
回答by Felix
If I remember correctly, there is a strict standard limit for the memory usage in V8 of around 1.7 GB, if you do not increase it manually.
如果我没记错的话，如果不手动增加内存使用量，V8 中的内存使用量有一个严格的标准限制，大约为 1.7 GB。
In one of our products we followed this solution in our deploy script:
在我们的一个产品中，我们在部署脚本中遵循了这个解决方案：
node --max-old-space-size=4096 yourFile.js
There would also be a new space command but as I read here: a-tour-of-v8-garbage-collectionthe new space only collects the newly created short-term data and the old space contains all referenced data structures which should be in your case the best option.
还有一个新的空间命令，但正如我在这里读到的：a-tour-of-v8-garbage-collection新空间只收集新创建的短期数据，旧空间包含所有引用的数据结构，这些数据结构应该在你的情况是最好的选择。
回答by Kamiel Wanrooij
Just in case anyone runs into this in an environment where they cannot set node properties directly (in my case a build tool):
以防万一有人在无法直接设置节点属性的环境中遇到这种情况（在我的情况下是构建工具）：
NODE_OPTIONS="--max-old-space-size=4096" node ...
You can set the node options using an environment variable if you cannot pass them on the command line.
如果无法在命令行上传递节点选项，则可以使用环境变量设置节点选项。
回答by Maksim Luzik
If you want to increase the memory usage of the node globally - not only single script, you can export environment variable, like this:export NODE_OPTIONS=--max_old_space_size=4096
如果你想全局增加节点的内存使用——不仅仅是单个脚本，你可以导出环境变量，像这样：export NODE_OPTIONS=--max_old_space_size=4096
Then you do not need to play with files when running builds like
npm run build.
然后在运行像
npm run build.
回答by Astr-o
I encountered this issue when trying to debug with VSCode, so just wanted to add this is how you can add the argument to your debug setup.
我在尝试使用 VSCode 进行调试时遇到了这个问题，所以只想添加这就是如何将参数添加到调试设置中的方法。
You can add it to the runtimeArgsproperty of your config in launch.json.
您可以将其添加到runtimeArgs.config 文件的属性中launch.json。
See example below.
请参阅下面的示例。
{
"version": "0.2.0",
"configurations": [{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceRoot}\server.js"
},
{
"type": "node",
"request": "launch",
"name": "Launch Training Script",
"program": "${workspaceRoot}\training-script.js",
"runtimeArgs": [
"--max-old-space-size=4096"
]
}
]}
回答by duchuy
i was struggling with this even after setting --max-old-space-size.
即使在设置 --max-old-space-size 之后，我也在为此而苦苦挣扎。
Then i realised need to put options --max-old-space-size before the karma script.
然后我意识到需要在 karma 脚本之前放置选项 --max-old-space-size 。
also best to specify both syntaxes --max-old-space-size and --max_old_space_size my script for karma :
也最好同时指定语法 --max-old-space-size 和 --max_old_space_size 我的 karma 脚本：
node --max-old-space-size=8192 --optimize-for-size --max-executable-size=8192 --max_old_space_size=8192 --optimize_for_size --max_executable_size=8192 node_modules/karma/bin/karma start --single-run --max_new_space_size=8192 --prod --aot
reference https://github.com/angular/angular-cli/issues/1652
回答by Nicholas Porter
Here are some flag values to add some additional info on how to allow more memory when you start up your node server.
这里有一些标志值，用于添加一些关于如何在启动节点服务器时允许更多内存的附加信息。
1GB - 8GB
1GB - 8GB
#increase to 1gb
node --max-old-space-size=1024 index.js
#increase to 2gb
node --max-old-space-size=2048 index.js
#increase to 3gb
node --max-old-space-size=3072 index.js
#increase to 4gb
node --max-old-space-size=4096 index.js
#increase to 5gb
node --max-old-space-size=5120 index.js
#increase to 6gb
node --max-old-space-size=6144 index.js
#increase to 7gb
node --max-old-space-size=7168 index.js
#increase to 8gb
node --max-old-space-size=8192 index.js
回答by Chandru
I had a similar issue while doing AOT angular build. Following commands helped me.
我在进行 AOT 角度构建时遇到了类似的问题。以下命令帮助了我。
npm install -g increase-memory-limit
increase-memory-limit
Source: https://geeklearning.io/angular-aot-webpack-memory-trick/
来源：https: //geeklearning.io/angular-aot-webpack-memory-trick/
回答by NathanQ
fwiw, finding and fixing a memory hog with something like memwatchmight help.
fwiw，使用memwatch 之类的东西查找和修复内存占用可能会有所帮助。
回答by Umang Patwa
Steps to fix this issue (In Windows) -
解决此问题的步骤（在 Windows 中）-
- Open command prompt and type
%appdata%press enter - Navigate to
%appdata%> npm folder - Open or Edit
ng.cmdin your favorite editor - Add
--max_old_space_size=8192to the IF and ELSE block
- 打开命令提示符并键入
%appdata%按回车 - 导航到
%appdata%> npm 文件夹 ng.cmd在您喜欢的编辑器中打开或编辑- 添加
--max_old_space_size=8192到 IF 和 ELSE 块
Your node.cmdfile looks like this after the change:
node.cmd更改后您的文件如下所示：
@IF EXIST "%~dp0\node.exe" (
"%~dp0\node.exe" "--max_old_space_size=8192" "%~dp0\node_modules\@angular\cli\bin\ng" %*
) ELSE (
@SETLOCAL
@SET PATHEXT=%PATHEXT:;.JS;=;%
node "--max_old_space_size=8192" "%~dp0\node_modules\@angular\cli\bin\ng" %*
)
回答by Amar Powar
The following environment variable can be used:
可以使用以下环境变量：
NODE_OPTIONS= --max-old-space-size=8192 .
Another note: console.log()also consumes memory in the terminal.
另一个注意事项：console.log()也会消耗终端中的内存。
just tried to comment the console.log() on terminal. because that will also take memory.
只是试图在终端上评论 console.log() 。因为那也会占用内存。
