我是否已达到浏览器中 JavaScript 可以处理的对象大小的限制？

Have I reached the limits of the size of objects JavaScript in my browser can handle?

提问by Matt Ball

I'm embedding a large array in <script>tags in my HTML, like this (nothing surprising):

我<script>在 HTML 的标签中嵌入了一个大数组，就像这样（没什么奇怪的）：

<script>
    var largeArray = [/* lots of stuff in here */];
</script>

In this particular example, the array has 210,000 elements. That's well below the theoretical maximum of 2³¹- by 4 orders of magnitude. Here's the fun part: if I save JS source for the array to a file, that file is >44 megabytes (46,573,399 bytes, to be exact).

在此特定示例中，数组有 210,000 个元素。这远低于理论最大值 2 ³¹- 4 个数量级。这是有趣的部分：如果我将数组的 JS 源代码保存到一个文件中，那么该文件将大于 44 兆字节（准确地说是 46,573,399 字节）。

If you want to see for yourself, you can download it from GitHub. (All the data in there is canned, so much of it is repeated. This will not be the case in production.)

如果你想亲自看看，你可以从 GitHub 下载。（那里的所有数据都是固定的，其中很多是重复的。生产中不会出现这种情况。）

Now, I'm really not concerned about servingthat much data. My server gzips its responses, so it really doesn't take all that long to get the data over the wire. However, there is a really nasty tendency for the page, once loaded, to crashthe browser. I'm not testing at all in IE (this is an internal tool). My primary targets are Chrome 8 and Firefox 3.6.

现在，我真的不关心提供那么多数据。我的服务器 gzip 其响应，因此通过网络获取数据确实不需要那么长时间。但是，页面一旦加载就会使浏览器崩溃，这是一种非常令人讨厌的趋势。我根本没有在 IE 中进行测试（这是一个内部工具）。我的主要目标是 Chrome 8 和 Firefox 3.6。

In Firefox, I can see a reasonably useful error in the console:

在 Firefox 中，我可以在控制台中看到一个相当有用的错误：

Error: script stack space quota is exhausted

Error: script stack space quota is exhausted

In Chrome, I simply get the sad-tab page:

在 Chrome 中，我只是得到了 sad-tab 页面：

Cut to the chase, already

切入正题，已经

• Is this reallytoo much data for our modern, "high-performance" browsers to handle?
• Is there anything I can do* to gracefully handle this much data?

• 对于我们现代的“高性能”浏览器来说，这真的是太多数据无法处理吗？
• 我能做些什么*来优雅地处理这么多数据？

Incidentally, I was able to get this to work (read: not crash the tab) on-and-off in Chrome. I really thought that Chrome, at least, was made of tougher stuff, but apparently I was wrong...

顺便说一句，我能够让它在 Chrome 中时断时续地工作（阅读：不会使选项卡崩溃）。我真的认为至少 Chrome 是由更坚固的东西制成的，但显然我错了......

---

---

Edit 1

编辑 1

@Crayon: I wasn't looking to justify whyI'd like to dump this much data into the browser at once. Short version: either I solve this one (admittedly not-that-easy) problem, or I have to solve a whole slew of other problems. I'm opting for the simpler approach for now.

@Crayon：我不想证明为什么我想一次将这么多数据转储到浏览器中。简短版本：要么我解决这个（不可否认，那么容易）问题，要么我必须解决一大堆其他问题。我现在选择更简单的方法。

@various: right now, I'm not especially looking for ways to actually reduce the number of elements in the array. I know I could implement Ajax paging or what-have-you, but that introduces its own set of problems for me in other regards.

@various：现在，我并不是特别在寻找实际减少数组中元素数量的方法。我知道我可以实现 Ajax 分页或 what-have-you，但这在其他方面给我带来了一系列问题。

@Phrogz: each element looks something like this:

@Phrogz：每个元素看起来像这样：

{dateTime:new Date(1296176400000),
 terminalId:'terminal999',
 'General___BuildVersion':'10.05a_V110119_Beta',
 'SSM___ExtId':26680,
 'MD_CDMA_NETLOADER_NO_BCAST___Valid':'false',
 'MD_CDMA_NETLOADER_NO_BCAST___PngAttempt':0}

@Will: but I have a computer with a 4-core processor, 6 gigabytes of RAM, over half a terabyte of disk space ...and I'm not even asking for the browser to do this quickly - I'm just asking for it to work at all!?

@Will：但我有一台带有 4 核处理器、6 GB 内存、超过半 TB 磁盘空间的计算机......我什至没有要求浏览器快速执行此操作 - 我只是在问让它工作！?

---

---

Edit 2

编辑 2

Mission accomplished!

任务完成！

With the spot-on suggestions from Juanas well as Guffa, I was able to get this to work! It would appear that the problem was just in parsingthe source code, not actually working with it in memory.

在Juan和Guffa的现场建议下，我能够让它发挥作用！问题似乎只是在解析源代码，而不是在内存中实际使用它。

To summarize the comment quagmire on Juan's answer: I had to split up my big array into a series of smaller ones, and then Array#concat()them, but that wasn't enough. I alsohad to put them into separate varstatements. Like this:

总结一下 Juan 回答的评论泥潭：我不得不将我的大阵列拆分为一系列较小的阵列，然后再拆分Array#concat()它们，但这还不够。我还必须将它们放入单独的var语句中。像这样：

var arr0 = [...];
var arr1 = [...];
var arr2 = [...];
/* ... */
var bigArray = arr0.concat(arr1, arr2, ...);

To everyone who contributed to solving this: thank you.The first round is on me!

感谢所有为解决此问题做出贡献的人：谢谢。第一轮在我身上！

---

---

_{*other than the obvious: sending less data to the browser}

_{*除了显而易见的：向浏览器发送更少的数据}

回答by Juan Mendes

Here's what I would try: you said it's a 44MB file. That surely takes more than 44MB of memory, I'm guessing this takes much over 44MB of RAM, maybe half a gig. Could you just cut down the data until the browser doesn't crash and see how much memory the browser uses?

这就是我要尝试的：你说这是一个 44MB 的文件。这肯定需要超过 44MB 的内存，我猜这需要超过 44MB 的 RAM，可能是半个演出。你能不能把数据删减直到浏览器不崩溃，看看浏览器使用了多少内存？

Even apps that run only on the server would be well served to not read a 44MB file and keep it in memory. Having said all that, I believe the browser should be able to handle it, so let me run some tests.

即使只在服务器上运行的应用程序也不会读取 44MB 的文件并将其保存在内存中。说了这么多，我相信浏览器应该能够处理它，所以让我运行一些测试。

(Using Windows 7, 4GB of memory)

（使用 Windows 7，4GB 内存）

First TestI cut the array in half and there were no problems, uses 80MB, no crash

第一次测试我把阵列切成两半，没有问题，使用 80MB，没有崩溃

Second TestI split the array into two separate arrays, but still contains all the data, uses 160Mb, no crash

第二个测试我把数组拆分成两个单独的数组，但仍然包含所有数据，使用160Mb，没有崩溃

Third TestSince Firefox said it ran out of stack, the problem is probably that it can't parse the array at once. I created two separate arrays, arr1, arr2 then did arr3 = arr1.concat(arr2); It ran fine and uses only slightly more memory, around 165MB.

第三次测试由于Firefox 说它用完了堆栈，问题可能是它无法立即解析数组。我创建了两个单独的数组， arr1, arr2 然后做了 arr3 = arr1.concat(arr2); 它运行良好，只使用了稍微多一点的内存，大约 165MB。

Fourth TestI am creating 7 of those arrays (22MB each) and concatting them to test browser limits. It takes about 10 seconds for the page to finish loading. Memory goes up to 1.3GB, then it goes back down to 500MB. So yeah chrome can handle it. It just can't parse it all at once because it uses some kind of recursion as can be noticed by the console's error message.

第四次测试我正在创建 7 个这样的数组（每个 22MB）并将它们连接起来以测试浏览器限制。页面完成加载大约需要 10 秒钟。内存上升到 1.3GB，然后又下降到 500MB。所以是的 chrome 可以处理它。它只是不能一次全部解析它，因为它使用了某种递归，正如控制台的错误消息所指出的那样。

AnswerCreate separate arrays (less than 20MB each) and then concat them. Each array should be on its own var statement, instead of doing multiple declarations with a single var.

回答创建单独的数组（每个小于 20MB），然后将它们连接起来。每个数组都应该在它自己的 var 语句上，而不是使用单个 var 进行多个声明。

I would still consider fetching only the necessary part, it may make the browser sluggish. however, if it's an internal task, this should be fine.

我仍然会考虑只获取必要的部分，这可能会使浏览器变得迟钝。但是，如果是内部任务，这应该没问题。

Last point: You're not at maximum memory levels, just max parsing levels.

最后一点：你不是在最大内存级别，只是最大解析级别。

回答by Guffa

Yes, it's too much to ask of a browser.

是的，对浏览器的要求太多了。

That amount of data would be managable if it already was data, but it isn't data yet. Consider that the browser has to parse that huge block of source code while checking that the syntax adds up for it all. Once parsed into valid code, the code has to run to produce the actual array.

如果该数据量已经是数据，但还不是数据，那么该数据量将是可管理的。考虑到浏览器必须在检查语法加起来的同时解析那个巨大的源代码块。一旦解析为有效代码，就必须运行代码以生成实际数组。

So, all of the data will exist in (at least) two or three versions at once, each with a certain amount of overhead. As the array literal is a single statement, each step will have to include all of the data.

因此，所有数据将同时存在于（至少）两个或三个版本中，每个版本都有一定的开销。由于数组文字是单个语句，因此每个步骤都必须包含所有数据。

Dividing the data into several smaller arrays would possibly make it easier on the browser.

将数据分成几个较小的数组可能会使浏览器更容易。

回答by Matěj Zábsky

Do you really need all the data? can't you stream just the data currently needed using AJAX? Similar to Google Maps - you can't fit all the map data into browser's memory, they display just the part you are currently seeing.

您真的需要所有数据吗？您不能使用 AJAX 仅流式传输当前需要的数据吗？与谷歌地图类似 - 您无法将所有地图数据放入浏览器的内存中，它们仅显示您当前看到的部分。

Remember that 40 megs of hard data can be inflated to much more in browser's internal representation. For example the JS interpreter may use hashtable to implement the array, which would add additional memory overhead. Also, I expect that the browsers stores both source code and the JS memory, that alone doubles the amount of data.

请记住，在浏览器的内部表示中，40 兆字节的硬数据可以膨胀到更多。例如，JS 解释器可能使用哈希表来实现数组，这会增加额外的内存开销。此外，我希望浏览器同时存储源代码和 JS 内存，仅此一项会使数据量增加一倍。

JS is designed to provide client-side UI interaction, not handle loads of data.

JS 旨在提供客户端 UI 交互，而不是处理大量数据。

EDIT:

编辑：

Btw, do you really think users will like downloading 40 megabytes worth of code? There are still many users with less than broadband internet access. And execution of the script will be suspended until all the data is downloaded.

顺便说一句，你真的认为用户会喜欢下载 40 兆字节的代码吗？仍然有许多用户无法访问宽带互联网。脚本的执行将暂停，直到所有数据下载完毕。

EDIT2:

编辑2：

I had a look at the data. That array will definitely be represented as hashtable. Also many of the items are objects, which will require reference tracking...that is additional memory.

我看了一下资料。该数组肯定会表示为哈希表。还有许多项目是对象，这将需要引用跟踪......这是额外的内存。

I guess the performance would be better if it was simple vector of primitive data.

我想如果它是原始数据的简单向量，性能会更好。

EDIT3: The data could certainly be simplified. The bulk of it are repeating strings, which could be encoded in some way as integers or something. Also, my Opera is having trouble just displaying the text, let alone interpreting it.

EDIT3：当然可以简化数据。其中大部分是重复的字符串，可以以某种方式编码为整数或其他东西。此外，我的 Opera 无法仅显示文本，更不用说解释它了。

EDIT4: Forget the DateTime objects! Use unix era timestamps or strings, but not objects!

EDIT4：忘记 DateTime 对象！使用 unix 时代时间戳或字符串，但不要使用对象！

EDIT5: Your processor doesn't matter because JS is single-threaded. And your RAM doesn't matter either, most browsers are 32bit, so they can't use much of that memory.

EDIT5：您的处理器无关紧要，因为 JS 是单线程的。你的 RAM 也不重要，大多数浏览器都是 32 位的，所以它们不能使用太多的内存。

EDIT6: Try changing the array indices to sequential integers(0, 1, 2, 3...). That might make the browser use more efficient array data structure. You can use constants to access the array items efficiently. This is going to cut down the array size by huge chunk.

EDIT6：尝试将数组索引更改为连续整数（0、1、2、3...）。这可能会使浏览器使用更高效的数组数据结构。您可以使用常量有效地访问数组项。这将大大减少数组大小。

回答by Bitsplitter

Try retrieving the data with Ajax as an JSON page. I don't know the exact size but I've been able to pull large amounts of data into Google Chrome that way.

尝试使用 Ajax 作为 JSON 页面检索数据。我不知道确切的大小，但我已经能够通过这种方式将大量数据提取到谷歌浏览器中。

回答by Raynos

Use lazy loading. Have pointers to the data and get it when the user asks.

使用延迟加载。拥有指向数据的指针并在用户询问时获取它。

This technique is used in various places to manage millions of records of data.

这种技术在很多地方都被用来管理数百万条数据记录。

[Edit]

[编辑]

I found what I was looking for. Virtual scrolling in the jqgrid. That's 500k records being lazy loaded.

我找到了我要找的东西。jqgrid 中的虚拟滚动。那是 500k 条记录被延迟加载。

回答by Shadow Wizard is Ear For You

I would try having it as one big string with separator between each "item" then use split, something like:

我会尝试将它作为一个大字符串，每个“项目”之间有分隔符，然后使用split，例如：

var largeString = "item1,item2,.......";
var largeArray = largeString.split(",");

Hopefully string won't exhaust the stack so fast.

希望字符串不会这么快耗尽堆栈。

Edit: in order to test it I created dummy array with 200,000 simple items (each item one number) and Chrome loaded it within an instant. 2,000,000 items? Couple of seconds but no crash. 6,000,000 items array (50 MB file) made Chrome load for about 10 seconds but still, no crash in either ways.

编辑：为了测试它，我创建了包含 200,000 个简单项目（每个项目一个数字）的虚拟数组，Chrome 立即加载了它。2,000,000 件物品？几秒钟，但没有崩溃。6,000,000 个项目数组（50 MB 文件）使 Chrome 加载了大约 10 秒，但仍然没有任何崩溃。

So this leads me to believe the problem is not with the array itself but rather it's contents.. optimize the contents to simple items then parse them "on the fly" and it should work.

所以这让我相信问题不在于数组本身，而在于它的内容..将内容优化为简单的项目，然后“动态”解析它们，它应该可以工作。