在 JavaScript 中处理大型（12K+ 行）数组
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/10485333/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Processing a large (12K+ rows) array in JavaScript
提问by S16
The project requirements are odd for this one, but I'm looking to get some insight...
这个项目的要求很奇怪，但我希望得到一些见解......
I have a CSV file with about 12,000 rows of data, approximately 12-15 columns. I'm converting that to a JSON array and loading it via JSONP (has to run client-side). It takes many seconds to do any kind of querying on the data set to returned a smaller, filtered data set. I'm currently using JLINQ to do the filtering, but I'm essentially just looping through the array and returning a smaller set based on conditions.
我有一个 CSV 文件，其中包含大约 12,000 行数据，大约 12-15 列。我正在将其转换为 JSON 数组并通过 JSONP 加载它（必须在客户端运行）。对数据集进行任何类型的查询以返回较小的、经过过滤的数据集都需要很多秒。我目前正在使用 JLINQ 进行过滤，但我基本上只是遍历数组并根据条件返回一个较小的集合。
Would webdb or indexeddb allow me to do this filtering significantly faster? Any tutorials/articles out there that you know of that tackles this particular type of issue?
webdb 或 indexeddb 会允许我显着更快地进行此过滤吗？您知道的任何教程/文章都可以解决这种特定类型的问题？
采纳答案by Andrew
http://square.github.com/crossfilter/(no longer maintained, see https://github.com/crossfilter/crossfilterfor a newer fork.)
http://square.github.com/crossfilter/（不再维护，请参阅https://github.com/crossfilter/crossfilter以获得更新的分支。）
Crossfilter is a JavaScript library for exploring large multivariate datasets in the browser. Crossfilter supports extremely fast (<30ms) interaction with coordinated views, even with datasets containing a million or more records...
Crossfilter 是一个 JavaScript 库，用于在浏览器中探索大型多元数据集。Crossfilter 支持与协调视图的极快（<30 毫秒）交互，即使数据集包含一百万或更多记录......
回答by buley
If you require loading an entire data object into memory before you apply some transform on it, I would leave IndexedDB and WebSQL out of the mix as they typically both add to complexity and reduce the performance of apps.
如果您需要在对其应用一些转换之前将整个数据对象加载到内存中，我会将 IndexedDB 和 WebSQL 排除在外，因为它们通常都会增加复杂性并降低应用程序的性能。
For this type of filtering, a library like Crossfilterwill go a long way.
对于这种类型的过滤，像Crossfilter这样的库会大有帮助。
Where IndexedDB and WebSQL can come into play in terms of filtering is when you don't need to load, or don't want to load, an entire dataset into memory. These databases are best utilized for their ability to index rows (WebSQL) and attributes (IndexedDB).
当您不需要或不想将整个数据集加载到内存中时，IndexedDB 和 WebSQL 可以在过滤方面发挥作用。这些数据库最适合用于索引行 (WebSQL) 和属性 (IndexedDB) 的能力。
With in browser databases, you can stream data into a database one record at a time and then cursor through it, one record at a time. The benefit here for filtering is that this you means can leave your data on "disk" (a .leveldbin Chrome and .sqlitedatabase for FF) and filter out unnecessary records either as a pre-filter step or filter in itself.
在浏览器数据库中，您可以一次一条记录将数据流式传输到数据库中，然后通过光标一次一条记录。过滤的好处是，这意味着您可以将数据保留在“磁盘”上（.leveldbChrome 中的 a 和.sqliteFF 的数据库）并过滤掉不必要的记录，作为预过滤步骤或过滤器本身。
回答by Dawson Toth
This reminds me of an article John Resig wrote about dictionary lookups (a real dictionary, not a programming construct).
这让我想起了 John Resig 写的一篇关于字典查找的文章（一个真正的字典，而不是一个编程结构）。
http://ejohn.org/blog/dictionary-lookups-in-javascript/
http://ejohn.org/blog/dictionary-lookups-in-javascript/
He starts with server side implementations, and then works on a client side solution. It should give you some ideas for ways to improve what you are doing right now:
他从服务器端实现开始，然后在客户端解决方案上工作。它应该为您提供一些改进您目前正在做的事情的方法的想法：
- Caching
- Local Storage
- Memory Considerations
- 缓存
- 本地存储
- 内存注意事项
