.net 抛出了“System.OutOfMemoryException”类型的异常

Exception of type 'System.OutOfMemoryException' was thrown

提问by vbNewbie

I have suddenly been getting the memory exception errors for two programs running on different machines and even though it seems there is enough memory, it still shows up. I am creating multiple threads in the program so not sure if this appropriate for this forum but could it be something else related to visual studio or is it definitely memory issue.The one program runs on my desktop using visual studio 2008 with 2 gb ram. The other is running on a windows 2003 server with 4 GB ram using visual basic 2008 express. Now the module takes a large xml file that is read into a string and then split and stored in a string array. Now the number of chunks can be upto 10000. Now I know this is big, but I have been running this for over a month now and never had the issue. The only other possible related issue I noticed was that I was running out of space on my harddrive but that was quickly solved with a cleanup. Oh yes the processor for my machine is a duo core set at 2.13 GHZ. It is a console program that makes multiple webrequests but the memory problem arises in one specific module only as I explained above.

我突然收到在不同机器上运行的两个程序的内存异常错误，即使看起来有足够的内存，它仍然出现。我在程序中创建了多个线程，所以不确定这是否适合这个论坛，但它可能是与 Visual Studio 相关的其他东西，或者它绝对是内存问题。一个程序在我的桌面上使用 Visual Studio 2008 和 2 GB 内存运行。另一个是使用visual basic 2008 express 在具有4 GB 内存的Windows 2003 服务器上运行。现在该模块将一个大型 xml 文件读入一个字符串，然后拆分并存储在一个字符串数组中。现在块的数量可以达到 10000。现在我知道这很大，但是我已经运行了一个多月了，从来没有遇到过这个问题。我注意到的唯一其他可能的相关问题是我的硬盘空间不足，但通过清理很快解决了这个问题。哦，是的，我机器的处理器是双核，频率设置为 2.13 GHZ。它是一个控制台程序，可以发出多个 web 请求，但内存问题只出现在一个特定模块中，正如我上面所解释的。

Public Shared Function textLoad(ByVal _html As String) As Boolean
    Try 
            //_html is the filestream that was read in

        Dim defaultHeading = "xmlns:gnip=""http://www.gnip.com/schemas/2010"" xmlns=""http://www.w3.org/2005/Atom"""
        Dim header_of_xml As String = "<?xml version=""1.0"" encoding=""utf-8""?>" & vbNewLine & "<entry " & defaultHeading & ">"
        Dim footer_of_xml As String = "</entry>"
        Dim entry As String = String.Empty
        Dim xrs As XmlReaderSettings = New XmlReaderSettings()
        Dim dupeArray As New ArrayList
        Dim stringSplitter() As String = {"</entry>"}

        //split the file content based on the closing entry tag
        sampleResults = Nothing
        sampleResults = _html.Split(stringSplitter, StringSplitOptions.RemoveEmptyEntries)
        entryResults.Clear()

        If getEntryId(sampleResults) Then
           // the following loops seem clumsy but I wanted to dedupe the lists to //ensure that I am not adding duplicate entries and I do this by going to the getEntryID //method and extracting the ids and then comparing them below 

            For i As Integer = 0 To sampleResults.Count - 1
                For j As Integer = 0 To idList.Count - 1
                    If sampleResults(i).Contains(idList.Item(j)) AndAlso Not dupeArray.Contains(idList.Item(j)) Then
                        dupeArray.Add(idList.Item(j))
                        entry = sampleResults(i)

I did look at taskmanager for identifying resources used by this program and this is what is going on:

我确实查看了 taskmanager 以识别该程序使用的资源，这就是正在发生的事情：

Parser.exe CPU = 34 MEM usage = 349,500 K

Parser.exe CPU = 34 MEM 使用量 = 349,500 K

nothing else intensive is running

没有其他密集型正在运行

Edit _-

编辑_-

Figured out exactly where the problem originates:

找出问题的确切来源：

**sampleResults = _html.Split(stringSplitter, StringSplitOptions.RemoveEmptyEntries)**

Can anyone notice anything wrong with this??

任何人都可以注意到这有什么问题吗？？

回答by matt.dolfin

The Split method allocates memory for a string object for each element in the returned array. If you're going to run into a memory issue, it's going to be on that line. Since you're allocating memory for potentially 10,000 large strings at once, on a potentially fragmented heap, it's not surprising that you might come to a point where it just can't find enough contiguous space to allocate for the next string (which would result in the exception you are getting). Do you really need all of those strings at once? Or can you do something like this loop:

Split 方法为返回数组中的每个元素为字符串对象分配内存。如果您要遇到内存问题，它将在该线上。由于您一次为潜在的 10,000 个大字符串分配内存，因此在潜在的碎片堆上，您可能会发现无法找到足够的连续空间来分配下一个字符串（这将导致在您得到的例外情况下）。您真的一次需要所有这些字符串吗？或者你可以做这样的循环：

1. Read one line at a time (or into a buffer) to build a string.
2. Stop when you've found the first "chunk".
3. Get the info you need from the chunk. (I'm assuming you don't need the whole thing, you're just interested in getting something out of it?).
4. Continue reading the html (Restart at #1), reading it into the same string variable so that the first chunk can get garbage collected.

1. 一次读取一行（或读入缓冲区）以构建一个字符串。
2. 当您找到第一个“块”时停止。
3. 从块中获取您需要的信息。（我假设你不需要整个东西，你只是想从中得到一些东西？）。
4. 继续阅读 html（从 #1 重新开始），将其读入相同的字符串变量，以便第一个块可以被垃圾收集。

If you can implement your solution that way, you're issue should disappear.

如果您可以以这种方式实施您的解决方案，那么您的问题应该会消失。

回答by CodesInChaos

Note that an OOM doesn't mean you're out of physical RAM, but that you can't fit a new memory block of the desired size into the address space.

请注意，OOM 并不意味着您没有物理 RAM，而是您无法将所需大小的新内存块放入地址空间。

There are a number of possible causes for that

有很多可能的原因

• Heap fragmentation
• the 2GB limit of a 32 bit process
• a maximum memory size set in the config

• 堆碎片
• 32 位进程的 2GB 限制
• 在配置中设置的最大内存大小

And then there are fake OOM exceptions. For example GDI+ returns out-of-memory errors for a wide class of problems including invalid parameters. These are turned into an OOM exception too.

然后还有假的 OOM 异常。例如，GDI+ 会针对包括无效参数在内的多种问题返回内存不足错误。这些也变成了 OOM 异常。

回答by rsbarro

Have you tried profiling the memory usage as your application is running using the .NET Performance Counters? You can fire up the Performance Monitor (in Windows 2008 R2 it's under Administrative Tools > Performance Monitor) and add counters under .NET Memory for Gen 0 Heap Size, Gen 1 Heap Size, Gen 2 Heap Size and Large Object Heap size. As your application runs, you can use these counters to see where the memory allocations are.

您是否尝试过在应用程序运行时使用 .NET 性能计数器分析内存使用情况？您可以启动性能监视器（在 Windows 2008 R2 中，它位于管理工具 > 性能监视器下）并在 .NET 内存下为第 0 代堆大小、第 1 代堆大小、第 2 代堆大小和大对象堆大小添加计数器。当您的应用程序运行时，您可以使用这些计数器来查看内存分配的位置。

Since you are working with strings, it's important to take into account that strings over 85,000 bytes will get allocated on the Large Object Heap. All of the other heaps are compacted by the garbage collector, but for performance reasons the Large Object Heap is never compacted. That basically means that objects will be cleared from the LOH by the GC as they are disposed, but the remaining objects are never moved from their original locations on the LOH. This behavior of the CLR can lead to LOH fragmentation and eventually OOM exceptions.

由于您正在处理字符串，因此重要的是要考虑到超过 85,000 字节的字符串将被分配到大对象堆上。所有其他堆都由垃圾收集器压缩，但出于性能原因，大对象堆从不压缩。这基本上意味着对象将在被处理时被 GC 从 LOH 中清除，但剩余的对象永远不会从它们在 LOH 上的原始位置移动。CLR 的这种行为会导致 LOH 碎片化并最终导致 OOM 异常。

Here's a simple example for how LOH framgentation can lead to OOM exceptions. Say we have 100 total units available on the LOH, and we allocate 10 units and then 5 units, leaving us with 85 left. The CLR must allocate the memory continuously, so in our example right now we're taking up the first 15 units of the LOH. We try to allocate 86 units for a really big object, and we get an OOM exception. Realizing that we don't have enough space, we clear the 10 units we first allocated, leaving us with 95 total units avaible. We try again to allocate 86 units, but since the CLR doesn't compact the LOH and the CLR is asking for 86 continuous units, we'll still get OOM exception. We won't be able to allocate our 86 units until we clear the second 5 units as well. This is an oversimplified example, but if you Google LOH fragmentation or check this articleyou can get a more in depth explanation.

下面是 LOH 分片如何导致 OOM 异常的一个简单示例。假设我们在 LOH 上总共有 100 个单位可用，我们分配了 10 个单位，然后分配了 5 个单位，剩下 85 个单位。CLR 必须连续分配内存，因此在我们的示例中，现在我们占用 LOH 的前 15 个单元。我们尝试为一个非常大的对象分配 86 个单元，但我们得到了一个 OOM 异常。意识到我们没有足够的空间，我们清除了最初分配的 10 个单位，剩下 95 个可用单位。我们再次尝试分配 86 个单元，但由于 CLR 没有压缩 LOH，并且 CLR 要求 86 个连续单元，我们仍然会遇到 OOM 异常。在我们清除第二个 5 个单位之前，我们将无法分配我们的 86 个单位。这是一个过于简单的例子，文章你可以获得更深入的解释。

I can't say for certain if this is the problem you are running into, but I would definitely check out the Performance Monitor to see where your memory is being allocated. If you know for a fact you're allocating strings or other objects in excess of 85,000 bytes, then try to split them up further so they stay off the LOH.

我不能肯定这是否是您遇到的问题，但我肯定会检查性能监视器以查看您的内存分配位置。如果您知道要分配超过 85,000 字节的字符串或其他对象，请尝试进一步拆分它们，使它们远离 LOH。

回答by visitor

I've encoutered this very same problem. Thinking about moving to 64 bit OS. I have some tricks to delay this exception:

我遇到了同样的问题。考虑迁移到 64 位操作系统。我有一些技巧可以延迟这个异常：

1. the most important one is that when heavily working with strings (especially long) use ref to transfer from one method to another. It segnificantly reduces memory and performance.

1. 最重要的是，当大量处理字符串（尤其是长字符串）时，使用 ref 从一种方法转移到另一种方法。它显着降低了内存和性能。

2. You can use AppDomain to store data. this doubles your memory capasity.

2.您可以使用AppDomain来存储数据。这使你的记忆能力加倍。

3. Whenever you can - instead of creating objects and strings (WebRequests/ Response) use the same memoryStream or buffer all over again. Allocate one with the max size possile (even *2 the max expected size). the allocation (using new) of Stream/StringBuilder/strings/classes/Buffers Suffocate the heap.

3. 只要有可能 - 不再创建对象和字符串（WebRequests/Response），而是再次使用相同的 memoryStream 或缓冲区。分配一个具有最大可能大小（甚至 *2 最大预期大小）。Stream/StringBuilder/strings/classes/Buffers的分配（使用new）使堆窒息。

回答by Jordan

This article can be usefull, it describe something than can throw the Exception:

这篇文章很有用，它描述了一些可以抛出异常的东西：

http://kb.insideofthefuture.com/Assuntos/outofmemoryexception.htm

http://kb.insideofthefuture.com/Assuntos/outofmemoryexception.htm