Java MySQL JDBC 内存泄漏

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/5765990/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-10-30 12:39:32  来源:igfitidea点击:

Java MySQL JDBC Memory Leak

javamysqljdbcmemory-leaks

提问by Jordan

Ok, so I have this program with many (~300) threads, each of which communicates with a central database. I create a global connection to the DB, and then each thread goes about its business creating statements and executing them.

好的,所以我有这个程序有很多(~300)个线程,每个线程都与一个中央数据库进行通信。我创建了一个到 DB 的全局连接,然后每个线程开始处理它的业务,创建语句并执行它们。

Somewhere along the way, I have a massive memory leak. After analyzing the heap dump, I see that the com.mysql.jdbc.JDBC4Connection object is 70 MB, because it has 800,000 items in "openStatements" (a hash map). Somewhere it's not properly closing the statements that I create, but I cannot for the life of me figure out where (every single time I open one, I close it as well). Any ideas why this might be occurring?

在此过程中的某个地方,我遇到了大量内存泄漏。分析堆转储后,我看到 com.mysql.jdbc.JDBC4Connection 对象为 70 MB,因为它在“openStatements”(哈希映射)中有 800,000 个项目。某处它没有正确关闭我创建的语句,但我终生无法弄清楚在哪里(每次我打开一个,我也关闭它)。任何想法为什么会发生这种情况?

采纳答案by MJB

You know unless MySQL says so, JDBC Connections are NOT thread safe. You CANNOT share them across threads, unless you use a connection pool. In addition as pointed out you should be try/finally guaranteeing all statements, result sets, and connections are closed.

您知道,除非 MySQL 这么说,否则 JDBC 连接不是线程安全的。您不能跨线程共享它们,除非您使用连接池。此外,正如所指出的,您应该尝试/最终保证所有语句、结果集和连接都已关闭。

回答by Vlad Balan

I had exactly the same problem. I needed to keep 1 connection active for 3 threads and at the same time every thread had to execute a lot of statements (the order of 100k). I was very careful and I closed every statement and every resultset using a try....finally... algorithm. This way, even if the code failed in some way, the statement and the resultset were always closed. After running the code for 8 hours I was suprised to find that the necessary memory went from the initial 35MB to 500MB. I generated a dump of the memory and I analyzed it with Mat Analyzer from Eclipse. It turned out that one com.mysql.jdbc.JDBC4Connection object was taking 445MB of memory keeping alive some openStatements objects wich in turn kept alive aroun 135k hashmap entries, probably from all the resultsets. So it seems that even if you close all you statements and resultsets, if you do not close the connection, it keeps references to them and the GarbageCollector can't free the resources.

我遇到了完全相同的问题。我需要为 3 个线程保持 1 个连接处于活动状态,同时每个线程都必须执行大量语句(100k 的顺序)。我非常小心,我使用 try....finally... 算法关闭了每个语句和每个结果集。这样,即使代码以某种方式失败,语句和结果集也始终关闭。运行代码 8 小时后,我惊讶地发现所需的内存从最初的 35MB 增加到 500MB。我生成了内存转储,并使用 Eclipse 中的 Mat Analyzer 对其进行了分析。事实证明,一个 com.mysql.jdbc.JDBC4Connection 对象占用了 445MB 的内存,使一些 openStatements 对象保持活动状态,而这些对象又在 135k 个哈希映射条目(可能来自所有结果集)周围保持活动状态。

My solution: after a long search I found this statement from the guys at MySQL:

我的解决方案:经过长时间的搜索,我从 MySQL 的人那里找到了以下声明:

"A quick test is to add "dontTrackOpenResources=true" to your JDBC URL. If the memory leak goes away, some code path in your application isn't closing statements and result sets."

“快速测试是将“ dontTrackOpenResources=true”添加到您的 JDBC URL。如果内存泄漏消失,您的应用程序中的某些代码路径不会关闭语句和结果集。

Here is the link: http://bugs.mysql.com/bug.php?id=5022. So I tried that and guess what? After 8 hours I was around 40MB of memory required, for the same database operations. Maybe a connection pool would be advisible, but if that's not an option, this is the next best thing I came around.

这是链接:http: //bugs.mysql.com/bug.php?id=5022。所以我试过了,你猜怎么着?8 小时后,对于相同的数据库操作,我需要大约 40MB 的内存。也许连接池是可取的,但如果这不是一个选项,这是我遇到的下一个最好的事情。

回答by Pete Wilson

Once upon a time, whenever my code saw "server went away," it opened a new DB connection. If the error happened in the right (wrong!) place, I was left with some non-free()d orphan memory hanging around. Could something like this account for what you are seeing? How are you handling errors?

曾几何时,每当我的代码看到“服务器消失”时,它就会打开一个新的数据库连接。如果错误发生在正确(错误!)的地方,我就会留下一些非 free()d 孤儿内存。像这样的事情可以解释你所看到的吗?你如何处理错误?

回答by Don

Without seeing your code (which I'm sure is massive), you should really consider some sort of more formal thread pooling mechanism, such as Apache Commons pool framework, Spring's JDBC framework, and others. IMHO, this is a much simpler approach, since someone else has already figured out how to effectively manage these types of situations.

在没有看到您的代码(我确定这些代码很大)的情况下,您真的应该考虑某种更正式的线程池机制,例如 Apache Commons 池框架、Spring 的 JDBC 框架等。恕我直言,这是一种更简单的方法,因为其他人已经想出了如何有效地管理这些类型的情况。