python 如何高效使用 MySQLDB SScursor?
声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow
原文地址: http://stackoverflow.com/questions/1808150/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to efficiently use MySQLDB SScursor?
提问by Sylvain
I have to deal with a large result set (could be hundreds thousands of rows, sometimes more).
They unfortunately need to be retrieved all at once (on start up).
我必须处理一个大的结果集(可能是数十万行,有时更多)。
不幸的是,它们需要一次全部检索(在启动时)。
I'm trying to do that by using as less memory as possible.
By looking on SO I've found that using SSCursor
might be what I'm looking for, but I still don't really know how to exactly use them.
我试图通过使用尽可能少的内存来做到这一点。
通过查看 SO 我发现 usingSSCursor
可能是我正在寻找的东西,但我仍然不知道如何确切地使用它们。
Is doing a fetchall()
from a base cursor or a SScursor the same (in term of memory usage)?
Can I 'stream' from the sscursor my rows one by one (or a few by a few) and if yes,
what is the best way to do so?
是fetchall()
从基本游标还是从 SScursor 执行相同的操作(就内存使用而言)?
我可以从 sscursor 一行一行(或几行)“流式传输”我的行,如果可以,
最好的方法是什么?
回答by unutbu
I am in agreement with Otto Allmendinger's answer, but to make explicit Denis Otkidach's comment, here is how you can iterate over the results without using Otto's fetch() function:
我同意 Otto Allmendinger 的回答,但为了明确 Denis Otkidach 的评论,这里是如何在不使用 Otto 的 fetch() 函数的情况下迭代结果的方法:
import MySQLdb.cursors
connection=MySQLdb.connect(
host="thehost",user="theuser",
passwd="thepassword",db="thedb",
cursorclass = MySQLdb.cursors.SSCursor)
cursor=connection.cursor()
cursor.execute(query)
for row in cursor:
print(row)
回答by Otto Allmendinger
Definitely use the SSCursor when fetching big result sets. It made a huge difference for me when I had a similar problem. You can use it like this:
在获取大结果集时,一定要使用 SSCursor。当我遇到类似的问题时,这对我产生了巨大的影响。你可以这样使用它:
import MySQLdb
import MySQLdb.cursors
connection = MySQLdb.connect(
host=host, port=port, user=username, passwd=password, db=database,
cursorclass=MySQLdb.cursors.SSCursor) # put the cursorclass here
cursor = connection.cursor()
Now you can execute your query with cursor.execute()
and use the cursor as an iterator.
现在您可以执行查询cursor.execute()
并将游标用作迭代器。
Edit: removed unnecessary homegrown iterator, thanks Denis!
编辑:删除了不必要的本土迭代器,谢谢丹尼斯!
回答by Yuda Prawira
Alternatively, you can use SSCursor
outside the connection object (it is pretty important when you already define connection and dont want all the connection use SSCursor
as a cursorclass).
或者,您可以SSCursor
在连接对象之外使用(当您已经定义了连接并且不想将所有连接SSCursor
用作游标类时,这非常重要)。
import MySQLdb
from MySQLdb.cursors import SSCursor # or you can use SSDictCursor
connection = MySQLdb.connect(
host=host, port=port, user=username, passwd=password, db=database)
cursor = SSCursor(connection)
cursor.execute(query)
for row in cursor:
print(row)