Python 在pickle文件中保存和加载多个对象？

Saving and loading multiple objects in pickle file?

提问by hamidfzm

I have a class that serves players in a game, creates them and other things.

我有一个班级，为游戏中的玩家服务，创造他们和其他东西。

I need to save these player objects in a file to use it later. I've tried the pickle module but I don't know how to save multiple objects and again loading them? Is there a way to do that or should I use other classes such as lists and save and load my objects in a list?

我需要将这些播放器对象保存在一个文件中以供以后使用。我已经尝试过 pickle 模块，但我不知道如何保存多个对象并再次加载它们？有没有办法做到这一点，或者我应该使用其他类（例如列表）并在列表中保存和加载我的对象？

Is there a better way?

有没有更好的办法？

采纳答案by Tim Peters

Using a list, tuple, or dict is by far the most common way to do this:

使用列表、元组或字典是迄今为止最常用的方法：

import pickle
PIK = "pickle.dat"

data = ["A", "b", "C", "d"]
with open(PIK, "wb") as f:
    pickle.dump(data, f)
with open(PIK, "rb") as f:
    print pickle.load(f)

That prints:

那打印：

['A', 'b', 'C', 'd']

However, a pickle file cancontain any number of pickles. Here's code producing the same output. But note that it's harder to write and to understand:

然而，一个pickle 文件可以包含任意数量的pickle。这是产生相同输出的代码。但请注意，它更难编写和理解：

with open(PIK, "wb") as f:
    pickle.dump(len(data), f)
    for value in data:
        pickle.dump(value, f)
data2 = []
with open(PIK, "rb") as f:
    for _ in range(pickle.load(f)):
        data2.append(pickle.load(f))
print data2

If you do this, you're responsible for knowing how many pickles are in the file you write out. The code above does that by pickling the number of list objects first.

如果你这样做，你有责任知道你写出的文件中有多少泡菜。上面的代码通过首先酸洗列表对象的数量来做到这一点。

回答by Lutz Prechelt

Two additions to Tim Peters' accepted answer.

蒂姆·彼得斯接受的答案的两个补充。

First, you need not store the number of items you pickled separately if you stop loading when you hit the end of the file:

首先，如果您在到达文件末尾时停止加载，则无需单独存储您腌制的项目数：

def loadall(filename):
    with open(filename, "rb") as f:
        while True:
            try:
                yield pickle.load(f)
            except EOFError:
                break

items = loadall(myfilename)

This assumes the file contains only pickles; if there's anything else in there, the generator will try to treat whatever else is in there as pickles too, which could be dangerous.

这假设文件只包含泡菜；如果里面还有其他东西，生成器也会尝试把里面的其他东西也当作泡菜，这可能很危险。

Second, this way, you do not get a list but rather a generator. This will load only one item into memory at a time, which is useful if the dumped data is very large -- one possible reason why you may have wanted to pickle multiple items separately in the first place. You can still iterate over itemswith a forloop as if it were a list.

其次，通过这种方式，您不会得到一个列表，而是一个生成器。这一次只会将一个项目加载到内存中，如果转储的数据非常大，这会很有用——这可能是您首先想要分别腌制多个项目的一个可能原因。您还可以遍历items一个for循环就好像它是一个列表。

回答by lyfing

I will give an object-oriented demo using pickleto store and restore one or multi object:

我将给出一个面向对象的演示，pickle用于存储和恢复一个或多个object：

class Worker(object):

    def __init__(self, name, addr):
        self.name = name
        self.addr = addr

    def __str__(self):
        string = u'[<Worker> name:%s addr:%s]' %(self.name, self.addr)
        return string

# output one item
with open('testfile.bin', 'wb') as f:
    w1 = Worker('tom1', 'China')
    pickle.dump(w1, f)

# input one item
with open('testfile.bin', 'rb') as f:
    w1_restore = pickle.load(f)
print 'item: %s' %w1_restore

# output multi items
with open('testfile.bin', 'wb') as f:
    w1 = Worker('tom2', 'China')
    w2 = Worker('tom3', 'China')
    pickle.dump([w1, w2], f)

# input multi items
with open('testfile.bin', 'rb') as f:
    w_list = pickle.load(f)

for w in w_list:
    print 'item-list: %s' %w

output:

输出：

item: [<Worker> name:tom1 addr:China]
item-list: [<Worker> name:tom2 addr:China]
item-list: [<Worker> name:tom3 addr:China]

回答by Mike McKerns

It's easy if you use klepto, which gives you the ability to transparently store objects in files or databases. It uses a dict API, and allows you to dumpand/or loadspecific entries from an archive (in the case below, serialized objects stored one entry per file in a directory called scores).

如果您使用klepto，这很容易，它使您能够透明地将对象存储在文件或数据库中。它使用 dict API，并允许您dump和/或load存档中的特定条目（在下面的情况下，序列化对象在名为 的目录中每个文件存储一个条目scores）。

>>> import klepto
>>> scores = klepto.archives.dir_archive('scores', serialized=True)
>>> scores['Guido'] = 69 
>>> scores['Fernando'] = 42
>>> scores['Polly'] = 101
>>> scores.dump()
>>> # access the archive, and load only one 
>>> results = klepto.archives.dir_archive('scores', serialized=True)
>>> results.load('Polly')
>>> results
dir_archive('scores', {'Polly': 101}, cached=True)
>>> results['Polly']
101
>>> # load all the scores
>>> results.load()
>>> results['Guido']
69
>>>

回答by N.S

Try this:

尝试这个：

import pickle

file = open('test.pkl','wb')
obj_1 = ['test_1', {'ability', 'mobility'}]
obj_2 = ['test_2', {'ability', 'mobility'}]
obj_3 = ['test_3', {'ability', 'mobility'}]

pickle.dump(obj_1, file)
pickle.dump(obj_2, file)
pickle.dump(obj_3, file)

file.close()

file = open('test.pkl', 'rb')
obj_1 = pickle.load(file)
obj_2 = pickle.load(file)
obj_3 = pickle.load(file)
print(obj_1)
print(obj_2)
print(obj_3)
file.close()

回答by cs95

If you're dumping it iteratively, you'd have to read it iteratively as well.

如果你迭代地转储它，你也必须迭代地阅读它。

You can run a loop(as the accepted answershows) to keep unpickling rows until you reach the end-of-file (at which point an EOFErroris raised).

您可以运行一个循环（如已接受的答案所示）以保持解压行，直到到达文件末尾（此时 anEOFError被引发）。

data = []
with open("data.pickle", "rb") as f:
    while True:
        try:
            data.append(pickle.load(f))
        except EOFError:
            break

Minimal Verifiable Example

最小可验证示例

import pickle

# Dumping step
data = [{'a': 1}, {'b': 2}]
with open('test.pkl', 'wb') as f:
    for d in data:
        pickle.dump(d, f)

# Loading step
data2 = []
with open('test.pkl', 'rb') as f:
    while True:
        try:
            data2.append(pickle.load(f))
        except EOFError:
            break

data2
# [{'a': 1}, {'b': 2}]

data == data2
# True

---

---

Of course, this is under the assumption that your objects have to be pickled individually.You can also store your data as a single list of object, then use a single pickle/unpickle call (no need for loops).

当然，这是在您的对象必须单独腌制的假设下。您还可以将数据存储为单个对象列表，然后使用单个 pickle/unpickle 调用（不需要循环）。

data = [{'a':1}, {'b':2}]  # list of dicts as an example
with open('test.pkl', 'wb') as f:
    pickle.dump(data, f)

with open('test.pkl', 'rb') as f:
    data2 = pickle.load(f)

data2
# [{'a': 1}, {'b': 2}]