Python可以pickle lambda函数吗？

Can Python pickle lambda functions?

提问by Lars

I have read in a number of threads that Python pickle/cPicklecannot pickle lambda functions. However the following code works, using Python 2.7.6:

我在许多线程中读到 Python pickle/cPickle不能pickle lambda 函数。但是，以下代码使用 Python 2.7.6 有效：

import cPickle as pickle

if __name__ == "__main__":
    s = pickle.dumps(lambda x, y: x+y)
    f = pickle.loads(s)
    assert f(3,4) == 7

So what is going on? Or, rather, what is the limit of pickling lambdas?

那么发生了什么？或者，更确切地说，酸洗 lambda 的限制是什么？

[EDIT] I think i know why this code runs. I forgot (sorry!) i am running stackless python, which has a form of micro-threads called tasklets executing a function. These tasklets can be halted, pickled, unpickled and continued, so i guess (asked on the stackless mailing list) that it also provides a way to pickle function bodies.

[编辑] 我想我知道为什么这段代码会运行。我忘了（抱歉！）我正在运行无堆栈 python，它具有一种称为 tasklets 的微线程形式，用于执行一个函数。这些 tasklet 可以被暂停、腌制、取消腌制和继续，所以我猜（在无堆栈邮件列表中询问）它还提供了一种腌制函数体的方法。

回答by Ned Batchelder

No, Python can't pickle lambda functions:

不，Python 不能pickle lambda 函数：

>>> import cPickle as pickle
>>> s = pickle.dumps(lambda x,y: x+y)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle function objects

Not sure what you did that succeeded...

不知道你做了什么成功...

回答by Mike McKerns

Yes, python can pickle lambda functions… but only if you have something that uses copy_regto register howto pickle lambda functions -- the package dillloads the copy_regyou need into the pickle registry for you, when you import dill.

是的，蟒蛇可以泡制lambda函数......但只有当你有一些用途copy_reg登记如何咸菜lambda函数-包dill加载copy_reg你需要到泡菜注册表中的你，当你import dill。

Python 2.7.8 (default, Jul 13 2014, 02:29:54) 
[GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> import dill  # the code below will fail without this line
>>> 
>>> import pickle
>>> s = pickle.dumps(lambda x, y: x+y)
>>> f = pickle.loads(s)
>>> assert f(3,4) == 7
>>> f
<function <lambda> at 0x10aebdaa0>

get dill here: https://github.com/uqfoundation

在这里获取莳萝：https: //github.com/uqfoundation

回答by Saim Raza

Python can pickle lambdas.We will cover Python 2 and 3 separately as implementation of pickle are different in different Python versions.

Python 可以腌制 lambda。我们将分别介绍 Python 2 和 3，因为泡菜的实现在不同的 Python 版本中是不同的。

• Python 2.7

• 蟒蛇 2.7

pickleuses pickle registrywhich is nothing but a mapping from typeto the function to use for serializing (pickling) objects of that type. You can see pickle registryas:

pickle使用pickle 注册表，它只是从type到用于序列化（pickling）该类型对象的函数的映射。您可以将pickle注册表视为：

>> pickle.Pickler.dispatch

{bool: <function pickle.save_bool>,
 instance: <function pickle.save_inst>,
 classobj: <function pickle.save_global>,
 float: <function pickle.save_float>,
 function: <function pickle.save_global>,
 int: <function pickle.save_int>,
 list: <function pickle.save_list>,
 long: <function pickle.save_long>,
 dict: <function pickle.save_dict>,
 builtin_function_or_method: <function pickle.save_global>,
 NoneType: <function pickle.save_none>,
 str: <function pickle.save_string>,
 tuple: <function pickle.save_tuple>,
 type: <function pickle.save_global>,
 unicode: <function pickle.save_unicode>}

To pickle custom types, Python provides copy_regmodule to register our functions. You can read more about it here. By default, copy_regmodule supports pickling of the following additional types:

为了pickle自定义类型，Python提供了copy_reg模块来注册我们的函数。您可以在此处阅读更多相关信息。默认情况下，copy_reg模块支持以下附加类型的酸洗：

>> import copy_reg
>> copy_reg.dispatch_table

{code: <function ipykernel.codeutil.reduce_code>,
 complex: <function copy_reg.pickle_complex>,
 _sre.SRE_Pattern: <function re._pickle>,
 posix.statvfs_result: <function os._pickle_statvfs_result>,
 posix.stat_result: <function os._pickle_stat_result>}

Now, type of lambdafunctions is types.FunctionType. However, the builtin function for this type function: <function pickle.save_global>is not able to serialize lambda functions. Therefore, all third party libraries like dill, cloudpickle, etc override the inbuilt method to serialize lambda functions with some additional logic. Let's import dilland see what it does.

现在，lambda函数类型是types.FunctionType。但是，此类型的内置函数function: <function pickle.save_global>无法序列化 lambda 函数。因此，所有第三方库（如dill、cloudpickle等）都会覆盖内置方法，以使用一些附加逻辑来序列化 lambda 函数。让我们导入dill并查看它的作用。

>> import dill
>> pickle.Pickler.dispatch

{_pyio.BufferedReader: <function dill.dill.save_file>,
 _pyio.TextIOWrapper: <function dill.dill.save_file>,
 _pyio.BufferedWriter: <function dill.dill.save_file>,
 _pyio.BufferedRandom: <function dill.dill.save_file>,
 functools.partial: <function dill.dill.save_functor>,
 operator.attrgetter: <function dill.dill.save_attrgetter>,
 operator.itemgetter: <function dill.dill.save_itemgetter>,
 cStringIO.StringI: <function dill.dill.save_stringi>,
 cStringIO.StringO: <function dill.dill.save_stringo>,
 bool: <function pickle.save_bool>,
 cell: <function dill.dill.save_cell>,
 instancemethod: <function dill.dill.save_instancemethod0>,
 instance: <function pickle.save_inst>,
 classobj: <function dill.dill.save_classobj>,
 code: <function dill.dill.save_code>,
 property: <function dill.dill.save_property>,
 method-wrapper: <function dill.dill.save_instancemethod>,
 dictproxy: <function dill.dill.save_dictproxy>,
 wrapper_descriptor: <function dill.dill.save_wrapper_descriptor>,
 getset_descriptor: <function dill.dill.save_wrapper_descriptor>,
 member_descriptor: <function dill.dill.save_wrapper_descriptor>,
 method_descriptor: <function dill.dill.save_wrapper_descriptor>,
 file: <function dill.dill.save_file>,
 float: <function pickle.save_float>,
 staticmethod: <function dill.dill.save_classmethod>,
 classmethod: <function dill.dill.save_classmethod>,
 function: <function dill.dill.save_function>,
 int: <function pickle.save_int>,
 list: <function pickle.save_list>,
 long: <function pickle.save_long>,
 dict: <function dill.dill.save_module_dict>,
 builtin_function_or_method: <function dill.dill.save_builtin_method>,
 module: <function dill.dill.save_module>,
 NotImplementedType: <function dill.dill.save_singleton>,
 NoneType: <function pickle.save_none>,
 xrange: <function dill.dill.save_singleton>,
 slice: <function dill.dill.save_slice>,
 ellipsis: <function dill.dill.save_singleton>,
 str: <function pickle.save_string>,
 tuple: <function pickle.save_tuple>,
 super: <function dill.dill.save_functor>,
 type: <function dill.dill.save_type>,
 weakcallableproxy: <function dill.dill.save_weakproxy>,
 weakproxy: <function dill.dill.save_weakproxy>,
 weakref: <function dill.dill.save_weakref>,
 unicode: <function pickle.save_unicode>,
 thread.lock: <function dill.dill.save_lock>}

Now, let's try to pickle lambda function.

现在，让我们尝试pickle lambda 函数。

>> pickle.loads(pickle.dumps(lambda x:x))
<function __main__.<lambda>>

It WORKS!!

有用！！

In Python 2 we have two versions of pickle-

在 Python 2 中，我们有两个版本pickle-

import pickle # pure Python version
pickle.__file__ # <install directory>/python-2.7/lib64/python2.7/pickle.py

import cPickle # C extension
cPickle.__file__ # <install directory>/python-2.7/lib64/python2.7/lib-dynload/cPickle.so

Now, let's try to pickle lambda with C implementation cPickle.

现在，让我们尝试使用 C 实现pickle lambda cPickle。

>> import cPickle
>> cPickle.loads(cPickle.dumps(lambda x:x))
TypeError: can't pickle function objects

What went wrong? Let's see the dispatch table of cPickle.

什么地方出了错？让我们看看 的调度表cPickle。

>> cPickle.Pickler.dispatch_table
AttributeError: 'builtin_function_or_method' object has no attribute 'dispatch_table'

The implementation of pickleand cPickleis different. Importingdill makes only Python version of picklework. The disadvantage of using pickleinstead of cPickleis that it can be as much as 1000 timesslower than cPickle.

的实施pickle和cPickle是不同的。Importingdill 只制作 Python 版本的pickle工作。使用pickle代替的缺点cPickle是它可能比 cPickle 慢1000 倍。

• Python 3.6

• 蟒蛇 3.6

In Python 3, there is no module named cPickle. We have pickleinstead which also doesn't support pickling of lambdafunctions by default. Let's see it's dispatch table:

在 Python 3 中，没有名为cPickle. pickle相反，我们有lambda默认情况下也不支持酸洗功能。让我们看看它的调度表：

>> import pickle
>> pickle.Pickler.dispatch_table
<member 'dispatch_table' of '_pickle.Pickler' objects>

Wait. I tried looking up dispatch_tableof picklenot _pickle. _pickleis the alternative and faster C implementation of pickle. But we haven't imported it yet! This C implementation is imported automatically, if it is available, at the end of pure Python picklemodule.

等待。我试着查找dispatch_table的picklenot _pickle。_pickle是 pickle 的替代和更快的 C 实现。但是我们还没有导入它！如果可用，此 C 实现将在纯 Pythonpickle模块的末尾自动导入。

# Use the faster _pickle if possible
try:
    from _pickle import (
        PickleError,
        PicklingError,
        UnpicklingError,
        Pickler,
        Unpickler,
        dump,
        dumps,
        load,
        loads
    )
except ImportError:
    Pickler, Unpickler = _Pickler, _Unpickler
    dump, dumps, load, loads = _dump, _dumps, _load, _loads

We are still left with the question of pickling lambdas in Python 3. The answer is you CAN'Twith the native pickleor _pickle. You will need to import dillor cloudpickleand use that instead of the native pickle module.

我们仍然面临在 Python 3 中酸洗 lambdas 的问题。答案是您不能使用本机pickle或_pickle. 您将需要导入dill或cloudpickle并使用它而不是本机 pickle 模块。

>> import dill
>> dill.loads(dill.dumps(lambda x:x))
<function __main__.<lambda>>

I hope this clears all the doubts.

我希望这能消除所有疑虑。

回答by Sandro

Even though it might be obvious I would like to add an other possible solution. As you probably know lambda functions are just anonymous function declarations. If you don't have many lambdas that are used only once and it wouldn't add much noise to your code you could just name your lambda and pass the name of it (without the parentheses) like this:

尽管这可能很明显，但我想添加其他可能的解决方案。您可能知道 lambda 函数只是匿名函数声明。如果您没有很多只使用一次的 lambda，并且不会给您的代码增加太多噪音，您可以像这样命名您的 lambda 并传递它的名称（不带括号）：

import cPickle as pickle

def addition(x, y):
    return x+y


if __name__ == "__main__":
    s = pickle.dumps(addition)
    f = pickle.loads(s)
    assert f(3,4) == 7

The name also adds more semantic and you wouldn't need an additional dependency like Dill. But only do that if that outweighs the added noise of the additional function(s).

该名称还增加了更多语义，您不需要像 Dill 这样的额外依赖项。但只有当这超过附加功能的附加噪音时才这样做。

回答by juan Isaza

Install dill

安装莳萝

$ pip install dill

Touch a file

触摸文件

touch yeah.p

Now run this python3 script,

现在运行这个python3脚本，

import dill

dill.dump(lambda x:x+1, open('yeah.p', 'wb'))
my_lambda = dill.load(open('yeah.p', 'rb'))
print(my_lambda(2))  # 3

回答by TamarV

what worked for me (windows 10, python 3.7) was to pass a function instead of a lambda function:

对我有用的（windows 10，python 3.7）是传递一个函数而不是 lambda 函数：

def merge(x):
    return Image.merge("RGB", x.split()[::-1])

transforms.Lambda(merge)

instead of:

代替：

transforms.Lambda(lambda x: Image.merge("RGB", x.split()[::-1]))

no dill or cPickel needed.

不需要莳萝或 cpickel。