Python 在 TensorFlow 中有什么方法可以只初始化未初始化的变量？

In TensorFlow is there any way to just initialize uninitialised variables?

提问by Daniel Slater

The standard way of initializing variables in TensorFlow is

在 TensorFlow 中初始化变量的标准方法是

init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)

After running some learning for a while I create a new set of variables but once I initialize them it resets all my existing variables. At the moment my way around this is to save all the variable I need and then reapply them after the tf.initalize_all_variables call. This works but is a bit ugly and clunky. I cannot find anything like this in the docs...

经过一段时间的学习后，我创建了一组新变量，但是一旦我初始化它们，它就会重置我所有现有的变量。目前我解决这个问题的方法是保存我需要的所有变量，然后在 tf.initalize_all_variables 调用后重新应用它们。这有效，但有点丑陋和笨重。我在文档中找不到这样的东西......

Does anyone know of any good way to just initialize the uninitialized variables?

有谁知道初始化未初始化变量的任何好方法？

采纳答案by mrry

There is no elegant* way to enumerate the uninitialized variables in a graph. However, if you have access to the new variable objects—let's call them v_6, v_7, and v_8—you can selectively initialize them using tf.initialize_variables():

没有优雅*的方法来枚举图中未初始化的变量。但是，如果您可以访问新的变量对象——让我们称它们为v_6、v_7、 和v_8——您可以使用 有选择地初始化它们tf.initialize_variables()：

init_new_vars_op = tf.initialize_variables([v_6, v_7, v_8])
sess.run(init_new_vars_op)

---

---

* A process of trial and error could be used to identify the uninitialized variables, as follows:

* 试错过程可用于识别未初始化的变量，如下所示：

uninitialized_vars = []
for var in tf.all_variables():
    try:
        sess.run(var)
    except tf.errors.FailedPreconditionError:
        uninitialized_vars.append(var)

init_new_vars_op = tf.initialize_variables(uninitialized_vars)
# ...

...however, I would not condone such behavior :-).

...但是，我不会容忍这种行为:-)。

回答by Poik

UPDATE:TensorFlow 0.9 has a new method that "fixes" all this but only if you're using a VariableScopewith reuseset to True. tf.report_uninitialized_variableswhich can be used in one line with sess.run( tf.initialize_variables( list( tf.get_variable(name) for name in sess.run( tf.report_uninitialized_variables( tf.all_variables( ) ) ) ) ) )

UPDATE：TensorFlow 0.9有一个新的方法“修复”所有这一切，但只有当您使用的是VariableScope与reuse设置为True。tf.report_uninitialized_variables可以在一行中使用sess.run( tf.initialize_variables( list( tf.get_variable(name) for name in sess.run( tf.report_uninitialized_variables( tf.all_variables( ) ) ) ) ) )

or more intelligently through the ability to specify the variables you expect to be initialized:

或者更智能地通过指定您期望初始化的变量的能力：

def guarantee_initialized_variables(session, list_of_variables = None):
    if list_of_variables is None:
        list_of_variables = tf.all_variables()
    uninitialized_variables = list(tf.get_variable(name) for name in
                                   session.run(tf.report_uninitialized_variables(list_of_variables)))
    session.run(tf.initialize_variables(uninitialized_variables))
    return unintialized_variables

This is still less ideal than actually knowing which variables are and are not initialized and taking care of that properly, but in the case of misdirection like the optimclasses (see below) it may be hard to avoid.

这仍然不如实际了解哪些变量已初始化和未初始化并正确处理这些变量更理想，但在诸如optim类（见下文）之类的误导的情况下，可能难以避免。

Also note, tf.initialize_variables cannot evaluate tf.report_uninitialized_variables, so both of them have to be run within the context of the session to work.

另请注意，tf.initialize_variables 无法评估 tf.report_uninitialized_variables，因此它们都必须在会话的上下文中运行才能工作。

---

---

There is an inelegant but concise way to do it. Before introducing your new variables run temp = set(tf.all_variables())and afterwards run sess.run(tf.initialize_variables(set(tf.all_variables()) - temp)). These together will only initialize any variables created after the temp value is assigned.

有一种不优雅但简洁的方法来做到这一点。在引入新变量之前运行temp = set(tf.all_variables())，然后运行sess.run(tf.initialize_variables(set(tf.all_variables()) - temp))。这些一起只会初始化在分配临时值后创建的任何变量。

I've been playing with transfer learning, so I wanted a quick way to do it too, but this is the best way I could find. Especially when using things like AdamOptimizer, which doesn't give you easy (or any, I'm not sure) access to the variables it uses. So the following actually shows up in my code. (I initialize the new layer's variables explicitly, and run it once to show the initial error before transfer learning. Just for a sanity check.)

我一直在玩迁移学习，所以我也想要一种快速的方法，但这是我能找到的最好的方法。特别是在使用 AdamOptimizer 之类的东西时，它不会让您轻松（或任何，我不确定）访问它使用的变量。所以以下实际上显示在我的代码中。（我显式地初始化了新层的变量，并在迁移学习之前运行它一次以显示初始错误。只是为了进行完整性检查。）

temp = set(tf.all_variables())
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
#I honestly don't know how else to initialize ADAM in TensorFlow.
sess.run(tf.initialize_variables(set(tf.all_variables()) - temp))

And it solves all my problems.

它解决了我所有的问题。

EDIT:@Lifu_Huang's answerstates the proper way to fix my problem. Theoretically, you should use tf.train.Optimizer.get_slot_namesand tf.train.Optimizer.get_slot:

编辑：@Lifu_Huang 的回答说明了解决我的问题的正确方法。理论上，你应该使用tf.train.Optimizer.get_slot_names和tf.train.Optimizer.get_slot：

optim = tf.train.AdadeltaOptimizer(1e-4)
loss = cross_entropy(y,yhat)
train_step = optim.minimize(loss)
sess.run(tf.initialize_variables([optim.get_slot(loss, name)
                                  for name in optim.get_slot_names()])

This however gives me AttributeError: 'NoneType' object has no attribute 'initializer'. I'll make edits when I figure out what I did wrong, so you don't make my mistakes.

然而，这给了我AttributeError: 'NoneType' object has no attribute 'initializer'。当我弄清楚我做错了什么时，我会进行编辑，这样你就不会犯我的错误。

回答by Lifu Huang

For the case @Poik mentioned, when variables are created by optimizers so that they cannot be accessed directly, a cleaner solution is to use tf.train.Optimizer.get_slot.

对于@Poik 提到的情况，当优化器创建变量而无法直接访问它们时，更简洁的解决方案是使用tf.train.Optimizer.get_slot.

Some optimizer subclasses, such as MomentumOptimizerand AdagradOptimizerallocate and manage additional variables associated with the variables to train. These are called Slots. You can use tf.train.Optimizer.get_slot_names()to get all slot names a optimizer has and then use tf.train.Optimizer.get_slotto get the variable allocated for these slots.

一些优化器子类，例如MomentumOptimizer和AdagradOptimizer分配和管理与要训练的变量相关联的附加变量。这些被称为插槽。您可以使用tf.train.Optimizer.get_slot_names()来获取优化器拥有的所有插槽名称，然后使用它tf.train.Optimizer.get_slot来获取为这些插槽分配的变量。

回答by u4841393

I've come up with a method for TensorFlow r0.11:

我为 TensorFlow r0.11 提出了一种方法：

def get_uninitialized_variables(variables=None):
    """Get uninitialized variables as a list.

    Parameters
    ----------
    variables : collections.Iterable[tf.Variable]
        Return only uninitialized variables within this collection.
        If not specified, will return all uninitialized variables.

    Returns
    -------
    list[tf.Variable]
    """
    sess = tf.get_default_session()
    if variables is None:
        variables = tf.all_variables()
    else:
        variables = list(variables)
    init_flag = sess.run(
        tf.pack([tf.is_variable_initialized(v) for v in variables]))
    return [v for v, f in zip(variables, init_flag) if not f]

回答by Taras Kucherenko

I think that the easiest way is to create all the training operators first and initialize variables afterwards.

我认为最简单的方法是先创建所有训练运算符，然后再初始化变量。

For example, I solved the problem of layer-wise pretraining with Adam Optimizer in the following way:

例如，我通过以下方式解决了 Adam Optimizer 分层预训练的问题：

 # create an optimizer
 pretrain_optimizer =  tf.train.AdamOptimizer(learning_rate=learning_rate)

 # Make an array of the trainers for all the layers
 trainers=[pretrain_optimizer.minimize(loss_reconstruction(ae.run_less_layers(ae._input_, i+1), ae.run_less_layers(ae._input_, i+1, is_target=True)), global_step=tf.contrib.framework.get_or_create_global_step(), name='Layer_wise_optimizer_'+str(i)) for i in xrange(len(ae_shape) - 2)]

 # Initialize all the variables
 sess.run(tf.global_variables_initializer())

回答by Salvador Dali

TF does not have a function that does exactly what you want, but you can easily write one:

TF没有完全按照您的要求执行的功能，但您可以轻松编写一个：

import tensorflow as tf

def initialize_uninitialized(sess):
    global_vars          = tf.global_variables()
    is_not_initialized   = sess.run([tf.is_variable_initialized(var) for var in global_vars])
    not_initialized_vars = [v for (v, f) in zip(global_vars, is_not_initialized) if not f]

    print [str(i.name) for i in not_initialized_vars] # only for testing
    if len(not_initialized_vars):
        sess.run(tf.variables_initializer(not_initialized_vars))

Here I extract all global variables, iterate all of them and check whether they are already initialized. After this I get a list of uninitialized variables which I initialize. I also print variables that I am going to initialize for debugging purposes.

在这里，我提取所有全局变量，迭代它们并检查它们是否已经初始化。在此之后，我得到了一个我初始化的未初始化变量列表。我还打印了为了调试目的而要初始化的变量。

---

---

You can easily verify that it works as expected:

您可以轻松验证它是否按预期工作：

a = tf.Variable(3, name='my_var_a')
b = tf.Variable(4, name='my_var_b')

sess = tf.Session()
initialize_uninitialized(sess)
initialize_uninitialized(sess)

c = tf.Variable(5, name='my_var_a') # the same name, will be resolved to different name
d = tf.Variable(6, name='my_var_d')
initialize_uninitialized(sess)

print '\n\n', sess.run([a, b, c, d])

This will print all the unitialized variables before initializing them and the last sess.run will make sure persuade you that all variables are initialized.

这将在初始化之前打印所有未初始化的变量，最后的 sess.run 将确保说服您所有变量都已初始化。

---

---

You can also use tf.report_uninitialized_variables()to write a similar function. A sketch of it is here.

你也可以用tf.report_uninitialized_variables()写一个类似的函数。它的草图在这里。

回答by kmario23

Btw, if you want to initialize only a single tensor (for e.g. tf.Variable) that hasn't been initialized using tf.global_variables_initializer(), then you can use your_tensor.initializerin the sess.run()as in the following example:

顺便说一句，如果你想初始化只有一个张量（例如，对于 tf.Variable使用尚未初始化）tf.global_variables_initializer()，那么你可以使用your_tensor.initializer在sess.run()如下面的例子：

In [196]: weights = tf.Variable(tf.zeros(shape=(3, 4)), name='weights')

In [197]: with tf.Session() as sess:
     ...:     sess.run(weights.initializer)
     ...:     print(weights.eval())
     ...:     

# the result
[[ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]