Python 如何使用 TensorFlow 获得稳定结果，设置随机种子
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/36288235/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to get stable results with TensorFlow, setting random seed
提问by Waanders
I'm trying to run a neural network multiple times with different parameters in order to calibrate the networks parameters (dropout probabilities, learning rate e.d.). However I am having the problem that running the network while keeping the parameters the same still gives me a different solution when I run the network in a loop as follows:
我正在尝试使用不同的参数多次运行神经网络，以校准网络参数（辍学概率、学习率 ed）。但是我遇到的问题是，当我在循环中运行网络时，在保持参数相同的情况下运行网络仍然给我一个不同的解决方案，如下所示：
filename = create_results_file()
for i in range(3):
g = tf.Graph()
with g.as_default():
accuracy_result, average_error = network.train_network(
parameters, inputHeight, inputWidth, inputChannels, outputClasses)
f, w = get_csv_writer(filename)
w.writerow([accuracy_result, "did run %d" % i, average_error])
f.close()
I am using the following code at the start of my train_network function before setting up the layers and error function of my network:
在设置网络的层和误差函数之前，我在 train_network 函数的开头使用以下代码：
np.random.seed(1)
tf.set_random_seed(1)
I have also tried adding this code before the TensorFlow graph creation, but I keep getting different solutions in my results output.
我也尝试在 TensorFlow 图创建之前添加此代码，但我在结果输出中不断得到不同的解决方案。
I am using an AdamOptimizer and am initializing network weights using tf.truncated_normal. Additionally I am using np.random.permutationto shuffle the incoming images for each epoch.
我正在使用 AdamOptimizer 并使用tf.truncated_normal. 此外，我正在使用np.random.permutation为每个时代打乱传入的图像。
回答by mrry
Setting the current TensorFlow random seed affects the current default graph only. Since you are creating a new graph for your training and setting it as default (with g.as_default():), you must set the random seed within the scope of that withblock.
设置当前的 TensorFlow 随机种子只会影响当前的默认图。由于您正在为训练创建一个新图并将其设置为默认值 ( with g.as_default():)，因此您必须在该with块的范围内设置随机种子。
For example, your loop should look like the following:
例如，您的循环应如下所示：
for i in range(3):
g = tf.Graph()
with g.as_default():
tf.set_random_seed(1)
accuracy_result, average_error = network.train_network(
parameters, inputHeight, inputWidth, inputChannels, outputClasses)
Note that this will use the same random seed for each iteration of the outer forloop. If you want to use a different—but still deterministic—seed in each iteration, you can use tf.set_random_seed(i + 1).
请注意，这将为外for循环的每次迭代使用相同的随机种子。如果您想在每次迭代中使用不同但仍具有确定性的种子，您可以使用tf.set_random_seed(i + 1).
回答by ssegvic
Deterministic behaviour can be obtained either by supplying a graph-level or an operation-level seed. Both worked for me. A graph-level seed can be placed with tf.set_random_seed. An operation-level seed can be placed e.g, in a variable intializer as in:
可以通过提供图形级或操作级种子来获得确定性行为。两者都为我工作。可以使用tf.set_random_seed放置图形级种子。操作级种子可以放置在例如变量初始化器中，如下所示：
myvar = tf.Variable(tf.truncated_normal(((10,10)), stddev=0.1, seed=0))
回答by Tensorflow Support
Tensorflow 2.0 Compatible Answer: For Tensorflow version greater than 2.0, if we want to set the Global Random Seed, the Command used is tf.random.set_seed.
Tensorflow 2.0 兼容答案：对于大于 2.0 的 Tensorflow 版本，如果我们要设置全局随机种子，使用的命令是tf.random.set_seed。
If we are migrating from Tensorflow Version 1.x to 2.x, we can use the command,
tf.compat.v2.random.set_seed.
如果我们从 迁移Tensorflow Version 1.x to 2.x，我们可以使用命令，
tf.compat.v2.random.set_seed。
Note that tf.functionacts like a re-run of a program in this case.
请注意，tf.function在这种情况下，它的作用类似于程序的重新运行。
To set the Operation Level Seed (as answered above), we can use the command, tf.random.uniform([1], seed=1).
要设置操作级别种子（如上所述），我们可以使用命令tf.random.uniform([1], seed=1).
For more details, refer this Tensorflow Page.
有关更多详细信息，请参阅此Tensorflow 页面。
回答by Zhihui Shao
Please add all random seed functions before your code:
请在您的代码之前添加所有随机种子函数：
tf.reset_default_graph()
tf.set_random_seed(0)
random.seed(0)
np.random.seed(0)
I think, some models in TensorFlow are using numpy or the python random function.
我认为，TensorFlow 中的某些模型正在使用 numpy 或 python 随机函数。
回答by Luke
It seems as if none of these answers will work due to underlying implementation issues in CuDNN.
由于 CuDNN 中的潜在实现问题，这些答案似乎都不起作用。
You can get a bit more determinism by adding an extra flag
你可以通过添加一个额外的标志来获得更多的确定性
os.environ['PYTHONHASHSEED']=str(SEED)
os.environ['TF_CUDNN_DETERMINISTIC'] = '1' # new flag present in tf 2.0+
random.seed(SEED)
np.random.seed(SEED)
tf.set_random_seed(SEED)
But this still won't be entirely deterministic. To get an even more exact solution, you need use the procedure outlined in this nvidia repo.
但这仍然不是完全确定的。要获得更精确的解决方案，您需要使用此 nvidia repo 中概述的过程。
