Python 在 TensorFlow 中计算交叉熵
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/42521400/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
Calculating Cross Entropy in TensorFlow
提问by David Kaftan
I am having a hard time with calculating cross entropy in tensorflow. In particular, I am using the function:
我在计算张量流中的交叉熵时遇到了困难。特别是，我正在使用该功能：
tf.nn.softmax_cross_entropy_with_logits()
Using what is seemingly simple code, I can only get it to return a zero
使用看似简单的代码，我只能让它返回零
import tensorflow as tf
import numpy as np
sess = tf.InteractiveSession()
a = tf.placeholder(tf.float32, shape =[None, 1])
b = tf.placeholder(tf.float32, shape = [None, 1])
sess.run(tf.global_variables_initializer())
c = tf.nn.softmax_cross_entropy_with_logits(
logits=b, labels=a
).eval(feed_dict={b:np.array([[0.45]]), a:np.array([[0.2]])})
print c
returns
返回
0
My understanding of cross entropy is as follows:
我对交叉熵的理解如下：
H(p,q) = p(x)*log(q(x))
Where p(x) is the true probability of event x and q(x) is the predicted probability of event x.
其中 p(x) 是事件 x 的真实概率，q(x) 是事件 x 的预测概率。
There if input any two numbers for p(x) and q(x) are used such that
如果使用 p(x) 和 q(x) 的任意两个数字输入，则
0<p(x)<1 AND 0<q(x)<1
there should be a nonzero cross entropy. I am expecting that I am using tensorflow incorrectly. Thanks in advance for any help.
应该有一个非零交叉熵。我期待我错误地使用 tensorflow。在此先感谢您的帮助。
回答by Franck Dernoncourt
In addition to Don's answer (+1), this answer written by mrrymay interest you, as it gives the formula to calculate the cross entropy in TensorFlow:
除了 Don 的答案 (+1) 之外，mrry 写的这个答案可能会让您感兴趣，因为它给出了计算 TensorFlow 中交叉熵的公式：
An alternative way to write:
xent = tf.nn.softmax_cross_entropy_with_logits(logits, labels)...would be:
softmax = tf.nn.softmax(logits) xent = -tf.reduce_sum(labels * tf.log(softmax), 1)However, this alternative would be (i) less numerically stable (since the softmax may compute much larger values) and (ii) less efficient (since some redundant computation would happen in the backprop). For real uses, we recommend that you use
tf.nn.softmax_cross_entropy_with_logits().
另一种写法：
xent = tf.nn.softmax_cross_entropy_with_logits(logits, labels)...将是：
softmax = tf.nn.softmax(logits) xent = -tf.reduce_sum(labels * tf.log(softmax), 1)然而，这种替代方案将 (i) 数值稳定性较差（因为 softmax 可能计算出更大的值）和 (ii) 效率较低（因为在反向传播中会发生一些冗余计算）。对于实际用途，我们建议您使用
tf.nn.softmax_cross_entropy_with_logits().
回答by Don Reba
Like they say, you can't spell "softmax_cross_entropy_with_logits" without "softmax". Softmax of [0.45]is [1], and log(1)is 0.
就像他们说的，如果没有“softmax”，你就不能拼写“softmax_cross_entropy_with_logits”。的 Softmax[0.45]是[1]，log(1)是0。
Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both.
NOTE:While the classes are mutually exclusive, their probabilities need not be. All that is required is that each row of
labelsis a valid probability distribution. If they are not, the computation of the gradient will be incorrect.If using exclusive
labels(wherein one and only one class is true at a time), seesparse_softmax_cross_entropy_with_logits.WARNING:This op expects unscaled logits, since it performs a
softmaxonlogitsinternally for efficiency. Do not call this op with the output ofsoftmax, as it will produce incorrect results.
logitsandlabelsmust have the same shape[batch_size, num_classes]and the same dtype (eitherfloat16,float32, orfloat64).
测量离散分类任务中的概率误差，其中类是互斥的（每个条目都在一个类中）。例如，每张 CIFAR-10 图像都标有一个且只有一个标签：图像可以是狗或卡车，但不能同时是两者。
注意：虽然这些类是互斥的，但它们的概率不必如此。所需要的只是每一行
labels都是一个有效的概率分布。如果不是，梯度的计算将是不正确的。如果使用exclusive
labels（其中一次只有一个类为真），请参阅sparse_softmax_cross_entropy_with_logits。警告：此操作需要未缩放的 logits，因为它
softmax在logits内部执行on以提高效率。不要用 的输出调用这个操作softmax，因为它会产生不正确的结果。
logits并且labels必须具有相同的形状[batch_size, num_classes]和相同的D型细胞（无论是float16，float32，或float64）。
