Python 如何将 Keras 损失输出记录到文件中
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/38445982/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
How to log Keras loss output to a file
提问by BigBoy1337
When you run a Keras neural network model you might see something like this in the console:
当您运行 Keras 神经网络模型时，您可能会在控制台中看到如下内容：
Epoch 1/3
6/1000 [..............................] - ETA: 7994s - loss: 5111.7661
As time goes on the loss hopefully improves. I want to log these losses to a file over time so that I can learn from them. I have tried:
随着时间的推移，损失有望改善。我想随着时间的推移将这些损失记录到一个文件中，以便我可以从中学习。我试过了：
logging.basicConfig(filename='example.log', filemode='w', level=logging.DEBUG)
but this doesn't work. I am not sure what level of logging I need in this situation.
但这不起作用。我不确定在这种情况下我需要什么级别的日志记录。
I have also tried using a callback like in:
我也尝试过使用回调，如：
def generate_train_batch():
while 1:
for i in xrange(0,dset_X.shape[0],3):
yield dset_X[i:i+3,:,:,:],dset_y[i:i+3,:,:]
class LossHistory(keras.callbacks.Callback):
def on_train_begin(self, logs={}):
self.losses = []
def on_batch_end(self, batch, logs={}):
self.losses.append(logs.get('loss'))
logloss=LossHistory()
colorize.fit_generator(generate_train_batch(),samples_per_epoch=1000,nb_epoch=3,callbacks=['logloss'])
but obviously this isn't writing to a file. Whatever the method, through a callback or the logging module or anything else, I would love to hear your solutions for logging loss of a keras neural network to a file. Thanks!
但显然这不是写入文件。无论采用何种方法，通过回调或日志记录模块或其他任何方式，我都希望听到您将 keras 神经网络的丢失记录到文件中的解决方案。谢谢！
采纳答案by Marcin Mo?ejko
There is a simple solution to your problem. Every time any of the fitmethods are used - as a result the special callback called History Callbackis returned. It has a field historywhich is a dictionary of all metrics registered after every epoch. So to get list of loss function values after every epoch you can easly do:
您的问题有一个简单的解决方案。每次使用任何fit方法时，都会返回名为History Callback的特殊回调。它有一个字段history，它是每个 epoch 之后注册的所有指标的字典。因此，要在每个 epoch 后获取损失函数值列表，您可以轻松执行以下操作：
history_callback = model.fit(params...)
loss_history = history_callback.history["loss"]
It's easy to save such list to a file (e.g. by converting it to numpyarray and using savetxtmethod).
将此类列表保存到文件中很容易（例如，通过将其转换为numpy数组并使用savetxt方法）。
UPDATE:
更新：
Try:
尝试：
import numpy
numpy_loss_history = numpy.array(loss_history)
numpy.savetxt("loss_history.txt", numpy_loss_history, delimiter=",")
UPDATE 2:
更新 2：
The solution to the problem of recording a loss after every batch is written in Keras Callbacks Documentationin a Create a Callbackparagraph.
Keras Callbacks 文档中的Create a Callback段落中写了每批后记录损失问题的解决方案。
回答by Alex Glinsky
You can use CSVLoggercallback.
您可以使用CSVLogger回调。
as example:
例如：
from keras.callbacks import CSVLogger
csv_logger = CSVLogger('log.csv', append=True, separator=';')
model.fit(X_train, Y_train, callbacks=[csv_logger])
Look at: Keras Callbacks
查看：Keras 回调
回答by Benjamin Striner
Old question, but here goes. Keras history output perfectly matches pandas DataSet input.
老问题，但在这里。Keras 历史输出与 Pandas DataSet 输入完美匹配。
If you want the entire history to csv in one line:
pandas.DataFrame(model.fit(...).history).to_csv("history.csv")
如果您希望在一行中将整个历史记录转换为 csv：
pandas.DataFrame(model.fit(...).history).to_csv("history.csv")
Cheers
干杯
回答by Nagabhushan Baddi
You can redirect the sys.stdout object to a file before the model.fit method and reassign it to the standard console after model.fit method as follows:
您可以在 model.fit 方法之前将 sys.stdout 对象重定向到一个文件，并在 model.fit 方法之后将其重新分配给标准控制台，如下所示：
import sys
oldStdout = sys.stdout
file = open('logFile', 'w')
sys.stdout = file
model.fit(Xtrain, Ytrain)
sys.stdout = oldStdout
回答by Sayantan Das
Best is to create a LambdaCallback:
最好是创建一个LambdaCallback：
from keras.callbacks import LambdaCallback
txt_log = open('loss_log.txt', mode='wt', buffering=1)
save_op_callback = LambdaCallback(
on_epoch_end = lambda epoch, logs: txt_log.write(
{'epoch': epoch, 'loss': logs['loss']} + '\n'),
on_train_end = lambda logs: txt_log.close()
)
Now,Just add it like this in the model.fit function:
现在，只需在 model.fit 函数中像这样添加它：
model.fit(...,callbacks = [save_op_callback])
回答by Rishabh Jain
So In TensorFlow 2.0, it is quite easy to get Loss and Accuracy of each epoch because it returns a History object. Its History.history attribute is a record of training loss values and metrics values at successive epochs, as well as validation loss values and validation metrics values
所以在 TensorFlow 2.0 中，很容易得到每个 epoch 的 Loss 和 Accuracy，因为它返回一个 History 对象。它的 History.history 属性记录了连续 epoch 的训练损失值和度量值，以及验证损失值和验证度量值
If you have validation Data
如果您有验证数据
History = model.fit(trainX,trainY,validation_data = (testX,testY),batch_size= 100, epochs = epochs,verbose = 1)
train_loss = History.history['loss']
val_loss = History.history['val_loss']
acc = History.history['accuracy']
val_acc = History.history['val_accuracy']
If you don't have validation Data
如果您没有验证数据
History = model.fit(trainX,trainY,batch_size= 100, epochs = epochs,verbose = 1)
train_loss = History.history['loss']
acc = History.history['accuracy']
Then to save list data into text file use the below code
然后将列表数据保存到文本文件中，使用以下代码
import numpy as np
train_loss = np.array(loss_history)
np.savetxt("train_loss.txt", train_loss, delimiter=",")
