Python Keras Sequential 模型中使用的验证数据是什么？

What is validation data used for in a Keras Sequential model?

提问by danidc

My question is simple, what is the validation datapassed to model.fit in a Sequential model used for?

我的问题很简单，什么是验证数据传递给model.fit在顺序模型用于？

And, does it affect how the model is trained (normally a validation set is used, for example, to choose hyper-parameters in a model, but I think this does not happen here)?

并且，它是否会影响模型的训练方式（例如，通常使用验证集来选择模型中的超参数，但我认为这里不会发生这种情况）？

I am talking about the validation set that can be passed like this:

我说的是可以这样传递的验证集：

# Create model
model = Sequential()
# Add layers
model.add(...)

# Train model (use 10% of training set as validation set)
history = model.fit(X_train, Y_train, validation_split=0.1)

# Train model (use validation data as validation set)
history = model.fit(X_train, Y_train, validation_data=(X_test, Y_test))

I investigated a bit, and I saw that keras.models.Sequential.fitcalls keras.models.training.fit, which creates variables like val_accand val_loss(which can be accessed from Callbacks). keras.models.training.fitalso calls keras.models.training._fit_loop, which adds the validation data to the callbacks.validation_data, and also calls keras.models.training._test_loop, which will loop the validation data in batches on the self.test_functionof the model. The result of this function is used to fill the values of the logs, which are the values accessible from the callbacks.

我调查了一下，我看到了keras.models.Sequential.fitcall keras.models.training.fit，它创建了像val_accandval_loss这样的变量（可以从回调中访问）。keras.models.training.fit还调用keras.models.training._fit_loop，将验证数据添加到callbacks.validation_data，还调用keras.models.training._test_loop，这将self.test_function在模型的上批量循环验证数据。此函数的结果用于填充日志的值，这些值是可从回调访问的值。

After seeing all this, I feel that the validation set passed to model.fitis not used to validate anything during training, and its only use is to get feedback on how the trained model will perform in every epoch for a completely independent set. Therefore, it would be okey to use the same validation and test set, right?

看到这一切后，我觉得传递给的验证集model.fit在训练过程中并没有用来验证任何东西，它唯一的用途是获得关于训练模型在每个 epoch 中对于一个完全独立的集的表现如何的反馈。因此，使用相同的验证和测试集是可以的，对吗？

Could anyone confirm if the validation set in model.fit has any other goal besides being read from the callbacks?

任何人都可以确认除了从回调中读取之外，model.fit 中的验证集是否还有其他目标？

采纳答案by petezurich

If you want to build a solid model you have to follow that specific protocol of splitting your data into three sets: One for training, one for validationand one for final evaluation, which is the test set.

如果你想建立一个实体模型，你必须遵循将数据分成三组的特定协议：一组用于训练，一组用于验证，一组用于最终评估，即测试集。

The idea is that you train on your training data and tune your model with the results of metrics (accuracy, loss etc) that you get from your validation set.

这个想法是你对你的训练数据进行训练，并使用你从验证集中获得的指标（准确度、损失等）的结果来调整你的模型。

Your model doesn't "see" your validation set and isn′t in any way trained on it, but you as the architect and master of the hyperparameters tune the model according to this data. Therefore it indirectly influences your model because it directly influences your design decisions.You nudge your model to work well with the validation data and that can possibly bring in a tilt.

您的模型不会“看到”您的验证集，也不会以任何方式对其进行训练，但是您作为超参数的架构师和大师根据这些数据调整模型。因此，它会间接影响您的模型，因为它会直接影响您的设计决策。您推动模型与验证数据配合良好，这可能会导致倾斜。

Exactly that is the reason you only evaluate your models final score on data that neither your model nor you yourself has used – and that is the third chunk of data, your test set.

正是这就是为什么你只评估你的模型最终分数的原因，你的模型和你自己都没有使用过——这是第三块数据，你的测试集。

Only this procedure makes sure you get an unaffected view of your models quality and ability to generalize what is has learned on totally unseen data.

只有这个过程才能确保您不受影响地了解模型质量，并能够概括在完全看不见的数据上学到的东西。

回答by blackHoleDetector

This YouTube video explains what a validation set is, why it's helpful, and how to implement a validation set in Keras: Create a validation set in Keras

这个 YouTube 视频解释了验证集是什么，它为什么有用，以及如何在 Keras 中实现验证集：在 Keras 中 创建验证集

With a validation set, you're essentially taking a fraction of your samples out of your training set, or creating an entirely new set all together, and holding out the samples in this set from training.

使用验证集，您实际上是从训练集中取出一小部分样本，或者一起创建一个全新的集，并从训练中保留该集中的样本。

During each epoch, the model will be trained on samples in the training set but will NOT be trained on samples in the validation set. Instead, the model will only be validatingon each sample in the validation set.

在每个 epoch 期间，模型将在训练集中的样本上进行训练，但不会在验证集中的样本上进行训练。相反，模型只会在验证集中的每个样本上进行验证。

The purpose of doing this is for you to be able to judge how well your model can generalize. Meaning, how well is your model able to predict on data that it's not seen while being trained.

这样做的目的是让您能够判断您的模型的泛化能力。意思是，您的模型能够对在训练时未看到的数据进行预测的能力如何。

Having a validation set also provides great insight into whether your model is overfitting or not. This can be interpreted by comparing the accand lossfrom your training samples to the val_accand val_lossfrom your validation samples. For example, if your accis high, but your val_accis lagging way behind, this is good indication that your model is overfitting.

拥有验证集还可以深入了解您的模型是否过度拟合。这可以通过比较来解释acc，并loss从你的训练样本的val_acc和val_loss从您的验证样品。例如，如果你的分数acc很高，但你val_acc落后很多，这很好地表明你的模型过度拟合。

回答by hafiz031

I think an overall discussion on train-set, validation-set and test-set will help:

我认为对训练集、验证集和测试集的全面讨论将有所帮助：

• Train-Set:The data-set on which the model is being trained on. This is the onlydata-set on which the weights are updated during back-propagation.
• Validation-Set (Development Set):The data-set on which we want our model to perform well. During the training process we tune hyper-parameters such that the model performs well on dev-set (but don't use dev-set for training, it is only used to see the performance such that we can decide on how to change the hyper-parameters and after changing hyper-parameters we continue our training on train-set). Dev-set is only used for tuning hyper-parameters to make the model eligible for working well on unknown data (here dev-set is considered as a representative of unknown data-set as it is not directly used for training and additionally saying the hyper-parameters are like tuning knobs to change the way of training) and no back-propagation occurs on dev-set and hence no direct learning from it.
• Test-Set:We just use it for unbiased estimation. Like dev-set, no training occurs on test-set. The only difference from validation-set (dev- set) is that we don'teven tune the hyper-parameters here and just see how well our model have learnt to generalize. Although, like test-set, dev-set is not directly used for training, but as we repeatedly tune hyper-parameters targeting the dev-set, our model indirectly learns the patterns from dev-set and the dev-set becomes no longer unknown to the model. Hence we need another fresh copy of dev-setwhich is not even used for hyper parameter tuning, and we call this fresh copy of dev-set as test set. As according to the definition of test-set it should be "unknown" to the model. But if we cannot manage a fresh and unseen test set like this, then sometimes we say the dev-set as the test-set.

• 训练集：正在训练模型的数据集。这是在反向传播期间更新权重的唯一数据集。
• 验证集（开发集）：我们希望我们的模型在其上表现良好的数据集。在训练过程中，我们调整超参数，使模型在 dev-set 上表现良好（但不要使用 dev-set 进行训练，它仅用于查看性能，以便我们决定如何更改超参数） -参数，在更改超参数后，我们继续对训练集进行训练）。dev-set 仅用于调整超参数，使模型有资格在未知数据上很好地工作（这里 dev-set 被认为是未知数据集的代表，因为它不直接用于训练，另外说超-参数就像调整旋钮以改变训练方式）并且在开发集上没有发生反向传播，因此没有直接从中学习。
• 测试集：我们只是将它用于无偏估计。与开发集一样，测试集上不进行训练。与验证集（开发集）的唯一区别是我们甚至没有在这里调整超参数，而只是看看我们的模型已经学会了如何泛化。虽然像测试集一样，开发集不直接用于训练，但是当我们反复调整针对开发集的超参数时，我们的模型间接地从开发集学习模式，开发集不再是未知的到模型。因此我们需要另一个新的 dev-set 副本它甚至不用于超参数调整，我们称这个开发集的新副本为测试集。根据测试集的定义，模型应该是“未知的”。但是如果我们不能像这样管理一个新的和看不见的测试集，那么有时我们会说开发集作为测试集。

Summarizing:

总结：

• Train-Set:Used for training.
• Validation-Set / Dev-Set:Used for tuning hyper-parameters.
• Test-Set:Used for unbiased estimation.

• 训练集：用于训练。
• 验证集/开发集：用于调整超参数。
• 测试集：用于无偏估计。

Again some practical issues here:

这里又是一些实际问题：

• For training you may collect data from anywhere. It is okay if your all collected data are not from the same domain where the model will be used. For example if the real domain is the photos taken with smartphone camera, it is not necessary to make data-set with smartphone photos only. You may include data from internet, high-end or low-end cameras or from anywhere.
• For dev-set and test-set it is necessaryto reflect the real domain data where the model will be practically used. Also it should contain all possible cases for better estimation.
• Dev-set and test-set need not to be that large. Just ensure that it is almost covering all cases or situations that may occur in real data. After ensuring it try to give as much data as you can to build train-set.

• 对于训练，您可以从任何地方收集数据。如果您收集的所有数据都不是来自将使用模型的同一个域，那也没关系。例如，如果真正的领域是智能手机相机拍摄的照片，则没有必要仅使用智能手机照片制作数据集。您可以包含来自互联网、高端或低端相机或来自任何地方的数据。
• 对于开发集和测试集，有必要反映模型实际使用的真实领域数据。它还应该包含所有可能的情况，以便更好地估计。
• 开发集和测试集不需要那么大。只要确保它几乎涵盖了真实数据中可能出现的所有情况或情况。在确保它尝试提供尽可能多的数据来构建训练集。