Python 在 scikit learn 中使用混淆矩阵作为交叉验证中的评分指标
声明：本页面是StackOverFlow热门问题的中英对照翻译，遵循CC BY-SA 4.0协议，如果您需要使用它，必须同样遵循CC BY-SA许可，注明原文地址和作者信息，同时你必须将它归于原作者（不是我）：StackOverFlow
原文地址: http://stackoverflow.com/questions/40057049/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me):
StackOverFlow
using confusion matrix as scoring metric in cross validation in scikit learn
提问by user128751
I am creating a pipeline in scikit learn,
我正在 scikit learn 中创建一个管道，
pipeline = Pipeline([
('bow', CountVectorizer()),
('classifier', BernoulliNB()),
])
and computing the accuracy using cross validation
并使用交叉验证计算准确性
scores = cross_val_score(pipeline, # steps to convert raw messages into models
train_set, # training data
label_train, # training labels
cv=5, # split data randomly into 10 parts: 9 for training, 1 for scoring
scoring='accuracy', # which scoring metric?
n_jobs=-1, # -1 = use all cores = faster
)
How can I report confusion matrix instead of 'accuracy'?
如何报告混淆矩阵而不是“准确性”？
回答by Xema
You could use cross_val_predict(See the scikit-learn docs) instead of cross_val_score.
您可以使用cross_val_predict（请参阅 scikit-learn 文档）而不是cross_val_score.
instead of doing :
而不是做：
from sklearn.model_selection import cross_val_score
scores = cross_val_score(clf, x, y, cv=10)
you can do :
你可以做 ：
from sklearn.model_selection import cross_val_predict
from sklearn.metrics import confusion_matrix
y_pred = cross_val_predict(clf, x, y, cv=10)
conf_mat = confusion_matrix(y, y_pred)
回答by lejlot
Short answer is "you cannot".
简短的回答是“你不能”。
You need to understand difference between cross_val_scoreand cross validation as model selection method. cross_val_scoreas name suggests, works only on scores. Confusion matrix is not a score, it is a kind of summary of what happened during evaluation. A major distinction is that a score is supposed to return an orderable object, in particular in scikit-learn - a float. So, based on score you can tell whether method b is better from a by simply comparing if b has bigger score. You cannot do this with confusion matrix which, again as name suggests, is a matrix.
您需要了解cross_val_score作为模型选择方法的交叉验证和交叉验证之间的区别。cross_val_score顾名思义，仅适用于score。混淆矩阵不是一个分数，它是对评估过程中发生的事情的一种总结。一个主要的区别是分数应该返回一个可排序的对象，特别是在 scikit-learn-a float 中。因此，根据分数，您可以通过简单地比较 b 是否具有更大的分数来判断方法 b 是否比 a 更好。你不能用混淆矩阵来做到这一点，顾名思义，它是一个矩阵。
If you want to obtain confusion matrices for multiple evaluation runs (such as cross validation) you have to do this by hand, which is not that bad in scikit-learn - it is actually a few lines of code.
如果您想获得多次评估运行（例如交叉验证）的混淆矩阵，您必须手动完成，这在 scikit-learn 中并没有那么糟糕——它实际上是几行代码。
kf = cross_validation.KFold(len(y), n_folds=5)
for train_index, test_index in kf:
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]
model.fit(X_train, y_train)
print confusion_matrix(y_test, model.predict(X_test))
回答by BottleNick
What you can do though is to define a scorer that uses certain values fromthe confusion matrix. See here [link]. Just citing the code:
什么可以做，虽然是定义使用特定值的得分手，从混淆矩阵。请参阅此处 [链接]。仅引用代码：
def tp(y_true, y_pred): return confusion_matrix(y_true, y_pred)[0, 0]
def tn(y_true, y_pred): return confusion_matrix(y_true, y_pred)[1, 1]
def fp(y_true, y_pred): return confusion_matrix(y_true, y_pred)[1, 0]
def fn(y_true, y_pred): return confusion_matrix(y_true, y_pred)[0, 1]
scoring = {'tp' : make_scorer(tp), 'tn' : make_scorer(tn),
'fp' : make_scorer(fp), 'fn' : make_scorer(fn)}
cv_results = cross_validate(svm.fit(X, y), X, y, scoring=scoring)
This will perform the cross validation for each of these four scorers and return the scoring dictionary cv_results, e.g., with keys test_tp, test_tn, etc. containing the confusion matrices' values from each cross-validation split.
这将执行交叉验证用于这四个得分的并返回得分词典cv_results，例如，连键test_tp，test_tn等等。从每个交叉验证分裂包含混淆矩阵值。
From this you could reconstruct an average confusion matrix, but the cross_val_predictof Xemaseems more elegant for this.
从此，你可以重建一个平均混淆矩阵，但cross_val_predict的Xema似乎这更优雅。
Note that this will actually not work with cross_val_score; you'll need cross_validate(introduced in scikit-learn v0.19).
请注意，这实际上不适用于cross_val_score; 你需要cross_validate（在 scikit-learn v0.19 中引入）。
Side note: you could use oneof these scorers (i.e. one element of the matrix) for hyper-parameter optimization via grid search.
旁注：你可以使用一个这些得分手为通过网格搜索超参数优化（矩阵即一个元件）。
*EDIT: true negatives are returned at [1, 1], not [0, 0]
*编辑：真正的负数在 [1, 1] 处返回，而不是 [0, 0]
回答by Harvey
I think what you really want is average of confusion matrices obtained from each cross-validation run. @lejlot already nicely explained why, I'll just upgrade his answer with calculation of mean of confusion matrices:
我认为您真正想要的是从每次交叉验证运行中获得的混淆矩阵的平均值。@lejlot 已经很好地解释了原因，我将通过计算混淆矩阵的均值来升级他的答案：
Calculate confusion matrix in each run of cross validation. You can use something like this:
在每次交叉验证中计算混淆矩阵。你可以使用这样的东西：
conf_matrix_list_of_arrays = []
kf = cross_validation.KFold(len(y), n_folds=5)
for train_index, test_index in kf:
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]
model.fit(X_train, y_train)
conf_matrix = confusion_matrix(y_test, model.predict(X_test))
conf_matrix_list_of_arrays .append(conf_matrix)
On the end you can calculate your mean of list of numpy arrays (confusion matrices) with:
最后，您可以使用以下方法计算 numpy 数组（混淆矩阵）列表的平均值：
mean_of_conf_matrix_arrays = np.mean(conf_matrix_list_of_arrays, axis=0)
