本文共 4134 字,大约阅读时间需要 13 分钟。
confusion matrix(混淆矩阵)在分类问题中能比单纯的准确率更全面地反应模型的好坏,本文主要目的是在caffe中用python添加confusion matrix层。
完整代码如下:
import caffeimport jsonimport numpy as npimport sysimport sklearn.metricsclass PythonConfMat(caffe.Layer): """ Compute the Accuracy with a Python Layer """ def setup(self, bottom, top): # check input pair if len(bottom) != 2: raise Exception("Need two inputs.") self.num_labels = bottom[0].channels params = json.loads(self.param_str) self.test_iter = params['test_iter'] self.conf_matrix = np.zeros((self.num_labels, self.num_labels)) self.current_iter = 0 def reshape(self, bottom, top): # bottom[0] are the net's outputs # bottom[1] are the ground truth labels # Net outputs and labels must have the same number of elements if bottom[0].num != bottom[1].num: raise Exception("Inputs must have the same number of elements.") # accuracy output is scalar top[0].reshape(1) def forward(self, bottom, top): self.current_iter += 1 # predicted outputs pred = np.argmax(bottom[0].data, axis=1) accuracy = np.sum(pred == bottom[1].data).astype(np.float32) / bottom[0].num top[0].data[...] = accuracy # compute confusion matrix self.conf_matrix += sklearn.metrics.confusion_matrix(bottom[1].data, pred, labels=range(self.num_labels)) if self.current_iter == self.test_iter: self.current_iter = 0 sys.stdout.write('\nCAUTION!! test_iter = %i. Make sure this is the correct value' % self.test_iter) sys.stdout.write('\n"param_str: \'{"test_iter":%i}\'" has been set in the definition of the PythonLayer' % self.test_iter) sys.stdout.write('\n\nConfusion Matrix') sys.stdout.write('\t'*(self.num_labels-2)+'| Accuracy') sys.stdout.write('\n'+'-'*8*(self.num_labels+1)) sys.stdout.write('\n') for i in range(len(self.conf_matrix)): for j in range(len(self.conf_matrix[i])): sys.stdout.write(str(self.conf_matrix[i][j].astype(np.int))+'\t') sys.stdout.write('| %3.2f %%' % (self.conf_matrix[i][i]*100 / self.conf_matrix[i].sum())) sys.stdout.write('\n') sys.stdout.write('Number of test samples: %i \n\n' % self.conf_matrix.sum()) # reset conf_matrix for next test phase self.conf_matrix = np.zeros((self.num_labels, self.num_labels)) def backward(self, top, propagate_down, bottom): pass
注意:此python脚本应放置在$caffe_root/python目录下。
若caffe还没有编译,可以打开makefile.config中 WITH_PYTHON_LAYER=1
的开关,然后make编译,再然后make pycaffe
编译。
make clean
,然后用WITH_PYTHON_LAYER=1 make all -j&& make pycaffe
编译caffe源码。 在测试网络的prototxt中添加把混淆矩阵层添加进去,列子如下
layer { type: 'Python' name: 'py_accuracy' top: 'py_accuracy' bottom: 'fc101' bottom: 'label' python_param { # the module name -- usually the filename -- that needs to be in $PYTHONPATH module: 'python_confmat' # the layer name -- the class name in the module layer: 'PythonConfMat' # this is the number of test iterations, it must be the same as defined in the solver. param_str: '{"test_iter":100}' } include { phase: TEST }}
注意:test_iter乘以batch size应cover所有测试集样本。
/build/tools/caffe test
工具计算测试集的confusion matrix这个需要提前把测试机生成lmdb文件。生成后用如下姿势使用
CAFFE_ROOT=/home/***/caffe MODEL_ROOT=/home/***/work/resnet_18_age_7 $CAFFE_ROOT/build/tools/caffe test --model=$MODEL_ROOT/ResNet-18-test.prototxt \ --weights ./model_lrstep6/resnet18_age_finetue_iter_43883.caffemodel \ --iterations 1392 --gpu=1
caffe test中不需要solver.prototxt,所以在命令中需要指定--iterations
. 这里的iterations需要与PythonConfMat层中的test_iter相同。
注意:调用test命令可能出现import error:no module named python_confmat
解决方案:终端输入export PYTHONPATH=/home/xxx/caffe/python即可。最后的效果如下
Reference: