BACKGROUND & AIMS:
:Receiver operating characteristic (ROC) analysis is well established in the evaluation of systems involving binary classification tasks. However, medical tests often require distinguishing among more than two diagnostic alternatives. The goal of this work was to develop an ROC analysis method for three-class classification tasks. Based on decision theory, we developed a method for three-class ROC analysis. In this method, the objects were classified by making the decision that provided the maximal utility relative to the other two. By making assumptions about the magnitudes of the relative utilities of incorrect decisions, we found a decision model that maximized the expected utility of the decisions when using log-likelihood ratios as decision variables. This decision model consists of a two-dimensional decision plane with log likelihood ratios as the axes and a decision structure that separates the plane into three regions. Moving the decision structure over the decision plane, which corresponds to moving the decision threshold in two-class ROC analysis, and computing the true class 1, 2, and 3 fractions defined a three-class ROC surface. We have shown that the resulting three-class ROC surface shares many features with the two-class ROC curve; i.e., using the log likelihood ratios as the decision variables results in maximal expected utility of the decisions, and the optimal operating point for a given diagnostic setting (set of relative utilities and disease prevalences) lies on the surface. The volume under the three-class surface (VUS) serves as a figure-of-merit to evaluate different data acquisition systems or image processing and reconstruction methods when the assumed utility constraints are relevant.
背景与目标:
:接收机工作特性(ROC)分析在涉及二进制分类任务的系统评估中已得到充分确立。但是,医学测试通常需要在两个以上的诊断替代方案之间进行区分。这项工作的目的是为三类分类任务开发一种ROC分析方法。基于决策理论,我们开发了一种用于三类ROC分析的方法。在此方法中,通过做出相对于其他两个对象提供最大效用的决策来对对象进行分类。通过对错误决策的相对效用的大小进行假设,我们发现了一个决策模型,当使用对数似然比作为决策变量时,该模型最大化了决策的预期效用。该决策模型由以对数似然比为轴的二维决策平面和将该平面分为三个区域的决策结构组成。在决策平面上移动决策结构,这相当于在两类ROC分析中移动决策阈值,并计算真实的1类,2类和3类分数定义了3类ROC曲面。我们已经表明,生成的三类ROC曲面与两类ROC曲线具有许多特征;即,使用对数似然比作为决策变量会导致决策的最大预期效用,并且给定诊断设置(相对效用和疾病患病率的集合)的最佳操作点位于表面。当假定的效用约束相关时,三级表面下的体积(VUS)可作为评估不同数据采集系统或图像处理和重构方法的一个品质因数。