Investigating the impact of measurement error on AUC estimation using hybrid ROC models: An application to medical and industrial domains

2.1 HE hybrid ROC curve

Let $X$ and $Y$ be the two markers considered from the health (H) and disease (D) populations, respectively. Here, $X$ is assumed to follow a Half-Normal distribution, i.e., $X\sim HN\left({\sigma }_H\right)$, and $Y$ is assumed to follow an Exponential distribution, i.e., $Y\sim Exp\left({\lambda }_D\right)$. In this setting, ${\sigma }_H$ and ${\lambda }_D$ denote the scale parameters associated with respective distributions. Based on these distributional assumptions, the intrinsic measures of the hybrid ROC curve arising from the Half-Normal and Exponential model are defined as follows

where ‘t’ denotes the threshold value, which can be obtained by using the expression in Eq. (1), i.e., $t={\sigma }_H{\Phi }^{-1}\left(1-\frac{FPR\left(t\right)}{2}\right)$. Substituting the expression ‘t’ into Eq. (2), then the expression of the HEROC curve is defined as

where, ${\Phi }^{-1}$ is the inverse cumulative standard normal distribution function and ${\sigma }_H$, ${\sigma }_D$ are the scale parameters of H and D populations, respectively. The AUC expression for the HEROC curve can be obtained by integrating the Eq. (3) over (0, 1).

$\theta ={\int }_0^1\exp \left(-\frac{{\sigma }_H}{{\sigma }_D}{\Phi }^{-1}\left(1-\frac{FPR\left(t\right)}{2}\right)\right) d$

here, ${\theta }_{HE}$ is the AUC of HEROC curve. Let $FPR\left(t\right)=x\left(t\right),$ then

$\begin{array}{c}{\theta }_{HE}={\int }_0^1\exp \left(-\frac{{\sigma }_H}{{\sigma }_D}{\Phi }^{-1}\left(1-\frac{x\left(t\right)}{2}\right)\right)dt\\ ={\int }_0^1\exp \left(-\frac{\sqrt{2{\sigma }_H}}{{\sigma }_D}er{f}^{-1}(1-x(t\left)\right)\right)dx\left(t\right)\end{array}$

Assume $t=er{f}^{-1}\left(x\left(t\right)-1\right)\Rightarrow dx\left(t\right)=-\frac{2}{\sqrt{\pi }}{e}^{-t^2}dt$

By substituting the above expressions in ${\theta }_{HE}$, the integral will become

$\begin{array}{c}{\theta }_{HE}={\int }_0^1\exp \left(-\frac{\sqrt{2}{\sigma }_H}{{\sigma }_D}.t\right).\left(-\frac{2}{\sqrt{\pi }}{e}^{-t^2}\right)dt\\ =\frac{2}{\sqrt{\pi }}{\int }_{-\infty }^0\exp \left(kt-t^2\right)dt\end{array}$

where, $k=-\frac{\sqrt{2}{\sigma }_H}{{\sigma }_D}$. On further simplification using Mathematica, one can get the AUC expression as follows, in Eq. (4)

2.2 HR hybrid ROC curve

In this setup, $X\sim HN\left({\sigma }_H\right)$ and $Y\sim RL\left({\sigma }_D\right)$, where ${\sigma }_H$, ${\sigma }_D$ are the scale parameters of the Half-Normal and Rayleigh distributions, respectively. The intrinsic measures for this hybrid ROC curve are defined as

From Eq. (5), the expression for t is $t={\sigma }_H{\Phi }^{-1}\left(1-\frac{FPR}{2}\right)$ and after simplifying Eq. (6), the expression for the HR hybrid ROC curve is obtained as

where, $\beta =\frac{{\sigma }_H}{{\sigma }_D}$. The AUC expression of this HRROC curve is obtained by integrating the Eq. (7) and is defined as as

$\begin{array}{c}{\theta }_{HR}={\int }_0^{1}ROC\left(t\right)dt\\ ={\int }_0^1\exp \left(-\frac{{\beta }^2}{2}{\left({\Phi }^{-1}\left(1-\frac{FPR}{2}\right)\right)}^2\right)dt\end{array}$

here, ${\theta }_{HR}$ is the AUC expression of HRROC curve, on further simplification, the closed form for ${\theta }_{HR}$ is as follows

Using the maximum likelihood estimates of the parameters ${\sigma }_H,{\sigma }_D$ and${\sigma }_H,$ ${\lambda }_D$, the natural estimators of ${\theta }_{HE}$ and ${\theta }_{HR}$ are given by from Eq. (4 and 8)

${\hat{\theta }}_{HE}=\left(1-erf\left(\frac{{\hat{\sigma }}_H}{\sqrt{2{\hat{\sigma }}_D}}\right)\right)\exp \left(\frac{{\hat{\sigma }}_H^2}{2{\hat{\sigma }}_D^2}\right)\&{\hat{\theta }}_{HR}=\frac{1}{\sqrt{1+{\hat{\beta }}^2}}$

With the help of Taylor series expansion, the expected values of ${\hat{\theta }}_{HE}$ and ${\hat{\theta }}_{HR}$ can be easily expressed as $E\left({\hat{\theta }}_{HE\left(or\right)HR}\right)={\theta }_{HE\left(or\right)HR}+O\left(n^{{*}^{-1}}\right)$, $n^{*}=\min \left(m,n\right);$ where, m and n denote the number of samples in H and D populations, respectively. However, if the sample observations are measured with errors, then the expected values of ${\hat{\theta }}_{HE\left(\text{or}\right)HR}$ may not be reliable and lead to biased outcomes. In order to correct this bias in ${\hat{\theta }}_{HE\left(or\right)HR}$ we derived bias-corrected approximations, which are clearly explained in the following sections.

2.3 Proposed bias- corrected approximations

3.1 Best classification scenario

The best-case classification scenario represents an ideal setting in which the underlying populations are well-separated, enabling the classification model to assign observations to their respective groups with high accuracy and reliability. In this context, the markers exhibit strong discriminatory power, and the overlap between the two distributions is minimal, resulting in optimal classification conditions. For the HEROC curve, samples observations were generated from $X\sim \text{HN}\left(2.5\right)$ and $Y\sim \text{Exp}\left(5.2\right)$, whereas for the HRROC curve samples were drawn from $X\sim \text{HN}\left(1.5\right)$ and $Y\sim \text{RL}\left(2.1\right).$ In both cases, observations were simulated at various sample sizes m = n = {25, 50, 100, 300, 500, 1000}. To stimulate the existence of MEs in the data, error-contaminated observations were generated from a normal distribution at different levels ${\sigma }_u={\sigma }_v=\left(0.6, 1.5, 2.0\right)$ to reflect realistic situations where the markers are subject to small instrument variability (mild), notable device or observer inconsistences (moderate), and substantial external noise or environment influence (severe). The errors were assumed independent between populations and are added into the original data to assess the performance of the bias-corrected estimators. The detailed simulation results for both ROC curves are presented in Tables 1 and 2.

The results across both hybrid ROC curves show that MEs lead to a notable downward bias in the estimated AUC, reflecting the loss of diagnostic accuracy. However, the proposed bias-corrected estimators effectively counteracted this distortion, yielding AUC estimates that closely approximated the true values across all sample sizes and contamination levels. For example, in HEROC curve at sample size = 100 and ME level 1.5, the estimated AUC $\left({\hat{\theta }}_{H{E}_{ME}}\right)$ is obtained as 0.63903, whereas the true AUC $\left({\theta }_{HE}\right)$ is 0.85248, demonstrating a significant loss of accuracy due to presence of MEs. To correct this bias, the proposed bias-corrected estimator was applied, yielding AUC $\left({\tilde{\theta }}_{HE}\right)$ =0.84539 close to the true AUC with a minimal MSE of 0.000050. Similar trends were also observed for all the sample sizes, where the contaminated AUC estimates were significantly biased downward and the proposed estimator improved accuracy substantially. A graphical comparison of true and contaminated ROC curves across different sample sizes for both hybrid ROC curves were presented in Figs. 1 and 2. The solid lines represent the actual ROC curves, while the dotted lines denote the contaminated ROC curves of various sample sizes. The plots consistently illustrate that the presence of MEs causes a noticeable downward shift in the ROC curves, indicating reduced classification accuracy. In addition, Figs. 3 and 4 provide a graphical summary of bias patterns across different sample sizes under varying ME levels. These plots show that the proposed bias-corrected estimators exhibit minimal bias across all sample sizes compared to the contaminated ones, thereby demonstrating robustness in approximating the true diagnostic performance. This visual trend highlights the detrimental effect of measurement contamination and underscores the necessity of employing correction strategies to recover the true diagnostic performance.

Close

Fig. 1.

The true and contaminated HEROC curves at different sample sizes for the best classification.

Export to PPT

Close

Fig. 2.

The true and contaminated HRROC curves at different sample sizes for the best classification.

Export to PPT

Close

Fig. 3.

The bias comparison of contaminated and corrected AUC at different sample sizes for the best classification of HEROC.

Export to PPT

Close

Fig. 4.

The bias comparison of contaminated and corrected AUC at different sample sizes for the best classification of HRROC.

Export to PPT

3.2 Worst classification scenario

This setting represents a challenging diagnostic environment where the distributions of the healthy and diseased populations exhibit substantial overlap. For the HEROC curve, both populations were simulated with the same parameter 2.0, while in the HRROC curve, the parameters were set as 0.9, resulting in identical distributions. As in the best-case scenario, simulations were performed across the same range of sample sizes and error levels. The results are summarized in Tables 3 and 4.

Even under severe overlap, the proposed bias-corrected estimators demonstrated strong performance. While the contaminated AUC estimates were significantly biased downward due to the effect of overlapping distributions and ME, the bias-corrected values consistently exhibited minimal deviation from the true AUC. Notably, the advantage of bias correction became more pronounced with increasing sample size, as reflected in the reduced MSE. Figs. 5 and 6 illustrate how overlapping distributions, when combined with measurement errors, further degrade diagnostic performance and underscore the difficulty of maintaining classification accuracy under such adverse conditions. In particular, the contaminated ROC curves often displayed non-regular or improper shapes, deviating from the expected monotonicity of true ROC curves. This irregularity arises due to high overlap between the H and D populations. Such improper ROC behavior further emphasizes the need for bias correction to ensure reliable diagnostic inference. Figs. 7 and 8 further illustrate the bias trends across different sample sizes, showing that while the contaminated estimators remain heavily biased, the proposed bias-corrected estimators achieve minimal bias, even in worst-case conditions.

Close

Fig. 5.

The true and contaminated HEROC curves at different sample sizes for the worst classification.

Export to PPT

Close

Fig. 6.

The true and contaminated HRROC curves at different sample sizes for the worst classification

Export to PPT

Close

Fig. 7.

The bias comparison of contaminated and corrected AUC at different sample sizes for the worst classification of HEROC.

Export to PPT

Close

Fig. 8.

The bias comparison of contaminated and corrected AUC at different sample sizes for the worst classification of HRROC.

Export to PPT

Across both hybrid ROC frameworks, the simulation findings confirm that MEs substantially distort the estimated AUC. The proposed bias-corrected estimators not only mitigate this distortion but also maintain high accuracy and efficiency across a range of sample sizes and contamination levels.

4.1 HEROC curve under MEs

Here, we considered two datasets, namely, the Gallstone Dataset (GD) and the Stroke Prediction Dataset (SPD). The GD was obtained from the University of California, Irvine (UCI) Machine Learning Repository (Esen et al., 2024), while the SPD was sourced from the Kaggle (Fedesoriano 2022).

Dataset 2: SPD

Comprises data from 5110 individuals, of which 249 had experienced a stroke, and 4861 were classified as non-stroke cases. Among the available clinical variables, the feature average ‘glucose levels’ was selected for analysis, as it serves as a key indicator in assessing stroke risk.

Since real datasets with explicit MEs are difficult to obtain, we simulated error observations at ${\sigma }_u={\sigma }_v=0.6$ to ensure their presence in the analysis and are incorporated into the respective datasets. This gives us a situation where the variables in the dataset were affected due to the presence of MEs. The distributional adequacy of the contaminated observations was assessed using the Kolmogorov-Smirnov test, and the resulting p-values are reported in Table 5. In our results, the reported p-values were consistently above 0.05, confirming that the error-contaminated observations retained distributional adequacy for the models considered.

Table 5. Goodness of fit of the HEROC curve.

Distribution type	Dataset	p-value	Fit of the distribution
Half-normal (healthy)	GD	0.43913	Yes
	SPD	0.49481
Exponential (diseased)	GD	0.47108	Yes
	SPD	0.52819

The introduction of MEs in both datasets resulted in a notable downward bias in the estimated AUC, thereby degrading classification performance. The corresponding results are summarized in Table 6. For instance, in the GD, the true AUC $\left({\theta }_{HE}\right)$ was observed to be 0.75031, which dropped significantly to 0.58344 upon contamination $\left({\hat{\theta }}_{H{E}_{ME}}\right)$. However, after applying the proposed approximation, the corrected AUC $\left({\tilde{\theta }}_{HE}=0.74213\right)$ closely approximated the true value, with substantially reduced bias and MSE. A similar pattern was observed in the Stroke dataset, further confirming the effectiveness of the proposed methodology in mitigating the impact of measurement errors. To validate the statistical assessments of the contaminated and corrected AUCs, paired bootstrap resampling with 1000 replications was performed, and the corresponding confidence intervals are also reported in Table 6. For the GD, the improvement in discriminatory ability was ${\Delta }_{HE}$ = 0.1587 (95% CI: 0.1204, 0.1946), while, for the stroke dataset, the improvement was even more pronounced with ${\Delta }_{HE}$=0.3568 (95% CI:0.3117, 0.4029). In both cases, the confidence intervals exclude zero, confirming that the improvements are statistically significant.

Table 6. Comparison of AUCs between contaminated and bias-corrected estimator of HEROC curve.

Datasets	${\theta }_{HE}$	${\hat{\theta }}_{H{E}_{ME}}$	Bias	MSE	${\tilde{\theta }}_{HE}$	Bias	MSE	${\Delta }_{HE}$ (95% CI)
GD	0.75031	0.58344	-0.16687	0.02785	0.74213	-0.00818	0.000067	0.1587 (0.1204, 0.1946)
SPD	0.67317	0.31210	-0.36107	0.13037	0.66887	-0.00430	0.000018	0.3568 (0.3117, 0.4029)

${\theta }_{HE}$ – Actual AUC; ${\hat{\theta }}_{H{E}_{ME}}$ – Estimated AUC with ME; ${\tilde{\theta }}_{HE}$ – Bias-corrected AUC; ${\Delta }_{HE}$- Difference in AUC with 95% CI

Furthermore, the visual representation of the true and contaminated ROC curves for both datasets are depicted in Fig. 9. It is clearly seen that the contaminated ROC curves deviate from the original, indicating the poor performance of the classifiers due to the measurement distortion.

Close

Fig. 9.

True and contaminated HEROC curves for the real datasets.

Export to PPT

4.2 HRROC curve under MEs

Here also, two datasets have been considered to examine the proposed methodology under the half-normal & rayleigh distribution framework. The datasets are: the Predictive maintenance dataset (PMD) (Agarwal 2022) and the weather forecast dataset (WFD) (Ahmad 2022), both obtained from Kaggle.

Dataset 2: WFD

Consists of approximately 2,500 daily weather observations. Each record includes key meteorological variables such as temperature, humidity, pressure, wind speed, and a binary label indicating rainfall occurrence. For this study, daily average temperature was selected as the continuous variable due to its known sensitivity to sensor noise and critical role in precipitation forecasting.

As in the case of the HEROC curve, MEs were incorporated into each dataset at same measurement level 0.6. To check the goodness of fit of the contaminated data, we applied the Kolmogorov-Smirnov test for each dataset, and the obtained p-values are reported in Table 7. Similar to the previous case, all the obtained p-values were greater than 0.05, indicating that the contaminated observations remained consistent with the assumed Half-Normal and Rayleigh distributions. The corresponding AUC estimates, both contaminated and bias-corrected, are summarized in Table 8.

Table 7. Goodness of fit of the HRROC curve.

Distribution type	Dataset	p-value	Fit of the distribution
Half-Normal (Healthy)	PMD	0.36720	Yes
	WFD	0.50245
Rayleigh (Diseased)	PMD	0.44352	Yes
	WFD	0.47682

Table 8. Comparison of AUCs between contaminated and bias-corrected estimator of HRROC curve.

Datasets	${\theta }_{HR}$	${\hat{\theta }}_{H{R}_{ME}}$	Bias	MSE	${\tilde{\theta }}_{HR}$	Bias	MSE	${\Delta }_{HR}$ (95% CI)
PMD	0.72310	0.41002	-0.31308	0.09802	0.71002	-0.01308	0.000171	0.3000 (0.2917, 0.3083)
WFD	0.59042	0.34012	-0.25030	0.06265	0.58121	-0.00921	0.000085	0.2411 (0.2339, 0.2483)

${\theta }_{HR}$ – Actual AUC; ${\hat{\theta }}_{H{R}_{ME}}$- Estimated AUC with ME; ${\tilde{\theta }}_{HR}$ – Bias-corrected AUC; ${\Delta }_{HR}$- Difference in AUC with 95% CI

Here also, similar pattern has been identified. Across both the datasets, the bias-corrected AUCs were notably closer to the true AUC values. From the bootstrap confidence intervals, the results indicate that the improvements in AUC were statistically significant for both datasets. Fig. 10 illustrates the HRROC curves for the two datasets, clearly showing the impact of MEs on classification performance. The contaminated ROC curves exhibit a visible deviation from the true curves, indicating a loss of discriminative ability.

Close

Fig. 10.

True and contaminated HRROC curves for real datasets.

Export to PPT

The findings across both the HE and HR ROC frameworks clearly indicate that the presence of MEs compromises the accuracy of diagnostic evaluation, resulting in a downward bias in the estimated AUC. The proposed bias-corrected estimators effectively address this issue, producing AUC estimates that are substantially closer to the true values, with notably reduced bias and MSE.