Study on the prediction of congenital cardiac abnormalities using various Machine learning models

2.1 Computational resources

The algorithm is built in a Jupyter Notebook using Anaconda and trained on a PC with Python v3.11 in a TPU-v3:8 environment with 16 GB of RAM. It utilizes TensorFlow and Keras, along with an 8-core Tensor Processing Unit (TPU), to speed up machine learning tasks.

2.2 Study population selection and data acquisition process

In this study, a dataset from an open-source database is utilized. It includes ultrasound evaluations of female patients at 12–14 weeks of gestational age (Stoean et al., 2022). This dataset has been referenced in other studies as well (Hutchinson et al., 2017; Ungureanu et al., 2023). The dataset includes 6,720 images taken from original video files showing four main views of the heart.: the atrioventricular flows in the four-chamber view, the aorta in the left ventricular outflow tract plane, the intersection of the great vessels in the right ventricular outflow tract plane (shaped like an “X”), and the arterial arches in the three-vessel plane (shaped like a “V”). An additional “other” class was also included, containing random frames that do not fit the specified categories. To diagnose heart defects, the study employed techniques like rhythmic tapping, palpation, auscultation, and physical examination, which are important for the early evaluation of congenital heart defects (CHDs) and help direct further imaging (Fig. 1).

Close

Fig. 1

Diagnosis of heart abnormalities.

Export to PPT

Fig. 2 shows a holistic block diagram of the proposed CHD detection, classification, and therapy paradigm.

Close

Fig. 2

Flowchart showing the proposed design of the model.

Export to PPT

2.3 Preprocessing

Pre-processing was done by 3 techniques, namely, K-Nearest Neighbors (KNN) estimation, “Min_Max” Normalization and One Hot encoding. KNN estimates the similarity of two data points and can identify similar cases within the obtained dataset. Then the similar data point is replaced by a missing value. In simpler language, it finds the closest point of the reference points, marked as “K”, which is based on the nearby reference points.

Min-max normalization is a popular data preprocessing technique employed to scale numerical features within a fixed range, typically between 0 and 1. The process involves two main steps. Firstly, the minimum and maximum values of the feature of interest are determined within the dataset. Subsequently, each data point in the feature is scaled using the following formula: $$X_{\mathit{normalized}}=\frac{X-X_{\mathit{min}}}{{X_{\mathit{max}}-X}_{\mathit{min}}}$$ X = original value of the feature

X_{min =} the minimum value of the feature in the dataset.

X_max = the maximum value of the feature in the dataset, and.

X_normalized = the normalized value of the feature.

This normalization method proves beneficial when dealing with features that exhibit varying scales, enabling them to be compared on a level playing field without compromising the original data distribution. It finds widespread application in machine learning algorithms, particularly those reliant on gradient descent techniques, where the scale of features can significantly influence model performance. The last technique that is used during the pre-processing stage is “One hot encoding”, in which data analysis is conducted to convert categorical variables into a binary representation, involving each category within a variable as a binary vector, where only one element is marked as “1” while all others are “0”. This technique is particularly useful when dealing with categorical data in machine learning algorithms that require numerical input. Transforming categorical variables into this binary format, allows algorithms to interpret and process the data more effectively. For instance, if we have a variable like “color” with categories red, green, and blue, one hot encoding would represent each color as a distinct binary vector, facilitating analysis and modeling tasks.

2.4 Proposed model

The proposed machine learning approach to the detection of CHD involves using a supervised learning algorithm. The structure of this algorithm is outlined in Algorithm 1. It is trained using historical data of newborns with CHD, along with factors such as gestational age, birth weight, and maternal health history.

Close

Fig. 3

Proposed flow diagram showing the detection stage.

Export to PPT

2.5 Comparative analysis

A comparison of CHD prediction models in a certain population includes accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. The Heart DL Model (CDLM) is compared to the Novel Healthcare Framework (NHF), Decision Support System for Early Prediction (DSSEP), Machine Learning-Based Discharge Prediction (MLBDP), and Predictive Analysis of Congenital Heart Defects (PACHD) (Nurmaini et al., 2022).

Simulation is done with Matlab r2022a. The ratio of true positives (TP) to total diseased patients, presented as a percentage, measures a test's sensitivity (S_e) in CHD prediction. A more sensitive test or treatment is more accurate. $$S_e=\frac{\mathit{TP}}{\mathit{TP}+FN}$$ Specificity measures the ability to correctly identify true negative (TN) cases. It was calculated using $$\mathit{Sp}=\frac{\mathit{TN}}{\mathit{TN}+FP}$$ The Positive Predictive Value (PPV) reflects the accuracy of a test in identifying individuals with CHD and was computed as $$\mathit{PPV}=\frac{\mathit{TP}}{\mathit{TP}+FP}$$ NPV assesses the proportion of true negatives among negative predictions and was calculated using $$\mathit{NPV}=\frac{\mathit{TN}}{\mathit{TN}+FN}$$

3.1 Sensitivity Measurements

Table 1 presents the evaluation of sensitivity percentages for each model in predicting heart disease. The results of the sensitivity evaluation indicate that CDLM significantly outperformed the other models, achieving a sensitivity range of 87.05 % to 91.55 %.

In contrast, NHF showed a sensitivity of 59.81 % to 70.29 %, while DSSEP maintained a more stable sensitivity around 79.81 % to 81.02 %. MLBDP and PACHD exhibited lower sensitivity, with MLBDP ranging from 66.10 % to 73.25 % and PACHD from 56.95 % to 64.95 %. The average sensitivity for all the models is plotted in Fig. 4.

Close

Fig. 4

Average sensitivity of models.

Export to PPT

3.2 Specificity evaluation

Table 2 summarizes the specificity percentages for the models, which measure the ability to correctly identify negative cases. Most models showed a decline in specificity as the number of images increased, indicating challenges in accurately identifying negative cases with larger datasets. CDLM, however, consistently maintained higher specificity across different image counts, demonstrating robust effectiveness in recognizing negative situations.

3.3 Positive Predictive Value (PPV)

Table 3 shows the PPV for the models, which indicates the proportion of true positives among all positive predictions. CDLM demonstrated consistently high PPV values, ranging from 85.32 % to 89.25 %. NHF’s PPV declined from 70.58 % to 65.51 %, indicating challenges in maintaining predictive accuracy. DSSEP started strong at 85.40 % but dropped to 75.05 %. MLBDP and PACHD exhibited lower PPV, with MLBDP ranging from 66.30 % to 72.48 % and PACHD from 52.10 % to 60.26 %.

Negative prediction values(NPV%) for proposed models are summarized in Table 4. CDLM exhibited the highest NPV values among the proposed models. That ranges from 82.66 % to 96.66 %. NPV for NHF decreased from 66.95 % to 61.25 %. DSSEP maintained a reasonable NPV from 79.02 % to 88.12 %, while both PACHD and MLBDP displayed lower NPV values.

Table 5 presents the findings of miss rates (%) across various healthcare frameworks. Miss rates represent the percentage of positive cases that are incorrectly classified as negative, indicating the model's failure to identify actual cases of congenital heart disease (CHD). In this evaluation, the proposed cardiac deep learning model (CDLM) achieved the lowest miss rates, ranging from 10.15 % to 10.92 %. NHF had significantly higher miss rates, ranging from 35.63 % to 36.85 %. DSSEP’s miss rates varied between 19.21 % and 20.48 %, while PACHD had the highest miss rates, between 39.47 % and 41.05 %.

Fig. 5 compares five healthcare frameworks' performance characteristics. Recall indicates the percentage of real positive cases the model properly identifies out of all positive cases. NHF detects 63.79 of positive cases with sensitivity. Specificity quantifies how many true negative cases the model accurately identifies out of all negative cases. For instance, DSSEP has 80.25 specificity, identifying 80.25 of negative cases. PPV, or precision, estimates the proportion of real positive instances among all model-identified positive cases.

Close

Fig. 5

Comparison of the output of models.

Export to PPT

MLBDP has a PPV of 59.87, meaning 59.87 of the positive results are real positives. NPV is the percentage of model-identified negative scenarios that are true. PACHD has an NPV of 89.25, meaning 89.25 of its negative situations are true negatives. The false negative rate measures the percentage of results that are false negatives. CDLM misses 89.25 of positive cases while misclassifying them as negative.

4.1 Performance variability among models

The results show that sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) varied significantly across models. CDLM consistently outperformed the other models in sensitivity (89.45 %) and specificity (88.25 %), indicating its robustness in correctly identifying both positive and negative cases. This suggests that CDLM may be particularly effective for clinical applications where both accurate identification of disease and avoidance of false positives are crucial. Conversely, models like NHF and PACHD demonstrated lower sensitivity and higher miss rates, especially at larger image counts. This decline could suggest that these models may not generalize well with increased data variability or complexity. It raises concerns about their reliability in clinical settings, where accurate identification of CHD is critical for timely intervention.

4.2 Impact of dataset Size

The analysis revealed that larger dataset sizes often reduce model sensitivity and positive predictive value (PPV). This decline may stem from the challenges of training on diverse data, which can create prediction variability. In contrast, the CDLM model maintained more stable performance, suggesting better generalization for real-world use. Variability in specificity across models also points to difficulties in identifying negative cases as datasets expand. This highlights the need for ongoing evaluation and adjustment to sustain model performance. Codes from the ICD (International Classification of Diseases) in administrative information are frequently used for the detection of coronary heart disease (CHD) (Bhatt et al., 2023). However, these codes might incorrectly categorize individuals with true positive (TP) cases of heart disease. Improving CHD surveillance can be achieved by accurately detecting CHD in administrative documents with machine learning (ML) techniques. ML enhances the precision of identifying true positive cases compared to relying solely on ICD codes. This approach increases the relevance of findings from large datasets for the CHD patient group, thereby strengthening public health monitoring efforts (Shi et al., 2023).

Cardiovascular disease is the leading cause of death worldwide, and predicting it from clinical data is challenging. Machine learning (ML) is effective for diagnosing and predicting heart disease (Griffeth et al., 2023). Previous studies have applied ML techniques to forecast heart illness. In one study, eight ML classifiers were used to identify key features that enhance prediction accuracy (Mohsin et al., 2023). Neural network models like Naïve Bayes and Radial Basis Functions achieved accuracies of 95 % and 91 %, respectively. Vector Quantization demonstrated a 99 % reliability rate, outperforming other methods for predicting cardiovascular issues. The goal of predicting cardiovascular disease is to save lives, improve outcomes, and optimize healthcare resources. Key contributions include early intervention, personalized treatment, technological advancements, and ongoing research to mitigate the impact of coronary heart disease (Srinivasan et al., 2023).