5.1 Simplifying ASD analysis with Chi-Square test

The chi-square test is a statistical method used to determine the presence of a significant association between two categorical variables by comparing observed frequencies with expected frequencies, assuming no connection between the variables. It evaluates the null hypothesis (H₀) which imagines that no relationship exists between the categorical variables in the data, implying they are independent, while the alternative hypothesis (H₁) suggests an association between the variables. In this research, we used chi-square test in the opposite direction, where significance level of p < 0.05 indicates a rejection of the null hypothesis, implying a dependency between the variables. Essentially, a lower probability threshold (p < 0.05) is chosen to detect any significant associations between the categorical variables.

In this research, variables like gender and used_app_before were excluded from further analysis based on chi-square test results. For instance, the chi-square value for gender was low (0.004), with a corresponding p-value of 0.943, significantly higher than 0.05, indicating no statistically significant association with autism. Similarly, the chi-square statistic for “used_app_before” was moderate (1.05), and its p-value (0.300) exceeded 0.05, affirming no statistically significant association between this variable and autism (Table 2 & Table 3).

In this case, since the p-values for all variables are significantly less than 0.05, it indicates that there is a highly significant association between each of these variables (A1_Score, A2_Score, A3_Score, A4_Score, A5_Score, A6_Score, A7_Score, A8_Score, A9_Score) and the outcome variable. This suggests that these variables are important predictors or indicators of the outcome to autism in the dataset being analyzed.

5.2 Model evaluation measures

We used various performance measures to ensure that developed ASD diagnostic model works effectively. These measures collectively guide us in refining and optimizing the model for accurate and reliable diagnosis.

a. True Positive (TP): TP, also known as sensitivity, represents the cases where the model correctly predicts individuals with ASD. A True Positive indicates that the model correctly identifies individuals who have ASD, enabling timely intervention and support (Ahmed et al., 2022).

b. True Negative (TN): TN, also known as specificity, represents the cases where the model correctly predicts individuals without ASD. A true negative suggests that the model accurately identifies individuals who do not have ASD, avoiding unnecessary interventions and treatments (Ahmad Fayaz et al., 2023).

c. False Positive (FP): These are cases where the model incorrectly predicts individuals as having ASD when they do not have. A false positive may lead to unnecessary stress and concern for individuals and their families, as well as potential misallocation of resources for interventions (Maenner et al. 2020).

d. False Negative (FN): These are cases where the model incorrectly predicts individuals as not having ASD when they actually have. False negatives are concerning because they may result in missed opportunities for early intervention and support, delaying diagnosis and treatment for individuals with ASD.

e. Positive Predictive Value (Precision): Precision measures the proportion of individuals predicted to have ASD who actually have the disorder. High precision ensures that individuals identified as having ASD are more likely to truly have the disorder, reducing the risk of over diagnosis (Atlam et al., 2024).

f. Negative Predictive Value (NPV): NPV measures the proportion of individuals predicted not to have ASD who truly do not have the disorder. NPV reflects the model's ability to rule out ASD for individuals without the disorder, complementing the information provided by sensitivity and specificity.

g. Area under the Receiver Operating Characteristic curve (AUROC): AUROC evaluates the model's ability to distinguish between individuals with and without ASD across different decision thresholds. A higher AUROC value indicates better discrimination ability, with 1 representing perfect discrimination and 0.5 representing random chance.

6.1 Random Forest model

Random forest is an ensemble learning technique that consists of multiple decision trees. Each tree is trained on a random subset of the training data and a random subset of features. Through a voting mechanism, predictions from individual trees are aggregated, resulting in a final prediction. This ensemble approach enhances prediction accuracy and robustness (Abdelwahab et al., 2024). We applied Random Forest to our ASD dataset and obtained the results described in Table 4. The developed random forest model’s performance is measured using various evaluation metrics. These metrics provide valuable insights into the model's effectiveness in distinguishing between instances of “absence of ASD” and “presence of ASD.”.

• Precision Analysis: Our analysis revealed a precision of 0.96 for the class “absence of ASD,” indicating that 96 % of instances predicted as not having ASD were indeed true negatives. Conversely, the precision for the class “presence of ASD” was 0.98, demonstrating that 98 % of instances predicted as having ASD were true positives.

• Recall Assessment: In terms of recall, the model achieved a recall of 0.98 for the class “absence of ASD,” correctly identifying 98 % of all instances not associated with ASD. For the class “presence of ASD,” the recall was 0.96, indicating that 96 % of all actual ASD instances were correctly identified by the model.

• F1-Score Evaluation: The F1-score, a balanced measure of precision and recall, yielded scores of 0.97 for both classes. This highlights the model's ability to minimize both false positives and false negatives in ASD diagnosis, contributing to its overall effectiveness.

• Support Analysis: Examining the support values revealed 179 instances of “absence of ASD” and 200 instances of “presence of ASD” within the dataset. These support values provide insights into the distribution of data and the prevalence of each class.

• Accuracy Examination: Our model achieved an overall accuracy of 97 %, indicating its high level of correctness in predicting both classes collectively. This underscores the model's reliability in accurately diagnosing ASD across the dataset.

• Average Metrics Interpretation: Additionally, both macro and weighted averages yielded consistent scores of 0.97, reflecting the model's consistent performance across all classes. These average metrics provide a comprehensive overview of the model's effectiveness in ASD diagnosis.

We also conducted an assessment of the AUROC curve to evaluate the performance of the random forest model in the context of ASD diagnosis (Fig. 3a-d). The AUROC curve, with an AUC = 1.00, underscores the model's exceptional capability to accurately differentiate between individuals with ASD and those without ASD across various thresholds of discrimination. This perfect AUC value signifies the model's ability to achieve a true positive rate of 1 (sensitivity) and a false positive rate of 0 (specificity), thereby ensuring precise predictions for all instances in the dataset. In practical terms, the flawless performance of the Random Forest model in identifying individuals with ASD while minimizing misclassifications of non-ASD cases highlights its robust reliability and efficacy for ASD diagnosis.

Close

Fig. 3

AUROC curve for ASD classification using (a) random forest, (b) logistic regression, (c) gradient boosting, (d) extra tree classifier.

Export to PPT

Overall, these findings demonstrate the strong performance of the random forest machine learning model in accurately diagnosing ASD. With high precision, recall, F1-score, support, accuracy, and average metrics the model showcases promising potential for ASD diagnosis in clinical settings.

6.2 Logistic regression model

Logistic regression is a linear binary classification model which gives the probability that an instance belongs to a particular class using a logistic (sigmoid) function. Logistic regression estimates the coefficients for each feature, which represent the impact of that feature on the log-odds of the target class. Logistic Regression could be used to model the probability of a child having ASD based on the given features (Ansarullah et al., 2024). We applied logistic regression to our ASD dataset and obtained the results described in Table 5.

• Precision: The precision metric reflects the proportion of true positive predictions out of all positive predictions made by the model. For class 0 (indicating the absence of ASD), the precision is 0.96, indicating that 96 % of instances predicted as not having ASD were indeed true negatives. Similarly, for class 1 (indicating the presence of ASD), the precision is also 0.96, demonstrating that 96 % of instances predicted as having ASD were true positives.

• Recall: Recall, also known as sensitivity, measures the model's ability to correctly identify actual positive instances. With a recall of 0.96 for both class 0 and class 1, the model accurately identified 96 % of all instances not associated with ASD and 96 % of all actual ASD instances, respectively.

• F1-Score: The F1-score, a harmonic mean of precision and recall, provides a balanced measure of the model's accuracy. Both class 0 and class 1 exhibit F1-scores of 0.96, indicating a high level of precision and recall for both classes.

• Support: Support denotes the number of actual occurrences of each class in the dataset. In this case, there are 179 instances of class 0 and 200 instances of class 1, contributing to a total of 379 instances.

• Accuracy: Accuracy measures the overall correctness of the model's predictions. With an accuracy of 0.96, the model achieved a high level of correctness in predicting both classes collectively.

• Macro and Weighted Averages: Both macro and weighted averages yield consistent scores of 0.96 for precision, recall, and F1-score, reflecting the model's consistent performance across all classes.

We also conducted an assessment of the AUROC curve to evaluate the performance of the logistic regression model in the context of ASD diagnosis (Fig. 3b). The AUROC curve plots the true positive rate (sensitivity) against the false positive rate (1 − specificity) for different threshold values. The AUC = 0.99 implies that the logistic regression model performed remarkably well in correctly identifying individuals with ASD while minimizing misclassifications of non-ASD cases. The high AUC value underscores the model's reliability and effectiveness in ASD diagnosis, making it a valuable tool for clinical decision-making and patient care.

6.3 Gradient Boosting Classifier

The Boosting Classifier is a machine learning algorithm belonging to the ensemble learning methods. It operates by sequentially combining multiple weak learners, typically decision trees with shallow depth, in a sequential manner to create a strong predictive model. In this method, each weak learner is trained to rectify errors made by preceding ones. This is accomplished by fitting each new model to the residual errors of the ensemble, progressively diminishing the overall prediction error. The algorithm iteratively minimizes a loss function using gradient descent optimization, thereby adjusting the parameters of each weak learner to enhance the model's performance (Ansarullah et al., 2022). We applied gradient boosting classifier to our ASD dataset and obtained the results described in Table 6.

• Precision: For class 0 (representing the absence of ASD), the precision was 0.97, indicating that 97 % of instances predicted as not having ASD were indeed true negatives. Conversely, for class 1 (indicating the presence of ASD), the precision was 0.98, signifying that 98 % of instances predicted as having ASD were true positives.

• Recall: The recall for class 0 was 0.98, suggesting that 98 % of all instances not associated with ASD were correctly identified by the model. For class 1, the recall was 0.97, indicating that 97 % of all actual ASD instances were accurately identified by the model.

• F1-Score: The F1-score, a harmonic mean of precision and recall, yielded scores of 0.98 for both class 0 and class 1. This balanced measure reflects the model's ability to minimize both false positives and false negatives in ASD diagnosis.

• Support: The dataset comprised 179 instances of class 0 and 200 instances of class 1, providing insights into the distribution of data and the prevalence of each class.

• Accuracy: The model achieved an overall accuracy of 98 %, demonstrating its high level of correctness in predicting both classes collectively.

• Macro and Weighted Averages: Both macro and weighted averages yielded consistent scores of 0.98 for precision, recall, and F1-score, indicating the model's consistent performance across all classes.

In our analysis utilizing the Gradient Boosting Classifier on the ASD dataset, we generated an AUROC with an exceptional AUC value of 1.00 which demonstrates the model's remarkable ability to accurately distinguish between individuals with ASD and those without ASD (Fig. 3c). In summary, the AUROC curve with an AUC value of 1.00 generated by the Gradient Boosting Classifier underscores the model's exceptional reliability and accuracy in ASD diagnosis, making it a highly promising tool for clinical applications.

6.4 Extra Trees Classifier

The Extra Trees Classifier, short for Extremely Randomized Trees Classifier, is a machine learning algorithm that belongs to the ensemble learning family. In Extra Trees Classifier, rather than selecting the best split among a subset of features, as in Random Forest, it randomly selects candidate splits for each feature. This randomness during the tree building process leads to extra diversity among the trees in the ensemble, hence the name “Extra Trees.” By introducing more randomness into the tree construction process, Extra Trees Classifier reduces the variance of the model at the expense of a slight increase in bias. This can sometimes lead to improved generalization performance, especially when dealing with noisy or high-dimensional datasets (Syed Immamul Ansarullah, 2019). This research applied the Extra Trees Classifier model to analyse the ASD dataset, from which we derived the following results as described in Table 7.

• Precision: The precision metric indicates the proportion of true positive predictions out of all positive predictions made by the model. For class 0 (representing the absence of ASD), the precision is 0.97, suggesting that 97 % of instances predicted as not having ASD were indeed true negatives. Similarly, for class 1 (indicating the presence of ASD), the precision is 0.98, indicating that 98 % of instances predicted as having ASD were true positives.

• Recall: Recall, also known as sensitivity, measures the proportion of true positive predictions out of all actual positive instances in the dataset. A recall of 0.98 for class 0 means that 98 % of all instances not associated with ASD were correctly identified by the model. Conversely, a recall of 0.97 for class 1 indicates that 97 % of all actual ASD instances were correctly identified by the model.

• F1-Score: The F1-score, a harmonic mean of precision and recall, provides a balanced assessment of the model's performance. With F1-scores of 0.98 for both classes, the model demonstrates a high level of accuracy in distinguishing between instances of “absence of ASD” and “presence of ASD.”

• Support: Support refers to the number of actual occurrences of each class in the dataset. In this case, there are 179 instances of class 0 and 200 instances of class 1, indicating a relatively balanced distribution between the two classes.

• Accuracy: Accuracy measures the overall correctness of the model's predictions. With an accuracy of 98 %, the Extra Trees Classifier model achieved a high level of correctness across all predictions, further emphasizing its effectiveness in ASD diagnosis.

• Macro and Weighted Averages: Both macro and weighted averages yielded consistent scores of 0.98 for precision, recall, and F1-score, reflecting the model's consistent performance across all classes and accounting for the distribution of instances within each class.

In our analysis utilizing the Extra Tree Classifier model on the ASD dataset, we generated the AUROC with an AUC value of 1.00 (Fig. 3d), which indicates perfect discrimination by the model, making it highly reliable for ASD diagnosis. Overall, these results highlight the robust performance of the Extra Trees Classifier model in accurately diagnosing ASD, with high precision, recall, F1-score, support, accuracy, and average metrics across both classes.