Enhanced major depressive disorder diagnosis: A hybrid deep learning framework utilizing EEG and fMRI

2.1 Preprocessing

The presented model starts with the preprocessing of EEG and fMRI signals. Due to the different nature of these two signals, the proposed method employs two different mechanisms for preprocessing these signals, which are described in this section.

2.1.1 EEG preprocessing

In most MDD diagnosis studies based on EEG, signals from 64 or 128 electrodes are used to record brain activities. However, using this configuration presents complexities and challenges. Preparing the patient for recording signals from 64 or 128 electrodes is time-consuming, and the use of conductive gels complicates the recording process. The prefrontal lobe exhibits high correlation with emotional processes, making it sufficient to rely on signals recorded by electrodes in this region for the MDD detection process. Additionally, the absence of hair in this area eliminates the need for conductive gels. Therefore, in this study, an EEG recording device with three electrodes (Fp1, Fpz, and Fp2) was used. The device used for EEG signal recording and the electrode positions have been shown in Fig. 3.

Close

Fig. 3.

EEG signal recording in the proposed diagnostic system (a) The used device and (b) Electrode positions.

Export to PPT

All EEG signals were collected with a frequency of 250 Hz. We selected this sampling rate according to MDD EEG analysis studies and established that it enables sufficient detection of brain activity frequency bands that operate below 100 Hz for MDD diagnosis. Higher sampling rates for this application do not improve diagnostic information while requiring increased data storage and processing power. To preprocess the input signals and remove the destructive effects of noise, each signal was first passed through a low-pass filter with a cutoff frequency of 1 Hz, and then a high-pass filter with a cutoff frequency of 40 Hz was applied. We applied a 4^th-order Butterworth filter configuration for both low-pass and high-pass frequency filtering. The Butterworth filter provided the best choice because it delivered a maximally flat passband response. A 1 Hz low-pass filter functioned to eliminate both slow drifts and DC offsets present in the recorded signal. The filtering technique is widely used in EEG preprocessing procedures. The low 1Hz cutoff frequency used in this study stands below the cutoff frequencies applied in other research, which employed either 0.5Hz or greater values. The selected low cutoff frequency aimed to eliminate all very slow drifts and DC offsets from the signal. The 40 Hz high-pass filter function removed high-frequency noise components, including muscle artifacts and line noise from the signal. The chosen cutoff frequency stands as a typical selection in EEG analysis. Subsequently, any potential noise in the resulting signals was eliminated using discrete wavelet decomposition. To achieve this, the EEG signal was decomposed into four levels using the Haar wavelet function (Zhang et al., 2019) and then reconstructed using the inverse wavelet transform. We selected the Haar wavelet because it offers both easy implementation and efficient computation. The analysis used four decomposition stages, which were determined by observing signal decomposition through visual assessment and their ability to remove unwanted noise from EEG signals. We selected this decomposition level because it filters out eye blinks and muscle movements that appear predominantly in higher decomposition levels. The outcome of this process is an EEG signal free from noise.

After noise removal from the input signals, the spectrogram is extracted. For this aim, the Short-Time Fourier Transform (STFT) paradigm is utilized to explain the features of each signal and generate its spectrogram matrix. The STFT framework provides a good technique for expressing signal fluctuations in the temporal domain. This model provides a good feed for methods of deep learning such as CNN and is ideal for expressing interpretable features that show signal variations in closely spaced locations. The spectrogram produced by this approach is a time-frequency matrix created by leveling the intensities of STFT units of the signal. To generate this matrix, the input signal is first separated into sections of equal length using a windowing technique. Following that, the Fourier transformation is calculated for each segment. STFT may be characterized as follows given a signal (Le-Roux et al., 2010):

(1)

$STF{T}_{S\left(n\right)}\left(m,w\right)=\sum _{n=-\infty }^{+\infty }S\left(n\right)\times w\left(n-m\right)e^{-jwn}$

The normalized signal is represented by S[n] in the equation above. Moreover, the windowing operator centered at m is indicated by w[m]. The signal’s spectrogram may be derived by squaring its scales once the STFT has been calculated using the previous equation (Le-Roux et al., 2010).

(2)

$SP{G}_{S\left(n\right)}\left(m,w\right)={\left(STF{T}_{S\left(n\right)}\left(m,w\right)\right)}^2$

A spectrogram matrix, with the vertical and horizontal dimensions signifying frequency and time, respectively, is the outcome of the equation. We applied STFT calculations using a Hamming window of 256 samples with a 50% overlap setting. We selected this combination of window size and overlap to achieve optimal results between time and frequency domain resolution. STFT analysis often uses 50% overlap because this combination provides better time resolution with weaker frequency resolution. The spectrogram required a frequency range between 0 to 40 Hz for its production. The selected frequency bands, which are important for MDD analysis, fit within this frequency range.

2.2 CNN-based feature extraction

After preprocessing the simultaneous EEG and fMRI signals, two CNN models with similar structures will be used for feature extraction from the preprocessed signals. In this step, each EEG and fMRI signal is fed into a separate CNN model to extract features and represent each signal as a descriptive vector. In the following text, the feature extraction model for EEG signals will be represented as $CN{N}_{EEG}$ and the feature extraction model for fMRI images will be represented as $CN{N}_{fMRI}$. The base architecture for the employed CNNs used for extracting features from EEG and fMRI signals has been illustrated in Fig. 4. Both CNNs have the same number and type of layers, but their hyperparameters differ.

Close

Fig. 4.

The base architecture for the employed CNNs used for extracting features from EEG and fMRI signals.

Export to PPT

The feature extraction model is fed with preprocessed signal data. In the $CN{N}_{EEG}$, spectrogram matrices are used as inputs, while the $CN{N}_{fMRI}$ is fed with preprocessed fMRI images. Both proposed CNN models utilize three convolutional blocks to extract feature maps from the input signals. Each convolutional module includes a sequence of convolution, Batch Normalization (BN), ReLU, and pooling layers. After extracting the feature maps, three consecutive fully connected (FC) layers have been employed to further extract structural attributes and compress them. Thus, the FC1 layer transforms the feature maps into vector form, and then the extracted features are compressed using the FC2 and FC3 layers. Finally, the activations resulting from FC3 are used as the attributes extracted from the input EEG/fMRI signals and are considered as the output of the feature extraction model. As mentioned, both CNNs employed by the presented approach consist of the same combination of layer types, but the parameters of their layers are configured differently. The parameter configuration details for each of the layers$CN{N}_{EEG}$and $CN{N}_{fMRI}$ have been provided in Table 1.

The parameter configurations for the CNN_EEG and CNN_fMRI models were determined by a grid search strategy. In this regard, various hyperparameter settings of the CNNs were examined using the training loss metric. The examined hyperparameters for tuning the model include the dimensions and number of convolutional filters, the type and dimensions of the pooling layers, the type of activation layers, and the size of FC layers. Additionally, various settings for training-related parameters of mini-batch size and optimizer were considered in the tuning step. Details for the search bounds of each configurable parameter of the CNNs have been listed in Table 2.

The investigations based on the grid search strategy and according to the search space employed (Table 2) revealed that feature extraction from fMRI images requires a higher number of filters in the convolutional layers compared to EEG signals to extract a greater number of patterns from the input images. Therefore, the number of filters in the CNN_fMRI convolutional layers was set to 64, while the CNN_EEG model effectively extracts features using 32 filters in its convolutional layers.

Furthermore, the nature of spectrogram matrices in EEG signals allows achieving desirable results by utilizing the average pooling function in the CNN_EEG layers. On the other hand, feature extraction from fMRI images is more efficiently performed using the Max pooling function in the CNN_fMRI model. Additionally, considering the larger dimensions and more detailed features present in fMRI images compared to EEG signals, the length of the feature vectors extracted from fMRI images was set to twice the number of feature vectors extracted from EEG signals.

At the end of the feature extraction step, the two feature vectors generated by the CNN_EEG and CNN_fMRI models are merged to create a single feature vector with a length of 300.

2.3 SVM and GA-based classification

Upon feature extraction and fusion from simultaneous EEG-fMRI data, an SVM model developed by a genetic algorithm is utilized for MDD identification. The link between the input elements and the target categories in the research subject under discussion is nonlinear. Consequently, a nonlinear kernel, such as the radial basis function (RBF) kernel, is used to tackle this problem with SVM. In the SVM model, the kernel of RBF is expressed as follows (Farquad et al., 2012):

The nonlinear multiplication of the two vectors of features for instances $x_i$ and $x_j$ is represented by the equation above as $x_i.x_j$. Furthermore, the symbol $\gamma$ denotes the kernel function’s parameter. Additionally, there can be an imbalance in the number of instances in the positive and negative categories. Classification mistakes may arise due to this imbalance. The SVM adjusts a correction factor independently for each category to solve this problem. In this instance, the SVM model’s optimisation issue may be explained as follows (Farquad et al., 2012).

The hyperplane’s normal vector is denoted by w in the equation above, and the margin coefficient is determined by b. Furthermore, $d_i$ specifies the matching label, and xi denotes the i-th training sample. In addition, ${\xi }_i$ represents the slack parameter. Lastly, the adjustment variables for both the negative and positive categories are denoted by C- and C+, respectively. The model offered suggests that the RBF parameter γ, along with the correction variables $C_{+}$ and $C_{-}$, can be used to improve an SVM model. The suggested approach uses GA to solve this procedure, which can be expressed as a problem of optimization.

A real-valued GA served as the optimization method for SVM hyperparameters. The continuous nature of SVM hyperparameters (including $C_{+}$, $C_{-}$, and γ) makes Genetic Algorithms an optimal choice for their optimization. Real-valued GAs function most effectively in exploring continuous spaces because they directly represent parameters as floating-point numbers rather than binary or integer values. The search process becomes more efficient since the encoding and decoding steps are eliminated. The GA applied its optimization process to SVM kernel parameters by maximizing the validation set classification accuracy. The GA demonstrates efficient parameter space exploration capabilities, which makes it appropriate for this task.

Considering the three tunable kernel parameters, each chromosome will have a length of three. The variables $C_{+}$ and $C_{-}$ in each chromosome have search ranges set to [0.01, 36000], whereas the parameter γ has a range set to [0.0001, 35]. Since the fitness function measures the worth of a solution, defining it may be thought of as the essential step in addressing optimization issues. The validation error criteria serve as the basis for the fitness function definition in the suggested technique. Therefore, to assess each chromosome’s fitness, an SVM is first set up using the parameters that are listed in that chromosome. After that, the training samples are used to train the preset SVM model. Ultimately, the error in validation is regarded as the value of fitness when the trained model is subjected to validation samples.

The amount of validation instances for which the SVM output differs from the true class value is indicated by the symbol E in the equation above. The parameter N also shows the overall number of validation samples. In the suggested strategy, the GA’s goal is to set up the SVM model to minimize Eq. (5)‘s output. Considering the presented coding for the chromosomes and their fitness metric, the GA’s steps for optimizing the SVM are as follows:

Step 1) The initial population of chromosomes is generated randomly based on the defined boundaries for the optimization variables $C_{+}$, $C_{-}$, and γ.

Step 2) The fitness of each chromosome in the population is calculated based on Eq. (5).

Step 3) The number of $N_p=0.7\times N$ parents are selected from the current population using the roulette wheel selection algorithm, where N represents the population size.

Step 4) Each selected pair of parents from the previous step undergoes one-point crossover to generate two new offspring chromosomes. Then, the fitness of the offspring chromosomes is calculated.

Step 5) In each offspring chromosome, a mutation operation is performed with a probability of 0.01. During this step, if the mutation probability is satisfied, one of the optimization variables, $C_{+}$, $C_{-}$, or γ is randomly selected, and its value is replaced with a randomly generated number within the specified bounds. Finally, the fitness of the mutated chromosome is calculated using Eq. (5).

Step 6) The entire set of existing chromosomes is sorted in ascending order based on their fitness. Then, N chromosomes with lower fitness values are selected as the new population for the next generation.

Step 7) The chromosome with the lowest fitness discovered so far is considered the current best-found solution.

Step 8) If one of the following conditions is met, terminate the algorithm; otherwise, go to Step 3:

• The number of algorithm iterations reaches the threshold G.

• The fitness of the best discovered solution reaches zero.

Following the preceding procedures, the SVM model is trained with the configuration found in the optimal solution, and this model is then utilized to identify MDD in new instances.

2.4 Complexity analysis and resource requirement

This section examines computational complexity together with resource needs for the proposed MDD diagnosis framework based on hybrid deep learning from a practical implementation perspective.

Multiple factors create the computational complexity within the proposed model design. The preprocessing steps for EEG signals consist of filtering operations ($O\left(n\right)$) and wavelet denoising ($O(n\times \log (n\left)\right)$) and STFT-based spectrogram generation ($O(m\times k\times \log (k\left)\right)$) where $n$ represents EEG signal length and $m$ denotes window count and $k$ stands for window size. On the other hand, fMRI preprocessing through fMRIPrep follows a complex pipeline. The exact complexity of this model remains challenging to determine but it requires more computational power than EEG preprocessing. Users can find complete information regarding fMRIPrep in (Esteban et al., 2019). The CNN feature extraction process in the model includes convolutional layers operating at complexity $O(H\times W\times C_{in}\times K^2\times C_{out})$ while FC layers contribute $O(n_{in}\times n_{out})$ complexity based on input neuron count $n_{in}$ and output neuron count $n_{out}$. $H$ and $W$ denote input feature map height and width, $C_{in}$ represents input channel number, $K$ indicates kernel size and $C_{out}$ indicates output channel number. Feature fusion has negligible cost. The SVM classification process follows a computational pattern of $O(n_{sv}\times d)$ that relies on $n_{sv}$ support vectors and feature vector dimension $d$. The computational requirements of GA-based hyperparameter optimization run in the system’s offline mode. The proposed model’s main computational complexity stems from CNN feature extraction, particularly within $CN{N}_{fMRI}$. Real-world deployment requires optimizing this stage because it determines the overall efficiency.

The necessary resources consist of hardware equipment and software applications, together with data storage facilities. The training process took place on a workstation equipped with an Intel core i7 13700H CPU alongside an NVIDIA GeForce RTX 4050 GPU that had 6 gigabytes of VRAM. The computation of CNN requires graphical processing unit (GPU) acceleration to be effective. MATLAB 2020a with Deep Learning Toolbox served as the software component. The system also requires at least 8 gigabytes of RAM and data storage space to contain datasets, together with preprocessed data and trained models, and training file records. When implementing the model in real-world clinical settings, the speed of making predictions becomes the most important factor. For successful operation, models must be optimized along with efficient inference methods. Available hardware equipment within clinical settings plays an important role. The data acquisition process needs to be standardized to achieve results that can be generalized. Model explainability builds trust. Data privacy stands as the most important ethical issue alongside other considerations. Future research efforts will focus on deploying the model by addressing aspects such as compression techniques and performance speed, and explainable model behavior, together with ethical considerations.

2.5 Database and evaluation metrics

The database used in this study consists of 150 simultaneous EEG-fMRI signals. The EEG signals were collected using a portable three-electrode device. The sampling frequency for the EEG signals is set to 250 Hz. The fMRI images in the database were acquired using two Siemens 3T scanners. The data for these samples were collected using an EPI sequence, where each image slice has a thickness of 4 millimeters, and a total of 34 slices were obtained. The time of repetition (TR) is set to 2 seconds, and the time of echo (TE) is 30 ms. Additionally, the matrix size for each fMRI slice is 64×64. Moreover, for each patient, a T1-weighted image with a matrix size of 256x256, TR=1.9 seconds, and TE=2.26 milliseconds is available. The age range of the participants is between 21 and 76 years. Out of the total samples, 83 samples belong to female participants, and 67 samples belong to male participants. Furthermore, in the mentioned database, there are 94 samples belonging to healthy individuals and 56 samples belonging to individuals with MDD. All samples were recorded prior to the commencement of treatment for the disease.

A 10-fold cross-validation method was applied during the trials. During the experiments, we have used cross-validation with an 80/10/10 split for training/validation/test instances. In other words, in each fold of the cross-validation, 80% and 10% of instances were used for training and validating the model, respectively. After training the model, the remaining 10% of instances were used as unseen test instances. It should be noted that in each fold, a new 10% portion of instances was used for testing the model, meaning that after completing the 10 folds, all instances have been used in the test phases. Following every iteration, the test instances’ real labels and the labels provided by the suggested model were contrasted. Each test sample fits under either of the following groups under these circumstances:

• 1.

  True Positive (TP): A test sample belonging to an MDD patient that has been correctly identified as having the disease by the proposed method.

• 2.

  False Positive (FP): A test sample belonging to a healthy individual that has been mistakenly classified as a patient by the proposed method.

• 3.

  True Negative (TN): A test sample belonging to a healthy individual that has been correctly labeled as healthy by the proposed method.

• 4.

  False Negative (FN): A test sample of an MDD patient that has been mistakenly classified as healthy by the proposed method.

Based on the above conditions, the metrics of accuracy, precision, recall, and F-measure can be used to evaluate the performance of the diagnostic system. These metrics are formulated as follows:

In Eq. (6), the accuracy metric describes the system’s ability to correctly identify each of the positive (patient) or negative (healthy) categories. The precision metric in Eq. (7) indicates the proportion of correctly identified patient outputs among the total outputs classified as patients by the system. Additionally, the Recall metric in Eq. (8) describes how many of the actual patient samples the detection system was able to correctly identify. Lastly, the F-Measure metric in Eq. (9) provides the overall performance of the diagnostic system in identifying MDD, represented as the harmonic mean of the precision and recall metrics.

3.1 Results

According to the procedure described in section 4-1, the evaluation of the proposed method in detecting MDD was performed. During the experiments, the population size and the number of iterations in the genetic algorithm were set to 150 and 300, respectively, to optimize the configuration of SVM. Fig. 5 illustrates the variations of GA fitness during different iterations. In this graph, the horizontal axis represents the number of GA generations, and the vertical axis depicts the minimum fitness and the average fitness of the population during different iterations. As shown in this figure, the genetic algorithm can reach the best solution after 103 iterations, resulting in a validation error of 0.73%. The decreasing trend of the average population fitness in different iterations also indicates the successful performance of the optimization algorithm in moving towards the global optimum.

Close

Fig. 5.

The graph of GA fitness variations during different iterations for optimizing the configuration of SVM.

Export to PPT

To evaluate the effectiveness of the techniques employed in the proposed diagnostic system, the performance of this model has been compared with the following scenarios:

• SVM-GA-EEG: In this scenario, only features extracted from EEG signals are used for MDD detection, and fMRI images are disregarded.

• SVM-GA-fMRI: In this scenario, only features extracted from fMRI images are used for MDD detection, and EEG signal features are disregarded.

• SVM-EEG-fMRI: In this scenario, features extracted from EEG-fMRI signals are classified using an SVM model with a radial basis function (RBF) kernel. In other words, in this scenario, the optimization of the SVM model configuration is not performed by GA.

• Random forest (RF)-EEG-fMRI: In this scenario, features extracted from EEG-fMRI signals are classified using a random forest classifier.

• Multilayer perceptron (MLP)-EEG-fMRI: In this scenario, features extracted from EEG-fMRI signals are classified using an MLP neural network. This MLP model has two hidden layers with 10 and 8 neurons, and is trained by the Levenberg-Marquardt algorithm.

It should be noted that the above scenarios are comparisons with different configurations of our proposed model (e.g., using only EEG or only fMRI). The remainder of this section is dedicated to presenting the results of evaluating the proposed approach with the above comparison cases. The average accuracy of the suggested approach for recognizing MDD in the instances in comparison to alternative approaches has been shown in Fig. 6. It should be mentioned that the data shown in this figure and the other figures in this part are the average of the findings from ten cross-validation folds. According to Fig. 6, the suggested technique outperforms the methods that were examined, with an accuracy of 98.67% in determining the health or disease condition of the database instances. On the other hand, if the optimized proposed SVM model is replaced with the conventional SVM model with the RBF kernel, the detection accuracy will decrease to 94.67%. Similarly, if MDD detection is based on either the EEG or fMRI feature set alone, the detection accuracy will be 94%. In contrast, if the proposed classifier is replaced with RF and MLP models, the detection accuracy will be 93.33% and 92%, respectively. This shows that using the genetic algorithm to optimize the SVM configuration in the introduced approach can outperform the conventional SVM classifier and increase the accuracy by at least 4%. Additionally, comparing the accuracy of the proposed hybrid method with the scenario where either the EEG or fMRI feature set is used demonstrates that the combination employed in the proposed method can result in more accurate extraction of MDD-related features, leading to a minimum increase of 4.67% in detection accuracy. These findings demonstrate the potency of the strategies included in the suggested approach to raising MDD detection accuracy.

Close

Fig. 6.

Comparing the average accuracy of different methods in MDD detection.

Export to PPT

On the other hand, the closest method to the proposed method in terms of accuracy is the SVM-EEG-fMRI approach, where an SVM model with a radial basis function is used to classify EEG-fMRI features. Thus, support vector machine, compared to other conventional learning models such as Random Forest and Multi-Layer Perceptron, can classify the features extracted by CNN_EEG and CNN_fMRI with higher accuracy, and the optimized configuration of this model in the proposed method has been able to demonstrate this superiority more prominently.

More thorough insights into how well classification techniques detect MDD may be obtained via a confusion matrix. The suggested method’s matrix of confusion, along with alternative approaches for identifying database samples, has been shown in Fig. 7.

Close

Fig. 7.

(a-f) Confusion matrix of different methods in MDD diagnosis after 10 cross-validation iterations.

Export to PPT

The true labels of the test instances are shown in each column of these confusion matrices, while the estimates produced by each classification technique are shown within the rows. The healthy class is represented by the first row and column, while the MDD class is represented by the second row and column. For instance, the suggested technique successfully classified all 94 samples in Fig. 7(a) that belonged to the class of healthy (the total of values in the matrix’s first column).

In contrast, Fig. 7(d) shows that the base SVM model misclassified 6 samples from this class and assigned them to the MDD class. According to Fig. 7(a), it can be observed that the proposed method only made errors in classifying 2 out of 150 database samples, which belonged to the MDD class. On the other hand, the number of FP samples in the proposed method is zero. This means that the MDD samples detected by the proposed method are 100% correct, making the results of MDD diagnosis more reliable. The interpretation of classification results for the other presented cases in Fig. 7 can be done similarly. Overall, comparing these confusion matrices indicates that the proposed method outperforms other methods in classifying samples from both the healthy and MDD classes and has increased the accuracy of diagnosis by at least 4%.

As shown in Fig. 7, the proposed model proved to show a lower FP rate (FPR) and FN rate (FNR) than the compared scenarios. This is especially important in the case of MDD diagnosis, because both types of errors have important clinical consequences.

• FNs: The implications of FN predictions in patients with MDD are severe. MDD requires early diagnosis and treatment; hence, timely diagnosis and intervention are important. Reducing FNs can therefore assist the model in providing early and right care to the patients.

• FPs: Even though comparably less severe than FNs, FPs can have serious consequences as well. They can produce avoidable concerns and stress in patients who then require additional tests that may incur costs or even risks. In addition, the occurrence of FPs can put a lot of pressure on the health sector and cause a misallocation of available treatment. As a result, the model will enable investigators to avoid FPs, thus saving time and healthcare resources.

These results show that the techniques used in the proposed model can be used to reduce both false-positive and false-negative cases, which will enhance the ability to diagnose MDD and ultimately enhance patient care.

The average accuracy, recall, and F-measure measures have been shown in Fig. 8. These graphs show how various approaches fare overall in terms of classification rate metrics. Table 3 also includes the numerical outcomes of the tests carried out in this section. This table compares the suggested method’s performance to a few earlier investigations.

Close

Fig. 8.

Average precision, recall, and F-measure metrics.

Export to PPT

As seen by Table 3 and Fig. 8‘s comparison of accuracy, precision, recall, and F-Measure metrics, the suggested approach is superior to the other techniques in its ability to identify MDD patients. These findings suggest that the suggested approach can raise the accuracy, recall, and F-Measure parameters. The suggested method’s precision rating of 1 shows that applying the suggested approach guarantees the accuracy of the method’s positive outcomes. Furthermore, a higher recall suggests that a greater percentage of MDD samples have been accurately identified using the suggested procedure.

The Receiver Operating Characteristic (ROC) curve that emerged from the database sample categorization has been shown in Fig. 9. This graphic shows that compared to the other approaches, the suggested method has a greater area under the ROC curve, a lower FPR, and a higher true positive rate (TPR). Therefore, it can be said that the approach suggested in this article has a better average accuracy when it comes to accurately identifying MDD individuals.

Close

Fig. 9.

ROC curve for MDD detection.

Export to PPT

3.2 Statistical analysis

A more thorough examination of the suggested method’s effectiveness and importance in relation to the baseline techniques can be obtained through statistical analysis. One-way Analysis of Variance (ANOVA) was employed for this purpose. In this test, first, the predictions made by the models for all test instances are organized as a matrix. Each row of the matrix corresponds to a test sample, and each column to either the proposed method or one of the baseline techniques. The predicted labels of our model and the compared methods are arranged in this manner. The accuracy values for each test instance are then computed in this matrix by comparing each predicted label with the sample’s ground-truth label. In this instance, the output is characterized as matching the context label if it is +1 and non-matching if it is -1. Each column of the accuracy matrix underwent a normalcy test using a quantile-quantile (Q-Q) diagram. The test’s findings demonstrated that each model column’s distribution is normal, allowing one-way ANOVA to be utilized for statistical analysis.

The test produced a significant effect ($F= 10.76, p = 2.35\times {10}^{-10}$), from which it is possible to conclude that there is a statistically significant difference ($p<0.05$) in the accuracy of at least two approaches. Because the one-way ANOVA test is unable to identify the models from which this difference originates, a multiple comparison analysis was employed to conduct a deeper investigation.

Tukey’s honestly significant difference (HSD) was employed as the post-hoc analysis in the multiple comparison study to determine which of the particular models had statistically distinct levels of accuracy. When comparing pairings of groups after a significant F-test in an examination of variance, this post-hoc test is perfect. Since Tukey’s HSD accounts for the family-wise inaccuracy rate, the likelihood of a minimum of one Type I mistake (FPs) for each individual comparison is kept within the acceptable alpha level (0.05 for this study), making it preferred over other post-hoc tests. Tukey’s HSD provides a workable method for determining which model performs noticeably worse or better by regulating this error rate. As a result of Tukey’s HSD test, we were able to determine specific models that accounted for the significant differences that were revealed in the ANOVA test. The test results have been displayed in Fig. 10, which indicates that the proposed method outperforms the compared scenarios.

Close

Fig. 10.

Multiple comparison analysis on the accuracy of the MDD diagnosis models.

Export to PPT

This analysis proves the effectiveness of each technique used in the proposed method in the significant improvement of the accuracy of the diagnosis system. For instance, the significant difference between the accuracy of the proposed model and the “SVM-GA-EEG” and “SVM-GA-fMRI” cases, proves that using simultaneous EEG/fMRI for MDD diagnosis, results in a more accurate identification. Additionally, comparison with the “SVM-EEG-fMRI” shows the effectiveness of optimizing SVM hyperparameters on leveraging the accuracy.

3.3 Discussion

The trained hybrid deep learning model in the current study provides the potential of improving the outcome of diagnosing MDD. Using the EEG and fMRI data and using machine learning algorithms, the model has shown higher accuracy compared to the previous methods. The next subsections explore the real-world significance of these findings, as well as the conditions that facilitated the model’s success, its limitations, and potential future research directions.

The proposed hybrid deep learning model has proved to be very efficient in diagnosing MDD, with a classification accuracy of 98.67%. This improvement over what has been known for so long is actually a testament to what this model can do to mental health care. Thus, the model can be helpful to clinicians in the early identification of the potential disorder, in the differentiation between various types of the disorder, and in the development of management strategies. Such early intervention is critical in enhancing the patients’ prognosis. In addition, the model’s accuracy can improve the diagnostic agreement and decrease dependence on the clinicians’ subjective estimations.

The combination of EEG and fMRI data, together with the suggested hybrid deep learning architecture, has played a crucial role in attaining the model’s high performance. The CNN-based feature extraction was able to capture intricate features of both the EEG and the fMRI signals and combined them to provide a holistic representation of the brain activity. This approach helped to increase the model’s accuracy in distinguishing between healthy and depressed individuals by a large percentage. The usage of the genetic algorithm in the optimization of the SVM classifier added to the model’s accuracy by tuning the parameters of the classifier.

This shows that the proposed method yields better results than other machine learning methods like SVM, random forest, and MLP when the same feature set from EEG-fMRI data is used. For example, the proposed GA-based SVM tuning model performed 4% more accurate than the conventional method of SVM. The model’s high precision, recall, and F-measure values of 1, 0.9818, and 0.9867, respectively, demonstrate its robustness and effectiveness in identifying MDD cases. Also, the proposed method showed at least 1 percent improvement in the diagnosis accuracy compared to the methods presented in (Čukić et al., 2020), (Mahato & Paul, 2019), (Bandopadhyay et al., 2020), and (Mahato et al., 2020), proving the effectiveness of utilizing EEG and fMRI data for diagnosing MDD simultaneously.

The use of AI, in particular, deep learning in the proposed framework has several important benefits for MDD diagnosis.

• Enhanced Diagnostic Accuracy: The proposed hybrid deep learning model has a higher diagnostic accuracy compared with conventional approaches, which can result in earlier and more accurate diagnosis of MDD. This may help with timely intervention as well as enhance treatment results.

• Improved Treatment Outcomes: The timely and accurate diagnosis helps clinicians to start proper treatment regimens in a better manner, which may lead to better patient outcomes, shorter treatment span, and overall quality of life in case of MDD.

• Personalized Medicine: Based on the patient’s own data, such as EEG and fMRI, AI can help in developing the patient’s individual treatment plan. This can pertain to the client demographics where treatment options depend on the client’s need, gender, or age, such as dosage of medication, type of therapy, length of treatment, and so on.

• Reduced Diagnostic Bias: AI algorithms will help to reduce the influence of human biases in the diagnostic activity, thus making the latter more accurate. This can assist in minimizing diagnostic disparities and enhancing the quality of care of patients with MDD.

We should emphasize that the method described in this paper is a computational diagnostic tool that operates on the existing EEG and fMRI data. It does not take direct action or even handle the patients in any form or another. Hence, there are no direct side effects associated with the method itself. But important questions remain concerning the practical use of AI in health, such as privacy, fairness and accountability, and the weights and measures of AI in diagnosis and treatment.

3.4 Limitations, ethical considerations, and future directions

It is necessary to state the limitations of the proposed model, although the results look quite promising. The small number of participants in the study reduces the ability of the proposed model to be generalized across various population types. The model performance might be affected by the various EEG and fMRI acquisition parameters, which differ between different medical centers. The mysterious (black-box) structure of deep learning models hinders a clear understanding of extracted features since researchers find it difficult to establish their neurobiological relationships with MDD. The unclear nature of this method prevents scientists from grasping disease mechanisms better.

One vital ethical matter in this field centers around protecting subjects privacy data. The protection of patient data confidentiality, together with data security, stands as the most vital priority. To guarantee clinical success, strict compliance must be maintained with all data privacy and data management best practices. The ethical problem with AI-driven decisions includes the lack of interpretability of their underlying processes. Medical practitioners must understand the reasoning behind each diagnosis made by the model when working in a clinical setting. Research into Explainable AI (XAI) techniques should take place to enhance the transparency and gain trust in AI-based diagnostic systems. The responsible clinical implementation depends on this essential aspect.

The model faces an overfitting risk because the available sample size remains limited. A model becomes overfit when it memorizes training data by learning its noise and specific characteristics, thus performing poorly on new data. We utilized techniques like regularization in the CNNs together with cross-validation in the implementation to face the overfitting problem, but this issue may arise if we plan to use specific features, such as age, along with other features as inputs. The model requires independent validation on different population datasets to determine its ability to generalize across different groups.

The model needs further development, which requires several research directions to improve its clinical effectiveness. The essential requirement for improved model performance rests in validating the model across extensive datasets that represent diverse patient populations. The study requires data from various clinical centers that use different acquisition methods and treat patients from different demographic groups. A prospective clinical trial evaluation needs to occur to assess the model’s real-world clinical performance and its effects on patient results. Such clinical trials will generate important data points related to model performance measurements and their potential benefits for diagnostic precision and treatment preparation.

The integration of supplementary data types would improve MDD diagnosis accuracy and establish a complete understanding of the disorder. The next versions of the model should incorporate genetic information together with clinical scores obtained from standardized depression scales and lifestyle data. A combined data analysis strategy would show how various elements combine to develop MDD. The inclusion of genetic information allows doctors to classify patients according to their individual depression susceptibility so they can receive better individualized treatment.

The interpretability challenge needs to be resolved as a last step. Research must explore XAI techniques to understand how the model reaches its decisions. The visualization of CNN features together with the identification of major brain regions and connectivity patterns that influence MDD diagnosis would generate useful knowledge for both clinical practitioners and neuroscientists. The implementation of this approach would establish model trust and create opportunities to discover MDD neurobiological mechanisms. Research will explore age use as input data alongside proper regularization approaches for minimizing overfitting effects to enhance cross-age generalizability of the model.