Accuracy assessment of RFerns, NB, SVM, and kNN machine learning classifiers in aquaculture

2.1 Data collection

Samples were taken from 3 different farms bimonthly for 1 year during 2018–2019. Ten fish were sampled during each sampling from every farm. Moribund individuals with typical signs of disease (such as swimming slowly and separate from the herd, darkened in color, lesions on the body, problems in the eyes) were taken. Disease-causing bacteria from these fish were isolated and diagnosed. During the samplings, the physico-chemical parameters of the water [water temperature (T), pH, total dissolved solids (TDS), dissolved oxygen (DO) and saturation (Sat)] were measured from the farm entrance, fishpond, and outlet of the farm with the help of YSI brand multi parameter sensor. The pathogenic bacteria were isolated and identified during laboratory examinations of the sampled fish and their group numbers are listed in Table 1.

No bacteria were detected in the specimens collected during April (Table 1). This undetected disease situation was designated as group number 1 since it was a reference for classifiers.

2.2 Machine learning classifiers

There are two categories of ML classifiers. There are two types of learning, supervised and unsupervised. Instead of utilizing unlabeled data, as in unsupervised learning, supervised learning uses labeled data to produce predictions or classifications for new data (Kumar et al., 2020). Supervised learning may be used for tasks like as regression, classification, and grouping. If there are exactly two classes, the procedure is termed binary classification; if there are more than two, it is called multiple classifications (Prabhakaran, 2016). Support vector machine (SVM), naive bayes (NB), and K-nearest neighbor (kNN) are the three most used supervised learning classification methods (Binkhonain and Zhao, 2019). As fish are afflicted with bacterial groups and suggest six distinct classes depending to their health, several classification approaches have been developed to tackle this issue. The kNN, NB, RFerns, and SVM classification models were used in this investigation.

The kNN (Altman, 1992) is a controlled ML algorithm that can be used for both classification and regression purposes. It is an algorithm that attempts to classify new samples along with similar ones. The working principle of this method is the assignment of data that is new in a previously created sample set to the cluster that has the closest (k) distance to it. The distance between these two data can be determined using distance functions such as the Euclidean distance, Euclidean, Manhattan, and Minkowski distance. The kNN formula can be seen in Eq. (1).

A statistical classification technique employing the similarity attribute is the NB classifier (Rish, 2001). It is favored in many disciplines for categorization difficulties despite being straightforward. Bayes' theorem is used to express the NB classifier (Berrar, 2018). The NB classification is based on Bayes' theorem where $y_i,j∊\left\{1,\cdots ,j\right\}$ is a discrete variable that represents one of the j classes. The X property is expressed by the $X=\left(x_1,x_2,\cdots ,x_m\right)$ feature vector, which consists of m properties. The probability of the probability $p(y_j\mathrm{I}X)$ according to the Bayes theorem for the possible value $y_j$ is specified in Eq. (2).

When the prediction of the target class Y' is calculated by multiplying the conditional probabilities for all properties, the basic equation in equation (3) for NB classifier is formed (Deng et al., 2015; Arpacı & Kalıpsız, 2018).

In the NB classifier used for the estimation of the target class $Y^{\prime }$ , the equation (3) can be simplified as in equation (4), since the denominator is not dependent on a single class and the denominator is jointly involved in the process in all calculations (Deng et al., 2015; Arpacı & Kalıpsız, 2018).

According to Equality 5, the total number of educational qualifications on the class label is |N|,the number of educational qualifications in class $y_i$ is $\left|N_j\right|$ . After applying Equality 4 for each class and calculating the probabilities, the class with the greatest probability in the resulting values is selected as the target class (Deng et al., 2015; Arpacı & Kalıpsız, 2018). The RFerns is a machine learning technique was proposed by Özuysal et al. (2007) for matching similar parts between two photos of the same scene, enabling one to identify specific items or trace them on movies. Although it was originally used for image processing, a number of modifications have been made that allow the use of the RFerns algorithm in general ML problems and an R package has been developed by De Leeuw (2009). The efficacy of RFerns is based on the idea that, when the tests are chosen at random, the approach's power stems more from the fact that mixing groups of binary tests allows for greater classification rates than the tree structure itself. So, in the proposed d-RFs, each layer's neuron is replaced by a fern, and each layer is composed of several fern species. Let us assume that the value of features $f_n$ is the outcome (0 or 1) of a simple binary test, I ( $d_{n,1})$  < I ( $d_{n,2})$ , depending only on the intensities of two pixel locations ( $d_{n,1})$ and ( $d_{n,2})$ (Eq. (6) of the input image I (Kim & Ko, 2020).

Given the class label Ck, NB makes the simplifying assumption that features are conditionally independent. The class of conditional densities (fi|Ck) is typically simple to learn, but this independence assumption is typically untrue, and the resultant approximation frequently substantially underestimates the real posterior probability. Therefore, Özuysal et al. (2007) organized the features into groups made up of L small sets of size S to to make the problem manageable while taking these dependencies into account. These collections are known as ferns $F_l=\left\{f_1,1,f_1,2,\cdots ,f_1,s\right\}$ . By assuming that the groups are conditionally independent, a technique known as semi-NB may be used to calculate the joint probability for the characteristics in each fern (Eq. (7)).

We acquire the class-conditional distributions for each of the ferns from Eq. (7), and we use the Bayes rule to combine their outputs using semi-naive Bayes to generate the posterior distribution.

Each fern in testing is made up of a few binary tests and returns a probability for the input vector, which is a member of one of the classes that was discovered during learning. The semi-naive Bayes model is used to integrate these fern replies, and the class with the highest probability value is then selected (Eq. (8)) (Kim and Ko, 2020).

A well-liked and flexible supervised machine learning technique, i.e., SVM may be used to both regression and classification. The foundations of SVM were laid by Vapnik (1995). The SVM has been employed in a variety of applications since it has numerous trustworthy learning qualities and predicts positive trial outcomes (Kulkarni et al., 2004; Chen & Wang, 2007). The SVM is generally preferred for small and medium-sized classification problems. In N-dimensional space, the SVM essentially discovers a hyperplane that clearly divides data points from different classes. A hyperplane is a plane that separates two parts of n-dimensional data linearly. A line in two dimensions and a plane in three dimensions are called hyperplanes. Theoretical support for the SVM algorithm is provided below (Equation (9).

the n-dimensional characteristic vectors in the real number field $x_i$ and $y_i$ , $x_i\in X$ and , $y_i\in \left\{-1,+1\right\}$ . When the analyzed data set is expressed with a linear relationship, can be used the linear Eqs. (10) and (11).

3.1 Training success of models

Two different metrics, i.e., accuracy and Kappa were used for the training success of the models. The minimum, first quartile, median, mean, third quartile and maximum values of the results obtained for both accuracy and Kappa metrics of the models evaluated with the cross-validation technique are given in Table 2. When evaluating the Mean value of Accuracy and Kappa metrics, the most successful models are RFerns and SVM classifiers. It is seen that the worst classifier model is NB. The graph showing the training achievements of the models comparatively is shown in Fig. 1.

Close

Fig. 1

Models’ comparison graph based on accuracy and Kappa metrics.

Export to PPT

3.2 Testing success of models

The models were tested with the testing dataset, which was never seen in the training phase. Confusion matrix tables obtained from the tests of the models with the testing dataset and the evaluation results obtained with various metrics are included. The same confusion matrix was obtained for SVM and RFerns classifiers and is shown in Table 3.

As seen in the confusion matrix (Table 3), all 5 observations belonging to class 1 were correctly classified. Similarly, 2 observations belonging to class 2, 6 observations belonging to class 3, 5 observations belonging to class 4 and 5 observations belonging to class 6 were successfully predicted in the accurate classes. Accuracy value of SVM and RFerns classifiers was 100%, Kappa value of 100% and p-value was 3.78E-14, which is very close to zero. In addition to these metrics, evaluations of other metrics are given in Table 4.

For good classifiers, accuracy, Kappa, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and balanced accuracy (B) metrics should be close to 1 (Cakir et al., 2021). These related assessment metrics were equal to 1 for all classes. This shows that the SVM and RFerns models tested with the test data were quite successful. Confusion matrix table of NB and kNN classifiers is shown in Table 5.

As seen in the confusion matrix in Table 4, all 5 observations belonging to class 1 were correctly classified. Similarly, 2 observations belonging to class 2, 6 observations belonging to class 3, 4 out of 5 observations belonging to class 4 and 4 out of 5 observations belonging to class 6 were successfully predicted in their relevant classes. However, it is seen that the models predicted 1 observation in class 5, which is in class 4, and 1 observation in class 5, which is in class 6. Accuracy value of NB and kNN classifiers was obtained 91.3%, Kappa value is 89.15% and p-value is 7.93E-11, which is very close to zero. In addition to these metrics, evaluations of other metrics are given in Table 6.

The sensitivity metric measures how many of all positive observations we classify as positive. The sensitivity values for class 4 and 6 were 0.8. This indicates that the NB and kNN models were in fact 20% incorrectly estimated that the group of bacteria in class 4 (Aeromonas sobria, Lactococcus garvieae, Yersinia ruckeri) was the group of bacteria in class 5 (Acinetobacter sp, Aeromonas hydrophila, Lactococcus garvieae). Similarly, the NB and kNN models show that the group of bacteria in class 6 (Acinetobacter sp, Aeromonas sobria, Lactococcus garvieae) is incorrectly estimated by 20% as the group of bacteria incClass 5 (Acinetobacter sp, Aeromonas hydrophila, Lactococcus garvieae). For class 4 and 6, the BA metric calculates the arithmetic mean of the sensitivity and specificity metrics of the relevant classes (arithmetic mean of 0.8 and 1) and gives the value 0.9. Other assessments may be considered successful because their metric value is > 90%. These related assessment metrics do not appear to equal 1 for all classes. This shows that the NB and kNN models tested with the test data were good but more unsuccessful compared to the SVM and RFerns. The SVM and RFerns ML algorithms achieved more accurate results (100% for both) during the testing phase, while kNN and NB algorithms achieved less accurate results (91.3% accuracy for both).

According to literature search, no study was found related to RFerns in aquaculture applications. In a reported study, the effects of water quality on livestock performance in freshwater ponds was studied with 8 different ML techniques. Among them 3 ML techniques (SVM, NB and kNN) were analogous with ours. In that study, SVM and NB were successfully applied with 80–83% accuracy but kNN was insufficient (71–82 %) (Rana et al., 2021). We obtained the compatible result with SVM. Furthermore, the consistent results were reported in the other studies with SVM (Chen et al., 2022; Fan and Liu, 2013; Saberioon et al., 2018; Costa et al., 2019). The SVM was adopted in recirculating aquaculture system with the 98.95% accuracy for optimal variable-flow regulation (Chen et al., 2022). Besides, Fan & Liu (2013) achieved automate fry counting with the 98.73% of the average counting accuracy rate by SVM. In a reported study (Saberioon et al., 2018) fish diets were evaluated by ML methods. While the SVM provided the best classifier with correct classification rate of 82%, the k-NN was the least (40%). Moreover, SVM obtained the best result with 97.19% correct oocyte recognition against kNN (94.40%) and NB (90.03%) (Costa et al., 2019). It can be concluded that SVM has the great potential to be an effectively viable technique in aquaculture applications.