Statistical probability prediction model for E-Learning and realtime proctoring using IoT devices

2.1 E-Learning

According to (Güllü et al., 2014; Liu and Yu 2023), e-learning is “the use of information and communication technologies to improve the quality of learning by enabling access to resources and services, as well as remote exchange and collaboration.” Additionally, it is defined as a range of platforms and applications used to deliver training and educational processes to students (Alqahtani and Rajkhan 2020). There is a rapid development in e-learning platforms, which leads to changes in education methods and the transfer of resources to students (Kong et al., 2023). Consequently, innovative technologies have been utilized in e-learning platforms to enhance learning effectiveness (Gamalel-Din 2010; Udupi et al., 2016; Hsu et al., 2018; Tsipianitis et al., 2025). E-learning is a brilliant innovation that enables students to access numerous resources under the guidance of instructors (Ghashim and Arshad 2023; Adel 2024). It has been confirmed that e-learning plays a key role in educational activities because it uses modern technologies in teaching (Maatuk et al., 2022). Some advantages of e-learning for students or universities include saving time and effort, enabling them to manage educational activities more easily, and accessing resources quickly (Ms and Toro 2013; Gautam 2020; Mukhtar et al., 2020). Additionally, it can reach all audiences, offering flexibility, functionality, and accessibility (Ghashim and Arshad 2023). E-learning enables institutions to accurately evaluate their students’ progress and ensure their skills and knowledge (Chang 2016). It has been confirmed that e-learning enhances traditional learning by making students’ resources and materials more accessible (Haryono et al., 2023).

Smart Homes, Healthcare 5.0, Blockchain, IoMT, IDS and other technologies are spreading across the globe (Khan et al., 2024). IoT is a set of devices using the Internet to communicate and locate each other (Ghashim and Arshad 2023). IoT is considered one of the innovative technologies that contribute to enhancing society (Wang et al., 2023). There are several key characteristics of the IoT, including dynamic changes, intelligence, interaction, heterogeneity, connectivity, enormous scale and sensing (Al-Taai et al., 2023). IoT can transform classic education paradigms by enabling the development and management of educational activities within institutions through the creation of materials and the facilitation of student participation (Ghashim and Arshad 2023). IoT has three components: hardware (actuators, embedded communication tools and sensors), Middleware (devices to process that store the data), and interpretation and visualization devices (Gubbi et al., 2013). The essential work of IoT is to sense the data through tags and sensors and send it to the cloud system. There are several types of IoT interactions that involve objects, including object-to-machine, object-to-object, and machine-to-machine interactions. There are three layers of IoT structures in education: application, network, and perception layers (Fig. 2) (Gubbi et al., 2013).

Close

Fig. 2.

Steps in feature extraction and classification on third party data set (Gubbi et al., 2013).

Export to PPT

Internet of Educational Things (IoET) is defined as IoT devices that are used by institutions to enhance infrastructure, teaching, and academics, as in Fig. 3.

Close

Fig. 3.

IoET adapted from (Desk 2018).

Export to PPT

Now, classrooms in educational institutions utilize IoT tools in academic activities, such as cameras, Radio Frequency Identification devices (RFID), and sensors. (Shaqrah and Almars 2022). Utilizing IoT in educational activities offers several advantages, including enhanced communication between students and instructors, effective management of the educational process, and improved learning outcomes (Ali et al., 2023). Academic institutions use IoT to create an innovative campus, enhance and develop educational systems, and expand on-campus services, including transportation, security, and energy consumption (Sneesl et al., 2022; Din et al., 2023; Fernández-Batanero et al., 2024; Rao and Elias-Medina 2024). It has been asserted that IoT is essential, providing a bright object in e-Learning systems shown in Fig. 4 and proposed by (Bayani et al., 2017).

Close

Fig. 4.

IoT services architecture in education (Bayani et al., 2017).

Export to PPT

2.2 Technical architecture

The ideology used in this study tends to converge the use of IoT Devices by E-Learning and Examination systems. Modern computational units such as Raspberry PI and Arduino play a significant role in this architecture. CNNs are the fundamental backbone of deep learning. Various applications utilize CNNs to yield value-added output and impressive results (Alangari and Khan 2021). Multiple applications, such as text mining, image information extraction, handwriting text mining, and optical character recognition from old manuscripts, utilize the CNN architecture. Several layers are introduced and added to the existing layers in various CNN architectures. One of the studies suggested by George and Prakash 2018 focuses on enhancing the existing CNN values with new layers. Several datasets contain CNN applications for identifying human faces and recognizing faces from blurred images. Several researchers have been working to detect the appearance of human figures using trained networks. These networks make use of comparisons with the existing pictures within the training dataset. These datasets are used to test the accuracy of the trained model. A nearly 73.1% accuracy is achieved by various trained models suggested by (Mabrouk and Zagrouba 2018). (Mabrouk and Zagrouba 2018) The model’s accuracy was enhanced up to 75% when a neural network was applied to it. Several additional studies are working in a similar dimension. (Booranawong et al., 2018) suggested that various use cases use deep learning to ensure better results under several circumstances. The author’s experiments reveal various groupings in which adjustments can lead to proper results based on the previous inputs into the neural network layer.

Various algorithms and new techniques have been designed in the block to classify the images and video files. Detection of objects and the classification of multiple scenes are widespread in all the models proposed (Feng et al., 2017). The most commonly used descriptors in identification and classification were presented by (Ternes et al., 2019). The main idea behind the classification was to identify several features and commonly track anything found suspicious. The semantic ideas used to identify these images use image classification (Hsu et al., 2018). These classifiers are determined based on the adaptation of the neural network model. Frames and videos are identified in the model, which are challenging to understand in real life. The models prescribed in the previous cases utilize human intervention and object identification based on human-defined features. The data set comprising the objects was of limited size and did not cover all the aspects. Various researchers propose several low-level feature extraction models for object detection and its classification. The methodology used for detecting and recognizing students appearing in the examination in the real world comprises all the above-stated possible solutions. Identifying human activity in the classroom, recognizing its nature and significance, and categorizing it using various methods are integral to this study.

Cheating in examinations is a widespread phenomenon observed at several educational levels. Various researchers suggest that the availability of exam-related datasets and information is crucial for identifying inappropriate cheating methods. However, most research presented and published comprises static datasets not under computer vision techniques. There is a broad scope for introducing the identification of human activities in the classroom using computer vision recognition frameworks. Therefore, the real-time installation technique model presented in this study investigates a robust model that is helpful for the comprehensive diagnosis of Procter with minimal invasiveness. The unlawful exam attempts for students across any university in the world prohibit a student from appearing in the examination after he has been found cheating. We examined the code of conduct for the students appearing in an examination as outlined in the National Quality Framework of Saudi Arabia, along with the code of conduct for the invigilator. To ensure that the invigilator conducts the investigation properly, we aim to design a system that follows all the codes of conduct prescribed by the Ministry of Education of Saudi Arabia. Various features are considered before designing this model to ensure the applicability of the code of conduct. The entire room is regarded as the workspace for deploying the IOT devices, which is part of this architecture. Recent research has been done to provide and identify a complete model for detecting human activities across any domain.

Human activity identification is done by Hsu et al., 2018, which uses bargain detection. Several models incorporate useful features. These models cannot account for all possible situations in which information and accuracy can be lost during proctoring. Booranawong et al., 2018 suggested the two-dimensional CNN model. Noah et al., 2018; Agarwal et al., 2019; Du et al., 2019; Jalal et al., 2019 use CNNs to identify human activity and predict the possible situation. The object detection in these studies ensures the use of CNN to provide accurate prediction values. Human detection, as suggested by Noah et al., 2018; Al-Taai et al., 2023; Truss and Anderson 2023; Xu et al., 2023, utilizes new techniques that integrate areas with features such as size and human indicators. In several CNN models reported by various authors, a very high level of information is captured in every frame instance. The technique for fusion is applied in this case, including early and late detection. Some researchers have also investigated the use of temporal information for human detection (Jalal et al., 2019). Several enhanced authors, those who utilize 3-D classification of images from videos, also create advanced-level models. The supervised training of the model proposed by the researchers ensures that around 25% to 55% annotations can be achieved. For fully supervised architectures, the performance is enhanced abruptly (Kong et al., 2023). New features like identification of text from the images and videos, cognitive abilities like speech, identification of natural language from the photo and video frames, and the analysis of the labelling for the images are introduced with the help of a fully supervised architecture (Agarwal et al., 2019).

3.1 System setup

The general setup for creating an environment to test the identification of individual behavior during suspicious performance is made with the help of the Raspberry PI 4B model, which can connect to the wireless network. The integration of several circuits from four dimensions collects the video feed for the proctoring. The proposed model connects the circuit with the help of a high-definition camera and captures continuous motion. The video frames are passed to a training dataset for testing of posture detection. The prediction of suspicious activity from various students’ postures is possible with the help of the data collected by these cameras. The small computational unit can process the data collected by the high-definition camera in real-time.

The complete set is collected with the help of a wireless network to a centralized server system. As shown in Fig. 5, the dataset is used to detect any suspicious activity across the complete frames received.

Close

Fig. 5.

Experimental setup for the proposed model.

Export to PPT

The experimental setup relies on the use of edge devices and their integration with a computational unit for processing the input collected from the video sources. Since the project is related to its initial considerations, system integration with the devices is handled using the latest hardware. We have provided a computational unit, Raspberry Pi 4B, integrated with a high-end device for competition. 32 GB RAM, 8 GB graphics card, and an i7 Processing unit with 512 GB of SSD Memory enable fast execution to provide the least latency issues under processing. Since the surveillance cameras are installed at variable locations in the surveillance room, they are integrated with the edge computing device using wireless communication techniques. It ensures that the information flow is seamless and that minimal resistance is offered during the transfer of data through cable connections. Due to the wireless nature of the entire setup, the maintenance and logistics are well-suited to the requirements. The real deployment of the system can conduct surveillance and proctoring with minimal delay in the processing. However, the institute’s infrastructure remains a constraint. Once the product is successfully deployed, it is expected that the organization will provide funds for achieving technologically driven proctoring. The maintenance issue for the complete experimental model requires minimal effort, and the hardware involved in the experiment is relatively inexpensive. This ensures that the model is more likely to be adopted on financial grounds as well.

3.2 Data collection

The data sets used for training the model are third-party datasets passed to the entire logical code, as suggested by (Gupta and Agarwal 2023). EfficientNet, SPNASNet, EfficientNet, and MobileNet datasets were studied for the eligibility of the proposed model. The progression of modern classification involves categorizing data sets into two distinct categories: suspicious and non-suspicious. All the images trained with the dataset contain suspicious activities that the students can perform during the examination. The data used in this model training is labelled with the help of a data labelling technique. In this case, we have labelled the data for the input and output services. The prediction behavior for the future is facilitated with the help of data enablement, ensuring that both the input and output data sets are examined. The entire data set was classified into two classes, and 15 features were extracted during the training process. A predefined ImageNet dataset was used to train the complete model. Among all the selected ones, the most essential features are considered for training the complete model. These features are classified to identify the best methods required for detecting suspicious activity in the classroom.

Predicting behavior for any specific person typically involves several computational techniques using CNN and computer vision. The preprocessing of video content involves several elements, including frame extraction, resizing, and normalization. With the help of the preprocessing stage, the computational unit can receive the image. Feature extraction for the predicted system comprises object detection, pose estimation, and action recognition models. In our case, object detection is performed using the Faster R-CNN technique to locate various objects within the video frame received from the computational unit. In this case, we offer an algorithm that utilizes OpenPose and media points to analyze human body points and poses. A pre-trained model is used to recognize the complete actions performed by humans. ResNet is used to identify various patterns across the availability of multiple frames captured by the high-definition camera.

Since we are proposing an analytical reasoning approach for a predictive model of posture detection for suspicious activity, proctoring is facilitated with the help of IoT devices. The behavior is assumed to be a function related to time sequences of image frames in the form of a video. Temporal analysis utilizes the curriculum network to capture the dependencies between frames concerning the predictive behavior as prescribed in the neural network. This model enables supervised learning, where the data is labelled with various features and classified into suspicious or nonsuspicious activity. This classification model using CNN explained how to identify a student’s behavior for further prediction based on their posture. Since the video frames are collected for a large data set, annotation of the video is required to identify the trained patterns. The data labelling and training, in this case, are done with the help of the ResNet model, which contains several postures related to anomalous behavior by a student. To develop the model, this study makes use of OpenCV for processing the review and extracting the frames, PyTorch for building the model and training, OpenPose for the estimation of the pose acquired by the student during the examination, and MediaPipe as the backbone of posture detection and prediction.

3.3 Feasibility study

In the feasibility study, the implementation of an IoT-enabled proctoring environment is evaluated. Edge computing devices are integrated with ML algorithms to ensure that decision-making is automated and predictions are provided, facilitating the proctoring of real-time examinations in a university environment. The integration of the system deployed for surveillance, along with the existing infrastructure, will be helpful for handling digital proctoring and managing the prediction of words during assessment execution. The primary objective of the entire study is to identify the security, privacy, scalability, and assessment of examinations with the help of IoT devices and ML models.

4.1 Feature extraction using analytical technique

Once the video is captured, it is necessary to split the video into individual characters and frames for further analysis using mathematical tools. The use of feature extraction from the images and the frames is required to predict the behavior of the concern. Various tools and techniques are helpful for the prediction of behavior from the video frames acquired in the previous stage.

4.2 Behavior classification

After identifying the features extracted from the previous stage, these features must be classified into several different classifiers. The classifiers are generated using various techniques such as logistic classification, support vector machines (SVMs), or neural networks. Once the classifiers are set, the features generated from the previous stage can be used to identify the activity and predict the behavior.

4.3 Model optimization

After reducing the features and classification, a model can be trained to identify and predict a specific behavior that requires optimization techniques (Abdelhamid et al., 2023). The main feature of this optimization is to ensure that there is the least loss for any cross-entropy that might occur (Akram et al., 2024; El-Kenawy et al., 2024). There are several algorithms to identify and minimize the loss with the help of various features as per the equation below:

To train a predictive model, minimize a loss function: $w_{t+1}=w_t-\text{η}\nabla L\left(w_t\right)$

Here:

W: Value of parameter for the model at any step (t).

$\eta$: Rate of model learning.

$\nabla \text{L}\left({\text{w}}_{\text{t}}\right)$: Loss function gradient in contrast with ${\text{w}}_{\text{t}}$.

The current value of any specific parameter and the model’s learning rate (LR) can yield a gradient of loss for the parameters under consideration. The prediction function represented in this study is elaborated to identify the behavior of suspicious activity with the help of the patient, as given below. Integrating past probabilities with reference from the features selected and behavioral classification can determine the future predictive probability. There are several cases in which no specific irregular misconduct activity has been identified. In this case, the probability cannot be defined and remains unaltered.

4.4 Analytical treatment for the proposed model in prediction:

Case I: If t = 0 $\int P(X\_\left(t+1\right)=\int s_j$

Case II: if t = t > 1 $\int P(X\_\left(t+1\right)=\int P\left(X_t\right)$

Complete value is the Summation of the results: $\begin{array}{c}P(X)={{\displaystyle \sum }}_{i=1}^n{s}_n\\ P(X)={{\displaystyle \sum }}_{i=1}^{n}P\left(X_n\right)\end{array}$

However,

Case I: for $P\left(X\right)$ at t > 0 $P\left(X_t\right)=s_n$

Case II: for t = 0 $P\left(X_t\right)=0$

Case III: for t =1 $P\left(X\right)=P\left(X\right)$

Split Probability Approach for the proposed architecture: $\int P(X\_\left(t+1\right)=\int P\left(X_{1t}\right)+\int P\left(X_{2t}\right)+\int P\left(X_{3t}\right)+\int P\left(X_{4t}\right)$

The integration of complete probability analysis is split into several consecutive integrals. The complete response of the probability is achieved after the summation of the individual integral values. $P\left(X\right)={{\displaystyle \sum }}_{t=1}^{n}P\left(X_{nt}\right)$

The individual probability may vary for any section; however, for multiple sub-sections, it becomes easy to manage the probability analysis. The time sections can be split into various intervals where the sum of the intervals is achieved by successive addition.

Consider, total probability function P (X) is defined over the interval t such as:

Case I: 0 < t < n

For any n that belongs to the interval [0, n], we can expect the value to be distributed over the interval. However, if we can split the interval, the accuracy of the probability for tracking any strange behavior during proctoring will be more accurately observed.

Case II: t > 1

For any value of t, more than 1, it is possible that the proctoring has started, and the analysis has to be conducted. But the same rule for case 1 applies to this situation also.

Case III: [a < t < d]

In this case, the probability analysis can split the integral value over spread intervals for t as: $\int P(X\_\left(t+1\right)=\int P\left(X_a\right)+\int P\left(X_b\right)+\int P\left(X_c\right)+\int P\left(X_d\right)$

The split intervals can be created as per the analysis of the sections under consideration, $\begin{array}{l}P\left(X\right)={{\displaystyle \sum }}_{i=1}^n\left(s_{1n}+s_{2n}+s_{3n}+s_{4n}\right)\\ P\left(X\right)={{\displaystyle \sum }}_{i=1}^{n}P\left(X_n\right)\end{array}$

The analytical treatment for probability behavioral prediction refers to identifying the probability function P(X) that can be used to predict suspicious activity. The model is trained for various timestamp values ranging from [0 < t < n]. In Fig. 5, it is explained that the determination of probability for the inappropriate posture is done in a single direction. However, with the use of the split probability function, determination becomes more appropriate, and accuracy is increased.

5.1 Experiment setup

We have tested the system on an Intel i7 machine with 32GB of RAM and an 8GB graphics card integrated on the motherboard. The video frames are collected in real-time using a Raspberry PI 4B model in the examination room and processed simultaneously with the MediaPipe and OpenPose libraries in Python on the IoT device. The results are submitted to the prediction-enabled machine for further analysis using the predictive equations represented in Section 4 above. The complete statistical model is developed in terms of temporal analysis for behavioral predictions. In this model, we conducted the study using two different techniques in real-time. Initially, the collection of data frames is done with the help of real-time monitoring using IoT devices. We used OpenCV libraries to detect any suspicious motion activity in real-time. After collecting the video frames from the IoT devices, the data is submitted to the prediction system. The predictive behavior of humans and the temporal analysis are performed with the help of Mediapipe and OpenPose.

5.2 Evaluation metrics

In this approach, we have used supervised learning for the training of the datasets. The supervised learning data sets were evaluated for the accuracy, precision and recall, F1 score, and the confusion matrix for the analysis of the performance of the model trade. For further analysis and training of the model, we took 500 frame samples. Out of the total frames, 300 were considered for the division related to the training, testing, and validation. We have implemented the three models, EfficientNet, MobileNet, and SPNASNET, respectively, for experimental purposes. The results for the evaluation are represented with the help of the graphs below:

Table 2. and Fig. 6 refer to the preparation of the dataset based on the extracted frame from the experimental input. The training and testing input files are categorized as "Clear" and "Suspicious". In both cases, validation is performed using the minimum values extracted from the input frames of the video during proctoring from the source input.

Close

Fig. 6.

Dataset training and analysis frame split.

Export to PPT

In Table 3, the EfficientNet model performance has been reported. The complete set of input data is used to perform experiments, where Training loss, Validation Accuracy, and testing accuracy are calculated for epochs of variable size. The greater or smaller size of epochs might yield some inappropriate results. Thus, the size is considerable instead of the higher-end or lower-end.

Fig. 7 above represents the performance of the EfficientNet model on the prescribed architecture for the probability of expression to identify inappropriate behavior of the student during an examination. We considered four major factors in the performance evaluation of the complete task. The dataset is split into three series to experiment with the multiple-interval probability analysis.

Close

Fig. 7.

EfficientNet model performance – I (LR = 0.15).

Export to PPT

Table 4 and Fig. 8 represent the performance of the prescribed probabilistic approach in architecture. The model performance is evaluated on the dataset with the variable value of LR (0.0025) in this case.

Close

Fig. 8.

EfficientNet model performance – II (LR = 0.0025).

Export to PPT

A similar experiment for the proposed model is conducted using the MobileNet dataset model, and its performance is evaluated based on the training loss, validation accuracy, and testing accuracy, as shown in Table 5. The complete experiment was conducted in two phases with the split probability determination algorithm as per the equations proposed above.

Fig. 9 illustrates the model characteristics of the MobileNet model applied to the proposed architecture in this study. A comparative evaluation of the EfficientNet and MobileNet models proposes the idea of integrating different models and evaluating the proposed architecture.

Close

Fig. 9.

MobileNet model performance – I (LR = 0.148).

Export to PPT

Table 6 and Fig. 10 show the variable performance of the MobileNet model based on the LR value of 0.0031. The results achieved on this level are more significant compared to the EfficientNet model.

Close

Fig. 10.

MobileNet model performance – II (LR = 0.0031).

Export to PPT

Table 7 and Fig. 11 represent the SPNASNET model performance evaluation with an LR value of 0.0006 and Table 8 with Fig. 12 represents the SPNASNET model performance with LR value of 2E-13. The results are almost identical or slightly variant compared to the EfficientNet and MobileNet models.

Close

Fig. 11.

SPNASNET Model Performance – I (LR = 0.0006).

Export to PPT

Close

Fig. 12.

SPNASNET model performance – II (LR = 2E-13).

Export to PPT

After the complete training of the model from the total of 500 frames, we used the remaining 200 frames for testing. In this case, the model reported false and true values depending on the eligibility of model to identify any suspicious activity.

The efficiency of the proposed framework was measured using the true positive rate (TPR), true negative rate (TNR), positive prediction value (PPV), and negative prediction value (NPV). These values are calculate as per the equations below and represented in Table 9: $\begin{array}{l}TPR=\frac{TP}{(TP+FN)}\\ TNR=\frac{TN}{(TN+FP)}\\ PPV=\frac{TP}{(TP+FP)}\\ NPV=\frac{TN}{(TN+FN)}\end{array}$

With the help of the equations stated above, we calculated the factors as shown in Table 10, to evaluate the performance of the model trained in this study. The complete model results in an accuracy level of 0.88 which recommends that the model is very helpful for further processing. Due to the limitation of computational capacity, the testing was performed with the help of only 90 frames extracted from the videos that were collected by the IoT devices. These frames were traded and tested with the F1 score of 0.86 which is sufficient for less amount of tested data. It is further worth mentioning that the precision of the model received post 0.73 which suggests that there is a high scope of acceptance for this model. $\begin{array}{c}Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\\ Precision=\frac{TP}{TP+FP}\\ Recall=\frac{TP}{TP+FN}\\ F_1=\frac{2\cdot Precision\cdot Recall}{Precision+Recall}\end{array}$

We calculated the parameters such as accuracy, recall, precision and F1 score for the model suggested in this study based on the observations of the confusion matrix values. The results have been represented in Table 10. They correspond to good output from the existing computational resources input.