Earthquakes magnitude predication using artificial neural network in northern Red Sea area

2.1 Earthquake forecasting

Su and Zhu (2009) have studied the relationship between the maximum of earthquake affecting coefficient and site and basement condition. They have proposed a model based on artificial neural network. Itai et al. (2005) have proposed a multilayer using compression data for precursor signal detection in electromagnetic wave observation. Plagianakos and Tzanaki (2001) have applied chaotic analysis approach to a time series composed of seismic events occurred in Greece. After chaotic analysis, an artificial neural network was used to provide short term forecasting. Panakkat and Adeli (2007) have presented new neural network models for large earthquake magnitude prediction in the following month using eight mathematically computed parameters known as seismicity indicators. Lakshmi and Tiwari (2009) have studied a non-linear forecasting technique and artificial neural network methods to model dissection from earthquake time series. They have utilized these methods to characterize model behavior of earthquake dynamics in northeast India. Kulachi et al. (2009) have studied the relationship between radon and earthquake using an artificial neural networks model. Panakkat and Adeli (2009) have proposed a new recurrent neural network model to predict earthquake time and location using a vector of eight seismicity indicators as input.

Negarestani et al. (2002) have proposed layered neural networks to analysis the relationship between radon concentration and environmental parameters in earthquake prediction in Thailand which can give a better estimation of radon variations. Ozerdem et al. (2006) have studied the spatio-temporal electric field data measured by different stations and the regional seismicity. They have proposed neural network for classification and provide an accurate earthquake prediction. Ni et al. (2006) have used PCA-compressed frequency response functions and neural networks to investigate seismic damage identification.

Zhihuan and Junjing (1990) have considered earthquake damage prediction as fuzzy-random event. Jusoh et al. (2008) have investigated the existence of the ionospheric Total Electron Content (TEC) using GPS dual frequency data. The variation of TEC can be considered as an anomaly a few days or hours before the earthquake. Shimizu et al. (2008) have proposed using recursive sample-entropy technique for earthquake forecasting, where they have used the earth data based on VAN method. Turmov et al. (2000) have presented models of predication for earthquakes and tsunami based on simultaneous measurement of elastic and electromagnetic waves.

2.2 Overview of artificial neural network

Artificial neural network, usually called neural network, is a mathematical model that simulates the structure and functionality of biological neural networks. Neural network is a network of nodes – also called neuron – connected by directed links, where every link_i,j that connects node_i to node_j has a numeric weight w_i,j associated with it, also there is an activation function f associate with every node_j. The weight determines how much the input contributes to and affects the result of the activation function. The activation function will be applied to the sum of inputs multiple by the weights for all incoming links, as shown in Fig. 1. Any weight is called a bias weight whenever its input is a fixed value of −1, which can be used to shift the activation function regardless of the inputs.

Close

Figure 1

A simple mathematical model for a neuron.

Export to PPT

The activation function – also called transfer function – defines the output of that node given an input or set of inputs. The activation function needs to be nonlinear, otherwise the entire neural network becomes a simple linear function. Nonlinear activation function allows the neural network to deal with nontrivial problems using a small number of nodes. Neural Networks support a wide range of activation functions such as; step function, linear function, sign function, and sigmoid function, as shown in Fig. 2. The sigmoid function is considered the most popular activation function mainly for two reasons – firstly, it is differentiable so it possible to derive a gradient search learning algorithm for networks with multiple layers, and secondly, it is continuous-valued output rather than binary output produced by the hard-limiter.

Close

Figure 2

Some common activation functions.

Export to PPT

There are two main types of neural network structures; feedforward networks and recurrent networks. A feedforward network is acyclic network so it has no internal state other than the weights themselves. In the other side, a recurrent network is a cyclic network so the network has a state since the outputs are feeded back into the network. As result of that, recurrent networks support short-term memory unlike feedforward networks.

Feedforward networks are usually arranged in layers, where each neuron receives inputs only from the immediately preceding layer. Feedforward networks can be classified into: single layer feedforward neural networks – also called perceptron – and multilayer feedforward neural networks.

For perceptron, all the inputs connected directly to the output neurons and there is no hidden layer. In this manner, every weight affects only one output neuron. When there is a single output neuron, the network is called single perceptron, as shown in Fig. 3.

Close

Figure 3

Shows the design of the neural network.

Export to PPT

Perceptron is able to represent any linear separable function such as AND, OR, and NOT Boolean functions and majority function. On the other hand, perceptron is not able to represent XOR Boolean function since XOR is not linear separable. Despite perceptron limitations, perceptron has a simple learning algorithm that will fit the perceptron to any linearly separable function.

Hidden layers feed forward neural network – sometimes called backpropagation network – can be used to solve non-linear problems such XOR and many more difficult problems. Hidden layer feedforward neural network currently accounts for 80% of all neural network applications. A simple version of hidden layer feedforward neural network is when a single hidden layer is added and the discrete activation function is replaced with a nonlinear continuous one, as shown in Fig. 4.

Close

Figure 4

A 3-layer (single hidden layer) feedforward neural network.

Export to PPT

The biggest challenge in this type of network was the problem of how to adjust the weights from input to hidden units. Rumelhart et al. (1986) have proposed back propagation learning algorithm, where the errors for the neurons of the hidden layer are determined by back-propagating the errors of the neurons of the output layer.

There are many applications for neural networks such as marketing tactician, computer games, data compression, driving guidance, noise filtering, financial prediction, hand-written character recognition, pattern recognition, computer vision, speech recognition, words processing, aerospace systems, credit card activity monitoring, insurance policy application evaluation, oil and gas exploration, and robotics.

5.1 Data acquisition

For any prediction model, the data should be collected from reliable single or multiple sources. There are many public and private organizations which provide earthquake catalog and database for earthquakes over the world. Most of these sources are available over the Internet and provide advance search capabilities such as Advanced National Seismic System (ANSS), National Earthquake Information Center (NEIC), the Northern California Earthquake Data Center (NCEDC), and others. We have gotten the data from the Northern California Earthquake Data Center (NCEDC) where we limit search area with latitude between 28 and 32 and longitude between 32 and 36. Also we have limited collected data for every seismic event to the following; year, month, day, time, latitude, longitude, magnitude, depth. Although, earthquake catalog may provide more information about each seismic event, we believe that other information is irrelative to our prediction model and target. Furthermore, the process of building neural network model is time consuming since there are a large number of network configurations that need to be trained and tested. Without limiting our domain by classifying the data into relative and irrelative, the process of building our system will not be achievable within a reasonable time.

5.2 Preprocessing

After data acquisition, the data should be preprocessed which is mainly filtering first and then reformatting. Filtering is necessary to remove noisy and meaningless data. It also removes each seismic event in case part of its data is missing. However, if the missing part will not be used in some network configurations, the event will not be excluded for these configurations only. In other word, filtering criterions rely on the required data of network configurations for the next two phases. Even after filtering, the data may not be suitable for use directly either because; it has too much information and details that may drive our forecasting system away from the intended goal, or the critical information is hidden and needs to be cleared for features extraction. In order to deal with these two issues, reformatting is necessary to make the data ready for the next phase. For example, the latitude and longitude format is not suitable for the next phase because they provide a high degree of accuracy for the location – 100 × 100 units for each latitude or longitude – which is not necessary. In fact, it complicates the training process and drives the neural network to focus on these small details rather than magnitude prediction. In order to avoid that, we divide area of interest into 16 × 16 tiles. The latitude and longitude are replaced by the tile location. This reduces the number of possibilities by a ratio of 99.84%.

5.3 Features extraction

Feature Extraction is an important process which maps the original features of an event into fewer features which include the main information of the data structure. Feature extraction is used when the input data is too large to be processed or the input data have redundancy. In this manner, feature extraction is a special form of dimensionality reduction and redundancy removal. Features extracted are carefully chosen to allow the relevant information in order to perform the desired task.

We have identified a list of candidate features which can be tested in the next phase to eliminate poor quality sets and determine the best candidates. The features are the following:

• Earthquake sequence number: This is used to keep reserving the order of earthquake events in our study area. It will be used instead of the year, month, day, and time for reasons. First, the date and time is more complex data type for our neural network model than using the sequence number. Second, our focus is to predict the magnitude of the next earthquake rather than the time of the earthquake which means keeping the date and time is not necessary and the sequence number is sufficient for our target. Third, although the date and time logically may help to provide more accurate prediction, we found out after many experiments the sequence is easier to learn for the neural network configuration that we propose.

• Tile location: This feature identifies the location in which the earthquake events occurred. As we mentioned earlier, we divide the study area into 16 × 16 tiles instead of latitudes and longitudes. In this manner, each tile is represented by two numbers with a range of value between 0 and 15.

• Earthquake Magnitude: The main feature which should be considered since it represents the fluctuation of the magnitude.

• The source Depth: The source depth can be considered as a feature however relationship between this feature and the target magnitude (predicted magnitude) will be studied during the next phase.

It is very important to notice that, we study this feature over a predetermine set of neural network configuration. Based on the training and testing result we cancel some of these features and keep the other. This does not mean the selected features are the best ever for any neural network model.

5.4 Neural network training and testing

In this phase, neural network is constructed with a different set of configurations, then trained, and tested. We have used different feed forward neural network configuration with multi-hidden layers for training and testing. We consider the following parameters during neural network configuration and construction:

• Hidden layer: We have trained and tested feed forward neural network with one hidden layer, two hidden layers, and three hidden layers.

• Transfer function: Since we used Neural Network Toolbox 6 edition for Matlab 2008b as training and testing software tool, we have considered three Matlab transfer functions which are Logsig, Tansig, and Purelin.

• Delay of input data: The delay is the number of consecutive earthquakes that are entered to the neural network together as single input row. We are doing like this because feed forward neural network deals with each input row as a single entity. As a result of that neural network does not draw any relationship between different rows, so we enter multiple consecutive events together to allow feed forward neural network to figure out existing relationship between the events in each input row separately. We have considered different delay value from 0 to 10.

6.1 Benchmarks

As we mentioned before, to our best knowledge our prediction model is the first prediction model in the northern Red Sea area. Due to that, we need to compare the performance of our model with other methods that we propose too.

• Normal distributed random predictor: assuming that earthquakes magnitude distribution is normal, we used the normal distributed random predictor to generate random normal magnitude distribution on and we tested this method with different standard deviation, then we fixed the mean magnitude value (Fig. 16).

• Uniformly distributed random predictor: This method is similar to normally distributed random predictor however it follows uniform distribution. We also evaluated this method using different range value (difference between maximum and minimum possible value) as shown in Fig. 17.

• Moving average predictor: Moving average is used to analyze a series of values by creating a series of averages of different subsets of these values. A moving average may also use unequal weights for each value in the series to emphasize particular values. A moving average is commonly used with time series data to determine trends or cycles, for example in technical analysis of financial data. We have used a simple moving average which is the unweighted moving average. The simple moving average of period n is define as following: $$\text{Simple moving average}(i)=\frac{v_{i-1}+v_{i-2}+\cdots +v_{i-n}}{n}$$

• We have evaluated this method over the earthquake events as shown in Fig. 18 and Table 2.

• Curve fitting methods: In which several statistical methods for curve fitting are used such as linear regression, quadratic regression, cubic regression, and 4^th to 6^th degree regression. In all these methods, we try to construct a curve, or mathematical function, that has the best fit to the series of earthquakes magnitudes in the northern Red Sea area. The performance of these methods is shown in Table 2.

Close

Figure 16

Error estimation for normal distribution random predicator.

Export to PPT

Close

Figure 17

Error estimation for uniform distribution random predicator.

Export to PPT

Close

Figure 18

Error estimation for moving average predicator.

Export to PPT

6.2 Performance metrics

There are many performance metrics that can be used to evaluate and compare between forecasting methods such as time complexity, space complexity, and accuracy of the model. Although time – sometime space – is very important during training stage and may considered an obstacle, in this paper we focus on accuracy rather than time and space complexity because training stage is performed once. We have used two accuracy metrics to evaluate the performance over our model compared with other methods. These metrics are the following:

• Mean absolute error (MAE): It is a quantity used to measure how close predictions are to the target outcomes. The mean absolute error is defined as following: $$\text{MAE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n}|{\text{error}}_i|$$

• Mean squared error: It is a quantity used to measure the average of the square of the error from the target outcomes. The mean squared error is defined as following: $$\text{MSE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\text{error}}_i^2$$

• In MSE, the larger error will contribute more to the MSE value than small error. In other word, MSE penalizes larger errors and is generous toward small errors.

6.3 Performance results

In this section, we present the performance results for our neural network model compared to other forecasting methods. We have plotted the mean square error (MSE) and mean absolute error (MAE) for normally distributed random predictor as shown in Fig. 16, while Fig. 17 shows the mean square error (MSE) and mean absolute error (MAE) for uniformly distributed random predictor. We notice that, both methods start with the same performance since they represent the mean magnitude value at the beginning. However, normally distributed random predictor performs worse than uniformly distributed random predictor when the standard deviation value increase. These two figures illustrate clearly that earthquake magnitudes do not follow these statistical distributions which show how hard it is to predict magnitude using statistical tools. Furthermore, the two figures show that earthquake magnitude is not independent from pervious earthquake events so we cannot deal with earthquake events separately.

On the other hand, moving average performs much better than normally distributed random predictor and uniformly distributed random predictor as shown in Fig. 18. The figure shows the mean square error (MSE) and mean absolute error (MAE) for moving average method over different interval. The figure shows that moving average performs best when the interval value is four, so the last four earthquake magnitudes are enough to provide a best accuracy for moving average.

Other statistical fitting methods have been tested and their results are shown in Table 2. Similar to normally distributed random predictor and uniformly distributed random predictor, these statistical methods could not do better than moving average.

Finally, the performance of our neural network model is shown in Tables 3 and 4. Table 3 shows the accuracy for the top ten network configurations in terms of mean square error (MSE), while Table 4 shows the top ten network configurations in terms of mean absolute error (MAE). For the two tables, the second network configuration in Table 3 and the first one in Table 4 are the same. So that a network with the following configuration can be considered a better choice in terms of MSE and MAE together:

• Delay = 3.

• Number of neurons in the first hidden layer = 3.

• Number of neurons in the second hidden layer = 2.

• No third hidden layer.

• The transfer function is Tansig.

For the results shown in this section, we notice that neural network has better capabilities to model data that has nonlinear and complex relationships between variables and can handle interactions between variables. Neural networks do not present an easily-understandable model. They are more of “black boxes” where there is no explanation of how the results were derived. The performance results show there is a great potential of using neural network for earthquake forecasting in northern Red Sear area, which needs to be investigated more.