Fault Prediction Model BasedPhase Space Reconstruction

Fault Prediction Model Based RQ Phase Space Reconstruction and Least Squares

Support Vector Machine V

Yunhong GAO

College of Automation Engineering Nanjing University of Aeronautics and Astronautics

Nanjing, China

ggyyhh001@https://www.360docs.net/doc/4b13821579.html,

Yunhong GAO,Yibo LI

College of Automation Engineering Shenyang Institute of Aeronautical Engineering

Shenyang, China

lyb20040612@https://www.360docs.net/doc/4b13821579.html,

Abstract—C ombining phase space reconstruction theory and least

squares support vector machines (LSSVM) method, a novel fault prediction model is proposed in this paper. The model reconstructs phase space for fault characteristics time series of the system and fit the nonlinear relationship of phase point evolution by use of

least squares support vector machines according to the laws of phase space evolution. Fault prediction model based on gyroscope drift time series is established for single-step and multi-steps prediction compared with RBF neural network prediction results.

The results show that phase space reconstruction method can effectively determine the input and output vectors of prediction model, and in the case of limited samples, the fault prediction model established by the least squares support vector machine has

better accuracy and stronger generalization ability.

Keywords- phase space reconstruction; least squares support vector machines; fault prediction model; gyroscope drift

I.I NTRODUCTION

With fast development of high technique fields such as

aviation, aerospace and military field, a large number of

complex systems have became major technical equipment.

Not only these kinds of systems cost more but also

individual fault often bring chain reaction and finally, the

tasks process failed and the whole system paralysis. So it is necessary for these systems that the faults can be diagnosed

true and also inferior trend of these systems can be early

predicted, namely, fault prediction [1]. Currently, research on

fault prediction of systems is mainly divided into model-

based fault prediction and data-based fault prediction [2].

Model-based prediction method requires the system structure and the internal operation of law is well aware of. In this

method, accurate mathematical model of systems or

equipment can be set up and a typical representative of this

method is fault prediction method based on the Kalman filter [3]. In data-based fault prediction systems, complex mathematical model of the systems do not need to be set up and the prediction method has more application in these

areas such as the classic time-series methods, the gray model

theory, hidden Markov models, neural networks, support

vector machines. Support vector machine is a machine

learning algorithms based on statistical study theory and takes structural risk minimization principle for the foundation similar in form to the multilayer feed-forward networks. It can automatically resolve the problem of network structure, and stands out in aspects of small sample, nonlinear, high-dimensional problem and the generalization ability [4-5].

Least squares support vector machines (LSSVM) uses solutions of linear equations to replace the solution of quadratic programming problem of the standard SVM and the most advantage of the method is the training speed is much larger than the standard support vector machines. It is widely used in modeling, regression and parameters prediction and suitable for engineering applications [6]. However, the setting of input vector dimension are often according to the ways of experience or try when the prediction model is set up by use of support vector machine and certainly, the ways lack theoretical basis. In this paper, the fault prediction model based on the phase-space reconstruction and least squares support vector machine is proposed. The model uses the least squares support vector machines as a predictor to fit in the evolution trends of points in phase-space in the basis of phase space reconstruction.

II.P HASE SPACE RESTRUCTION THEORY AND LSSVM

TIME SERIES PREDICTION

Takens theorem considers evolution of any component of the system is decided by other components interacting with this component, therefore, the information of relevant component imply in the development process of this component, so the original rules of the system can be extracted and restored from a group of time-series data of a certain component. The one-dimensional time series is embedded to multi-dimensional phase space through reconstruction and the new system with same dynamic characteristics as original system can be obtained through the selection of a suitable embedding dimension(m) and time delay(τ)[7-8]. For time series, after the phase point by phase space reconstruction is described as follows:

τ

?

?

=

=τ

+

τ

+

τ+

)1

m

(

N,

,2,1

t

),

x,

,

x,

x,

(x

X1)-(m

t

2

t

t

t

t

"

"

(1)

The usual method of selecting time delayτincludes auto-correlation function method, multiple correlation function method, mutual information method. Embedding dimension m is calculated by the methods of GP algorithm, pseudo-

2009 Ninth International Conference on Hybrid Intelligent Systems

nearest-point method, correlation integral method and Cao method.

The prediction of time series is to use τ??)1m (N points X t and τ++=1)-(m 1t t x Y in R m to make up the sample (X t ,Y t ). We can use local method, ANN or SVM to estimate mapping f by these samples and build prediction model for one step or multi-step prediction. The basic idea of modeling and prediction by support vector machine is to build a non-linear mapping φ and map the data X to high dimensional feature space to realize the linear regression. In this way, the non-linear regression in low dimensional feature space is changed into linear regression in high dimensional feature space. The regression function is shown as in (2).

b ))x (,()x (f +φω= (2) The 2-norm of least squares support vector machine i ξ is lost function and equal sign is used to substitute the inequality in standard support vector machine constraint condition. Equation (3) is used to calculate w and b by sample data.

()N

,...,2,1i ,b x y .t .s ,

2121min i i T i n 1

i 2i T =ξ++φω=ξγ+ωω|

=

(3)

In order to get the solution easily, lagrange multiplier is introduced and from the condition KKT, we can find that:

()()0y b x ,0,x i i i T N

1i N

1

i i

i i i i =?ξ++φωγξ=α=αφα=ω|

|

== (4) The LSSVM regression function can be got by ()()b x ,x x y N 1

i i i +Ψα=|

= (5) Where ()()()

j T i j i x x x ,x φφ=Ψ, ()?Ψ is kernel function which fulfills the Mercer condition. The several common kernel functions are polynomial kernel functions, sigmoid kernel functions and radial basis function kernel functions. The research shows that the effect of gauss kernel functions is better than other kernel functions, when it is lack of progress prior knowledge. The gauss kernel function is shown as in (6), ? is perception variable, and it can be selected freely.

()

???1

·¨¨¨?§σ??=Ψ2j

i j i x x exp x ,x (6)

III. F AULT TREND PREDICTION MODEL BASED ON PHASE

SPACE RECONSTRUCTION AND LSSVM

A. Construction of Fault Trend Prediction Model

Given a time series of fault trend {x t }?t=1,2,…,N . Utilize the theory of phase space reconstruction to get τ??)1m (N sample couple composed the points X i and Y i values in m-dimensional space. The data before N TR are used to be training samples to estimate mapping f and the

last TR N -)1m (N τ??

are used to be testing samples to evaluate the effect of prediction [9]. After using the training samples to train LSSVM, the Regression Function is achieved as followed:

()b X ,X Y TR

N 1

i t i i t +Ψα=

|=?where t=1,…,N TR .

One step prediction model is:

()b X ,X Y TR

TR

TR N 1

i 1N i i 1N +Ψα=

|=++?

where }x ,,x ,x {X 1)-m (1N 1N 1N 1N TR TR TR TR τ++τ++++=".

L-steps prediction model is:

()

b X ,X Y TR

TR TR N 1

i L N i i L N +Ψα=|=++?

where }x ,,x ,x {X 1)-m (L N L N L N L N TR TR TR TR τ++τ++++=".

When we got the prediction result of fault prediction model, we can compare the fault thresholds. The fault threshold is always fixed in experience, and if it is beyond the threshold, we can judge that there are faults. The

prediction model of least squares support vector machine

only needs to reconstruct phase space parameters and

support vector machine parameters by optimization

criterions. And the network structure is fixed by support

vector machine algorithm automatically.

B. Selection of Fault Prediction Model Parameters

The main parameters that the Prediction model need to select is kernel function and its parameters, embedding dimension of time series m, delay time τand regularization parameter γof least squares support vector machines. In this paper, Gaussian kernel function is used. The kernel function parameters and the regularization parameters of LSSVM are made by use of chaotic particle swarm optimization and then are selected automatically according to root-mean-square error (RMSE) of the prediction results. In order to make the reconstructed phase space reflect the motion features of the system more enough, the key step of phase space reconstruction is selecting embedding dimension m and the size of delay time τ properly. If τis too much, any two adjacent delay coordinates of the time series would be irrelevant, which does not reflect the characteristics of the entire system; choice is too small, coordinates of any two adjacent points of the time series is very close, that will lead to data redundancy. Auto-correlation function can provide

the signal delay with its own the measurement from redundancy to irrelevant. Generally, the delay time that correlation function value is zero firstly or close to zero is determined as the time delay. Embedding dimension m is ensured by the use of false neighbor method. The basic principle is when the dimension from m is added up to m +1, inspect the adjacent points of the rail line which is the real point, which is a false neighbor, when the proportion of false neighbor is no longer reduced with the dimension increase, the embedding dimension can be confirmed [7].

IV. EXPERIMENTS

A. Data Preparation

Gyroscope is the core component of inertial navigation system and its reliability affects the performance of inertial navigation system directly. If we can judge the performance conditions of gyroscope earlier achieve early prediction and debug fault timely, it plays an important role in ensuring the completion of combating mission [10]. The random drift error of gyroscope is the major factor of affecting the accuracy of gyroscopes and the key parameters of measuring gyroscope normal or not and its reliability. So in this paper, we select the random drift error of gyroscope as fault characteristics in evaluating performance reliability to set up fault prediction model.

We collect a total of 324 random drift data as origin data in this paper through a certain rate gyroscope drifts testing. In order to reduce the modeling error, at first step we should normalize the original data and then reconstruct phase space according to Takens theory. In another word, m-dimensional phase space is reconstructed by one-dimensional time series, so we can get the study samples for prediction by deformation of original one-dimensional time. The input matrix and output matrix are described as follows after phase space is reconstructed:

??????o??????a=τ+τ?ττ+τ+τ+τ+1-N 1)-(m -N 11)-(m -N 1)-(m 222

1)-(m 111

x x x x x x x x x X #

##""? ?

?????o

??

???

?a=+τ+τN 31)-(m 21)-(m x x x Y #

. Where τ= 6, m = 7. Total of 287 group 7- dimension data are obtained, select 255 data sets as training data, and the remaining 32 sets as test data. According of training data, the optimal values of the regularization parameter γand Gaussian kernel function parameter ? are set 733.8 and 2000 respectively using of chaotic particle swarm algorithm. In order to illustrate further the effectiveness of the algorithm, select the RBF neural network to compare, where input layer is 7, the output layer is 1, function parameters is selected the best value for 1600 in accordance with test method.

B. Prediction Results and Analysis

Take 255 sets of data as training samples to set up the prediction model of gyroscope random drift. We use the 32 data sets back as a generalization test samples to predict fault trend of the gyroscope. We take RMSE error as principle, and construct of the fault prediction model using of the aforementioned parameters. The model fitting error and the single-step prediction error are 0.1211 and 0.070687 respectively. The specific results are shown as Fig. 1 and Fig. 2. Making use of RBF neural network to model and predict, model fitting error and forecast error are 0.093555 and 0.0753 respectively. Test results show that the generalization ability of LSSVM model is a little super than RBF neural network model.

For failure prediction, the ahead steps of fault prediction, which is the multi-steps failure prediction capability, is a key index of appraising the model quality. We use above data and model to check multi-steps failure trend prediction ability of models. The multi-steps prediction results are shown in Fig. 3 and Fig. 4 as follows; prediction errors (RMSE) are 0.58556 and 1.7199 respectively. Table 1 gives the Multi-steps prediction results and relative errors of LSSVM model and RBF model in the range of 10 steps, where prediction error of LSSVM model is in the range of 5% (4.2754%) for 6-steps, but the prediction error of RBF model is in the range for 2-steps. The results show that LSSVM failure prediction model in the multi-step fault trend prediction has the smaller prediction error and the stronger generalization ability.

Figure 1.

Fitting result based on LSSVM model

Figure 2. Single step prediction result based on LSSVM model

TABLE I.

M ULTI _STEPS PREDICTION RESULTS

1 2 3 4 5 6 7 8 9 10 ?

1.8736 1.757 1.612 1.4461 1.2652 1.0784 0.89153 0.71366 0.55223 0.41472 LSSVM

乘?

1.8615 1.7403 1.5943 1.4248 1.2358 1.0323 0.82019 0.60854 0.40246 0.20794 ? ? (%) 0.6453 0.9475 1.099 1.4692

2.3276 4.2754 8.0017 14.729 27.122 0.10178 RBF

乘?

1.8767 1.7947 1.7068 1.6011 1.4638 1.284 1.0583 0.78927 0.47144 49.861 ? ? (%)

0.1669 2.1457 5.8825 10.722 15.695 19.073 18.708 10.594 14.63 75.458

Figure 3. Multi-steps prediction result (LSSVM)

Figure 4. Multi-steps prediction result (RBF)

V. C ONCLUSION

In this paper, phase space reconstruction theory is used to ensure the fault features time series delay and embedding dimension and obtain the input and output vectors of support vector machine. The least squares support vector machine is utilized to set up fault trend prediction model for gyroscope fault trend data regression prediction. The single-step and

multi-steps prediction results show that the fault prediction models according to phase-space reconstruction and least squares support vector machines, has a higher prediction accuracy and generalization ability.

R EFERENCES

[1] ZHANG Jun-feng, HU Shou-song, “Nonlinear time series fault

prediction based on clustering and support vector machines,” Control Theory & Applications , vol.24, Jan.2007,pp. 64-68.

[2] Andrew K.S. Jardine, Daming L, Dragan, “A review on machinery

diagnostics and prognostics implementing condition-based maintenance”, Mechanical Systems and Signal Processing, vol.20,Jul.2006,pp: 1483–1510.

[3] Chen M Z, Zhou D H, Liu G P, “A new particle predictor for fault

prediction of nonlinear time varying systems”, Developments in Chemical and Engineering and Mineral Processing, vol.13,2005, pp: 379-388.

[4] MA Xiao-xiao, HUANG Xi-yue, CHAI Yi, “Fault Process Trend

Prediction Based on Support Vector Machines”, Acta Simulata Systematica Sinica , vol.14, Nov. 2002, pp:1548 - 1551.

[5] THISSENA U,BRAKELA R,”Using support vector machines for

time series prediction”, Chemo metrics & Intelligent Laboratory Systems, vol.6, Sep.2003, pp: 35-49.

[6] Baylar A, Hanbay D, Batan M, “Application of Least Square Support

Vector Machines in the Prediction of Aeration Performance of Plunging Overfall Jets from Weirs”, Expert Systems with Applications ,2008, doi: 10.1016/j.eswa.2008.10.061.

[7] HAN Min. Prediction Theory and Method of Chaotic Time

Series.Beijing: China WaterPower Press,2007.

[8] Stefania Tronci, “Reconstruction of chaotic time series by neural

models :a case study”, Neuro computing, vol.55, 2003, pp:581-591. [9] QU Wenlong, FAN Guangquan, YANG Bingru, “Research on

Complicated Time Series Prediction Based on Support Vector Machines”, Computer Engineering , vol.31, 2005 ,pp:1-3.

[10] BIAN Hongwei, XU Jiangning, ZHOU Gang, GU Yinglin, ZHU Tao.

“Research of AR Model and Neural Network on the On-line Diagnosis of Gyros”, Journal of Chinese Inertial Technology , vol.8, Apr, 2000,pp: 93–97.