Engineering Essay

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 65, NO. 3, MARCH 2018 2727

Electric Locomotive Bearing Fault Diagnosis Using a Novel Convolutional

Deep Belief Network Haidong Shao, Hongkai Jiang , Member, IEEE, Haizhou Zhang, and Tianchen Liang

Abstract—Bearing fault diagnosis is of significance to enhance the reliability and security of electric locomotive. In this paper, a novel convolutional deep belief network (CDBN) is proposed for bearing fault diagnosis. First, an auto-encoder is used to compress data and reduce the di- mension. Second, a novel CDBN is constructed with Gaus- sian visible units to learn the representative features. Third, exponential moving average is employed to improve the performance of the constructed deep model. The proposed method is applied to analyze experimental signals collected from electric locomotive bearings. The results show that the proposed method is more effective than the traditional methods and standard deep learning methods.

Index Terms—Convolutional deep belief network (CDBN), electric locomotive bearing, exponential moving average (EMA), fault diagnosis, feature learning.

NOMENCLATURE ANFIS Adaptive neuro fuzzy inference system. ANN Artificial neural network. BPNN Back propagation neural network. CDBN Convolutional deep belief network. CNN Convolutional neural network. CRBM Convolutional restricted Boltzmann machine. DAE Deep auto-encoder. DBN Deep belief network. EMA Exponential moving average. FD Frequency domain. PCA Principal component analysis. RBM Restricted Boltzmann machine. SVM Support vector machine. TD Time domain.

Manuscript received January 13, 2017; revised April 24, 2017 and June 26, 2017; accepted August 5, 2017. Date of publication August 25, 2017; date of current version December 15, 2017. This work was supported in part by the National Natural Science Foundation of China under Grant 51475368, in part by the Shanghai Engineering Research Center of Civil Aircraft Health Monitoring Foundation of China under Grant GCZX-2015-02, and in part by the Innovation Foundation for Doc- tor Dissertation of Northwestern Polytechnical University under Grant CX201710. (Corresponding author: Hongkai Jiang.)

The authors are with the School of Aeronautics, Northwestern Poly- technical University, Xi’an 710072, China (e-mail: [email protected]. edu.cn; [email protected]; [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIE.2017.2745473

I. INTRODUCTION

E LECTRIC locomotive is playing a more and more impor-tant role in the modern transportation. The key parts of electric locomotive usually get various faults due to the harsh operating conditions, which may result in great catastrophes. Bearing is one of the most widely used components in elec- tric locomotive [1]; thus, automatic and accurate fault diagnosis techniques are critically needed to ensure the safety of electric locomotive.

Vibration analysis is popular for bearing condition detection [2]–[4]. Intelligent diagnosis methods can effectively analyze the collected data and automatically provide accurate results [5], in which ANN, SVM, and ANFIS are the most widely ap- plied. Generally, two steps are necessary for bearing intelligent fault diagnosis: feature extraction and fault classification [6], [7]. Six features were proposed by Soualhi et al. [8] for rep- resenting the working conditions of motor bearing, and then the selected sensitive features were fed into ANN for fault di- agnosis. Prieto et al. [9] calculated 15 time-domain statistical parameters to characterize the bearing health conditions, then discriminant analysis was used for feature selection and ANN was adopted for fault detection. Boukra et al. [10] extracted time-frequency features and constructed ANN as the fault clas- sifier. A total of 24 features were developed by Lei et al. [11] to reveal bearing working conditions, and ANFIS was used for fault diagnosis. Rauber et al. [12] designed an original feature vector based on 26 statistical parameters, 72 envelope features, and 32 wavelet packet features, and then utilized SVM for identifying bearing faults. Ebrahimi et al. [13] used wavelet transform for feature extraction and PCA for feature fusion, then carried out SVM to diagnose bearing faults. Kang et al. [14] used wavelet packet transform for feature extraction and SVM for fault detection.

Although traditional intelligent methods such as ANN and SVM have been widely used in the fault diagno- sis of rotating machinery, they still have some inherent disadvantages:

1) The diagnosis performance of most traditional meth- ods depends heavily on the quality of the extracted and selected features. In engineering practice, the col- lected vibration signals are always complex and non- stationary with heavy noise [15]. Moreover, the fault categories of bearing often show not only the single

0278-0046 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

2728 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 65, NO. 3, MARCH 2018

fault but also the compound faults. Thus, very advanced signal processing techniques must be well mastered for effective fault feature extraction.

2) It is a challenging and time-consuming task to se- lect the most sensitive features in different diagno- sis issues. In most cases, feature selection depends largely on the engineering experience of diagnostic experts [16].

3) ANN and SVM belong to shallow learning models, which are difficult to effectively learn the complex nonlinear relationships [17]–[19]. Thus, there is an urgent need to construct deep architectures for bearing intelligent fault diagnosis.

Deep learning is a new machine learning method that has the great capacity to overcome the inherent disadvantages of traditional intelligent methods. The most obvious difference be- tween deep learning models and traditional models is that the former can automatically learn the valuable features from the raw data. In other words, deep learning models can get rid of the dependence on the prior knowledge about advanced signal processing techniques and domain experts. At present, various deep learning models can be divided into three main types [20]: DAE, DBN, and CNN. The three kinds of deep learning models are constructed with different base models, and they have their own properties for feature learning. DAE is easy to train and it is a purely unsupervised feature learning model [21]. DBN is a kind of probabilistic generative model which can acquire the joint distribution of the observed data and labels [22]. CNN has some attractive advantages such as shift-invariance and weight sharing [23].

In the last three years, different deep learning models have been gradually applied for intelligent fault diagnosis [21]–[23]. However, current deep learning models used in intelligent fault diagnosis field are mostly designed with the same base models, which are incapable of taking full advantages of different base models simultaneously. Thus, the research and development of new deep learning models is meaningful.

This paper aims to propose a novel method for automatic and accurate identification of electric locomotive bearing faults. First, an auto-encoder is used to compress the collected vibration data and learn the low-layer features. Then, a new CDBN model is designed with Gaussian visible units to learn the high-layer features. Finally, EMA is employed to improve the performance of the designed deep model. The contribution of this paper is that the proposed method takes full advantages of different deep learning models, and it can greatly get rid of the dependence on advanced signal processing techniques and manual feature extraction. The proposed method is applied to analyze the vi- bration signals collected from electric locomotive bearings. The results show that the proposed method is more superior in sta- bility and accuracy to the traditional methods and standard deep learning methods.

The rest of this paper is organized as follows. The standard CDBN theory is given briefly in Section II. In Section III, the proposed method is introduced. In Section IV, the exper- imental results are discussed. In Section V, the conclusions are given.

Fig. 1. Convolutional connections from the visible layer to the hidden layer in a CRBM with K 3× 3 filters.

II. CONVOLUTIONAL DEEP BELIEF NETWORK THEORY

The standard CDBN is a new hierarchical generative model that combines the advantages of DBN and CNN [24]. The stan- dard CDBN is constructed with some CRBMs, and each CRBM is an extension of standard RBM which overcomes the short- coming that all visible units must be related to all hidden units by different weights.

The standard CRBM consists of a visible layer V (input layer) and a hidden layer H . The visible layer consists of a NV ×NV matrix of binary units. The hidden layer is divided into K groups (H1 ,H2 , . . . , HK , each Hk is also called a feature map), and each hidden group is a NH ×NH matrix of binary units. Thus, the total number of hidden units in this CRBM is N 2H K. Each of the K hidden group is connected with a NW ×NW matrix called filter (NW = NV −NH + 1, W 1 ,W 2 , . . . ,WK ). Fig. 1 shows the convolutional connections in a standard CRBM with K 3× 3 filters. From Fig. 1, we can find that the connection weights are shared in each hidden group.

Similar to the standard RBM, the energy function of the stan- dard CRBM model is defined as follows [24]:

− log P (v,h) ∝ E(v,h)

= − K∑

k=1

NH∑

i,j=1

NW∑

r,s=1

hki,jW k r,svi+r−1,j+s−1

− K∑

k=1

bk

NH∑

i,j=1

hki,j − c NV∑

i,j=1

vi,j (1)

where v and h represent the visible vector and hidden vector, respectively. vi,j is the element in the ith row and the jth column of the matrix v, hki,j is the element in the ith row and the jth column of the kth hidden group, and Wkr,s is the element in the rth row and the sth column of the kth filter. bk is the bias of each hidden group, and all visible units share the same bias c.

The conditional probabilities of the standard CRBM can be calculated through one-step Gibbs sampling [24], which can be

SHAO et al.: ELECTRIC LOCOMOTIVE BEARING FAULT DIAGNOSIS USING A NOVEL CONVOLUTIONAL DEEP BELIEF NETWORK 2729

Fig. 2. Compression process of the collected vibration signal using an auto-encoder.

expressed as

P (hki,j = 1 |v ) = σ((W̃ k ∗ v)i,j + bk ) (2)

P (vi,j = 1 |h ) = σ ⎛

⎝ ∑

k,i,j

(Wk ∗ hk )i,j + c ⎞

⎠ (3)

where σ(x) = 1/(1 + e−x) is the sigmoid function, ∗ is the convolution operation, and W̃ ki,j = W

k NW −j+1 .

III. PROPOSED METHOD

In this paper, we develop a novel CDBN for locomo- tive bearing fault diagnosis. This method includes three parts: vibration data compression using an auto-encoder, novel CDBN construction, and general procedure of the proposed method.

A. Vibration Data Compression Using an Auto-Encoder

Massive vibration data will be acquired from the electric locomotive bearing after a long-term running. Thus, there is an urgent need for data compression methods which can reduce the data amount while retaining most useful information.

We consider that the auto-encoder is a new method that can effectively find the compressed representation of the input data. In this paper, we use the auto-encoder to reduce the dimensions of the collected vibration data and learn its low-layer features.

An auto-encoder is an unsupervised network consisting of an encoder and a decoder. The main data compression process us- ing the auto-encoder is shown in Fig. 2. Given a vibration signal X ∈ RN , the encoder transforms X into a compressed repre- sentation (hidden representation) Y ∈ RM (M < N ) through the following nonlinear function [16]

Y = sf ( W(1)X + b(1)

) (4)

sf (t) = 1/(1 + e−t) (5)

where X and Y are N-dimensional and M-dimensional vectors, respectively, W(1) is an M× N-dimensional weight matrix, and b(1) is an M-dimensional bias vector.

Fig. 3. Feature learning process of the Gaussian CRBM model.

Then, the compressed data Y is transformed back to a recon- struction vector Z ∈ RN by the decoder as follows:

Z = sf ( W(2)Y + b(2)

) (6)

where W(2) is an N × M-dimensional weight matrix and b(2) is an N-dimensional bias vector. The auto-encoder training aims to determine the parameter set θ = {W(1) ,b(1) ,W(2) ,b(2)} minimizing the reconstruction error between X and Z.

B. Novel CDBN Construction

Binary visible units are always employed in the standard CDBN [26], which is unsuitable to model real-valued data such as the collected vibration data [27]. Thus, we adopt Gaussian visible units for constructing new CDBN, which can capture the local features in different time scales of the analyzed signals. In order words, the new CDBN considers the two-dimensional (2-D) structure and periodic characteristics of the input data. A significant difference between the new CDBN and standard CNN is that the convolutional connections are employed in a generative Markov random filed structure. The energy function of this new model is modified as

Enew (v,h) = 1 2

NV∑

i,j=1

v2 i , j

− K∑

k=1

NH∑

i,j=1

NW∑

r,s=1

hki,jW k r,svi+r−1,j+s−1

− K∑

k=1

bk

NH∑

i,j=1

hki,j − c NV∑

i,j=1

vi,j . (7)

2730 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 65, NO. 3, MARCH 2018

Unlike the standard CRBM, the conditional probabilities of Gaussian CRBM are

Pnew (hki,j = 1 |v ) = σ((W̃ k ∗ v)i,j + bk ) (8)

Pnew (vi,j = 1 |h ) = N (∑

k,i,j

( Wk ∗ hk)

i,j + c, 1

) (9)

where N(μ, σ2) is the Gaussian distribution with mean μ and variance σ2 .

Fig. 3 shows the feature learning process of the Gaussian CRBM model (only the kth hidden group Hk is shown). A Gaussian CRBM can be seen as a three-layer network consisting of a Gaussian visible layer V , a binary hidden layer H , and a pooling layer P . The pooling layer can further reduce the data amount while retaining the most useful information. The pooling layer and hidden layer both have K groups. Each group size in the pooling layer is NP ×NP . The kth hidden group Hk is divided into several small blocks of size C × C (C is usually set to 1, 2, or 3), and each block is connected to one unit in the pooling layer. The input of the visible layer is the compressed data given by the auto-encoder. The outputs of the hidden layer and pooling layer are the learned higher layer features.

The training of Gaussian CRBM aims to find the optimal parameter set {Wk}, which can be updated as

[Wk ](q+1) = [Wk ](q) + η[ΔWk ](q+1) (10)

with

ΔWk = { ΔWkr,s

}

= {⟨

hki,j vi+r−1,j+s−1 ⟩

data − ⟨ hki,j vi+r−1,j+s−1

⟩ recon

}

(11)

where η is the learning rate, and [Wk ](q) denotes the kth weight matrix at the qth iteration and q represents the cur- rent iteration number. 〈·〉data and 〈·〉recon refer to the ex- pectations over the training data and reconstructed data, respectively.

It can be found that only the values obtained from the last iteration are considered for weight updating using the standard learning algorithm while ignoring all the other past weights, which will result in error oscillation and slow convergence. Weight smoothing and adaptive learning rate have been proved effective to overcome these limitations and further improve the generalization capability of the traditional neural networks [28]. Thus, in this paper, a modified learning algorithm using an EMA technique is developed. In the modified learning algorithm, the weights trained from the previous iterations will be totally con- sidered for updating the current weights. The modified updated equations for {Wk} are ⎧ ⎨

⎩

[Wk ](q+1) = [Wk ](q) + [η](q+1) × [ΔWk ](q+1)

[Wk ](q+1) ← 2×[W k ]( q + 1 ) +q×[W k ]( q )

q+2 , q ≥ 1 (12)

with {

[η](q+1) ← Rd [η](q) , if Error(q + 1) ≥ Error(q) [η](q+1) ← Ri [η](q) , if Error(q + 1) < Error(q)

(13)

where Error(q) represents the reconstruction error at the qth epoch. Ri and Rd are increasing factor and decreasing factor, respectively. According to [29]–[31] and our experimental tasks, the empirical values of increasing factor and decreasing factor are set as Ri ∈ [1, 10] and Rd ∈ [0.1, 1] in this paper.

In order to overcome the overcompleteness of the learned features, we have to regularize each hidden unit of Gaussian CRBM model for sparsity [24]–[26], which can constrain the information content of each feature map, defined as

λ

NH∑

i,j=1

( ρ− 1

T

T∑

t=1

K∑

k=1

E[hki,j ∣∣∣v(t) ]

)2 (14)

where E[·] is the conditional expectation operator, and v(t) is a sample in the training set with T total samples. λ and ρ are regularization constant and sparseness constant, respectively. By now, the novel CRBM has been designed. Similar to the standard DBN [27], the construction of the novel CDBN can be accomplished through successive training of each individual novel CRBM. Finally, the proposed CDBN can be used for bearing feature learning and fault classification.

C. General Procedure of the Proposed Method

In this paper, a novel CDBN is developed for locomotive bearing fault diagnosis. The framework is shown in Fig. 4 and the general procedures are summarized as follows:

� Step 1: Collect the vibration signal of electric locomotive bearing.

� Step 2: Auto-encoder is adopted to reduce the dimensions of the collected vibration data, and then the compressed data can be obtained.

� Step 3: Without any feature extraction, the compressed data are divided into training samples and testing samples separately.

� Step 4: Gaussian visible units are used to design novel CRBM, and the EMA technique is adopted to modify the standard learning algorithm.

� Step 5: Construct the novel CDBN based on a series of trained novel CRBMs, and then it is used for unsupervised feature learning of the training samples. The learned deep features are fed into a Softmax for fault classification.

� Step 6: Validate the performance of the proposed method using the testing samples.

IV. EXPERIMENTAL VALIDATION

A. Experimental Data Description

In order to validate the effectiveness of the proposed method, an experimental setup of electric locomotive bearing was built, as shown in Fig. 5. Fig. 6 shows three kinds of faulty bearings (rub faults using an electric discharge). The accelerometer with sensitivity of 100 mv/g is mounted on the load module adja- cently for measuring the vibration signals. Under the load of 9800 N, the vibration signals under all operating conditions were collected at a sampling frequency of 12.8 kHz, and the

SHAO et al.: ELECTRIC LOCOMOTIVE BEARING FAULT DIAGNOSIS USING A NOVEL CONVOLUTIONAL DEEP BELIEF NETWORK 2731

Fig. 4. Framework of the proposed method.

Fig. 5. Electric locomotive bearing experimental setup.

Fig. 6. Three kinds of faulty bearings.

TABLE I PARAMETERS OF ELECTRIC LOCOMOTIVE BEARING

Bearing specs

Inner race diameter

Outer race diameter

Roller diameter

Roller number

Contact angle

552732QT 160 mm 290 mm 34 mm 17 0°

sampling time is 32 s. More parameters of the locomotive bear- ing are listed in Table I.

In this study, by installing the testing bearing with different types of faults, eight bearing operating conditions are created, as listed in Table II. Based on the parameters and the rota- tion speed, the characteristic fault frequencies of the inner race, roller, and outer race can be calculated. The vibration signal of each condition is stably collected under corresponding speed. Each condition contains 400 samples, and each sample is a col- lected vibration signal containing 1024 data points. Random 300 samples of each condition are selected for training and the

2732 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 65, NO. 3, MARCH 2018

TABLE II DESCRIPTION OF THE BEARING OPERATION CONDITIONS

Operation conditions of electric locomotive bearing Rotation speed (r/min) Size of training/testing samples Label of condition

Normal condition 490 300/100 1 Slightly outer race fault 490 300/100 2 Severe outer race fault 481 300/100 3 Roller fault 531 300/100 4 Inner race fault 498 300/100 5 Compound faults (outer race and inner race) 525 300/100 6 Compound faults (inner race and roller) 640 300/100 7 Compound faults (outer race and roller) 521 300/100 8

Fig. 7. Raw waveforms of the eight bearing conditions. (a) Condition 1. (b) Condition 2. (c) Condition 3. (d) Condition 4. (e) Condition 5. (f) Condition 6. (g) Condition 7. (h) Condition 8.

remaining 100 for testing. Fig. 7 shows the raw waveforms of the eight bearing conditions (data points: 400 � 1024), which are very hard to be classified.

B. Comparison With the Traditional Methods

Different from the traditional methods (SVM, BPNN, and ANFIS), the proposed method focuses on the intelligent fault diagnosis without signal preprocessing or feature extraction. Two important points need to be pointed out: 1) The input of the proposed method is the raw vibration data, and the classifier is Softmax. 2) SVM, BPNN, and ANFIS have three inputs. The first is the raw data. The second is ten TD features extracted from each sample (1024 data points). The third is 13 FD features extracted from the corresponding spectrum (512 data points) of each sample. More details about the 10 TD features and 13 FD features can be seen in [32] and [11], respectively.

A total of ten trials are carried out for diagnosing each dataset, as listed in Table III. Fig. 8 shows the detailed diagnosis results in each trial. The average accuracy of the proposed method is 97.4375%, and it is much higher than BPNN, SVM, and ANFIS using raw data, which are 52.2875%, 56.0875%, and 56.1375%, respectively. After feature extraction, though the accuracies of BPNN, SVM, and ANFIS increase greatly, their results still cannot be compared with the proposed method. Fig. 9 shows the confusion matrix of the proposed method for the second trial. The influence of sample size on the performance of the

TABLE III DIAGNOSIS RESULTS OF DIFFERENT METHODS

Methods Average testing accuracy Standard deviation

The proposed method 97.4375% (7795/8000) 0.1350 BPNN with raw data 52.2875% (4183/8000) 1.1197 BPNN with TD features 85.0250% (6802/8000) 0.8371 BPNN with FD features 91.9000% (7352/8000) 0.6556 SVM with raw data 56.0875% (4487/8000) 0.2704 SVM with TD features 87.1125% (6969/8000) 0.2045 SVM with FD features 92.9625% (7437/8000) 0.2348 ANFIS with raw data 56.1375% (4491/8000) 0.2704 ANFIS with TD features 87.0875% (6967/8000) 0.2372 ANFIS with FD features 92.3250% (7386/8000) 0.2045

Fig. 8. Diagnosis results of the ten trials using different methods.

Fig. 9. Confusion matrix of the proposed method for the second trial.

proposed method is investigated in Table IV (Core i5, 16 GB memory).

It can be concluded that 1) the performance of BPNN, SVM, and ANFIS depends

heavily on feature extraction. Their diagnosis results may be further improved after selecting the sensitive features, but it is labor-intensive and time-consuming.

SHAO et al.: ELECTRIC LOCOMOTIVE BEARING FAULT DIAGNOSIS USING A NOVEL CONVOLUTIONAL DEEP BELIEF NETWORK 2733

TABLE IV DIAGNOSIS RESULTS BASED ON DIFFERENT SIZES OF TRAINING SAMPLES

Size of training / testing samples Average testing accuracy Time (s)

350/50 97.5500% (7804/8000) 244.83 325/75 97.6375% (7811/8000) 225.57 300/100 97.4375% (7795/8000) 208.29 275/125 97.0500% (7764/8000) 192.14 250/150 96.3875% (7711/8000) 175.56 225/175 94.8500% (7588/8000) 162.72 200/200 93.6125% (7489/8000) 150.28

Fig. 10. Projections of the learned deep features. (a) Two-dimensional and (b) three-dimensional.

TABLE V DIAGNOSIS RESULTS OF DIFFERENT DEEP LEARNING METHODS

Deep learning methods Average testing accuracy (%) Time (s)

The proposed method 97.4375% (7795/8000) 208.29 Standard DAE 90.7625% (7261/8000) 214.86 Standard DBN 88.1000% (7048/8000) 187.05 Standard CNN 91.2375% (7299/8000) 245.31

2) The proposed method shows higher accuracy and better stability than SVM, BPNN, and ANFIS. The reason is that the proposed method can adaptively learn the essential features from the raw data. In order to show the quality of the learned essential features, 2-D and three-dimensional projections are used for visualizing with PCA, as shown in Fig. 10.

2) From 200 training samples to 300 training samples, the accuracies of the proposed method keep growing and are always over 95%. Thus, the proposed method can effectively avoid over-fitting. Generally, larger size of the training sample will achieve higher testing accuracy and more computing time.

C. Comparison With Other Deep Learning Methods

It is necessary to compare the performance of the proposed method and other deep learning methods (standard DAE, DBN, and CNN), as shown in Table V. In this section, all the inputs are the raw data, and the classifiers are Softmax. In addition, the performance of different CDBN models is compared in Ta- ble VI. The error curves are shown in Fig. 11, and Fig. 12 shows

TABLE VI COMPARISON OF DIFFERENT CDBN MODELS AND LEARNING ALGORITHMS

Different CDBN models Average accuracy

Gaussian CDBN with modified algorithm (Proposed) 97.4375% Gaussian CDBN with standard algorithm 91.6125% Binary CDBN with modified algorithm 69.7250% Binary CDBN with standard algorithm (standard) 67.2250%

Fig. 11. Reconstruction error curves of different learning algorithms. (a) The first Gaussian CRBM. (b) The second Gaussian CRBM.

Fig. 12. Learned weights of the last filter group at the last iteration. (a) The standard learning algorithm. (b) The modified learning algorithm.

the learned weights of the last filter group in the first Gaussian CRBM at the last iteration. Although deep models need much training time due to the increase of hidden layers and units, the reconstruction error curve converges more quickly and the learned weights are more smoothly using the modified learning algorithm. These comparison results confirm the superiority of the proposed CDBN model.

Therefore, the proposed method is more effective than other deep learning methods. The superiority of the proposed method arises from two main respects: 1) the proposed model is a new deep learning model that combines the advantages of the stan- dard DAE, DBN, and CNN. 2) Gaussian visible units and modi- fied learning algorithm further enhance the generalization ability when dealing with vibration data.

The parameters of the proposed method are listed in Table VII. At present, there is not a systematic method to determine the structure of various deep learning models. In this paper, the structure of the proposed deep model is decided by experiments. Two key parameters λ and ρ are selected through the cross validation, as shown in Fig. 13.

2734 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 65, NO. 3, MARCH 2018

TABLE VII PARAMETERS OF THE PROPOSED METHOD

Description Value

The length of the compressed sample M 400 (20 � 20) The number of Gaussian CRBMs 2 Size of visible units in the first Gaussian CRBM 20 � 20 Size of each hidden group in the first Gaussian CRBM 14 � 14 The number of filter groups in the first Gaussian CRBM 9 Size of each filter in the first Gaussian CRBM 7 � 7 Size of visible units in the second Gaussian CRBM 7 � 7 Size of each hidden group in the second Gaussian CRBM 3 � 3 The number of filter groups in the second Gaussian CRBM 12 Size of each filter in the second Gaussian CRBM 5 � 5 Iteration times of each Gaussian CRBM 12 Initial learning rate/Increasing factor/Decreasing factor 0.05/1.05/0.95 Sparsity coefficient/Regularization constant 0.02/0.07

Fig. 13. Relationship between accuracy and two key parameters.

Fig. 14. Diagnosis performance under different compression cases.

The parameters of other methods are described as follows: 1) DAE: The structure is 1024-400-400-400-8, which is de-

cided by experiments. Iteration number is 100, and learn- ing rate is 0.1.

2) DBN: 1024-400-100-100-8, iteration number is 100, and learning rate is 0.1.

3) CNN: The size of the input feature map is 32 � 32. C1 layer contains 6 kernels, C3 contains 12 kernels, and the scales of P2 layer and P4 layer are both set to 2. The learning rate is 0.1, and iteration number is 200.

TABLE VIII DIAGNOSIS RESULTS BASED ON DIFFERENT COMPRESSION METHODS

Compression methods Average testing accuracy

Auto-encoder (1024→ 400) 97.4375% (7795/8000) RBM (1024→ 441) 96.1000% (7688/8000) Fast Fourier transform (1024→ 512→ 484) 96.7625% (7741/8000) Compressed sensing (1024→ 484) 97.0750% (7766/8000)

Note: First 484 data points in the corresponding spectra of each sample.

Fig. 15. Collected vibration data of Bearing 3 (During the 25th day to the 35th day).

D. Contribution of Data Compression

In practical engineering, it is important to strike a balance between the accuracy and efficiency. In this section, ten kinds of compression cases (sampling rates) are used to investigate the contribution of the auto-encoder, as shown in Fig. 14. Besides, another three popular compression methods are compared with the auto-encoder, as shown in Table VIII. It can be found that: 1) Auto-encoder can reduce data amount and improve analysis efficiency, and it is slightly better than RBM. 2) The processed data using fast Fourier transform and compressed sensing can also be used for effective feature learning; however, the two methods are regarded as signal processing techniques, and this paper aims to achieve intelligent fault diagnosis without any signal processing.

E. Validation on NASA Bearing Dataset

In this section, NASA bearing dataset is used to further demonstrate the superiority of the proposed method, which is from Prognostics Center Excellence [33].

An inner race defect is discovered in Bearing 3 of Testing 1, which is shown in Fig. 15 (During the 25th day to the 35th day). Three kinds of bearing operation conditions are created, which are health condition, slight degradation condition, and severe degradation condition. Each condition consists of 100 samples: the random 50 samples are adopted for training and the rest 50 samples for testing. Each sample is a collected vibration signal containing 1225 (35 � 35) data points.

A total of ten trials are performed for analyzing each bearing dataset, as shown in Table IX and Fig. 16. It can be found that the proposed method shows a significant superiority over other methods, which can accurately identify different fault severities and monitor the law of bearing performance degradation.

SHAO et al.: ELECTRIC LOCOMOTIVE BEARING FAULT DIAGNOSIS USING A NOVEL CONVOLUTIONAL DEEP BELIEF NETWORK 2735

TABLE IX DIAGNOSIS RESULTS FOR NASA BEARING DATASET

Methods Average testing accuracy Standard deviation

The proposed method 100.0% (1500/1500) 0 BPNN with FD features 92.07% (1381/1500) 1.5854 SVM with FD features 94.60% (1419/1500) 0.9661 ANFIS with FD features 94.40% (1416/1500) 0.7730

Fig. 16. Detailed diagnosis results on NASA bearing dataset.

V. CONCLUSION

In this paper, a novel CDBN was proposed for locomotive bearing fault diagnosis. First, an auto-encoder was employed to compress the collected vibration data. Second, a new CDBN model was constructed to learn the deep features. Finally, EMA was used to improve the performance of the constructed model.

The proposed method was applied to analyze the experimen- tal bearing signals. The results confirmed that the proposed method was more effective than the existing methods. With the rapid development of hardware technology, deep learning can be gradually applied in the new industrial scenario in the future. We will continue to study this topic.

REFERENCES

[1] Z. P. Li, J. L. Chen, Y. Y. Zi, and J. Pan, “Independence-oriented VMD to identify fault feature for wheel set bearing fault diagnosis of high speed locomotive,” Mech. Syst. Signal Process., vol. 85, pp. 512–529, Feb. 2017.

[2] J. Wang, Y. Y. Peng, and W. Qiao, “Current-aided order tracking of vi- bration signals for bearing fault diagnosis of direct-drive wind turbines,” IEEE Trans. Ind. Electron., vol. 63, no. 10, pp. 6336–6346, Oct. 2016.

[3] F. Immovilli, C. Bianchini, M. Cocconcelli, A. Bellini, and R. Rubini, “Bearing fault model for induction motor with externally induced vibra- tion,” IEEE Trans. Ind. Electron., vol. 60, no. 8, pp. 3408–3418, Aug. 2013.

[4] B. Zhang, C. Sconyers, C. Byington, R. Patrick, M. E. Orchard, and G. Vachtsevanos, “A probabilistic fault detection approach: Application to bearing fault detection,” IEEE Trans. Ind. Electron., vol. 58, no. 5, pp. 2011–2018, May 2011.

[5] F. Jia, Y. G. Lei, J. Lin, X. Zhou, and N. Lu, “Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data,” Mech. Syst. Signal Process., vol. 72–73, pp. 303–315, May 2016.

[6] M. Kang, M. R. Islam, J. Kim, J. M. Kim, and M. Pecht, “A Hybrid fea- ture selection scheme for reducing diagnostic performance deterioration caused by outliers in data-driven diagnostics,” IEEE Trans. Ind. Electron., vol. 63, no. 5, pp. 3299–3310, May 2016.

[7] D. He, R. Y. Li, and J. D. Zhu, “Plastic bearing fault diagnosis based on a two-step data mining approach,” IEEE Trans. Ind. Electron., vol. 60, no. 8, pp. 3429–3440, Aug. 2013.

[8] A. Soualhi, G. Clerc, and H. Razik, “Detection and diagnosis of faults in induction motor using an improved artificial ant clustering technique,” IEEE Trans. Ind. Electron., vol. 60, no. 9, pp. 4053–4062, Aug. 2013.

[9] M. D Prieto, G. Cirrincione, A. G. Espinosa, J. A. Ortega, and H. Henao, “Bearing fault detection by a novel condition-monitoring scheme based on

statistical-time features and neural networks,” IEEE Trans. Ind. Electron., vol. 30, no. 8, pp. 3398–3407, Aug. 2013.

[10] T. Boukra, A. Lebaroud, and G. Clerc, “Statistical and neural-network approaches for the classification of induction machine faults using the ambiguity plane representation,” IEEE Trans. Ind. Electron., vol. 60, no. 9, pp. 4034–4042, Sep. 2013.

[11] Y. G. Lei, Z. J. He, and Y. Y. Zi, “A new approach to intelligent fault diag- nosis of rotating machinery,” Expert Syst. Appl., vol. 35, no. 4, pp. 1593– 1600, Nov. 2008.

[12] T. W. Rauber, F. D. A. Boldt, and F. M. Varejao, “Heterogeneous feature models and feature selection applied to bearing fault diagnosis,” IEEE Trans. Ind. Electron., vol. 62, no. 1, pp. 637–646, Jan. 2015.

[13] B. M. Ebrahimi, M. J. Roshtkhari, J. Faiz, and S. V. Khatami, “Advanced eccentricity fault recognition in permanent magnet synchronous motors using stator current signature analysis,” IEEE Trans. Ind. Electron., vol. 61, no. 4, pp. 2041–2052, Apr. 2014.

[14] M. Kang, J. Kim, and J. M. Kim, “An FPGA-based multicore sys- tem for real-time bearing fault diagnosis using ultrasampling rate AE signals,” IEEE Trans. Ind. Electron., vol. 61, no. 4, pp. 2041–2052, Apr. 2014.

[15] N. P. Li, Y. G. Lei, J. Lin, and S. X. Ding, “An improved exponential model for predicting remaining useful life of rolling element bearings,” IEEE Trans. Ind. Electron., vol. 62, no. 12, pp. 7762–7773, Dec. 2015.

[16] F. Jia, Y. G. Lei, J. Lin, X. Zhou, and N. Lu, “Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data,” Mech. Syst. Signal Process., vol. 72–73, pp. 303–315, May 2016.

[17] C. Shang, F. Yang, D. X. Huang, and W. X. Lyu, “Data-driven soft sen- sor development based on deep learning technique,” J. Process Control, vol. 24, no. 3, pp. 223–233, Mar. 2014.

[18] Y. G. Lei, F. Jia, and J. Lin, “An intelligent fault diagnosis method using unsupervised feature learning towards mechanical big data,” IEEE Trans. Ind. Electron., vol. 63, no. 5, pp. 3137–3147, May 2016.

[19] Y. Bengio and A. Courville, “Representation learning: A review and new perspectives,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798–1828, Aug. 2013.

[20] J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural Netw., vol. 61, pp. 85–117, Jan. 2015.

[21] H. D. Shao, H. K. Jiang, F. A. Wang, and H. W. Zhao, “An enhance- ment deep feature fusion method for rotating machinery fault diagnosis,” Knowl.-Based Syst., vol. 119, pp. 200–220, Mar. 2017.

[22] V. T. Tran, F. AlThobiani, and A. Ball, “An approach to fault diagnosis of reciprocating compressor valves using Teager-Kaiser energy operator and deep belief networks,” Expert Syst. Appl., vol. 41, no. 9, pp. 4113–4122, Jul. 2014.

[23] O. Janssens, V. Slavkovikj, B. Vervisch, K. Stockman, M. Loccufier, and S. Verstockt, “Convolutional neural network based fault detection for rotating machinery,” J. Sound Vib., vol. 377, pp. 331–345, Sep. 2016.

[24] H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, “Convolutional deep belief networks for scalable unsupervised learning of hierarchical repre- sentations,” in Proc. 26th Annu. Int. Conf. Mach. Learn., Montreal, QC, Canada, Jun. 2009, pp. 609–616.

[25] M. Norouzi, M. Ranjbar, and G. Mori, “Stacks of convolutional restricted Boltzmann machines for shift-invariant feature learning,” in Proc. 2009 IEEE Conf. Comput. Vision Pattern Recog., Miami, FL, USA, Jun. 2009, pp. 2735–2742.

[26] H. Lee, P. T. Pham, L. Yan, and A. Y. Ng, “Unsupervised feature learn- ing for audio classification using convolutional deep belief networks,” in Proc. 22nd Int. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2009, pp. 1096–1104.

[27] G. E. Hinton, S. Osindero, and Y. W. The, “A fast learning algorithm for deep belief nets,” Neural Comput., vol. 18, no. 7, pp. 1527–1554, Jul. 2006.

[28] W. Bian and X. J. Chen, “Smoothing neural network for constrained non-Lipschitz optimization with applications,” IEEE Trans. Neural Netw. Learn., vol. 23, no. 3, pp. 399–411, Mar. 2012.

[29] H. C. Kuo, H. K. Chang, and Y. Z. Wang, “Symbiotic evolution-based design of fuzzy-neural transformer diagnostic system,” Electr. Power Syst. Res., vol. 72, no. 3, pp. 235–244, Dec. 2004.

[30] H. D. Shao, H. K. Jiang, F. A. Wang, and Y. N. Wang, “Rolling bearing fault diagnosis using adaptive deep belief network with dual-tree complex wavelet packet,” ISA Trans., vol. 69, pp. 187–201, Jul. 2017.

[31] N. Lopes and B. Ribeiro, “Towards adaptive learning with improved con- vergence of deep belief networks on graphics processing units,” Pattern Recog., vol. 47, no. 1, pp. 114–127, Jan. 2014.

2736 IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 65, NO. 3, MARCH 2018

[32] Y. G. Lei, Z. J. He, and Y. Y. Zi, “EEMD method and WNN for fault diagnosis of locomotive roller bearings,” Expert Syst. Appl., vol. 38, no. 6, pp. 7334–7341, Jun. 2011.

[33] [Online]. Available: http://ti.arc.nasa.gov/tech/dash/pcoe/prognostic- data-repository/

Haidong Shao was born in Zhejiang Province, China, in 1990. He received the B.S. degree in electrical engineering and automation and the M.S. degree in means of transport ap- plied engineering in 2013 and 2015, respec- tively, from Northwestern Polytechnical Univer- sity, Xi’an, China, where he is currently working toward the Ph.D. degree in means of transport applied engineering.

His current research interests include fault di- agnosis and intelligent prognosis.

Hongkai Jiang (M’17) received the Ph.D. de- gree in instrument science and technology from the Mechanical Engineering School, Xi’an Jiao Tong University, Xi’an, China, in 2006.

He is currently a Full Professor in the School of Aeronautics, Northwestern Polytechnical Uni- versity, Xi’an, China. From January 2011 to January 2012, he was a Visiting Scholar with the University of British Columbia, Vancouver, Canada. His current research interests include condition monitoring and dynamic signal pro-

cessing, fault diagnosis and health management, big data analysis, and maintenance support.

Haizhou Zhang was born in Anhui Province, China, in 1994. He received the B.S. degree in electrical engineering and automation in 2015 from Northwestern Polytechnical Univer- sity, Xi’an, China, where he is currently working toward the M.S. degree in means of transport applied engineering.

His current research interests include fault di- agnosis and signal processing.

Tianchen Liang was born in Shanxi Province, China, in 1993. He received the B.S. degree in computer science and technology from Wuhan University of Science and Technology, Wuhan, China, in 2016. He is currently working toward the M.S. degree in aeronautical engineering at Northwestern Polytechnical University, Xi’an, China.

His current research interests include fault di- agnosis and signal processing.