Research Paper

Roufiaa91

DrivingbehaviorrecognitionusingEEGdatafromasimulatedcar-followingexperiment.pdf

Home >English homework help >Research Paper

Contents lists available at ScienceDirect

Accident Analysis and Prevention

journal homepage: www.elsevier.com/locate/aap

Driving behavior recognition using EEG data from a simulated car-following experiment

Liu Yanga,1, Rui Mab, H. Michael Zhangb, Wei Guana,⁎, Shixiong Jianga

a MOE Key Laboratory for Urban Transportation Complex Systems Theory and Technology, Beijing Jiaotong University, Beijing 100044, China b Department of Civil and Environmental Engineering, University of California Davis, Davis, CA 95616, USA

A R T I C L E I N F O

Keywords: Driving behavior recognition Electroencephalography (EEG) Car-following behavior K-means Support vector machine

A B S T R A C T

Driving behavior recognition is the foundation of driver assistance systems, with potential applications in au- tomated driving systems. Most prevailing studies have used subjective questionnaire data and objective driving data to classify driving behaviors, while few studies have used physiological signals such as electro- encephalography (EEG) to gather data. To bridge this gap, this paper proposes a two-layer learning method for driving behavior recognition using EEG data. A simulated car-following driving experiment was designed and conducted to simultaneously collect data on the driving behaviors and EEG data of drivers. The proposed learning method consists of two layers. In Layer I, two-dimensional driving behavior features representing driving style and stability were selected and extracted from raw driving behavior data using K-means and support vector machine recursive feature elimination. Five groups of driving behaviors were classified based on these two-dimensional driving behavior features. In Layer II, the classification results from Layer I were utilized as inputs to generate a k-Nearest-Neighbor classifier identifying driving behavior groups using EEG data. Using independent component analysis, a fast Fourier transformation, and linear discriminant analysis sequentially, the raw EEG signals were processed to extract two core EEG features. Classifier performance was enhanced using the adaptive synthetic sampling approach. A leave-one-subject-out cross validation was conducted. The results showed that the average classification accuracy for all tested traffic states was 69.5% and the highest accuracy reached 83.5%, suggesting a significant correlation between EEG patterns and car-following behavior.

1. Introduction

Driving behavior recognition is widely studied in the field of transportation to improve traffic efficiency and safety. As such, driving behavior recognition is a critical component in personalized driver as- sistance systems, where driving behavior types are categorized by the types of driving maneuvers and other performance measures. With proper categorization, unsafe driving behaviors can be better identified so that future driver assistance systems are able to alert drivers and neighboring vehicles. There is also the possibility of developing auto- mated driving systems. For example, driving behavior recognition systems can detect and report the status of drivers to smart infra- structure and other manually driven or autonomous vehicles in a con- nected vehicle scenario. Such communication between vehicles and infrastructure would enhance the performance and reliability of trans- portation systems containing both manually driven and autonomous vehicles.

Typically, two types of data are available for driving behavior re- cognition: subjective questionnaire data and objective driving data. Driving behavior questionnaires (Reason et al., 1990) and driving skill inventories (Lajunen and Summala, 1995) are two common subjective evaluation methods utilized in driving behavior recognition. Zhang et al. (2009) proposed a quantitative method for analyzing driver characteristics based on driving behavior questionnaires. Traditionally, only one or two aspects of driving style have been considered in self- reported scales, such as aggressive or stressed driving styles. However, Taubman-Ben-Ari et al. (2004) constructed a multidimensional driving style inventory accounting for eight main aspects. Subsequently, Chung and Wong (2010) developed a five-aspect multidimensional driving style questionnaire. Martinussen et al. (2014) used both driving beha- vior questionnaires and driving skill inventories to identify different driver sub-groups.

Objective driving data (vehicle speed, acceleration, and position), collected from simulated or natural driving experiments, are the major

https://doi.org/10.1016/j.aap.2017.11.010 Received 15 April 2017; Received in revised form 27 October 2017; Accepted 8 November 2017

⁎ Corresponding author.

1 The analysis work in this paper was performed while the author was a visiting student at the University of California Davis.

E-mail addresses: [email protected] (L. Yang), [email protected] (R. Ma), [email protected] (H.M. Zhang), [email protected] (W. Guan), [email protected] (S. Jiang).

Accident Analysis and Prevention 116 (2018) 30–40

sources of driving behavior recognition data. Bar et al. (2011) presented a probabilistic driving style determination method using fuzzy rules in various simulated traffic situations. Chen et al. (2013) proposed a driving behavior modeling system based on graph theory that con- structs a driving habit graph. In natural driving experiments, in-vehicle sensors can provide the majority of the data utilized for driving action recognition, distraction detection, driver classification, and driving style recognition (Choi et al., 2007; Jensen et al., 2011; Van Ly et al., 2013). Recently, smartphones are being utilized as mobile sensors to recognize aggressive driving behavior (Johnson and Trivedi, 2011). GPS-based tracking devices can also help evaluate driver aggression (Constantinescu et al., 2010).

To compare the consistency of driving behavior classifications based on subjective evaluations and objective driving data, Wang et al. (2010) conducted a comparative driving behavior survey and a real-world driving experiment. The comparative results indicated that subjective evaluations are unreliable for parameterizing driver assistance systems.

Most existing studies assume that driving styles are fixed. Recently, Dorr et al. (2014) revealed that driving styles may vary with traffic conditions. Additionally, prevailing studies used either subjective eva- luations or objective driving data, whereas only a few studies have used physiological data such as electroencephalography (EEG) to identify driving behavior. EEG has been recognized as an effective, non-invasive technique for monitoring and assisting real-world driving (Lin et al., 2005; Papadelis et al., 2007; Kar et al., 2010; Wang et al., 2015). In Chuang et al. (2015), EEG data were used to detect driving states, and warnings were provided to real-world drivers as feedback. Relative to traditional driving data, EEG shows two advantages in recognizing driving behavior: (1) EEG data have higher temporal resolution, al- lowing for real time EEG-based classifications; and (2) EEG data can provide extra information (physiological and emotional) in addition to kinematic vehicle indices.

In this study, six traffic flow conditions were designed in a simulated car-following experiment and a two-layer EEG-based driving behavior recognition system was proposed. The relationship between the car- following behavior and EEG measures was identified by constructing a classifier to recognize EEG patterns associated with car-following be- haviors.

The objective of this study was to construct a driving behavior re- cognition system using EEG data and referencing the classification re- sults of driving behavior data. The remainder of this paper is organized as follows: Section 2 introduces the experimental setup; Section 3 de- scribes the methodology of the EEG-based driving behavior recognition system; Section 4 presents the classification results; and the discussion is presented in Section 5.

2. Experimental setup

2.1. Experimental apparatus

This experiment was conducted using the driving simulator at Beijing Jiaotong University (BJTU) (Li et al., 2016). The simulator consists of a full-size vehicle cabin with a real operating interface, an environmental noise and shaking simulation system, a vehicle dynamics simulation system, and a digital video replay system. It is a high-fidelity driving simulator with a linear motion platform ensuring one degree of freedom. The driving scenario is displayed on five screens around the vehicle with 300° surrounding vision.

A Neuroscan system was used to collect the EEG data, composed of a SynAmps2TM amplifier and an electrode cap. There were 64 channels in the electrode cap, and their locations followed the international 10–20 system. The BJTU simulator and electrode cap are shown in Fig. 1

2.2. Participants

Fifty-two (52) healthy participants were recruited for this

experiment. The average age of the participants was 35.21 years (S.D.2 = 6.88). Their average driving experience was 9.64 years (S.D. = 6.55), and their average annual mileage was greater than 30,000 km. Before the experiment, each of them was required to complete a 10-min practice drive using the simulator, and none of the subjects showed simulator sickness.

2.3. Scenario design and experimental procedure

An eight-kilometer two-way straight road was designed as the driving scenario for this research. There were two lanes, each 3.75 m wide. The speed limit was 80 km/h and the road was surrounded by urban landscape.

The leading vehicle followed six travel patterns in this study, in- cluding four steady phases (free flow, coherent-moving flow, synchro- nized flow and jam (Guan and He, 2008)) and two complementary states (recovery from traffic jam and collision avoidance). To present these traffic states, six time periods were designed with specific accel- erations and speed ranges for the leading vehicle (see Table 1). Fig. 2 shows the speed profile of the leading vehicle. To mimic a real-world driving environment, an additional vehicle was displayed 30 m ahead of the leading vehicle and random traffic was displayed on the opposite lane.

During the experiment, all participants were asked to drive ac- cording to their daily driving habits while obeying traffic rules. There was only one lane for the test vehicle to travel, and participants were not permitted to overtake the leading vehicle. The car-following process lasted 615 s. According to the speed profile of the leading vehicle, the car-following process was divided into six driving periods and the driving behavior recognition for each period was studied subsequently. Driving simulator and EEG data were collected simultaneously during the experiment.

2.4. Driving measures

2.4.1. Driving data acquisition Driving data were collected from the driving simulator with a

sampling rate of 60 Hz. Fourteen (14) key variables (see Table 2) were selected for further analyses, reflecting both longitudinal and lateral vehicle movements such as longitudinal acceleration and lateral lane deviation. Additionally, the relationship between the following and leading vehicles was evaluated using space headway, time headway, and relative speed.

Fig. 1. BJTU driving simulator and electrode cap on driver.

2 S.D. is the standard deviation.

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

2.4.2. Driving data analysis Driving skill and driving style were the two main aspects considered

in driving behavior classification (Zhang et al., 2009; Wang et al., 2010; Martinussen et al., 2014), where driving skill was indicated by the standard deviation of variables and driving style was measured by the mean value of the same variable set (Wang et al., 2010). In this study, driving stability was used instead of driving skill, as most prior studies have failed to consider the differences between traffic states and clas- sified driving behaviors equivalently according to individual drivers. However, this study investigated different traffic states (or periods) such that one driver may have different driving styles and stability in different traffic states. Finally, the mean values and standard deviations of the fourteen key variables in each driving period were calculated, where the mean values related to the driving style and the standard deviations referred to the driving stability.

2.5. EEG measures

2.5.1. EEG acquisition A 64-channel electrode cap connected to Curry 7 software was used

to collect EEG signals. The reference electrode was located between CZ and CPZ, the center of the scalp based on the international 10–20 system. The channel location on the scalp is shown in Fig. 3. EEG sig- nals were recorded at a sampling rate of 1000 Hz.

2.5.2. EEG preprocessing EEG signals are very weak and easily disturbed by eye movements

Table 1 Speed range and acceleration of leading vehicle in each driving period.

Period Period 1 Period 2 Period 3 Period 4 Period 5 Period 6 (Phase) (Free flow) (Coherent flow) (Synchronized flow) (Jam) (Recovery) (Collision avoidance)

Speed range (km/h) 70 50–60 20–30 5–10 0–70 70–0 Acceleration (m/s2) 1 ± 1 ± 0.5 ± 0.5 1 −6

Fig. 2. Speed profile of leading vehicle.

Table 2 Key variables selected from original driving simulator data.

Variable Unit

Longitudinal acceleration m/s2

Lateral acceleration m/s2

Degree of acceleration pedal ° Force of brake pedal N Head deviation °

Space headway m Time headway s Lane deviation m Distance to the yellow line m Steering wheel angle °

Speed m/s Relative speed m/s Acceleration m/s2

Deceleration m/s2

Fig. 3. Channel location of the 64-channel electrode cap.

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

or other outside disturbances. Hence, preprocessing the original noisy EEG data was essential. In this study, EEGlab toolbox and Matlab 2014a were utilized to process the original data. First, EEG data were re- sampled down to 512 Hz for simplicity (Wang et al., 2014) and a band- pass FIR filter (0.5–30 Hz) was used to remove noise (Kar et al., 2010). Then, bad channels (terminology defined in Delorme et al. (2006)) were removed and independent component analysis (ICA) was performed to decompose the EEG data. ADJUST was used to automatically identify artifacts (Mognon et al., 2011). Then, bad channels were re-inter- polated by averaging two neighboring channels. Finally, the EEG data were re-referenced to the average reference and baseline corrections were performed. This procedure generally followed the basic pre- processing strategy provided by Buiatti and Mognon (2011). The data samples before and after preprocessing are illustrated in Fig. 4. After preprocessing, the EEG waves were smoother, and bad channels were removed and re-interpolated to minimize potential biases in the next average referencing stage.

2.5.3. EEG data analysis A fast Fourier transformation (FFT) was applied to extract 48 base

EEG features (3 FFT values × 4 EEG bands × 4 Brain regions) from the preprocessed EEG measurements. First, FFT was performed separately for each driving period. Then, the amplitude, log-transformed power (Otmani et al., 2005) and power spectral density of δ-band (0.5–3 Hz), θ-band (4–7 Hz), α-band (8–13 Hz), and β-band (14–30 Hz) were computed for each period. According to Yang et al. (2016), channels were divided into four different brain regions, including the frontal, parietal, occipital and temporal regions (see Table 3). Finally, the EEG features were further extracted separately from these four brain re- gions. In the next section, two core EEG features were extracted using the linear discriminant analysis (LDA) algorithm. Fig. 5 presents the EEG data processing procedure.

3. EEG-based driving behavior recognition method

This study proposed a two-layer EEG-based driving behavior re- cognition method (see Fig. 6). Layer I used driving data to classify driving behavior. Layer II dealt with driving behavior recognition based

on EEG data. In Layer II, the classification results of Layer I, combined with EEG data, were applied as inputs to train the kNN classifier. Then, the well-trained model from the two-layer driving behavior recognition system could identify driving behavior types using EEG data in- dependently. The proposed method was composed of three stages: (1) data processing; (2) feature selection and extraction; (3) clustering and classification.

3.1. Layer I: driving behavior classification based on driving data

In Layer I, five groups of driving behavior were obtained based on driving data and the following procedure: first, K-means was used to form two clusters of base driving features, respectively the driving style and stability (i.e. aggressive versus unaggressive, stable versus un- stable). Then, support vector machine (SVM) and SVM recursive feature elimination (SVM RFE) were applied for feature selection and extrac- tion. Finally, the driving behavior types were classified using K-means.

3.1.1. Algorithm overview A. K-means clustering. The K-means algorithm is an unsupervised learning method aiming to classify n observations into K clusters by minimizing the squared error over all K clusters. Let X = {xi, i = 1, …, n} represent the n d-dimensional observations, C = {ck, k = 1, …, K} be the K clusters and μk denote the mean of cluster ck. The squared error between μk and the observations in cluster ck is formulated as (Jain, 2010):

∑= ∥ − ∥ ∈

J c x μ( )k x c

i k 2

i k (1)

The objective function is to minimize the squared error of all clusters:

∑ ∑= ∥ − ∥ = ∈

J C x μ( ) k

x c i k

i k (2)

B. Support vector machine(SVM). SVM is a binary supervised learning model widely applied for classification and regression analyses (Yeo et al., 2009). The core content of SVM aims to find a hyperplane with maximum margins capable of partitioning observations from one group to the opposite group. Let → … ⎯→⎯x y x y( , ), , ( , )n n1 1 be the training data of n observations, where →xi is a d-dimensional vector and yi is either 1 or −1. The hyperplane can then be written as ⎯→⎯ → − =w x b. 0, where ⎯→⎯w is the vector of the hyperplane and

∥ ⎯→⎯

∥

b w

is the offset of the hyperplane

from the origin along vector ⎯→⎯w . In linear SVM, two parallel hyperplanes ⎯→⎯ → − = ±w x b. 1 are selected to separate the observations into two groups. The distance between the two parallel hyperplanes is called the “margin”, which is measured by

∥ ⎯→⎯

∥w 2 . Maximizing

∥ ⎯→⎯

∥w 2 is

Fig. 4. EEG data before and after preprocessing. (a) Example of EEG raw data. (b) Example of preprocessed EEG data.

Table 3 Channels division of four brain regions (Yang et al., 2016).

Brain regions Channels

Frontal FP1, FP2, FPZ, FCZ, FZ, AF3, AF4, F1-F8, FC1-FC4 Parietal C1-C4, CZ, CP1-CP4, CPZ, P1-P4, PZ Occipital PO3-PO8, POZ, O1, O2, OZ, CB1, CB2 Temporal FT7, FT8, FC5, FC6, T7, T8, C5, C6, TP7, TP8, CP5, CP6, P5-P8

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

equivalent to minimizing ∥⎯→⎯ ∥w1 2

2. Thus, the objective function and constraint are formulated as follows:

∥ ⎯→⎯

∥wmin 1 2

2 (3)

⎯→⎯ → + ≥ = …s t y w x b i l. . ( . ) 1, 1, 2, ,i (4)

The observations that fall on these two parallel hyperplanes are called support vectors. ⎯→⎯w is treated as a weight vector, which is a linear combination of the training data.

C. SVM recursive feature elimination (SVM RFE). Feature selection is essential for reducing data dimensionality and improving predictor performances. The feature ranking technique is very popular among feature selection methods for its simplicity, scalability, and empirical success (Guyon and Elisseeff, 2003). In feature ranking methods, a threshold is set based on a feature ranking criterion. Once this criterion exceeds its threshold, the corresponding feature will be reserved. RFE is a feature ranking algorithm with an iterative process, and SVM RFE is a feature selection method applying RFE with the weight magnitude as the ranking criterion (Guyon et al., 2002). In SVM RFE, the weight vector ⎯→⎯w trained by the SVM classifier is used to calculate the ranking criterion wi

2. The value of wi

2 represents the contribution of each feature. The algorithm of SVM RFE is described as follows: first, the SVM classifier is trained to determine the weight vector wi; then, each feature's ranking criterion (wi

2) is calculated; finally, the features are ranked in descending order and the tail features are deleted.

3.1.2. Driving behavior feature selection and extraction The base driving features were the mean values and standard de-

viations of the key variables (see Table 2) in each period. For the sake of computational efficiency, it was necessary to eliminate trivial features using feature selection and extraction following the procedure below:

Step 1: K-means was utilized to separately classify base driving features for driving style and stability.

The core of this step was to set the initial clustering centers. For driving style, the 14-dimensional data were divided into two

categories: (1) longitudinal acceleration, lateral acceleration, degree of ac- celeration pedal, acceleration, speed, relative speed, head deviation, lane deviation, and steering wheel angle; and (2) space headway, time headway, distance to yellow line, force of brake pedal, and deceleration. Large mean values in Category 1 characterize driver preferences for accelerations and lateral deviations, referred to as “aggressive driving style”, whereas large mean values in Category 2 characterize the willingness of drivers to maintain a distance from the leading vehicle and yellow line, as well as their frequency of braking, referred to as “conservative driving style”. Hence, one initial center consists of the maximum of each dimension in Category 1 combined with the minimum of each dimension in Category 2, representing “typical-aggressive behavior”, whereas the other center consists of the minimum of each dimension in Category 1 and the maximum of each di- mension in Category 2, representing “typical-unaggressive behavior”.

Large standard deviations in driving stability indicate unstable driving behavior, as such one initial center consists of the minimum values in each dimension, which represents “typical-stable behavior”, whereas the other consists of the maximum values in each dimension and represents “typical-unstable behavior”.

After setting the initial centers, the aggressive, unaggressive, stable, and unstable groups were clustered using K-means.

Step 2: Use SVM RFE to select effective driving features. In Step 2, SVM RFE was applied to the 14 base features and the

coefficient of each base feature was obtained (⎯→⎯w ), then the 4 base features with the lowest wi

2 were eliminated. Step 3: Use SVM to extract driving style and stability features. After this elimination, another SVM was applied to the remaining

base features, and a new set of coefficients, ⎯→⎯w and b, were obtained. Single driving features (a driving style feature and driving stability feature, respectively) were then calculated as (⎯→⎯ → −w x b· ).

3.1.3. Driving-data-based classification process After feature selection and extraction, two final driving behavior

features were obtained: the driving style feature and driving stability feature. K-means was utilized to perform this classification. The pro- cedure is described below:

Fig. 5. EEG data processing procedure.

Fig. 6. Framework of two-layer EEG-based driving behavior recogni- tion system.

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

First, the driving style and stability features were standardized using the Z-score method.

Next, the outliers of each feature were identified and treated. Outliers may be too far from other data points and would then be classified as a solo group. This would be an undesirable classification result, as the goal was to identify a center group with (at most) four surrounding groups. To avoid such outlier-dominated classification results, outliers must be treated before the clustering centers are chosen. According to Chebyshev's inequality (Saw et al., 1984), 88.89% of data points will be within 3 standard deviations of the mean, regardless the probability distribution. Therefore, data points beyond 3 standard deviations from the mean can be identified as outliers. Instead of using or eliminating the original outlier data, all outliers were modified to the boundary of 3 standard deviations.3

Then, the initial clustering centers were set. Sample data were de- noted as (x, y), where x represented aggressiveness and y represented stability. The origin (0, 0) implied that both aggressiveness and stability were at an average level, such that (0, 0) was set as the initial center for

“typical-normal behavior”. All points landing on the vector ⎯ →⎯⎯⎯⎯⎯⎯ (1, 1) have

both aggressiveness and stability above the average level. Thus, in this study, (1, 1) was chosen as the initial center for “typical-aggressive- stable behavior”. Analogously, (−1, 1), (−1, -1), and (1, −1) were selected as the initial centers for “typical-unaggressive-stable beha- vior”, “typical-unaggressive-unstable behavior”, and “typical-ag- gressive-unstable behavior”, respectively. The driving behavior features distribution and initial clustering centers are shown in Fig. 7.

Finally, the second iteration clustering centers were selected. These were not selected as the initial centers because the initial centers do not contain any information from the data distribution. Conversely, the cluster centers after the second iteration do not necessarily make better center selections (Abu-Mostafa et al., 2012). Thus, only two iterations were conducted to efficiently and sufficiently generate the final clus- tering centers. The clustering results are introduced in Section 4.

3.2. Layer II: driving behavior classification based on EEG

In Layer II, the base EEG features combined with the group labels of Layer I were used as inputs. Since the input data were imbalanced, the adaptive synthetic sampling approach (ADASYN) (He et al., 2008) was adopted to balance the data size. Then, feature selection and extraction were conducted via LDA. The k-nearest-neighbor (kNN) approach was utilized as a classifier, and a leave-one-subject-out cross validation was applied for evaluation.

3.2.1. Data balancing The classification results of Layer I were imbalanced, as shown in

Fig. 8. Taking the sample distribution of period 2 as an example, 65.38% of samples were classified into Class 5, whereas only 1.92% of samples were classified into Class 2. Imbalanced data is a common problem in machine learning and can mislead the classifier into over- fitting (such that the classifier always generates results in the same majority class). Hence, ADASYN was used as a sampling technique to generate synthetic samples to balance the minority classes. An addi- tional introduction to ADASYN is given in Appendix A.

3.2.2. EEG feature selection and extraction The base EEG features included the amplitude, log-transformed

power, and power spectral density of four EEG bands in four brain re- gions. LDA was used to perform both the feature selection and extrac- tion. The feature selection based on LDA (Song et al., 2010) was con- ducted by defining a ranking criterion based on eigenvectors. After

feature selection, fifteen (15) EEG features were reserved. Then, using another LDA to transform the selected EEG features into a two-di- mensional space, two core EEG features were extracted. The LDA al- gorithm is presented in Appendix B.

3.2.3. EEG-based classification process After feature selection and extraction, two core EEG features were ob-

tained. The kNN was adopted as the classifier, and a leave-one-subject-out cross validation was utilized for evaluation. The kNN algorithm is a typical non-parametric method for data mining and classification. In kNN, every sample can be represented by its k nearest neighbors. The key concept of the kNN algorithm that when the majority of k nearest neighbors for sample x belong to class i, this sample also belongs to class i. For simplicity, one good choice for selecting k is an odd number near N , where N is the total number of samples (Abu-Mostafa et al., 2012). If N happens to be an odd number, then =k N . If not, either of the two odd numbers nearest N can be selected arbitrarily. After data balancing, the total number of sam- ples in each period were 115, 170, 99, 79, 118, 73, respectively. Taking the first period as an example, the odd numbers near 115 were 9 and 11, and 9 was selected. The parameter k of the kNN classifier was 9, 13, 9, 9, 11, 7 for Periods 1 through 6, respectively.

4. Results

4.1. Driving-data-based classification results

K-means was used to classify driving behaviors based on driving data. After setting the initial clustering centers and performing the iteration steps, five driving behavior groups (aggressive-stable, unaggressive-stable,

Fig. 7. Distribution of driving behavior features and initial clustering centers (five centers which represent “typical-aggressive-stable behavior” in the northeast, “typical-un- aggressive-stable behavior” in the northwest, “typical unaggressive-unstable behavior” in the southwest, “typical-aggressive-unstable behavior” in the southeast, and “typical- normal behavior” in the center) for Periods 1–6). (a) Period 1. (b) Period 2. (c) Period 3. (d) Period 4. (e) Period 5. (f) Period 6.

3 Solely from the perspective of finding proper clustering centers, outliers should be eliminated given their distance from other data points; however, the outliers in this study also represent certain types of driving behaviors and thus contain valuable information for classification purposes.

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

unaggressive-unstable, aggressive-unstable, and normal) were obtained via the locations of the initial clustering centers (see Fig. 9).

It was observed that each of the final clustering centers followed a si- milar compass point (or direction) to its corresponding initial center, re- spectively. Therefore, the five clusters maintain the names of their corre- sponding driving behavior groups. One exception was Period 4, where the clustering results of Period 4 contained only four clusters, and the “un- aggressive-stable behavior” group was not represented by any data point.

4.2. EEG-based classification results

The kNN approach combined with the leave-one-subject-out cross

validation was adopted for EEG-based classification. To evaluate the performance of the proposed method, the confusion matrix, overall accuracy, and precision of each class were selected as performance measures. For ∈ …i j N, {1, , },c Mij representing the confusion matrix elements, Cj representing the set of samples from class j, and h(Ik) de- noting the predicted output of the classifier for sample Ik (Gabriella et al., 2011), the equations are as follows:

= ∈ =M I C h I i| { : ( ) } |k j kij (5)

∑ ∑= = =

M CAccuracy / | | j

j 1

jj 1

c c

(6)

∑= =

i M MPrecision ( ) / j

ii 1

(7)

The confusion matrix of Period 1 is presented in Table 4, in which the classification results were derived from the balanced data. Classes 1–5 re- present the five driving behavior groups mentioned above. Each row re- presents a predicted class, and each column represents a true class. The last column is the precision of each predicted class, describing the exactness of the classifier. In Period 1, Class 3 has the highest precision (“unaggressive- unstable behavior”) at 91.7%. The summary of classification accuracy and precision is presented in Table 5. The confusion matrices of the other per- iods are shown in Appendix C.

5. Discussion

In this study, the driving behavior type of participants in a multiple- traffic flow state simulation experiment was predicted based on EEG. The average accuracy was 69.5%, with the highest accuracy reaching 83.5%. The average precision of all periods was greater than 60% (see Table 5). Considering that this was a five-class classification problem, random chance would have resulted in 1/5 =20%; thus, the accuracy of these results is

Fig. 8. Sample distribution of each period. Each color represents a class, and the length of the colored rectangles represents the number of samples in the corresponding class. (For interpretation of the re- ferences to color in this figure legend, the reader is referred to the web version of the article.)

Fig. 9. Results of K-means clustering analysis based on driving data, (a)–(f) for Periods 1 through 6, respectively. Each type of driving behavior is a different color. Red: “aggressive-stable behavior”; dark blue: “unaggressive-stable behavior”; green: “unaggressive-unstable behavior”; pink: “aggressive-unstable behavior”; light blue: “normal behavior”. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

Table 4 Confusion matrix of Period 1.

Accuracy 70.4% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class 1 8 1 1 2 5 47.1% Predicted Class 2 2 20 0 2 3 74.1% Class Class 3 1 0 22 1 0 91.7%

Class 4 8 1 1 17 1 60.7% Class 5 4 0 0 1 14 73.7%

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

considerably improved from random selections. Additionally, the average accuracy of the former four steady traffic states was 72.7%, whereas the average accuracy of the latter two complementary traffic states was 63.0% (less than the steady states). This difference suggests that EEG is more closely related to driving behavior in steady traffic states.

Previous driving behavior classification studies have regarded driving behavior types as constant personalities, seldom considering the influence of driving conditions (Chung and Wong, 2010; Wang et al., 2010; Jensen et al., 2011; Chen et al., 2013). However, driving styles depend strongly on driving conditions (Dorr et al., 2014), including different road classes and traffic flow states. It can be seen in Fig. 8 that the sample distribution in each traffic flow state is quite different, and thus, the driving behavior types varied with the traffic states.

Driving stability and style were the two main factors considered in this paper. When the driving stability is predicted to be unstable, it suggests that the driver is less cautious in that period, increasing the chance of unin- tentional driving errors or lapses. If the driving style is predicted to be ag- gressive, it indicates that the driver is impatient in that period and more likely to take risks and make intentional violations. Consequently, driving behaviors containing either unstable or aggressive driving can be regarded as unsafe. Of the five identified driving behavior types, “aggressive-stable behavior”, “unaggressive-unstable behavior”, and “aggressive-unstable be- havior” were marked as unsafe, allowing the EEG-based driving behavior recognition method to be applied to detecting driving safety.

In addition, the EEG-based driving behavior recognition method can help driving assistance systems to better understand which behavioral group a driver currently belongs to, improving system attunement to per- sonalized criteria for personalized driving assistance. For instance, a very smooth driving history indicates that a driver has good driving maneuvers; however, if this type of driver began performing comparatively rough driving maneuvers and producing higher standard deviations in driving speed, the driving assistance systems could trigger warnings or even inter- vene. Conversely, for a driver with frequent and fast shifts in style and

stability, such as a “sporty” driver, the criteria for triggering warnings should be set higher to make potential warnings and interventions less in- terruptive and more impactful. This application would allow the perfor- mance of driver assistance systems to be improved by recognizing different driving behavior types.

As EEG can measure physiological responses to specific driving condi- tions, many researchers have used EEG to detect risky driving behaviors, such as fatigued and distracted driving (Kar et al., 2010; Chuang et al., 2015; Garcés Correa et al., 2014; Almahasneh et al., 2014). The data ana- lysis methods used in these studies were different from the EEG-based driving behavior recognition method in this study, but the original data sources were similar. These EEG-based methods can cooperate with each other to build an integrated system for improving driver assistance systems. Additionally, in the current state of development, manned driving and unmanned driving will coexist for a lengthy period. In this situation, inter- vehicle communication is highly significant, and EEG-based systems can improve the efficiency of this communication, helping unmanned vehicles respond and adjust to human driving behaviors.

Furthermore, conventional EEG acquisition systems require elec- trodes to be in contact with the human scalp, an obstacle for practical applications. However, with the development of biopotential signal acquisition technology, non-contact EEG sensors have emerged in the last decade (Sullivan et al., 2007; Chi and Cauwenberghs, 2009, 2010). Sun and Yu (2014) proposed a nonintrusive driver assistance system capable of collecting EEG signals without physical contact with human skin. Using these advances, EEG signals will be more easily acquired.

The primary contribution of this study is to establish an EEG-based driving behavior recognition system, which can recognize different driving behavior types independently from EEG data once the classifier is well trained with driving and EEG data. Stability and aggressiveness measurements were extracted to characterize driving behavior in this study; however, other types of measurements utilizing the proposed framework could also be investigated.

The current study encountered some limitations. Its sole focus was on the car-following process, whereas other daily driving processes such as lane-changing and turning movements were not considered. Additionally, as a general issue in all driving simulator studies, drivers’ perceptions of the leading vehicle and their surroundings could be biased in a simulated environment.

Future research will be required to expand the current study by considering additional driving processes. Actual driving experiments should be conducted to eliminate possible biases in driver perceptions.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (NFSC, no. 71471014) and the Fundamental Research Funds for the Central Universities (no. 2017YJS093).

Appendix A. Adaptive synthetic sampling approach (ADASYN)

Over-sampling is one of the solutions to imbalanced data classification problem, which can get balanced data by adding samples to the minority class. Synthetic Minority Over-sampling Technique (SMOTE) is a representative over-sampling technique. SMOTE creates new minority class samples by interpolation method rather than replicating minority class samples, thus to avoid over-fitting (Jeatrakul and Wong, 2012). ADASYN (He et al.,

Table 5 Summary of classification accuracy and precision.

Period Accuracy Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Period 1 70.4% 47.1% 74.1% 91.7% 60.7% 73.7% Period 2 83.5% 71.1% 94.4% 85.0% 86.5% 78.9% Period 3 69.7% 53.8% 74.1% 90.9% 63.0% 50.0% Period 4 67.1% 62.5% – 73.1% 82.6% 45.5% Period 5 64.4% 61.9% 64.3% 62.5% 66.7% 71.4% Period 6 61.6% 66.7% 70.0% 46.2% 76.5% 42.9% Average 69.5% 60.5% 75.4% 74.9% 72.7% 60.4%

The number of samples representing Period 4 in Class 2 was zero, and thus, no precision value is given for Period 4 of Class 2.

Table A.1 Confusion matrix of Period 1.

Accuracy 70.4% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class 1 8 1 1 2 5 47.1% Predicted Class 2 2 20 0 2 3 74.1% class Class 3 1 0 22 1 0 91.7%

Class 4 8 1 1 17 1 60.7% Class 5 4 0 0 1 14 73.7%

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

2008) is an improved synthetic sampling method based on SMOTE, which can shift the classifier decision boundary toward the difficult to learn examples. There are two key steps in ADASYN:

Let {xi, yi}, i = 1, 2, …, n be the training data, where xi is a d-dimensional vector and yi is either 1 or -1. Let nl be the number of samples in majority class and ns be the number of samples in minority class, therefore nl + ns = n. Nl denotes the set of samples in majority class while Ns denotes the set of samples in minority class. For xi ∈ Ns, K nearest neighbors are found based on Euclidean distance.

Step 1: Calculate the density distribution r̂i :

Table A.2 Confusion matrix of Period 2.

Accuracy 83.5% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class1 27 0 0 0 11 71.1% Predicted Class2 0 34 0 0 2 94.4% class Class3 4 0 34 0 2 85.0%

Class4 1 0 0 32 4 86.5% Class5 2 0 0 2 15 78.9%

Table A.3 Confusion matrix of Period 3.

Accuracy 69.7% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class1 7 1 0 1 4 53.8% Predicted Class2 4 20 0 1 2 74.1% class Class3 1 0 20 0 1 90.9%

Class4 3 1 0 17 6 63.0% Class5 5 0 0 0 5 50.0%

Table A.4 Confusion matrix of Period 4.

Accuracy 67.1% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class1 5 0 0 0 3 62.5% Predicted Class2 0 0 0 0 0 - class Class3 2 0 19 1 4 73.1%

Class4 2 0 0 19 2 82.6% Class5 10 0 0 2 10 45.5%

Table A.5 Confusion matrix of Period 5.

Accuracy 64.4% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class1 13 5 0 0 3 61.9% Predicted Class2 3 18 2 0 5 64.3% class Class3 3 4 20 3 2 62.5%

Class4 1 0 0 20 9 66.7% Class5 2 0 0 0 5 71.4%

Table A.6 Confusion matrix of Period 6.

Accuracy 61.6% True class Precision

Class 1 Class 2 Class 3 Class 4 Class 5

Class1 6 1 0 2 0 66.7% Predicted Class2 2 14 1 0 3 70.0% class Class3 0 1 6 0 6 46.2%

Class4 3 0 1 13 0 76.5% Class5 3 1 4 0 6 42.9%

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

∑= =

r r rˆ /i i i

i 1

(A.1)

= = …r δ K i n/ , 1, ,i i s (A.2)

where δi is the number of samples that belongs to the majority class out of the K nearest neighbors of xi. The value of K is associated with the value of N, and can be N as practically accepted (Abu-Mostafa et al., 2012). ri is the ratio of δi to K. r̂i is the normalized ri, which can measure the complexity of learning for xi ∈ Ns.

Step 2: Randomly select one sample, xzi, from K nearest neighbors of xi and calculate the synthetic sample si:

= + − ×s x x x λ( )i i izi (A.3)

where λ is a random number and λ ∈ [0, 1]. ADASYN is designed for two-class problem (He et al., 2008). To adjust ADASYN for multi-class classification problem, one-against-one method is

introduced. First, find the majority class while other classes are marked as minority classes; then combine the majority class with a minority class one by one and apply ADASYN; finally, each minority class will have nearly equal number of samples with the majority class.

Appendix B. Linear discriminant analysis (LDA)

LDA is widely used in pattern recognition, statistics and dimension reduction before classification. The key idea of LDA is to find a linear combination of features to maximize between-class mean value and minimize within-class variance. Hence, these features can characterize each class of observations in different aspects. The within-class and between-class scatter matrix for C classes are expressed by Song et al. (2010):

∑ ∑= − − = =

S u u u u( ) ( )w i

i j

i i j

i T

1 1

(B.1)

∑= − − =

S n u u u u( ) ( )b i

i i i T

1 (B.2)

where ni is the number of samples in class i, ui j is the j-th sample of class i, ui is the mean of class i and u is the mean of all samples. The function of

LDA is as follows:

=S x λS xb w (B.3)

Let λ1 ≥ λ2 ≥ ⋯ ≥ λr be the eigenvalues of Eq. (10), X1, X2, …, Xr be the corresponding eigenvectors. Generally, the first d largest eigenvalues are selected to transform the sample into a d-dimensional vector.

Appendix C. Confusion matrix of each driving period

References

Abu-Mostafa, Y.S., Magdon-Ismail, M., Lin, H.T., 2012. Learning from Data. AMLBook, New York, USA.

Almahasneh, H., Chooi, W.T., Kamel, N., Malik, A.S., 2014. Deep in thought while driving: an EEG study on drivers’ cognitive distraction. Transp. Res. Part F: Traffic Psychol. Behav. 26, 218–226.

Bar, T., Nienhüser, D., Kohlhaas, R., Zöllner, J.M., 2011. Probabilistic driving style de- termination by means of a situation based analysis of the vehicle data. 14th IEEE Conference on Intelligent Transportation Systems 1698–1703.

Buiatti, M., Mognon, A., 2011. ADJUST: An Automatic EEG Artifact Detector Based on the Joint Use of Spatial and Temporal features.

Chen, S.W., Fang, C.Y., Tien, C.T., 2013. Driving behaviour modelling system based on graph construction. Transp. Res. Part C: Emerg. Technol. 26, 314–330.

Chi, Y.M., Cauwenberghs, G., 2009. Micropower non-contact EEG electrode with active common-mode noise suppression and input capacitance cancellation. Annual International Conference of the IEEE Engineering in Medicine and Biology Society 4218–4221.

Chi, Y.M., Cauwenberghs, G., 2010. Wireless non-contact EEG/ECG electrodes for body sensor networks. International Conference on Body Sensor Networks 297–301.

Choi, S., Kim, J., Kwak, D., Angkititrakul, P., Hansen, J.H.L., 2007. Analysis and classi- fication of driver behavior using in-vehicle can-bus information. Biennial Workshop on DSP for In-Vehicle and Mobile Systems 17–19.

Chuang, C.H., Huang, C.S., Ko, L.W., Lin, C.T., 2015. An EEG-based perceptual function integration network for application to drowsy driving. Knowl. Based Syst. 80, 143–152.

Chung, Y., Wong, J., 2010. Investigating driving styles and their connections to speeding and accident experience. J. East. Asia Soc. Transp. Stud. 8, 1944–1958.

Constantinescu, Z., Marinoiu, C., Vladoiu, M., 2010. Driving style analysis using data mining techniques. Int. J. Comput. Commun. Control 5, 654–663.

Delorme, A., Fernsler, T., Serby, H., Makeig, S., 2006. EEGLAB Tutorial. Dorr, D., Grabengiesser, D., Gauterin, F., 2014. Online driving style recognition using

fuzzy logic. 17th IEEE International Conference on Intelligent Transportation Systems 1021–1026.

Gabriella, C., Christopher, R.D., Lixin, F., Jutta, W., Cédric, B., 2011. Visual categoriza- tion with bags of keypoints. Workshop on statistical learning in computer vision, ECCV 179–191.

Garcés Correa, A., Orosco, L., Laciar, E., 2014. Automatic detection of drowsiness in EEG records based on multimodal analysis. Med. Eng. Phys. 36, 244–249.

Guan, W., He, S., 2008. Statistical features of traffic flow on urban freeways. Phys. A: Stat. Mech. Appl. 387, 944–954.

Guyon, I., Elisseeff, A., 2003. An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182.

Guyon, I., Weston, J., Stepthen, B., 2002. Gene selection for cancer classification. Mach. Learn. 46, 389–422.

He, H., Bai, Y., Garcia, E.A., Li, S., 2008. ADASYN: Adaptive synthetic sampling approach for imbalanced learning. Neural Networks. IJCNN 2008. (IEEE World Congress on Computational Intelligence) 1322–1328.

Jain, A.K., 2010. Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31, 651–666.

Jeatrakul, P., Wong, K.W., 2012. Enhancing classification performance of multi-class imbalanced data using the OAA-DB algorithm. IEEE World Congress on Computational Intelligence 1–8.

Jensen, M., Wagner, J., Alexander, K., 2011. Analysis of in-vehicle driver behaviour data for improved safety. Int. J. Veh. Saf. 5, 197.

Johnson, D.A., Trivedi, M.M., 2011. Driving style recognition using a smartphone as a sensor platform. 14th International IEEE Conference on Transportation Systems 1609–1615.

Kar, S., Bhagat, M., Routray, A., 2010. EEG signal analysis for the assessment and quantification of driver's fatigue. Transp. Res. Part F: Traffic Psychol. Behav. 13, 297–306.

Lajunen, T., Summala, H., 1995. Driving experience, personality, and skill and safety- motive dimensions in drivers’ self-assessments. Personal. Individ. Differ. 19, 307–318.

Li, X., Yan, X., Wu, J., Radwan, E., Zhang, Y., 2016. A rear-end collision risk assessment model based on drivers’ collision avoidance process under influences of cell phone use and gender - A driving simulator based study. Accid. Anal. Prev. 97, 1–18.

Lin, C.T., Wu, R.C., Liang, S.F., Chao, W.H., Chen, Y.J., Jung, T.P., 2005. EEG-based drowsiness estimation for safety driving using independent component analysis. IEEE

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40

Trans. Circuits Syst. I: Reg. Pap. 52, 2726–2738. Martinussen, L.M., Møller, M., Prato, C.G., 2014. Assessing the relationship between the

Driver Behavior Questionnaire and the Driver Skill Inventory: revealing sub-groups of drivers. Transp. Res. Part F: Traffic Psychol. Behav. 26, 82–91.

Mognon, A., Jovicich, J., Bruzzone, L., 2011. ADJUST: an automatic EEG artifact detector based on the joint use of spatial and temporal features. Psychophysiology 48, 229–240.

Otmani, S., Pebayle, T., Roge, J., Muzet, A., 2005. Effect of driving duration and partial sleep deprivation on subsequent alertness and performance of car drivers. Physiol. Behav. 84, 715–724.

Papadelis, C., Chen, Z., Kourtidou-Papadeli, C., Bamidis, P.D., Chouvarda, I., Bekiaris, E., Maglaveras, N., 2007. Monitoring sleepiness with on-board electrophysiological re- cordings for preventing sleep-deprived traffic accidents. Clin. Neurophysiol. 118, 1906–1922.

Reason, J., Manstead, A., Stradling, S., Baxter, J., Campbell, K., 1990. Errors and viola- tions on the roads: a real distinction? Ergonomics 33, 1315–1332.

Saw, J.G., Yang, M.C.K., Mo, T.C., Mo, T.S.E.C., 1984. Chebyshev inequality with esti- mated mean and variance. Am. Stat. 38, 130–132.

Song, F., Mei, D., Li, H., 2010. Feature selection based on linear discriminant analysis. International Conference on Intelligent System Design and Engineering Application 746–749.

Sullivan, T.J., Deiss, S.R., Cauwenberghs, G., 2007. A low-noise, non-contact EEG/ECG

sensor. Biomedical Circuits and Systems Conference 154–157. Sun, Y., Yu, X., 2014. An innovative nonintrusive driver assistance system for vital signal

monitoring. IEEE J. Biomed. Health Inform. 18, 1932–1939. Taubman-Ben-Ari, O., Mikulincer, M., Gillath, O., 2004. The multidimensional driving

style inventory - scale construct and validation. Accid. Anal. Prev. 36, 323–332. Van Ly, M., Martin, S., Trivedi, M.M., 2013. Driver classification and driving style re-

cognition using inertial sensors. IEEE Intelligent Vehicles Symposium 1040–1045. Wang, J., Lu, X., Xiao, Q., Lu, M., 2010. Comparison of driver classification based on

subjective evaluation and objective experiment. Transp. Res. Rec. 224, 1–13. Wang, S.Y., Zhang, Y.Q., Wu, C.X., Darvas, F., Chaovalitwongse, W.A., 2015. Online

prediction of driver distraction based on brain activity patterns. IEEE Trans. Intell. Transp. Syst. 16, 136–150.

Wang, Y.K., Chen, S.A., Lin, C.T., 2014. An EEG-based brain-computer interface for dual task driving detection. Neurocomputing 129, 85–93.

Yang, L., He, Z., Guan, W., Jiang, S., 2016. Exploring the Relationship Between EEG and Ordinary Driving Behavior: A Simulated Driving Study.

Yeo, M.V.M., Li, X., Shen, K., Wilder-Smith, E.P.V., 2009. Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf. Sci. 47, 115–124.

Zhang, L., Jianqiang, W., Furui, Y., Keqiang, L., 2009. A quantification method of driver characteristics based on driver behavior questionnaire. In: Intelligent Vehicles Symposium. IEEE. pp. 616–620.

L. Yang et al. Accident Analysis and Prevention 116 (2018) 30–40