Research Paper

Contents lists available at ScienceDirect

Accident Analysis and Prevention

journal homepage: www.elsevier.com/locate/aap

Modular design of fatigue detection in naturalistic driving environments

Hilal Al-libawya,b,⁎, Ali Al-Atabya, Waleed Al-Nuaimya, Majid A. Al-Taeea

a Department of Electrical Engineering and Electronics, University of Liverpool, Brownlow Hill, Liverpool L69 3GJ, UK b Department of Electrical Engineering, University of Babylon, Iraq

A R T I C L E I N F O

Keywords: Bayesian combiner Driver fatigue Machine learning Modular structure Performance optimisation Particle swarm optimisation

A B S T R A C T

Research in driver mental fatigue is motivated by the fact that errors made by drivers often have life-threatening consequences. This paper proposes a new modular design approach for the early detection of driver fatigue system taking into account optimisation of system performance using particle swarm optimisation (PSO). The proposed system is designed and implemented using an existing dataset that was simultaneously collected from participants and vehicles in a naturalistic environment. Four types of data are considered as fatigue-related metrics including: vehicle acceleration, vehicle rotation pattern, driver's head position and driver's head rota- tion. The driver's blink rate data is used in this work as a proxy for ground truth for the classification algorithm. The collected data elements are initially fed to input modules represented by ternary neural network classifiers that estimates alertness. A Bayesian algorithm with PSO is then used to combine and optimise detection per- formance based on the number of existing input modules as well as their output states. Performance of the developed fatigue-detection system is assessed experimentally with a small data samples of driver trips. The obtained results are found in agreement with the state-of-the-art in terms of accuracy (90.4%), sensitivity (92.6%) and specificity (90.7%). These results are achieved with significant design flexibility and robustness against partial loss of input data source(s). However, due to small sample size of dataset (N = 3), a larger dataset need to be tested with the same system framework to generalise the findings of this work.

1. Introduction

Reliable and robust driver fatigue detection systems are becoming essential requirements for road safety due to the dangerous and often fatal consequences of road accidents caused by fatigued drivers. The onset of mental fatigue is usually accompanied by slow reaction time, impaired judgement and may ultimately lead to falling asleep behind the steering wheel. Road accidents caused by fatigued drivers can have fatal and devastation consequences (Centre for Road Safety, 2016). Numerous driver fatigue symptoms have been reported in the literature along with relevant systems used to detect them (Abbood et al., 2014; Stork et al., 2015), with varying degree of success.

Different swarm optimisation methods have been proposed in the transportation sector to find the optimal traffic network situation (Sharma and Kumari, 2015; Sandberg and Wahde, 2008). However, only few studies have identified the use of optimisation techniques to enhance the performance level of the modular organisation (Durán et al., 2012). Despite the important contributions reported in these studies and others, identifying a practical, robust and flexible fatigue- detection approach remains an elusive goal.

In this work, a scalable modular design approach is considered to

build a system using a Bayesian combiner and a particle swarm opti- miser (PSO). This enables the utilisation of input modules depending on availability. The Bayesian combiner, which improves the detection accuracy with the aid of PSO, deals with the number of existing input modules as well as their states (i.e. alert, mild fatigue and fatigued). Unlike existing systems, this makes the proposed system flexible in terms of the available input data and more robust against losing one or more data sources.

The paper is organised as follows. A theoretical background on modularity, Bayesian algorithm and PSO is presented in Section 3. Section 4 details the study dataset. The overview for the proposed system architectures and methodology are described in Section 5. The obtained results are presented and discussed in Section 6. Finally, the work is concluded in Section 7.

2. Related work

Mental fatigue evaluation has been reported in a numerous amount of research articles (Meiring and Myburgh, 2015; Al-Libawy et al., 2016a, 2016b, 2017). The available driver fatigue-detection literature has agreed to categorise fatigue-related symptoms into three categories

https://doi.org/10.1016/j.aap.2018.08.012 Received 11 July 2017; Received in revised form 3 March 2018; Accepted 13 August 2018

⁎ Corresponding author at: Department of Electrical Engineering and Electronics, University of Liverpool, Brownlow Hill, Liverpool L69 3GJ, UK. E-mail address: [email protected] (H. Al-libawy).

Accident Analysis and Prevention 120 (2018) 188–194

Available online 28 August 2018 0001-4575/ © 2018 Elsevier Ltd. All rights reserved.

T

based on either generation or detection perspectives. The first category includes the biological symptoms which capture the development of driver fatigue using metrics such as heart rate, skin temperature and skin conductivity (Al-Libawy et al., 2015). Different detection techni- ques and devices are used to detect the biological signs of fatigue such as wearable devices or sensors built into the vehicle (Zhang, et al., 2017). The second is the behavioural category which is mainly noticed from driving style (Engelbrecht et al., 2015). Metrics such as steering wheel angle, acceleration pattern, lateral movements and braking pat- tern are the main measures used to quantify fatigue status of the driver (Meiring and Myburgh, 2015; Li et al., 2017). The last category is visual signs of drivers, such as facial features, percentage of eye closure, blink rate, yawn rate and others (Sigari et al., 2014).

More recent works of fatigue-detection systems have used more than one fatigue-related metric to improve the detection accuracy (Stork et al., 2015). The detection system fuses and combine the cal- culated metrics in three main levels. The first is the raw or filtered data level which is used when different sensors measure the same metric (Koenig et al., 2015). The second is the features level and can detect fatigue status from features even when they are extracted from different types of sensors (Yin et al., 2016). The last level is the abstract or the decision level which combines multi-module outputs to calculate an enhanced and accurate output. Several integrated driver fatigued de- tection systems have been developed that combine different classifiers decisions using different fatigue-related metrics (Craye et al., 2016). Despite the good results obtained from simulated environments, the robustness of these decision-combining systems was not tested under condition of partial loss of input data.

While several experiments were carried out to measure and quantify driver behaviour in real environments, only a small portion of these was focused on driver fatigue in naturalistic environments (Fu et al., 2016; Li et al., 2017). The work reported in Fu et al. (2016) adopted three fatigue-related physiological metrics (EEG, EMG and respiration rate), but the method for measuring these metrics is not very practical in real environments. The results in publication Li et al. (2017) are calculated based on one metric (steering wheel angle) which may not be available in many vehicles and require third party devices to be installed.

3. Theoretical background

This section presents a theoretical background for three key aspects of the proposed work: the importance of modularity, Bayesian com- biner and PSO.

3.1. Importance of modularity

The need for a system that can combine heterogeneous subsystems (modules) efficiently is behind the idea of “interdependence within and independence across modules” (Hatch, 2001) which is one of the defini- tions of modularity. Modular design is preferred over the integral de- sign due to its benefits including (Avigad, 2016; Pradhan et al., 2011):

(a) Reliability and robustness. The modular structure enhances the ro- bustness and improves the reliability of certain system if they are designed properly to maintain their functional mission even if they lose one or more of their modules.

(b) Flexibility. The need for a flexible system is an essential practical requirement (Sanchez and Mahoney, 1996). For a fatigue-detection system where a variety of detection methods are available, it is very important to design the system to be flexible to accommodate a variety of configurations for different working environments.

(c) Comprehensibility. In complex systems such as fatigue-detection systems, modular structure makes them more understandable and easier to handle on a module level.

(d) Independence. A primary motivation of using a modular structure is the independence of each individual module from other modules.

This feature is very helpful for modular systems to integrate het- erogeneous modules together without requiring agreement on in- ternal details.

(e) Abstraction level. This level is responsible for the interfacing be- tween modules. The abstraction and independence of the modules help to make the system more practical and flexible.

3.2. Bayesian combiner

The modular structure system needs a frame to be plugged in to produce the combined final output. Several combination algorithms have been proposed to combine heterogeneous sets of modules (e.g. majority voting, weighted majority voting, or Bayesian combiner) (Bahler and Navarro, 2000; Kuncheva, 2004). The Bayesian combiner is ideally suited for problems when the output of the modules is in- dependent even when the number of modules is dropped to two (Kim and Ghahramani, 2012).

The Bayesian combiner works at the abstraction level of the output of L modules, each module Mi predicts class label bi ∈ H, i = 1, …, L. So, any input set x ∈ Rn, can be combined, the L module outputs produce a vector b = [b1, …, bL]

T ∈ HL. Bayes’ theorem Vapnik and Vapnik (1998) of probability computes the posterior probability of module Mi based on the prior probability P(hj) where hj is the actual class label (hj ∈ H, j = 1, …, C and C is the number of classes); as well as based on the likelihood P(b|hj) of the evidence b. The independence assumption is maintained and allows the conditional probability of the module Mi labels the input x in class bi ∈ H to be presented as follows:

∏= … = =

bP h P b b b h P b h( | ) ( , , , | ) ( | )j L j i

L

i j1 2 1 (1)

Bayes’ rule can be described mathematically as follows:

=b b

b P h

P P h P

( ) ( | ) ( )

( ) j

j j

(2)

3.3. Particle swarm optimisation (PSO)

Particle swarm optimisation is a meta-heuristic algorithm devised by Kennedy and Eberhart in 1995 Engelbrecht (2006) which is inspired by the social behaviour of birds. Several versions of PSO were derived later to cover new applications or to address some limitations and challenges discovered with the original version (Poli et al., 2007).

PSO can find the optimal solution for an optimisation problem in a D-dimensional hyperspace. A swarm of N particles is recruited to find the best position according to the individual perspective (Pbest) and the overall perspective (Gbest) (Lazinica, 2009). Each particle tries to up- date its position (solution) to achieve the best fitness value and mini- mise the cost function. The update stochastic function is based on three parts: inertia part, self-knowledge part and team-work part. The update rule is determined as follows:

= + × − + × −+v c R x c R xwv (Pbest ) (Gbest )i k

i k

i k

i k

i k

i k1

1 1 2 2 (3)

= ++ +x x vi k

i k

i k1 1 (4)

where w is the inertia (habitual behaviour) weight, c1 is an acceleration constant of the self-knowledge (memory) component, c2 is an accel- eration constant of the team-work component, and R1 and R2 are random numbers.

PSO is used in this work to improve the performance of the Bayesian combiner. The cost function of the optimiser is a function of the com- biner accuracy, while the best solution will be represented in the best set of weights related to each module (it is not related to the inertia weight of the optimiser update rule in Eq. (3)).

H. Al-libawy et al. Accident Analysis and Prevention 120 (2018) 188–194

189

4. Study dataset

An existing dataset Transportation Research Board of the national Academics of Science (2017) is employed in this study: The Naturalistic Driving Study carried out by the Second Strategic Highway Research Program. The central goal of this dataset is to evaluate the role of driver performance and behaviour in traffic safety. It is the largest study of its kind with over 2000 drivers. The data acquisition system, which were used to collect this dataset include: forward radar; four video cameras, including one forward-facing, colour, wide-angle view; accelerometers; vehicle network information; GPS; onboard computer vision lane tracking, plus other computer vision algorithms; and data storage capabilities.

The selected data for this work is restricted by resource limitations and as well as the type of data (i.e. fatigue-related data). The dataset- generator institute offers six trips as an example dataset. Hence, this work adopts data collected for only three out of six trips, each of which last for 90 min. This period can be considered adequate for fatigue development over time. The excluded trips are deemed unsuitable for our study restrictions. These restrictions include: time of trip (less than 90 minutes), poor quality of camera video data or insufficient fatigue related metrics. So, three trips belonging to three different drivers are selected the data includes the driver data (head rotation and head po- sition) as well as the vehicle data (acceleration and rotation patterns).

The most common four metrics relevant to fatigue detection are considered in this study: 3-axes accelerometer, gyroscope, head posi- tion and head rotation. The first two metrics are collected with a sampling rate of 10 Hz, while the other two are dynamically calculated relative to a baseline with a sampling rate of 2–15 Hz. The head movement metrics (head position and head rotation) are chosen in this work because they are known to be fatigue related (Qiang et al., 2004). These metrics are pre-processed from missing slots and noise using a moving average with a 15 sample window.

5. System overview and methodology

A block diagram of the proposed system is shown in Fig. 1. As il- lustrated, the system incorporates several inertial and visual sensors, and is designed and implemented using an existing dataset offered by SHRP2NDS Transportation Research Board of the national Academics of Science (2017). The modular design of the proposed system is adopted for robustness, flexibility and practical implementation ad- vantages.

The system comprises three main stages. The first is the classifica- tion and labelling stage which consists of four classifications modules with their labelling component. The second stage combines the output of the classifiers based on the Bayesian combiner. Finally, the last stage is the PSO which is used to enhance the accuracy of the fatigue-

detection system by assigning an optimal set of weight combinations for the modules. These stages are described as follows.

5.1. Classification and labelling (Modules)

Four fatigue-related modules are built for this work to demonstrate the modularity structure as shown in Fig. 1. These modules are: (i) accelerometer data module, (ii) gyroscope data module, (iii) head po- sition data module and (iv) head rotation data module. The driver and the driven vehicle data are fed into four ternary artificial neural net- works (ANNs) classifiers, each of which generates a status output of an alert, mild fatigue or fatigued as shown in Fig. 2. The ANN classifier is a parallel, distributed computing approach inspired by human brain and the biological neural networks. It consists of processing elements called neurons which are interconnected together by weighted connections. It is a supervised learning algorithm used to learn systems by considering examples without the need for task-specific programming Wang (2003), Hecht-Nielsen (1989).

In the absence of any external ground truth to validate the data, it was necessary to use the video data itself to manually extract fatigue- related metrics. It is widely accepted that metrics such as blink rate, blink duration and percentage of eyelid closure over the pupil (PERCLOS) are reliable indicators of fatigue Tansakul and Tangamchit (2016), Stern et al. (1994), Haq and Hasan (2016), People et al. (1998), Wang et al. (2017). The resolution and quality of the available video data was such that the demonstrably most accurate of these measures, PERCLOS (People et al., 1998) could not be extracted, as the experi- ment had not been set up with such measurements in mind. Conse- quently, the data was processed manually to extract the blink frequency (in occurrences per minute), and used as a proxy for ground truth to label the data.

Six features are extracted and fed to each classifier: mean and standard deviation of each axis of the three-dimensional values of the fatigue-related metrics (i.e., acceleration, gyroscope, head-position and head-rotation). Different number of hidden layers and number of neu- rons in these layers are used to construct the four ANN classifiers. This variation in ANNs architectures are purposely adopted in addition to the different seeds of random generation of ANNs weight initialisation in order to maintain the independence of the modules decision.

5.2. Bayesian combiner

Outputs of the ANN classifiers are fed to the Bayesian combiner to improve system performance irrespective of the number input modules. Bayes’ rule can be further refined using Eqs. (1 and 2) to obtain P(hj ∣ b) as follows:

Fig. 1. Modular architecture of the proposed system.

H. Al-libawy et al. Accident Analysis and Prevention 120 (2018) 188–194

190

= ∏

= …=b b

P h P h P b h

P j( )

( ) ( ) ( )

, 1, , 3j j i

L i j1

(5)

In this work, three classes modules (classifiers) are implemented (c = 3) and different combinations sizes are attempted (L = 4, L = 3 and L = 2). The confusion matrix of each classifier is assumed to be an approximation of posterior probability. The Bayesian combiner can predict the combined output based on Eq. (5) with extra modification. Since the combined output depends on the outputs of all the modules, the probabilities of the modules’ outputs can be ignored and the probability of prediction β(x) for a certain input to be classified as a specific class will be proportional to:

∏∝ =

xβ P h P b h( ) ( ) ( )j j i

L

i j 1 (6)

The confusion matrix CMi of each classifier is calculated using the training set and a matrix of 3 × 3 (based on number of classes) is produced. The prior probability of the class hj can be considered as an estimate of Nj/N where Nj is the number of outputs classified as the class hj and N is the total number of outputs. Since the confusion matrix element cm j b

i , i can estimate the probability P(bi ∣ hj), this results in the

following:

∏∝ =

xβ N N N

( ) cm

j j

i

j b i

j1

4 , i

(7)

Finally, the class that gains maximum probability, will be chosen as the predicted label. Moreover, to satisfy the required condition for Bayes’ theory Kuncheva (2004), the generated outputs of the classifiers are considered independent. Such an assumption is maintained through utilisation of different sets of features and different seeds for the ANN initial weights. Since the Bayesian combiner that deals with abstracted output labels does not depend on the classification process, the number of input modules can vary depending on the available sources of driver and vehicle data, thus improving the flexibility and modularity attri- butes of the system.

5.3. Particle swarm optimiser

The fatigue-related metrics differ in their responses to fatigue, thus each module has a different weight in fatigue identification. Although the combiner usually enhances the performance of the ensemble clas- sifiers, a further step is needed to weight each classifier with its share or contribution to the combiner's performance. A particle swarm optimiser is adopted in this work to find the sets of optimal weights ω of each metrics/module. The fast implementation, low computational cost and the absence of a need for a derivative function are the reasons for choosing PSO.

A new modification is proposed in this work to give weights to the classifiers outputs by adding some changes to the combiner method. This rule is implemented by strengthening the true output class and weakening the false one for the module which is expected to have the

best response to fatigue and as follows:

= ⎧ ⎨ ⎩

× =

× ≠ˆ ω j bj bcm cmcmj bi i j bi iω j bi i, ,1 ,i ii i (8) whereˆcm j bi , i is the weighted confusion matrix element.The optimiser keeps updating the weights until the cost function ε isminimised below a certain value or the iteration of the PSO algorithm exceeds a set value.

The PSO algorithm is performed once in optimisation phase while the optimised weights are used in real-time phase. Thus, in this work, three sets of weights corresponding to three groups of input modules are updated in the optimisation phase. Then, the optimised weights are used based on available input modules.

6. Results and discussion

The findings have demonstrated ability to detect fatigue status from the dataset used in this study along with the trend of fatigue growth over time. The findings are detailed in the following subsections.

6.1. Classification and labelling

Each module predicts one of the three driver fatigue statuses using a trained classifier that is labelled based on blink rate level. Blink rate is calculated manually from the available sample videos by dividing the video into time slots of 15 min, randomly selecting five minutes from the 15 min slot, and are then averaged to produce one figure re- presenting the blink rate of the 15 min slot.

Three levels of blink rate is created to label three classes for the classifiers, as shown in Fig. 3. The trend of three trips is very clear to be growing with time of trip and in turn with fatigue evolution. It is worth pointing out that the three drivers have different trip durations and they and their blink rate growth have responded differently according to their fatigue attack. The selection of the three levels with the choice of boundaries needs more investigation and this will be achievable when more datasets and videos are available.

The four feed-forward ANN classifiers (modules) are built with different architecture (number of hidden layers and number of neurons in each hidden layer) to sustain the independence decision which is a requirement for the Bayesian combiner. Each module has six inputs to fit with the six features (mean and standard deviation of the 3-axes’ sensor data) and one ternary output for the three classes. A back-pro- pagation (BP) algorithm is used to train the modules. The BP algorithm is a method used to train ANNs and to calculate the error contribution of each neuron after dataset is processed. As a learning algorithm, back- propagation often use the gradient descent optimization algorithm to update the weights of neurons by calculating the gradient of the cost function. It is called back-propagation because the error is calculated at the output layer first and distributed back through the neural network layers (Hecht-Nielsen, 1989). The dataset is divided into two sets: 70%

Fig. 2. An example module.

H. Al-libawy et al. Accident Analysis and Prevention 120 (2018) 188–194

191

for training and the rest is used to test the system and calculate the performance of the classifiers.

Performance of the ANN classifiers was assessed in terms of accu- racy, sensitivity and specificity. These performance metrics can be de- scribed based on several terms including; true positive (tp: correctly identified), true negative (tn: correctly rejected), false negative (fn: in- correctly rejected), and false positive (fp: incorrectly identified). These metrics are defined briefly as follows Sokolova and Lapalme (2009), Al- Hyari et al. (2013, 2014):

(a) Accuracy – reflects how precisely the dataset is classified, measures the proportion of the correctly classified instances. In multi-classes, it represents the average per-class effectiveness of the classifier,

∑ +

+ + +=C 1 tp tn

tp fn fp tnj

C j j

j j j j1 (9)

(b) Sensitivity – measures the fraction of positive instances which are correctly classified to the amount of positive part of the dataset. In multi-classes, it represents the average per-class effectiveness of the classifier in order to identify the class labels,

∑ +=C

1 tp tp fnj

C j

j j1 (10)

(c) Specificity – measures the proportion of the negative instances which are correctly classified to all negative instances in the da- taset. In multi-classes, it reflects how effectively negatives are la- belled,

∑ +=C

1 tn tn fpj

C j

j j1 (11)

The performance metrics that are calculated for each class of the ternary classifiers are shown in Table 1. The overall metrics are then computed by averaging the individual metric over the three classes. It can be noticed that the performance of the visual data (i.e. head posi- tion and rotation) is superior to that of the inertial data (i.e. accel- erometer and gyroscope).

6.2. Combiner results

The Bayesian combiner utilises the output labels from the classifier stage to generate the final ternary decision. Possible module (classifier)

combinations were attempted to validate the modularity. Three com- bination sets were tested, starting with the presence of all four modules as one combination, followed by three and two modules per set, as shown in Table 2, in which performance metrics of the Bayesian com- biner for these combinations are summarised. It should be noted here that the values given in this table for the performance metrics are averaged to obtain one value per metric.

Table 2 demonstrates the validity and effectiveness of the proposed modular approach in capturing changes of fatigue-related features taking into account the growth of fatigue due to time spent driving. The visual metrics used in this study show a slight advantage over the in- ertial sensors’ metrics. It is further demonstrated in Table 2 that utili- sation of the Bayesian combiner has a significant positive impact on the fatigue-detection performance as compared to that of the individual classifiers. As expected, the best performance (accuracy: 90.4%, sensi- tivity: 92.6% and specificity: 90.7%) was obtained with utilisation of the four input modules and the poorest performance (accuracy: 77.1%, sensitivity: 82.9% and specificity: 77.5%) was obtained with two modules. It is also observed that the highest performance (accuracy: 80.5%, sensitivity: 85.1% and specificity: 80.9 %) achieved with two-

Fig. 3. Blink rate changes with driving time.

Table 1 Performance of individual classification modules.

Performance metrics ANN classifiers

Acceleration Gyroscope Head-position Head-rotation

Accuracy (%) 74.8 70.6 81.9 79.6 Sensitivity (%) 72.5 77.1 82.9 79.1 Specificity (%) 74.7 79.0 82.1 79.6

Table 2 Performance of combined classification modules.

No. of modules

Combinations of classifier modules

Bayesian combiner performance

Accuracy (%) Sensitivity (%) Specificity (%)

4 1, 2, 3, 4 90.4 92.6 90.7

3 1, 2, 3 88.5 91.5 88.9 1, 2, 4 85.7 88.8 86.2 1, 3, 4 85.9 88.9 86.9 2, 3, 4 86.3 90.0 88.0

2 1, 2 77.4 83.6 77.9 1, 3 76.3 83.1 76.7 1, 4 76.3 82.7 76.7 2, 3 77.4 83.4 77.8 2, 4 77.1 82.9 77.5 3, 4 80.5 85.1 80.9

H. Al-libawy et al. Accident Analysis and Prevention 120 (2018) 188–194

192

module combinations was with the existence of visual sensors (i.e. modules 3 and 4).

System resilience to random failure and the maintenance of func- tionality under this failure are important properties of a robust system. Based on the results demonstrated in Table 2, the developed system maintains its functionality even with losing randomly one or two of its resources.

6.3. PSO results

Fig. 4 shows a comparative view for accuracy metric changes of the Bayesian combiner before and after weights optimisation. The accuracy is improved slightly after selection of the optimal weights. The first cluster of bars consists of the four-module group, which represents one combination of modules, and this is the reason that it has no standard deviation bar. The second cluster shows the three-module group which consists of four combinations with their standard deviation bars. Fi- nally, the last cluster shows the two-module group which includes six combinations. Fig. 4 also illustrates the slight improvement in accuracy after selection of the optimal weights. It is worth mentioning that the highest improvement from optimisation is gained by the two-module group. The PSO assigns different weight values for different fatigue- related metrics which reflects differences in their effect. Hence, the PSO is an important and valuable added stage to enhance the system per- formance especially that PSO is performed during the training stage only, and will therefore not affect the system performance. Moreover, the improvement may be more significant with a different range of fatigue-related metrics.

Overall, the obtained results are found to be in agreement with the previously reported findings but with significant system design flex- ibility and robustness against losing one or more data sources. Performance assessment of the developed prototype showed that ac- curacy, sensitivity and specificity of the fatigue detection are only slightly affected by partial loss of input data.

7. Conclusions

Successful design, implementation and test of the proposed modular structure is approved by the demonstrated results. Different combina- tions of contributed modules is experimented and the results support the assumption of maintaining robustness and flexibility of the modular design in the implemented system. Hence, the performance of the op- timised Bayesian combiner is not affected dramatically when losing one or two of the modules. A new modification is applied to the Bayesian combiner to enable the particle swarm optimiser algorithm to work in conjunction with the combiner. The naturalistic environments of the collected data enrich the value of these findings. Despite the small sample size of driver trips used in this study, it serves as a successful pilot study in a naturalistic driving environment and forms a robust foundation future studies on personalisation of fatigue detection using larger datasets.

Driving style over hours of the day can also be integrated in the modular structure with the help of circadian cycle personalisation of individual drivers. These improvement and others are part of the on- going investigations and developments of the authors.

Acknowledgment

This research was supported by a PhD fellowship from Babylon University, Iraq; grant no. MOHESR-IQ-2013-1257.

References

Abbood, H., Al-Nuaimy, W., Al-Ataby, A., Salem, S.A., Al-Zubi, H.S., 2014. Prediction of driver fatigue: approaches and open challenges. In: 14th UK Workshop on Computational Intelligence (UKCI). 8–10 September, IEEE, Bradford, England. pp. 1–6.

Al-Hyari, A.Y., Al-Taee, A.M., Al-Taee, M.A., 2013. Clinical decision support system for diagnosis and management of chronic renal failure. In: IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT). IEEE. pp. 1–6.

Al-Hyari, A.Y., Al-Taee, A.M., Al-Taee, M.A., 2014. Diagnosis and classification of chronic renal failure utilising intelligent data mining classifiers. Int. J. Inf. Technol. Web Eng. 9 (4), 1–12.

Al-Libawy, H., Al-Nuaimy, W., Al-Ataby, A., Al-Taee, M.A., 2015. Estimation of driver alertness using low-cost wearable devices. In: Conference on Applied Electrical Engineering and Computing Technologies (AEECT). Jordan, IEEE. pp. 1–5.

Al-Libawy, H., Al-Ataby, A., Al-Nuaimy, W., Al-Taee, M.A., 2016a. Fatigue detection method based on smartphone text entry performance metrics. In: Proc. 9th International Conference on Developments in eSystems Engineering (DeSE 2016). Liverpool & Leeds, England, 31st August–2nd September.

Al-Libawy, H., Al-Ataby, A., Al-Nuaimy, W., Al-Taee, M.A., 2016b. HRV-based operator fatigue analysis and classification using wearable sensors. In: 13th International Multi-Conference on Systems, Signals & Devices (SSD). March 21–24, Leipzig, Germany, IEEE. pp. 268–273.

Al-Libawy, H., Al-Ataby, A., Al-Nuaimy, W., Al-Taee, M.A., 2017, March. Enhanced op- erator fatigue detection method based on computer-keyboard typing style. In: 14th International Multi-Conference on Systems, Signals Devices (SSD). Marrakech, Morocco. pp. 217–221.

Avigad, J., 2016. Modularity in Mathematics. http://www.andrew.cmu.edu/user/ avigad/Papers/modularity.pdf.

Bahler, D., Navarro, L., 2000. Combining heterogeneous sets of classifiers: theoretical and experimental comparison of methods. 17th National Conference on Artificial Intelligence (AAAI 2000), Workshop on New Research Problems for Machine Learning.

Centre for Road Safety, 2016, December. “NSW road fatalities report,” Tech. Rep. ac- cessed (01.01.18). http://roadsafety.transport.nsw.gov.au/downloads/road- fatalities-2106.pdf.

Craye, C., Rashwan, A., Kamel, M.S., Karray, F., 2016, September. A multi-modal driver fatigue and distraction assessment system. Int. J. Intell. Transp. Syst. Res. 14 (3), 173–194 10.1007/s13177-015-0112-9.

Durán, O., Pérez, L., Batocchio, A., 2012. Optimization of modular structures using par- ticle swarm optimization. Expert Syst. Appl. 39 (3), 3507–3515.

Engelbrecht, J., Booysen, M.J., van Rooyen, G.-J., Bruwer, F.J., 2015. Survey of smart- phone-based sensing in vehicles for intelligent transportation system applications. IET Intell. Transp. Syst. 9 (10), 924–935.

Engelbrecht, A.P., 2006. Fundamentals of Computational Swarm Intelligence. John Wiley & Sons.

Fu, R., Wang, H., Zhao, W., 2016. Dynamic driver fatigue detection using hidden Markov model in real driving condition. Expert Syst. Appl. 63, 397–411. Available from: http://www.sciencedirect.com/science/article/pii/S0957417416303293.

Haq, Z.A., Hasan, Z., 2016, August. Eye-blink rate detection for fatigue determination. 1st India International Conference on Information Processing (IICIP) 1–5.

Hatch, N.W., 2001. Design rules, volume 1: the power of modularity. Acad. Manag. Rev. 26 (1), 130–133.

Hecht-Nielsen, R., 1989. Theory of the backpropagation neural network. International 1989 Joint Conference on Neural Networks, vol. 1 593–605.

Kim, H.-C., Ghahramani, Z., 2012. Bayesian classifier combination. Artif. Intell. Stat. 619–627.

Koenig, A., Rehg, T., Rasshofer, R., 2015. Statistical sensor fusion of ECG data using automotive-grade sensors. Adv. Radio Sci. 13, 197.

Kuncheva, L.I., 2004. Combining Pattern Classifiers: Methods and Algorithms. John Wiley & Sons.

Lazinica, A., 2009. Particle Swarm Optimization. InTech Kirchengasse. Li, Z., Chen, L., Peng, J., Wu, Y., 2017. Automatic detection of driver fatigue using driving

operation information for transportation safety. Sensors 17 (6), 1212. Meiring, G.A.M., Myburgh, H.C., 2015. A review of intelligent driving style analysis

systems and related artificial intelligence algorithms. Sensors 15 (12), 30653–30682. Available from: http://www.mdpi.com/1424-8220/15/12/29822.

People, P.S., Dinges, D., Maislin, G., 1998. Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and the Basis for Alertness Management.

Poli, R., Kennedy, J., Blackwell, T., 2007. Particle swarm optimization. Swarm Intell. 1 (1), 33–57.

Pradhan, N., Dasgupta, S., Sinha, S., 2011. Modular organization enhances the robustness

Fig. 4. Bayesian combiner accuracy comparison before and after optimisation.

H. Al-libawy et al. Accident Analysis and Prevention 120 (2018) 188–194

193

of attractor network dynamics. EPL (Europhys. Lett.) 94 (3), 38004. Qiang, J., Zhiwei, Z., Peilin, L., 2004. Real-time nonintrusive monitoring and prediction

of driver fatigue. IEEE Trans. Veh. Technol. 53 (4), 1052–1068. Sanchez, R., Mahoney, J.T., 1996. Modularity, flexibility, and knowledge management in

product and organization design. Strateg. Manag. J. 17 (S2), 63–76. Sandberg, D., Wahde, M., 2008, June. Particle swarm optimization of feedforward neural

networks for the detection of drowsy driving. IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence) 788–793.

Sharma, R., Kumari, A., 2015. A review on traffic route optimizing by using different swarm intelligence algorithm. IJCSMC 4 (5), 271–277.

Sigari, M.H., Pourshahabi, M.R., Soryani, M., Fathy, M., 2014. A review on driver face monitoring systems for fatigue and distraction detection. Int. J. Adv. Sci. Technol. 64 (12), 73–100 10.14257/ijast.2014.64.07.

Sokolova, M., Lapalme, G., 2009. A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45 (4), 427–437.

Stern, J.A., Boyer, D., Schroeder, D., 1994. Blink rate: a possible measure of fatigue. Hum. Factors 36 (2), 285–297.

Stork, M., Skala, J., Weissar, P., Holota, R., Kubik, Z., 2015. Various approaches to driver fatigue detection: a review. In: International Conference on Applied Electronics (AE).

IEEE. pp. 239–244. Tansakul, W., Tangamchit, P., 2016. Fatigue driver detection system using a combination

of blinking rate and driving inactivity. J. Autom. Control Eng. 4 (1). Transportation Research Board of the national Academics of Science, Second Strategic

Highway Research Program, 2017. Naturalistic Driving Study. (accessed 01.04.17). https://insight.shrp2nds.us/.

Vapnik, V.N., Vapnik, V., 1998. Statistical Learning Theory. Wiley, New York, pp. 1. Wang, Y., Xin, M., Bai, H., Zhao, Y., 2017. Can variations in visual behavior measures be

good predictors of driver sleepiness? A real driving test study. Traffic Injury Prevent. 18 (2), 132–138.

Wang, S.-C., 2003. Artificial neural network. Interdisciplinary Computing in Java Programming. Springer, pp. 81–100.

Yin, H., Su, Y., Liu, Y., Zhao, D., 2016, March. A driver fatigue detection method based on multi-sensor signals. IEEE Winter Conference on Applications of Computer Vision (WACV) 1–7.

Zhang, J., Li, Y., Liu, Z., Zhang, Z., Wang, D., Luo, X., Zhou, M., Zhu, W., Salman, G., Hu, C., Wang, 2017. Design of a fatigue detection system for high-speed trains based on driver vigilance using a wireless wearable EEG. Sensors 17 (3), 486. Available from: http://www.mdpi.com/1424-8220/17/3/486.

H. Al-libawy et al. Accident Analysis and Prevention 120 (2018) 188–194

194