Driver Attention in Automatic Transmission Cars

profileKouniB
PersonalizedGearShiftingArchitectureforNextGenerationAutomaticTransmissionSystems.pdf

Personalized Gear Shifting Architecture for Next Generation Automatic Transmission Systems

Ayşegül Sarı AVL Research and Engineering

Istanbul, Turkey

[email protected]

Ahmet Taha Bilgiç AVL Research and Engineering

Istanbul, Turkey

[email protected]

Görkem Şafak AVL Research and Engineering

Istanbul, Turkey

[email protected]

Duygu Erateş AVL Research and Engineering

Istanbul, Turkey

[email protected]

Emre Kaplan AVL Research and Engineering

Istanbul, Turkey

[email protected]

Abstract—Personalization is one of the trending topics of nowadays. Artificial Intelligence based technologies enable us to personalize systems to reflect user desire and driving profile. In automotive domain, we see intelligent software takes place in many aspects of the vehicle including transmission systems. Today, most of the vehicles are produced with an automatic transmission system which works as programmed according to the development expertise but does not incorporate behavior feedbacks. This paper proposes a novel contribution to automatic transmission systems by incorporating driver feedback to achieve personalization. This way, next generation automatic transmis- sion systems can learn from user behavior taking their inputs into account and reflect under certain conditions. The system learns driver’s demands on the road via supervised learning and predicts driver’s desired gear according to the road conditions, user manipulations, and all relevant information gathered from the vehicle at run time. Learning desires of the driver can fit into the automatic transmission’s decision-making process without violating safety standards and the operational durability as well as leaving very small footprint in terms of memory, space and computation respecting to the limited capability of the environment that the method resides. The proposed method was tested in a realistic testing environment and the results are promising so that it can be deployed in a vehicle to extend automatic transmissions’ capabilities with personalization. In fact, personalized shifting leads to better customer experience and retention.

Index Terms—Artificial Intelligence, Automatic Transmission, Machine Learning, Personalization, Imbalanced Data Classifica- tion

I. INTRODUCTION

Various high-technology devices are going through digital

transformation and vehicles are no exception with many op-

portunities [1]. Automatic transmission equipped within cars is

one of the most important candidates in vehicles [2]. Most of

these developed advanced automatic transmissions, the trend is

on Artificial Intelligence (AI) based technologies [3]. One of

the trending topics in AI is personalization [4] which focuses

on acting upon user’s profile and desire. Car manufacturers

should consider that personalization should not be limited to

peripheral systems like infotainment devices, but also extend to

the powertrain elements. In this work, we bring personalization

and automatic transmissions together to build an intelligent

transmission system which is adaptable to its user.

Since transmission is one of the key components that affects

driving comfort and experience, we see that a personalized

transmission will greatly enhance driver’s experience by pro-

viding gear shifts which are completely modified according

to their previous demands and driving styles. Moreover, this

kind of system maybe promising in commercial purposes as

well. In this work, we propose an end to end system which

will learn driver’s actions and interventions to the automatic

transmission using AI-based classification technique.

In Methodology Section, we depict the main flow of the

processes in the proposed architecture. In order to learn

the behavior and demand of the user, the state-of-the-art

learning methods are utilized and adapted to work under

limited capability of transmission control unit (TCU). This is

especially crucial considering the full safety and operational

security requirements of the system. According to the output

of this personalization engine, it signals TCU to apply desired

upshifts or downshifts as it is applied by the user transparently.

Input parameters of the TCU and driver’s upshift and down-

shift intervention to the transmission will be used for training

the machine learning model and by learning driver’s personal

choices, desired shift changes will be done automatically.

For the proof of concept implementation of the proposed

system, an end to end real-time simulation system is used.

In particular, a Hardware in the Loop (HIL) system is used

because simulation environment plays an important role for

testing automotive software applications in terms of saving

time and money. Early detection of problems and acting cor-

respondingly provide the opportunity to enhance quality. Since

it is possible to test every scenario in simulation environment,

it is also beneficial for safety critical products. HIL is one of

the simulation environments that is used for the development

and testing of complex control systems [5]. Physical parts that

are connected to the control systems through actuators and

sensors, are replaced by software models in the HIL simulation

978-1-5386-5541-2/18/$31.00 ©2018 IEEE

201

Authorized licensed use limited to: Florida Institute of Technology. Downloaded on September 15,2021 at 19:07:58 UTC from IEEE Xplore. Restrictions apply.

systems [6].

We developed several AI methods and compared their

performance in software-based experiments. Even up to 95%

score is achieved when modelling different user interactions

and personalizations.

II. RELATED WORK

Most common approach to the personalized transmission

concept is using a learning mode which is activated at a

certain stage to learn the operating parameters of gear shifting

algorithm and data is stored in the memory to be used for

future control [7].

In [8], authors propose a method focusing on learning

driver’s actions in different travel situations such as lane

changing, overtaking. Data is collected from sensors in learn-

ing mode to learn the driving maneuver and used for training

process which will then anticipate driver behavior in a non-

learning mode and shift gears accordingly.

Utilizing the same methods in previous work, learning

driver’s characteristics during different travel routes is another

proposed concept where a learning phase occurs for collecting

sensor data during predefined routes such as travelling to work,

shopping center or school [9].

Driver classification (driver type evaluation) is another

concept in personalized transmission. The classification uses

various number of parameters, which can range from pedal

position, kick down behavior to seat position [10].

In [11], shifting lines are compared with pre-defined shift

lines of different drivers and according to the driver’s style,

curves in the gear shifting map are updated.

In the proposed method of this paper, the main difference

is having no dedicated learning mode. Instead of using a re-

stricted period for learning driver’s shift actions, continuously

learning method is aimed. Driver classification and vehicle

characteristics are not considered as significant inputs for the

concept. The learning algorithm does not alter shift lines in

shift maps; instead, the output of the personalization engine

is used to override the request of output of the gear shift

map where it is allowed by transmission control algorithm.

Proposed method also differs from similar works in not being

restricted only to certain travel routes and situations. Every

intervention of the driver to the transmission is considered.

III. METHODOLOGY

In a real-world personalized shifting engine life cycle, data

consisting parameters from TCU and requested gear (reqGear

in “Fig. 1”) from driver intervention is collected by person-

alization engine and stored in a measurement database. In

the meantime, a supervised learning model (Driver Behaviour

Learning block in “Fig. 1”) is trained periodically. This

training process continues until a confidence criterion is sat-

isfied and user confirmation is received. After personalization

engine is able to predict driver’s desired gear, the predicted

gear (predGear) is sent to Logic 1 symbolizing selection of

personalized gear. If driver does any tip-up or tip-down, logic 1

directly choose reqGear, if not, predGear is chosen as shifting

to be sent to TCU. Logic 2 checks the TCU conditions and

safety criteria. If shifting is allowed by TCU after safety

check, Next Gear Decision of TCU will be output of Logic 1.

Otherwise, no personalized shifting will be occurred, instead

desired gear from TCU is applied.

IV. IMPLEMENTATION AND TESTING

A. Data Acquisition Data acquisition and functionality tests are performed on a

HIL setup equipped with the TCU application software and a

plant model that consists of a recently developed state of the

art 7-speed dual clutch transmission and the corresponding

passenger car model. HIL follows a given driving cycle which

is a time-series signal of desired vehicle speeds (as depicted

in “Fig. 2”). In addition, pre-defined rules can be input to

change the behavior of the vehicle in certain situations. For

instance, tip up and tip down requests can be injected into

TCU application using these rules to emulate the behavior of

a user requesting another gear instead of the gear which is

calculated and decided by TCU gear selection mechanism.

We collected 4 hours of driving data using HIL with the

above-mentioned setup. Driving data generated using 4 differ-

ent driving cycle as a basis. In real life, these driving cycles

are the minority of the cases which consequently produce

imbalanced results since we are interested at these minority of

cases as training dataset. A practical solution on the other hand

and for the sake of rapid convergence, we generate and collect

data with higher ratio of tip-up and tip-down samples which

is produced by using aggressive driving cycles (not normal

driving behavior) including frequent speed changes.

Since HIL produces data with plenty of asynchronous

channels, we performed an interpolation and resampling with

sampling time of 40ms on the whole data (368487 samples).

B. Personalization Methods We aim to model user intervention state using feature set

consisting of engine speed, output shaft speed, gas pedal

Fig. 1. Personalized Shifting System on Vehicle

202

Authorized licensed use limited to: Florida Institute of Technology. Downloaded on September 15,2021 at 19:07:58 UTC from IEEE Xplore. Restrictions apply.

Fig. 2. A drive cycle used as an input for HIL

position, current gear and desired gear by TCU. The user

intervention state can be either of the classes: ‘no intervention’,

‘tip-up’ and ‘tip-down’.

Since we focus on prediction of minor classes with in-

terventions, drastic imbalance in the class populations (Ta-

ble I), makes this problem difficult. To solve this problem,

we performed following procedures. Data is split to train,

validation and test sets considering the intervention class

statistics. To handle imbalance, we either resample the training

data before modelling or use cost-sensitive models directly.

Boosting models are utilized beside the conventional models

like logistic regression or multi-layer perceptron.

1) Data splitting Splitting data into train and test sets is especially crucial

when the data is highly imbalanced. When the data is split

randomly, it is possible that samples from minor classes

diminish in the training. For that reason, the stratified splitting

method which splits the dataset without losing the overall

statistics of the target variable, is found to be suitable for this

problem. Our data is split to train, validation and test sets in

the ratio of 65:10:25 and stratified according to intervention

categories.

2) Resampling Resampling methods simply aim to balance the data classes

quantitatively, which can be both under sampling the majority

classes and over-sampling the minority classes. Using only

under-sampling is not efficient because the training data will

consist of about 800 sample which causes lack of information.

Therefore, applying over-sampling or using both are efficient

in terms of data utilization (Table II). Synthetic Minority

Over-sampling Technique (SMOTE) is applied to continuous

TABLE I COUNTS AND RATES OF TARGET CATEGORIES

Target categories No intervention Tip-up Tip-down

Count 366831 1253 403 Ratio 0.9955 0.0034 0.0011

variables as over-sampling method because it is better than

random over-sampling to not overfit to the specific instances

[12]. Although SMOTE is applicable if all variables are

continuous numerical variables, we apply a SMOTE-based

technique which [12] proposed in the same paper to handle

both categorical and numerical variables (SMOTE-NC, etc. in

section 6.1).

SMOTE-NC is applied alone and combined with Random

Under-sampling (RUS) to training data considering nominal-

categorical features. After SMOTE-NC and combination of

SMOTE-NC and under-sampling is applied, counts of each

category in training data is shown in the Table II.

3) Boosting Gradient boosting is an ensemble learning method to get

an iteratively strong model with starting a weak classifier,

focusing on the misclassifications of model and trying to fit

the residuals (real value - predicted value) in each iteration of

training [13]. Model iteratively fits more data and can catch the

minor patterns and learn difficult instances. Generally, decision

tree algorithms are used in boosting. As mentioned before,

personalized shifting behavior of a driver consists of some

decisions under similar conditions. This cause the appropriate

learning solution on the problem can be decision tree-based

classifiers. On the other hand, minor patterns which is hard to

learn for models can be handled by the boosting algorithms.

In the experiments, XGBoost [14] and CatBoost [15] are

applied as the new generation powerful gradient boosting tree

classifiers.

4) Cost-Sensitive Modelling In a real-world problem as AI model learns user’s behaviors

and demands, using synthetic data in training can make the

learning model weak when the output of the model will

directly be used for intervention to the real world. Giving

different costs according to weights of classes can be better al-

ternatives in modelling. For example, in a binary classification

assuming minority as positive class, false positive error for one

sample has accordingly higher importance than false negative

sample error, however, when a cost-insensitive classifier is

used, the model predicts all dataset as negative [16].

In the modelling personalized gear shift intervention, data is

three classes with two minorities. In the ratio of class weights

in training data, the high cost is given to misclassifications of

upshifts and downshifts, and the low cost to false positives of

minority classes. In other words, since the category ratio is

1:3:1000, the cost weights ratio is chosen as 1000:333:1. The

TABLE II TARGET CATEGORY COUNTS OF TRAIN SET BEFORE-AFTER RUS,

SMOTE-NC AND SMOTE-NC-RUS COMBINATION

Target category counts Train Set No intervention Tip-up Tip-down Original 238440 814 262

After RUS 262 262 262 After SMOTE-NC 238440 238440 238440 After combination 50000 50000 50000

203

Authorized licensed use limited to: Florida Institute of Technology. Downloaded on September 15,2021 at 19:07:58 UTC from IEEE Xplore. Restrictions apply.

implementation results show that using boosting algorithms

with cost sensitive approach can achieve confident models.

V. EXPERIMENTAL RESULTS

Concepts detailed in the previous part got into some com-

binations to reach the powerful model that can be used in the

personalization engine. Resampling and const-sensitive mod-

elling concepts are implemented as alternatives of each other

while logistic regression, multi-layer perceptron, XGBoost and

CatBoost used as modelling algorithms. The primary aim of our model is learning minorities well

because personalization engine changes the behavior of TCU

in these cases. On the other hand, misclassification of no-

intervention class is not problematic cases for the system

because the safety conditions are checked, and the gear

shifts occur if they are fully safe and eligible for shifting.

Considering all of these aspects, the recall score (i.e. the true

positive rate) of each class, especially the minority classes

is decided to be important but also the weighted accuracy

which is average of the recall scores of three categories. On the

other hand, precision metric and F1 score are not important to

evaluate models. Results of the selected method combinations

indicating the comparison are shown in the Table III. The scores show that the most powerful models are the cost-

sensitive XGBoost and cost-sensitive multi-layer perceptron

with average recall score greater than 0.94.

VI. CONCLUSIONS

Personalization may be considered as one of the most

appealing criteria that makes next generation products more

popular thanks to AI expansion in almost all fields of our

lives. A personalized automatic transmission concept is stud-

ied in this work incorporates AI into transmission to make

transmission intelligent. The gear shifting in a vehicle is one

of the most appropriate components giving the feeling of

personalization of their car to its driver. Main purpose of

the system is to eliminate user’s desire to interfere with the

automatic transmission by learning their previous interventions

and performing desired shift changes. Since driver’s manual

intervention to the automatic transmission is not a frequent

action, the generated training data is a good example of an

imbalanced data set which is also a challenge. Different meth-

ods, that are mostly used for imbalanced dataset problems, are

experimented such as resampling, boosting and cost-sensitive

modeling. The experimental results indicate that the proposed

system is promising. As a future work, we plan to deploy

this method to a development vehicle for demonstration. In

such a setup, the driver interventions will be used instead of

the upshift and downshift signals created by the HIL simula-

tion which is according to some predetermined programmed

conditions. In a real-world scenario, the correspondence for

the predetermined conditions are the user’s demands for gear

shifting in terms of tip-ups and tip-downs.

REFERENCES

[1] Pakusch, C., Bossauer, P., Shakoor, M., & Stevens, G. (2016, July). Using, Sharing, and Owning Smart Cars. In Proceedings of the 13th International Joint Conference on e-Business and Telecommunications (ICETE 2016), Lisbon, Portugal (pp. 26-28).

[2] Ha, S. H., & Jeon, H. T. (2013). Development of intelligent gear-shifting map based on radial basis function neural networks. International Journal of Fuzzy Logic and Intelligent Systems, 13(2), 116-123.

[3] Zhu, Z., & Xu, C. (2003). Experimental study on intelligent gear-shifting control system of construction vehicle based on chaotic neural network. Nature and Science, 1(1), 86-90.

[4] Adomavicius, G., & Tuzhilin, A. (1999). User profiling in personaliza- tion applications through rule discovery and validation. In Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 377-381). ACM.

[5] Cavina, N., Olivi, D., Corti, E., Mancini, G., Poggio, L., & Marcigliano, F. (2013). Development and implementation of hardware in the loop simulation for dual clutch transmission control units. SAE International Journal of Passenger Cars-Electronic and Electrical Systems, 6(2013- 01-0816), 458-466.

[6] Chen, R., Mi, L., & Tan, W. (2012). A new hardware-in-the-loop test system for electronic control unit of dual-clutch transmission vehicle. In Advanced Materials Research (Vol. 490, pp. 13-18). Trans Tech Publications.

[7] 维尔纳沃尔夫冈, 英戈索特, 迈克维特纳. (2007). Chinese Patent No. CN101680533A

[8] Bastian, A. (1993). German Patent No. DE4337164A1 [9] Richter, B., Bastian, A., (1993). German Patent No. DE4337163A1

[10] Henneken, M., Jauch, F., Herbster, K., Schuler, F., Mauz, T. (2000). German Patent No. DE4337163AI

[11] Borodani, P., Gianoglio, R., Giuliano, F., Lupo, M., (1999). European Patent No. EP0950839B1

[12] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.

[13] Freund, Y., Schapire, R., & Abe, N. (1999). A short introduction to boosting. Journal-Japanese Society For Artificial Intelligence, 14(771- 780), 1612.

[14] Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining (pp. 785-794). ACM.

TABLE III RESULTS OF MODEL TESTS ON THE POC DATA

Recalls Methods Weighted Accuracy Weighted F1 No Intervention Tip-down Tip-Up

Cost-sensitive Logistic Regression 0.5643 0.9279 0.1818 0.5831 0.9633 Logistic Regression to SMOTE-NC 0.8101 0.7886 0.8080 0.8338 0.7098

Cost-sensitive Multi-Layer Perceptron 0.9401 0.8923 0.9607 0.9674 0.9546 Cost-insensitive XGBoost 0.4087 0.9990 0.1717 0.0554 0.9929 XGBoost to SMOTE-NC 0.7678 0.9772 0.8181 0.5082 0.9749

Cost-insensitive XGBoost to SMOTE-NC-RUS combination 0.8167 0.9701 0.8384 0.6417 0.8975 Cost-Sensitive XGBoost 0.9467 0.8997 0.9697 0.9707 0.9237

Cost-Sensitive CatBoost without one-hot-encoder 0.8904 0.9609 0.9091 0.8013 0.9672 Cost-Sensitive CatBoost with one-hot-encoder gear states 0.8691 0.9628 0.8889 0.7557 0.9361

204

Authorized licensed use limited to: Florida Institute of Technology. Downloaded on September 15,2021 at 19:07:58 UTC from IEEE Xplore. Restrictions apply.

[15] Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2018). CatBoost: unbiased boosting with categorical features. In Advances in Neural Information Processing Systems (pp. 6638-6648).

[16] Ling, C. X., & Sheng, V. S. (2010). Cost-sensitive learning. Encyclope- dia of machine learning, 231-235.

205

Authorized licensed use limited to: Florida Institute of Technology. Downloaded on September 15,2021 at 19:07:58 UTC from IEEE Xplore. Restrictions apply.