Week 3 501

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 1

Trustworthy Data- and ML-Ops for Intelligent Transportation Systems and Logistics Antonio Emanuele Cinà , Member, IEEE, Giovanni Scodeller , Member, IEEE, Cecilia Caterina Pasquale , Member, IEEE, Silvia Siri , Senior Member, IEEE,

Davide Anguita , Senior Member, IEEE, Fabio Roli , Fellow, IEEE, Simona Sacone , Senior Member, IEEE, and Luca Oneto , Senior Member, IEEE

Abstract—The rapid evolution of Intelligent Transportation Systems and Logistics (ITS&L) has become a cornerstone of the modern social economy, relying heavily on the integra- tion of Data, Artificial Intelligence (AI), and, more specifically, Machine Learning (ML). This paper provides a comprehensive review of Trustworthy Data and Machine Learning Operations (DataOps and MLOps) in the ITS&L domain, underscoring their importance in improving efficiency, reliability, and decision- making precision within transportation and logistics services. We begin by identifying gaps in current literature, offering clear context for our contribution. Subsequently, we explore the complexities of DataOps and MLOps, discussing their necessity, key components, available tools, practical insights, and case studies relevant to ITS&L. Additionally, we address the critical issue of Trustworthiness in AI applications, examining methods and tools designed to strengthen confidence in AI systems - especially in real-world ITS&L scenarios. The paper concludes with a discussion of persisting challenges and future prospects in this rapidly advancing field, aiming to serve as a vital resource for researchers, industry practitioners, and policy makers. Overall, this work not only establishes a foundational understanding of DataOps and MLOps in ITS&L but also charts a path for further research and innovation in developing more efficient, sustainable, and trustworthy intelligent transportation and logistics systems.

Index Terms—Transportation and logistics, intelligent trans- portation, data, machine learning, artificial intelligence, DataOps, MLOps, LLMOps, FMOps, AgentOps trustworthy AI.

I. INTRODUCTION

I INTELLIGENT Transportation Systems and Logistics (ITS&L) are at the backbones of the contemporary social

economy [1]. These systems play a crucial role in ensuring the efficiency and functionality of modern services [2], [3]. They support industries, such as transporting raw materials

Received 26 June 2025; revised 28 June 2026; accepted 6 July 2026. This work was supported in part by the European Union (EU)–Next Generation European Union (NGEU) National Sustainable Mobility Center through Ital- ian Ministry of University and Research Decree no. 1033-17/06/2022 (Spoke 10) under Grant CN00000023; in part by the Project SERICS through National Recovery and Resilience Plan (NRRP) Ministero dell’università e della ricerca (MUR) Program funded by EU–NGEU under Grant PE00000014; in part by the Project FAIR through NRRP MUR Program funded by EU–NGEU under Grant PE00000013; and in part by the Project RAISE through NRRP MUR Program funded by EU–NGEU under Grant ECS00000035. The Associate Editor for this article was Y. Han. (Corresponding author: Luca Oneto.)

The authors are with the Department of Informatics, Bioengineering, Robotics and System Engineering (DIBRIS), Università of Genova, 16145 Genova, Italy (e-mail: [email protected]).

Digital Object Identifier 10.1109/TITS.2026.3711756

[4], facilitating e-commerce deliveries [5], and ensuring the timely distribution of medical supplies [6].

From the digitalization advancement of the late 90s’, which enabled the complete digitization of tracking and monitoring systems [7], to the most recent Internet of Things (IoT) devices that enable real-time monitoring of the status of the goods [8], ITS&L is nowadays rapidly producing a large amount of heterogeneous data [2], [9]. Such a data explosion has driven the adoption of Artificial Intelligence (AI), particu- larly Machine Learning (ML) and advanced data analytics, to improve operational efficiency, diagnostics, and predictive capabilities, moving toward more automated ITS&L opera- tions [10], [11]. The integration of Data and AI in ITS&L empowers organizations to make more informed decisions and deliver more competitive services, improving efficiency and performance across various domains such as traffic manage- ment [12], vehicle control [13], and warehousing [14]. For example, in traffic management, AI-driven predictive models optimize signal timings, reduce congestion, and decrease travel times [12]. In vehicle control, AI algorithms enable adaptive cruise control and autonomous navigation, enhancing safety and reducing driver workload [13]. In warehousing, intelli- gent systems streamline inventory management by automating stock tracking and optimizing space utilization, boosting operational efficiency and reducing costs [14]. For a more comprehensive collection of ITS&L applications that benefit from Data and AI, we refer to Section III and [10]. These systems have in common the use of Data and AI to process complex data patterns and provide solutions that support real- time decision-making and system optimization, becoming the core of the business for companies operating within ITS&L.

However, despite the inevitable hype accompanying Data and AI-based systems [10], [15], unlocking their full potential within the ITS&L domain brings multiple challenges [16]. In particular, data-related challenges in ITS&L applications reflect the Volume, Velocity, and Variety dimensions of the traditional 3V model of Big Data [17]. Volume stems from the large number of entities that must be continuously monitored, including vehicles, drivers, passengers, infrastructures, and carriers [18]. Velocity arises from the need for real-time con- trol, tracking, monitoring, and decision-making, since delayed information may result in outdated route recommendations, inefficient traffic control, missed delivery windows, or late emergency responses [19]. Variety derives from the heteroge-

© 2026 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

neous nature of the collected data, which may refer to highly different quantities as traffic flows, average speeds, individual trajectories, intermodal transport units. In addition, data can be measured with different devices, techniques, and formats, which can include tabular records, videos, images, logs, graphs representing mobility networks, and data generated at different time scales and frequencies [20]. For these reasons, systems are needed that can efficiently and effectively collect, store, and curate such data [1], [2], [19].

Furthermore, beyond data management, AI systems demand continuous monitoring of their performance both with tech- nical metrics [21] (e.g., accuracy [22], non-regressiveness [23], and sustainability [24]) and human-related metric [25] (e.g., robustness [26], fairness [27], and privacy [28]). Ful- filling these needs becomes fundamental to preserve adequate system behavior and inducing trust in AI use in the area of ITS&L applications [29], [30]. This is especially true when it comes to the adoption of state-of-the-art tools like Generative AI [31] (e.g., Large Language Models [32] or Multimodal Language Models [32]) to bridge the gap between results of AI and their actual actionability (e.g., to support the operators in ITS&L in natural language or with images/videos [33], [34]). The need to face this challenge is also becoming mandatory due to the recent effort of regulatory bodies to enforce regu- lations such as the AI-Act [35], [36], [37]. Consequently, it becomes essential to combine cultural philosophies, practices, and tools that enable ITS&L companies to deliver applications and services with high velocity [38], empowering them to better serve their customers and compete more effectively in the market [16].

Similar challenges have also been faced in the software development industry, leading to the introduction of Devel- opment Operations (DevOps) to enhance both the speed and quality of software engineering [39]. DevOps comprises two main components: Development (Dev), which includes software planning, development, and testing, and Operations (Ops), which is responsible for the continuous integration, deployment, and monitoring of the software [39]. When it comes to encompassing Data-driven software development and AI, researchers have extended traditional DevOps practices to include Data Operations [40], [41] (DataOps) for managing data workflows, and ML Operations [42], [43] (MLOps) for optimizing AI and ML pipelines.

DataOps defines a collection of practices and tools tailored to the design, deployment, and management of data pipelines in an agile, efficient, and usable manner [16]. DataOps estab- lishes processes for addressing evolving and real-time data requirements by ensuring that large, fast, and heterogeneous data streams can be reliably collected, organized, processed, and made available when needed, across diverse sources and under rapidly changing operational conditions [44], [45].

MLOps defines a set of practices and tools for deploying and maintaining ML models, which are essential components of AI-enabled systems, in production environments [46]. MLOps provides developers with functionalities for integrating ML models into scalable and automated infrastructures, enabling continuous learning from new data through updated techniques and thereby preserving reliability and performance over time [47]. Moreover, MLOps enables developers to seamlessly

Fig. 1. DataOps and MLOps in ITS&L. Heterogeneous data sources are governed through the DataOps lifecycle and subsequently support the MLOps lifecycle for model development, deployment, monitoring, and updating. The resulting AI-enabled services operate across the main ITS&L application domains, while TAI provides a governance layer grounded in robustness, explainability, fairness, privacy, and sustainability.

monitor different technical and human-related metrics, sup- porting the development and operationalization of Trustworthy AI (TAI)-based systems [48]. In this way, MLOps directly addresses the AI lifecycle challenges outlined above: it sup- ports the automation of model retraining pipelines to cope with continuously evolving data distributions, manages the com- plexity of large-scale ML deployments through reproducible and versioned workflows, and operationalizes the continuous monitoring of both technical metrics, such as accuracy and non-regressiveness, and human-related metrics, such as robust- ness, fairness, and privacy, thereby supporting compliance with regulatory frameworks such as the AI Act [35], [36], [37].

Together, DataOps and MLOps provide a comprehensive and complementary response to the challenges inherent to the ITS&L domain. DataOps governs the data layer by automat- ing the ingestion, processing, and storage of high-volume, high-velocity, and heterogeneous data streams, while MLOps governs the AI model lifecycle - from training and validation to deployment and monitoring - ensuring continuous perfor- mance, reproducibility, and compliance with trustworthiness requirements. They are therefore indispensable for ensuring the reliability and effectiveness of data-driven solutions, partic- ularly in applications where user safety, operational efficiency, and decision-making precision are of paramount importance [16], [47]. Figure 1 provides an end-to-end view of how DataOps, MLOps, and TAI jointly support trustworthy ITS&L applications.

In this work, we present a review of Trustworthy DataOps and MLOps for ITS&L able to

• conduct a review of the current surveys related to the con- tent of our work to pinpoint gaps in previously published surveys on this topic;

• provide the motivations and detailed description for DataOps and MLOps surveying both available tools and their current adoption within ITS&L application;

• provide motivations and a detailed description of TAI, surveying both the key metrics and their current adoption within ITS&L applications;

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 3

• underline the synergy between TAI, DataOps, and MLOps in the context of ITS&L;

• outline the unresolved challenges and future prospects in this burgeoning field of research.

Today, this review is timely and relevant, and we believe it can serve as a valuable resource for different audiences. It primarily targets early-career researchers seeking a structured entry point into the intersection of DataOps, MLOps, TAI, and ITS&L. At the same time, it may support industrial practition- ers interested in understanding the operational implications, tools, and case studies associated with these technologies, as well as senior researchers seeking to identify research gaps, future directions, and opportunities for real-world impact.

The rest of this review is organized as follows. Section II conducts a search for current reviews related to the content of our work aimed at clarifying our contribution. Section III focuses on presenting the main application areas in ITS&L and the diverse challenges they need to face demanding Trustworthy DataOps and MLOps solutions. Section IV delves into the DataOps, focusing on the reason for its need, its essential parts, the available tools (open source and propri- etary), some practical insights, and the current case studies in ITS&L. Section V delves into the MLOps, following the same structure of Section IV, with the addition of an in-depth analysis of a specific case of MLOps and LLMOps, which is the customization of MLOps for Large Language Models (LLMs) that are currently emerging as the next evolution of AI for ITS&L. Section VI focuses on the TAI, outlining methods and tools to integrate into the MLOps pipeline to enhance trust in AI-empowered systems, accompanied by examples of use cases in ITS&L. Section VII discusses open challenges and future perspectives for Trustworthy Data- and ML-Ops in ITS&L. Finally, Section VIII concludes the paper.

II. LITERATURE REVIEW

In the literature, it is possible to find many different reviews partially dealing with the topics covered in our work. For this reason, this section scrutinizes the current reviews to allow the reader to trace these works and better understand our novel contribution. In particular, we describe the search and selection criteria followed in this review and then we focus on the gaps in the current reviews that will be addressed in our work.

We searched for reviews in a specific time window using a series of keywords in the most important academic databases: Scopus1 and Google Scholar.2 In terms of time window, we restricted the search to the last 10 years. For the keywords to be searched, we applied the following criteria • papers containing in the title, abstract, or keywords one of

the following strings: “DataOps”, “Data Models”, “Data Collection”, “MLOps”, “LLMOps”, “FMOps”, “AIOps”, “Trustworthy AI”, “Trustworthy ML”, “Transportation Systems”; “Intelligent Transportation”; “Logistics”.

• papers containing in the title, abstract, or keywords one of the following strings: “review”, “sok”, “overview”, and “survey”.

1https://www.scopus.com/ 2https://scholar.google.com/

Moreover, papers cited in relevant papers found using the previous criteria have also been included. To reduce the risk of missing relevant surveys that adopt different terminology, the screening process also considered broader terms used in adjacent research areas, such as AI safety and governance, since relevant surveys may vary in terminology depending on their research community. However, because these broader terms often retrieve heterogeneous literature beyond the scope of this paper, we report only the keywords that are most prominent and directly related to operational AI lifecycles and ITS&L applications. In cases where multiple reviews address overlapping topics, we report the most recent or compre- hensive one. The search results are summarized in Table I, where each row represents a survey paper. A checkmark (3) indicates coverage of the four topics considered in this paper: ITS&L, DataOps, MLOps, and TAI. Several observations can be derived from Table I regarding the current reviews on the topics addressed in this work and our contribution.

Unlike existing surveys in the literature, which generally provide a broad overview of the entire spectrum of DataOps, MLOps, and TAI for general-purpose applications, our work focuses explicitly on the practical needs within the ITS&L sector. Existing works often treat TAI, DataOps, and MLOps as separate entities, overlooking their potential synergy. Some works, such as [19] and [20], analyze the relevance of DataOps in ITS&L applications and the associated challenges but do not delve into the role of TAI and MLOps development. Tao et al. [30] investigate the role of data quality in main- taining trustworthiness in data-driven systems but lack a focus on ITS&L. More recent studies have begun to link operational ML pipelines with trustworthiness. For example, Bayram and Ahmed [73] provide a comprehensive survey of robustness in MLOps systems, while Black et al. [70] examine fairness from a pipeline-aware perspective. However, these works do not address multiple TAI principles in an integrated manner, nor do they jointly analyze DataOps and MLOps within transportation and logistics applications. The most comprehensive work, Zhu et al. [19], covers three out of the four research domains investigated in this survey, excluding MLOps. Moreover, their discussion on TAI focuses solely on the robustness and privacy principles.

In this survey, we jointly focus on five trustworthiness dimensions: robustness, interpretability, fairness, sustainabil- ity, and privacy. We also review tools that support the assessment of these requirements, helping practitioners evalu- ate whether data pipelines, ML models, and deployed services satisfy these principles throughout their lifecycle.

III. INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS

In this section, we provide an overview of the main areas of ITS&L, drawing on several notable reviews in the field [1], [10], [13], [60], [74], [75]. This overview aims to frame the problems and opportunities related to Trustworthy Data- and ML-Ops, which are examined in detail in the following sections. Building on these existing works, we organize ITS&L applications into an operational taxonomy tailored to the scope of this survey. The objective is to group applications that face

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE I META-REVIEW OF EXISTING SURVEYS IN THE ITS&L, DATAOPS, MLOPS, AND TAI RESEARCH FIELDS

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 5

similar challenges in terms of DataOps, MLOps, and TAI, rather than to provide a purely descriptive classification of the domain. These applications share common needs, such as handling large and heterogeneous data, managing task complexity, and ensuring service trustworthiness. However, each area also presents specific operational requirements, for example in terms of real-time processing, scalability, safety, privacy, fairness, sustainability, or explainability. This taxon- omy is therefore used in the following sections to organize the discussion of case studies and to connect each application area with its most relevant requirements.

A. Vehicle Control Systems

Vehicle control systems involve vehicle-level technologies designed to improve vehicle performance and safety through automation and driver assistance [13]. Examples of these systems are: advanced driver-assistance systems [76], vehicle- to-vehicle or vehicle-to-everything [77], self driving cars [78], electronic stability control [79], and incident management [80]. These systems are developed and integrated into self- driving cars to increase efficiency, improve vehicle handling, and support humans to correct potential errors or improve their usage experience [78]. Furthermore, if connected together in a network [77], these systems can communicate with each other and the surrounding infrastructure for rapid detection, response, and mitigation of traffic incidents [80]. Vehicle technologies, including ML components, are rigorously tested, validated, and monitored in real time to detect and correct potential issues before they cause harm. Furthermore, from an operational perspective, these systems collect and process vast amounts of data, including sensitive information about drivers and their behaviors, thus demanding protection mechanisms (e.g., data encryption) to avoid unauthorized access and ensure compliance with safety and privacy regulations. Lastly, they must be able to scale in real time to accommodate varying traffic conditions, an increasing number of connected vehicles, and the dynamic nature of road environments [80].

B. Traffic Management

It focuses on optimizing traffic flow, reducing conges- tion, and improving overall road network efficiency [12]. For instance, these systems can manage traffic lights [81] and road diversions [82] to enhance road traffic and benefit drivers. Another example is dynamic toll pricing systems [83], which use traffic data to adjust toll rates [84]. These systems promote the use of highways when state roads are heavily congested, thereby balancing the traffic load across different road networks. Traffic management, however, does not only deal with road and city traffic. It also encompasses services and implementations in navigation systems [85], air transport systems [86], water transport systems [87], and rail systems [88]. In traffic management applications, we find the need for real-time data processing to provide timely and accurate rec- ommendations, which directly impacts the efficiency of traffic management and congestion reduction. Practically, and from an operational perspective, they require continuous processing of network-level traffic data to support congestion control,

routing efficiency, pricing decisions, and environmental objec- tives. At the same time, these systems face with privacy, security, fairness, and sustainability principles: privacy and security because of the sensitive nature of user data involved in the systems and their integrity [89], fairness because the inner algorithms and models should not favor users or areas in recommendations and pricing [90], and sustainability because of the environmental impact these systems can have on traffic pollution [91].

C. Transportation Management Systems

Unlike traffic management systems, which primarily focus on regulating traffic flows and network conditions, transporta- tion management systems aim to model travelers’ journey patterns across multimodal transportation networks, including car, bus, rail, and metro systems. These systems use such information to plan, optimize, and improve mobility services for users, including public transit routes, ride-sharing services, and transportation infrastructure [19], [92], [93], [94], [95]. These applications strongly rely on information collected from multiple, widespread, and heterogeneous data sources (e.g., social networks and smartphones) and on the ability to extract actionable insights from them. For instance, these systems can leverage large-scale data from social media to understand public opinions on transportation services or to detect traffic events (e.g., accidents, breakdowns, and road closures) and consequently improve planningpolicies [96], [97]. Decisions made based on these data have a strong societal and business impact. Starting from privacy, given the usage of sensitive data (e.g., personal data from transit cards), they must implement robust security measures to protect users data from unauthorized access, breaches, and misuse. Furthermore, they have to handle peak traffic times, where transportation demand increases, and future expansions without performance degradation, and allocate transportation resources in a way that benefits all communities fairly.

D. Traveler Information and Experience Systems

This category comprises user-facing systems that leverage algorithmic and AI-based methods to search for, simulate, plan, and recommend routes, destinations, or mobility options tailored to individual travelers’ needs and preferences, thereby enhancing the overall travel experience [13], [98], [99], [100], [101], [102], [103]. For example, these systems can offer drivers the fastest route to their destination, continuously updated in real-time to accommodate regular and unexpected traffic congestion, thus reducing overall travel costs and improving user satisfaction [101], [102]. Although these sys- tems may rely on traffic data and routing algorithms, their objective differs from that of traffic management systems: they primarily seek to optimize the information, guidance, and recommendations provided to individual users, rather than directly controlling traffic flows or coordinating network-level operations. The challenge is to deliver the service in real time to a large number of users, using the most recent data from a large number of sources [103], [104]. The focus in these applications is not on security and safety, as the

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

division mainly impacts user comfort or other metrics such as sustainability. However, they need to integrate explainable models to provide clear and understandable explanations for recommendations (e.g., route deviations), help users make informed decisions, and build trust in the system [105].

E. Freight and Supply Chain Logistics

These systems support the full supply-chain continuum, from managing cargo shipments across multiple transportation modes, including sea, road, air, inland waterways, and rail, to coordinating urban freight policies and optimizing ware- housing and last-mile delivery operations [13], [14], [106]. At the freight level, they can be used to reduce pollutant emissions, optimize energy consumption [107], and coordinate information exchange among seaport stakeholders [108]. At the warehousing level, smart systems leverage real-time data from RFID tags and IoT sensors to track goods, manage inventory levels, forecast demand, reduce waste, and optimize storage layouts [109], [110]. At the delivery level, last-mile systems integrate GPS data, real-time traffic updates, and customer feedback to dynamically adjust routes, reduce fuel consumption, and improve delivery reliability [111].

Across this spectrum, constraints such as energy efficiency, operational resilience, and business and environmental sus- tainability are of primary importance [112]. Disruptions, data compromises, or delays in data processing can propagate along the supply chain, causing logistics errors and poten- tially significant financial losses. Therefore, continuous data monitoring is essential to assess data integrity, while scalable implementations are needed to accommodate variable cargo volumes and adapt to changing demand patterns and regulatory requirements. These systems should also be designed to mit- igate biases that may lead to unequal treatment of customers or regions based on location, accessibility, or socioeconomic status [113], [114]. For example, last-mile delivery algorithms may unintentionally prioritize neighborhoods over less affluent areas because of disparities in road infrastructure, demand density, or service accessibility, resulting in poorer service quality for underserved communities. Addressing these risks requires incorporating fairness-aware constraints to support a more equitable distribution of logistics services across regions [113].

F. Emergency Management

These systems are essential to coordinate responses to unforeseen events and minimize their impact on communities [115], [116]. Largely applied in large-scale disaster contexts, they aim to ensure the timely delivery of emergency resources and rescue services into the affected regions so as to assist in rescue activities [117]. These systems connect more data sources together to collect data and use them to prevent and mitigate crises such as natural disasters [118], terrorist attacks [119], or public health emergencies [120]. The challenge lies in the timely collection and processing of data to rapidly extract information that can prevent or better manage emer- gencies [121]. Concurrently, they aim to ensure the integrity of collected data to avoid misinformation, which could lead

to ineffective or harmful responses. Lastly, these systems must preserve fairness in their decisions and operations, e.g., equity and fairness in humanitarian relief services [121].

IV. DATAOPS

Data has become crucial in ITS&L applications, driving the need for efficient process streamlining and operational efficiency [2], [122]. This involves seamless data collection, transfer, storage, and accessibility with minimal human inter- vention, facilitating complete automation of various logistical tasks [19], [45]. However, managing data in the ITS&L sector poses challenges [123]. These applications generate large volumes of data [2], which are important for scientific progress and process optimization. However, their architecture, involving a variety of interconnected components-such as vehicles, sensors, and digital machinery-creates a complex data environment characterized by high velocity and continuous streams [122]. As a result, advanced and scalable data manage- ment solutions are required [16], [123], and DataOps [124], [125] provides a promising solution within the ITS&L sector. It integrates practices and technologies to streamline the entire data lifecycle, from collection to analysis, ensuring efficient processing and availability of data generated [44]. In the end- to-end view presented in Figure 1, this section corresponds to the DataOps layer, where heterogeneous ITS&L data sources are transformed into reliable, monitored, and reusable data assets for downstream analytics and ML workflows.

In the following, we start by revisiting the underlying motivations for adopting DataOps, as seen in Section III and further detailed in Section IV-A. Subsequently, we delineate the various stages encompassed within a DataOps pipeline, outlined in Section IV-B. This is followed by an examination of the principal open-source (Section IV-C) and proprietary DataOps tools (Section IV-D). A comparative analysis of these tools is presented in Section IV-E, offering practical insights and implications. The section culminates with a presentation of current case studies in the ITS&L domain, elaborated in Sec- tion IV-F. This structured approach provides a comprehensive overview of the DataOps landscape, highlighting its practical applications and theoretical underpinnings.

A. Motivations

As seen in Section III, the adoption of advanced and scalable data management solutions from DataOps has been catalyzed by the proliferation of three primary factors [19], [84]: Volume, Variety, and Velocity. Although the services and applications developed behind the different areas of ITS&L are different in nature, they all must address these three factors during their development and service life cycle. For example, while volume is a common feature, variety and velocity are together particularly important when it comes to traffic and emergency management applications. These applications must continuously collect data from different sources and process it in real time to maintain quality of service and usefulness for users. In the following paragraphs we describe where these factors originate and some examples of their incorrect management.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 7

1) Volume: The massive volume of data within the ITS&L sector is experiencing exponential growth, a trend further accelerated by the emergence of new data sources [84]. Presently, ITS&L applications are leveraging data derived from three primary sources: • Novel Sensors [111]: this category encompasses a range

of devices such as Inertial Measurement Units (IMUs), On-Board Units (OBUs), GPS navigators, LIDARs, and various environmental sensors. These instruments are pivotal in gathering high-fidelity, real-time data pertinent to logistics operations;

• Social Data [111]: this source includes Geo-tagged infor- mation obtained from social media platforms. Such data is instrumental in extracting user trajectories and behav- ioral patterns, offering valuable insights into consumer behavior and logistics demands;

• Open Data [2]: this refers to datasets related to national public transportation services, which have become increasingly available due to the growing trend towards openness and transparency among government institu- tions. These datasets are crucial for understanding and optimizing public transportation logistics.

The integration of these diverse data sources has undeniably inaugurated a new epoch of possibilities, services, and applica- tions in the ITS&L field [19], [75]. However, this rapid influx of data simultaneously poses significant challenges. It requires the development of scalable solutions for the collection, man- agement, processing (including cleaning and cleansing), and dissemination of these data [45], [124]. Inefficient handling of large data volumes can lead to significant delays in data pro- cessing, resulting in outdated or inaccurate information being fed into decision-making processes. For example, in traffic management, outdated data can lead to incorrect traffic light timings or inaccurate route recommendations, exacerbating congestion and increasing travel times. Moreover, poor data volume management can overwhelm storage systems and lead to data loss or corruption, further compromising the reliability of the service. This can erode user trust and satisfaction, ultimately impacting the operational efficiency and financial viability of businesses reliant on these applications. This situa- tion underscores the inadequacies of traditional methodologies in handling the complexity and volume of current data streams in the ITS&L sector [19], [111].

2) Velocity: Regarding velocity, the ITS&L industry heav- ily depends on real-time data processing for critical decision- making processes [19]. Timely data processing is crucial for several operations, including route optimization [126], which seeks to minimize distances and travel times in real time; delivery time reduction [127], which tracks delivery schedules to improve customer satisfaction; traffic updates [20], which requires collecting real-time traffic information to avoid delays and ensure on-time deliveries; and the overall efficiency of logistics operations [111]. When the data processing mech- anisms fail to match this high velocity, the efficacy of the entire system is compromised, undermining its capability to support the intended applications [19], or leading to system crashes. For instance, in route optimization, delays in pro- cessing traffic updates can cause vehicles to follow suboptimal

paths, increasing travel time and fuel consumption. In delivery logistics, slow data processing can disrupt scheduling, leading to missed delivery windows and decreased customer satisfac- tion. Over time, the increasing velocity of data generation and the demands of associated services have highlighted the inadequacies of traditional methods and tools in addressing these challenges. This revelation underscores the need for innovative approaches in data processing within the ITS&L sector [128].

3) Variety: Finally, regarding variety, the ITS&L sector is witnessing an ever-increasing diversification in its data sources and types [19]. This trend marks a significant shift from the reliance on traditional tabular data formats to a more complex array of data types such as graph data [129] (utilized for representing complex relationships and network structures, essential in logistics for route optimization and net- work analysis), sequences [130] (sequential data, such as time series, are crucial for forecasting and trend analysis in logis- tics), logs [131] (log files provide detailed records of events, transactions, and operations, invaluable for auditing and pro- cess optimization), audio [132] (emerging as a data source for voice-based commands and communications in logistics operations) and natural language [133] (text data, including customer feedback and communication, are increasingly ana- lyzed for insights into logistics performance and customer satisfaction). These diverse data types necessitate sophisticated management strategies, including versioning (along with their metadata), processing, and linking [134], [135]. Integrating these varied data forms without losing any critical information is crucial, as such data can be pivotal for future uses and the development of innovative services. Failure to properly integrate and manage diverse data types can lead to incomplete or inaccurate analyses, negatively impacting decision-making processes. For instance, neglecting the integration of graph data might result in suboptimal route planning, missing out on critical network relationships [136]. Additionally, ignoring log files may prevent the identification of operational inefficiencies or security breaches [137]. Furthermore, the lack of a unified advanced approach to manage these varied data types can result in fragmented data silos [138], impeding comprehen- sive analysis and innovation. In conclusion, managing data variety represents one of the most formidable challenges for contemporary data pipelines in the ITS&L sector. It exposes the limitations of traditional methods and tools, underscoring the need for advanced solutions capable of handling the complexity and diversity of modern logistics data [128].

B. DataOps Lifecycle

Although there is widespread agreement within the data management community regarding the initiation (data acqui- sition) and termination points (analysis and visualization) of the data lifecycle, consensus on the intermediate stages is limited [16], [44]. Nonetheless, various proposed models of the DataOps lifecycle consistently depict an iterative work- flow compliant with agile principles [139], which facilitates continual refinement, adaptation, and enhancement of data processes to address evolving application requirements and challenges [41]. Within this paper, we propose a generalized

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

Fig. 2. DataOps abstract lifecycle.

DataOps process, depicted in Figure 2, designed to reflect the principles of DevOps for efficient software development. The proposed schema abstracts the DataOps lifecycle into eight distinct steps: ingestion, processing, storage, analysis and visualization, release, deployment, monitoring, and feedback.

1) Data Ingestion: Data ingestion in the DataOps pipeline involves planning, acquiring, and loading data (along with its metadata) from various sources into a centralized storage sys- tem [44]. It requires precise data requirement planning, which entails a thorough understanding of application-specific needs, potential data sources and acquisition methods, expected data volume, and relevant attributes. This stage corresponds to the well-known Plans phase in DevOps [39], where project objectives and requirements are defined.

2) Data Processing: The acquired data enters into a pro- cessing stage which entails the implementation of processing [44], cleaning [44], filtering [44], and refinement [44] proce- dures to ensure adherence to the required format and structure defined by the metadata [44]. Any inaccuracies or inconsisten- cies are tracked and possibly addressed, resulting in a dataset that is accurate and aligned with its metadata [44]. This stage aligns with the Code phase in DevOps [39], where developers write and integrate code for new features or improvements.

3) Data Storage: The data storage phase emerges as the central pillar for managing the data volume, velocity, and variety generated by data ingestion and processing [44], [140]. For this purpose, data storage architectures have gradually evolved over the years (i.e., relational database [141], data warehouse [142], data lake [143], and data lakehouse [144]), underscoring the dynamic nature of data storage solutions. Currently, data lakehouse [144] are the standard solution as they leverage the strengths of both data warehouses [142] and both data lakes [145], providing a centralized repository to store structured, semi-structured, and unstructured data. This solution ensures flexibility, scalability, and cost-effectiveness. The storage phase in DataOps aligns with the Build phase of DevOps [39], where a developer commits their code to a shared code repository.

4) Analysis and Visualization: The analysis and visualiza- tion phases align with the Test stage of DevOps pipelines [39]. Its role relies on inspecting the stored data to test the correct operativity of the DataOps pipeline. This step is of paramount importance since it allows to actually inspect and provide data to higher level services (e.g., MLOps), which leverage these data to generate value [146]. In fact, the primary role of the

DataOps substrate is to enable data-driven services, which can span from simple real-time tracking of a service to the delivery of the most advanced AI-based services.

5) Release, Deploy, Monitoring, and Feedback: The con- cluding stages of DataOps pipelines align exactly with the ones of the DevOps [39] pipeline, namely the Ops. First, the release phase ensures data consistency and verifies the proper functioning of the DataOps pipeline for deployment [16]. After release, the deployment phase involves the deployment of developed data pipelines into the designated testing or production environments [44]. Post-deployment, monitoring tools and frameworks are utilized to observe the health and performance of data pipelines [125]. Metrics such as data throughput, processing time, error rates, and alerting mech- anisms can be integrated to assess system performance and promptly notify teams of any issues or anomalies. Finally, the DataOps feedback loop involves the extraction of insights on data quality and performance from monitoring tools, coupled with the assimilation of feedback from end-users or down- stream systems to correct, improve, or better monitor the other steps of the DataOps pipeline [44].

C. Open-Source Solutions

This section reviews the most common and adopted open- source tools providing free access for implementing the DataOps pipeline, rendering them advantageous choices for development projects. These tools empower developers to construct and customize their data architectural framework while facilitating seamless integration with various third- party libraries and come with the support of the open-source community, documentation, and discussion groups [147]. It is important to note that a single tool often does not implement all components of the DataOps pipeline, and developers typi- cally combine multiple tools to achieve the complete pipeline [16], [20], [148].

1) Apache Kafka: Apache Kafka [149], [150] is a dis- tributed streaming platform widely acclaimed as a messaging system in real-time data processing. It supports the ingestion and monitoring phases of the DataOps pipeline. Its versatility and seamless ability to ingest data from diverse sources, including web applications and databases, is well known in both industry and research with more than 29 thousand stars on GitHub as of January 2025. The design of the data processing pipeline not only empowers Kafka to maintain high throughput and low latency while handling substantial data volumes but also enables smooth integration with a diverse array of Apache tools for future data processing. Finally, Kafka offers robust security measures, encompass- ing data encryption during transmission, authentication, and authorization.

2) Apache Hadoop Distributed File System: Apache Hadoop Distributed File System (HDFS) [151], [152] is a distributed file system that stores vast volumes of data across multiple cluster nodes. It supports the development of the storage phase of the DataOps pipeline, being the most used storage tool in data lakehouse [144], [145]. HDFS is a foundational component within the Apache framework, boasting more than 14 thousand stars on GitHub as of January

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 9

2025. The architecture of HDFS is designed to provide high reliability (through data replication) and scalability (through the possibility of adding nodes) in storing and maintaining large-scale data. HDFS natively deals with variety thanks to its ability to handle unstructured data, which includes tables, text, audio, video, images, and more.

3) Apache Spark: Apache Spark [153], [154] is an open- source framework for scalable in-memory data processing. With over 37 thousand stars on GitHub as of January 2025 it has proven to be the golden standard solution for implementing the processing phase of the DataOps pipeline. A key feature of Apache Spark is the distributed (i.e., ability to leverage multiple nodes) in-memory processing capabilities reducing the number of disk accesses and leading to a significant accel- eration in data processing. Finally, Apache Spark seamlessly integrates with various programming languages and cloud platforms, not limited to Apache’s offerings but extending to third-party tools.

4) Apache Flink: Apache Flink [155], [156] is an alter- native open-source framework to Apache Spark. Although it shares many similarities with Apache Spark, Apache Flink focuses on stream processing. Stream processing revolves around continuous data management which is crucial when the immediate processing of data streams is essential. With more than 24 thousand stars on GitHub as of January 2025, Flink is the golden standard in this setting. Stream processing reduces storage requirements compared to batch processing, but the choice between stream and batch processing may vary depending on specific task requirements, considering precision, velocity, and efficiency factors.

5) Ops Tools: The Ops tools are related to the release, deploy, monitor, and feedback steps in the DataOps lifecycle (see Figure 2), namely the ones that are entirely inherited from DevOps. Of course, in the DataOps lifecycle, some of them are the same used in DevOps while others are tailored toward the specific context. dbt (Data Build Tool) [157] is a command-line tool, focused on the release step, that empowers analysts and engineers to execute rigorous testing of data models to assess their reliability and accu- racy before deployment. Docker [158] and Kubernetes [159] jointly form an indispensable duo for deployment [160]. Docker provides container solutions for encapsulating applica- tions and dependencies. Kubernetes orchestrates containerized applications, furnishing scalability and adept management capabilities. Docker and scalability integration facilitates the deployment and scaling of containerized data processing appli- cations [160]. Prometheus [161], Apache Airflow [162], and Jenkins [163] are open-source tools designed for monitoring and alerting, providing real-time insights into pipeline perfor- mance and functionality. On the other hand, proprietary tools like Power BI [164] and Tableau [165] offer user-friendly integration, robust support, and advanced data visualization features. Finally, version control systems (e.g., Git [166] and SVN [167]) represent the final piece in the DataOps puzzle. They enable practitioners to establish an iterative feed- back loop, fostering continuous improvement in data-related operations.

D. Proprietary Solutions

This section is the counterpart of Section IV-C focused on the proprietary tools implementing all the DataOps steps of Figure 2 in a unique user-friendly solution. These proprietary solutions provide dedicated customer support and additional features specifically tailored to address the unique needs of enterprises, a single point of contact for any issues or concerns, and extensive documentation and training resources facilitating the learning curve [147]. However, it is important to consider potential drawbacks, such as licensing costs, potential vendor lock-in, and limited customization options compared to open- source solutions [168].

1) Google Cloud Dataflow (GCD): GCD [169] is a power- ful tool that shares functional similarities with Apache Spark and Apache Flink [155] while introducing its own distinctive features and architectural solutions. Similarly to its Apache counterparts, GCD has been meticulously designed to provide an efficient and low-latency approach to data manipulation while maximizing resource utilization. Contrarily to its Apache counterparts, it operates as a fully managed service on Google servers, integrating all the necessary components for building an efficient DataOps pipeline, which is a unique, easy-to- use solution. It represents a valid solution for users from a non-programming background having a user-friendly graphical user interface. GCD automatically provides cloud resources and automatically upscales or downscales the number of worker instances executing the job based on the data traf- fic. However, GCD can not be served on-premise, limiting its application to certain applications, i.e., enterprises with stringent privacy needs.

2) Amazon Elastic MapReduce (EMR): EMR [170] is an Amazon cloud-based data processing service, featuring technologies and functionalities similar to the Apache suite (e.g., Apache Hadoop and Apache Spark). Like GCD, EMR is a fully managed framework committed to simplifying and streamlining the data processing path that automatically man- ages the scalability and efficiency of the DataOps pipeline for various data processing tasks. For example, the Amazon S3 (Simple Storage Service) [171] is a data storage infrastructure, an alternative to HDFS, to retrieve and manage large amounts of data. Amazon Kinesis [172] is an Amazon service, an alternative to Apache Flink, designed to handle and process streams of large amounts of data. EMR offers fine-grained control over cluster size, allowing users to adjust the amount of data processed automatically and scale resources according to workload needs. EMR, as the GCP, can not be served on- premise.

3) Azure Data Factory (ADF): ADF [173] stands out as a robust cloud-based data integration service provided by Microsoft. Similarly to GCD and EMR, it offers a scalable and efficient cloud data processing solution, all while ensuring a user-friendly experience. This platform empowers users to collect data from various supported sources seamlessly and enables them to create, schedule, and manage data pipelines with the added benefit of automated workflow orchestration. Furthermore, it allows users to leverage the capabilities of the Apache Hadoop and Spark ecosystems within their data pipelines. This means that users can incorporate the power and

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE II COMPARISON OF REPRESENTATIVE DATAOPS TOOLS

versatility of these popular open-source frameworks directly into their workflows, enhancing the overall capabilities and efficiency of the data processing pipelines in the Azure Data Factory environment. ADF, contrary to GCD and EMR, allows for an in-premise deployment of the platform.

E. Solutions Comparison

The choice between open-source and proprietary DataOps solutions in ITS&L depends on the specific application and on the company competencies, requirements, budget constraints, and levels of control and customization needed [168], [174]. To facilitate comparison across the reviewed tools, Table II summarizes open-source and proprietary DataOps solutions in terms of lifecycle coverage, deployment model, end-to-end support, scalability, and main limitations.

The open-source solutions (Section IV-C), apart from being cost-effective, are surely the most popular choice for DataOps

being reliable, interoperable, strongly customizable, and sup- ported by a large and active community of developers [147]. A common limitation of these platforms is the requirement for advanced skills in integration, deployment, and maintenance, which restricts their adoption in many contexts. Specifically, open-source tools are generally designed for specific purposes, requiring practitioners to combine multiple components to build an end-to-end DataOps pipeline. For example, Apache Kafka is used for ingestion, Apache Spark for processing, Apache Hadoop HDFS for storage, Jenkins for analysis and visualization, dbt for release, Docker and Kubernetes for deployment, Apache Airflow for monitoring, and Git or SVN for feedback.

Proprietary solutions, on the other hand, provide user- friendly alternatives with enterprise-level support, enabling less specialized operators to deploy DataOps pipelines. These solutions are fully integrated and designed around “what the user wants to achieve” rather than “how it should be

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 11

implemented” [175]. Because of this fact, proprietary solutions are less customizable by design [176], even if interoperability and compatibility with open-source solutions are often guar- anteed. They scale very well both in terms of software, as the open source solutions, and in terms of hardware, as they offer both horizontal and vertical scalability in the cloud. They have the limitation of being closed sources [147], pushing toward a lock-in developing mode. Another limitation is the trade-off

between costs and benefits [168], [174], which is not always simple to estimate, taking into account also the problems of safety and security [147] and service continuity [177].

F. Case Studies

This section presents real case studies in ITS&L that have implemented DataOps solutions. These case studies encom- pass the 6 areas of ITS&L presented in Section III. In the next paragraphs, we present the case studies retrieved for each area in the last 10 years, highlighting complementary examples that demonstrate particularly complex data management chal- lenges. Notably, in most of the presented cases, the DataOps solutions implemented leverage open-source Apache technolo- gies, demonstrating their effective integration and adaptability for real-world scenarios.

1) Vehicle Control Systems: In this area, we found a total of 11 cases, distributed among fleet management [122], [178], [179], decision support and making [180], [181], [182], [183], vehicle incident identification [184], [185], and driver support [186], [187]. The platform of Yoo et al. [186] aims to provide drivers with real-time vehicle and road condition information. It collects and processes big data from connected cars, using a messaging system to handle large-capacity sensor data (e.g., OBU, CAN, camera, LiDAR, GPS) and traffic infor- mation. The system, built on the Hadoop ecosystem (Hadoop, Spark, Kafka), stores vehicle data in HDFS and uses Apache Spark for big data analysis. Apache Kafka facilitates data transfer between the vehicle and server, while Spark Streaming enables real-time data analysis. The DBDA platform described in [122] combines various data management and analytics tools, such as MongoDB, Apache Hadoop, and Apache Spark, to ensure reliable and continuous data collection, storage, processing, and analysis of data generated by vehicles. Given the large volume of data collected, the authors highlight the importance of monitoring data quality. To address this, the DBDA platform integrates automatic data quality control pro- cedures, including outlier and duplicate detection, to sanitize the ingested data. Lastly, the CARSTREAM project [180] integrates multiple services to support vehicle systems while monitoring the quality of the vast amount of collected data. The CARSTREAM architecture collects and processes various types of driving data, such as vehicle status, driver activity, and passenger-trip information. CARSTREAM is connected to over 30,000 vehicles distributed across 60 different cities in China, managing terabytes of driving data and provid- ing data analytics-based services. With nearly a billion data instances collected daily, CARSTREAM requires an efficient data operations pipeline capable of managing data collected during peak hours and resisting potential faults. To achieve this, CARSTREAM integrates tools such as Apache Spark

and Kafka for data streaming operations, and Apache Hadoop YARN for fault tolerance, processor isolation, and resource management.

2) Traffic Management: This area is the one that collects the highest number of case studies in the literature. We found a total of 19 cases, distributed between real-time urban traffic [20], [148], [188], [189], highway traffic detection and control [84], [190], [191], [192], [193], traffic prediction and simulation [194], [195], [196], [197], IoT smart transportation [198], [199], [200], [201], or air traffic flow [202], [203]. Nevertheless, according to the best knowledge of the authors the most notable ones are the following. Gillani and Niaz [148] implemented a real-time traffic data collection system city that utilizes Apache Kafka and Apache Spark to collect, process, and store data from approximately 400, 000 unique vehicles daily on Dublin roads, Ireland. Authors of [84] integrate Apache Spark and Hadoop to develop a DataOps pipeline to optimize highway traffic flows in Portugal by adjusting toll prices during peak hours, thereby encouraging drivers to use less congested routes. Amini et al. [20] discussed the integration of Apache Spark, HDFS, and Kafka to manage real-time traffic data from Munich’s highway network. A similar architecture has been utilized by Babar and Arif [198] to analyze traffic speed data in Aarhus city, Denmark, and within the SMASH project [188] to process large-scale traffic data for Adelaide and Victoria cities, Australia. Lastly, the BiDaML tool [194] integrates Microsoft’s Azure proprietary end-to-end architecture to ingest, process, store, visualize, and simulate traffic data in Victoria, Australia.

3) Transportation Management Systems: In this area, we found a total of 14 cases, distributed mostly in urban trans- portation analysis and planning [92], [93], [96], [189], [199], [202], [204], [205], [206], [207], [208], [209], [210], [211]. Among them, the BIGSEA [93] and ADMSv2 [189] projects are the most popular ones. BIGSEA is a DataOps solution for performance analysis and prediction of public transporta- tion services. The pipeline, which utilizes Apache Spark and HDFS, offers a scalable data management solution for European and Brazilian public transportation systems. The BIGSEA architecture has been later utilized and integrated by Fiore et al. [92] to tackle urban traffic data analysis and planning challenges of the municipality of Curitiba, a 1.8 million-inhabitants city in the south of Brazil. Qi et al. [96] offer a custom framework with efficient extraction and analysis of Twitter data for evaluating public opinions on trans- portation services. Despite the above example implementing only some stages, the ADMSv2 project [189] offers an end-to- end DataOps architecture to handle a variety of data sources, including massive and high-resolution traffic sensor data, bus trajectory data, transportation network data, and traffic inci- dents data from the Los Angeles Metropolitan Transportation Authority. It integrates several open-source frameworks to ingest (Apache Kafka), store (Apache Hadoop HDFS), manage (Apache Spark), and analyze data with custom web interfaces to efficiently support urban planning, public policy, and emer- gency response. T ärneberg et al. [199] utilize cloud-based IoT infrastructure in AWS to integrate multiple sensors and source data in Charlottesville, VA roads to regulate the red-green

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

12 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

patterns of traffic lights with the goal of increasing safety and minimizing wait/idle times. We also found some work around transportation management of metro, trains, or flights [202], [209], [210], [211] where the practical goal is to construct a DataOps pipeline for transportation delay forecasting and planning.

4) Traveler Information and Experience Systems: In this area, we found a total of 9 cases, all focusing on forecasting and recommendation systems for tourists [99], [103], [104], [212], [213], [214], [215], [216], [217]. These applications aim to enhance travelers’ experiences by providing accessible information from multiple sources. They face the challenge of integrating various data sources and delivering real-time information to users. For instance, the platform in [103] supports travel and hotel management planning by utilizing Hadoop and Spark to analyze hotel data and user comments. Malkawi and Alhajj [214] propose a web-based real-time ticket recommendation system that uses Apache Spark to retrieve information from various airline websites, allowing users to receive recommendations based on the lowest prices. Lastly, [217] integrates big data tools (i.e., Apache Spark and Apache Hadoop) in an iOS mobile app to provide users with essential information about tourist attractions and accommodations in Lampang Province, Thailand.

5) Freight and Supply Chain Logistics: In this area, we identified 14 cases, covering both the monitoring and planning of goods delivery [108], [110], [218], [219], [220], [221], [222], [223], [224], [225], [226], [227] and the optimization of freight management policies, such as pollution-emission reduction [107], [228]. The architecture proposed by Selvaraj et al. [222] combines Apache Hadoop, Hive, and Spark into a unified data-operations solution for truck management com- panies, supporting logistics optimization, cost and overhead reduction, profit-margin improvement, and load balancing. Provatas et al. [223] develop a data-operations framework for four distinct logistics case studies, including goods delivery by trucks for a Greek logistics company. The architecture uses Hadoop YARN for cluster and resource management, thereby supporting scalability under variable user demand, and Apache HBase to store event logs subsequently processed with Apache Spark. Sarabia-Jacome et al. [108] examine the Valencia seaport, one of the major ports on the Mediterranean coast, which handles millions of movements per year. To process large volumes of data within short time windows and support communication among seaport stakeholders, the proposed architecture integrates several open-source data- management tools, including Apache Hadoop and PostGIS for efficient storage, Apache Spark for streaming operations, and Apache Kafka for data ingestion and processing. The same architecture also supports warehousing and delivery within the seaport.

On the supply-chain and delivery side, Silva et al. [224] pro- pose a framework for optimizing supply-chain management, demonstrated through a real-world case study in the logistics innovation department of an automotive factory generating large volumes of data. Their architecture integrates Apache Spark for data cleansing and transformation and Hadoop HDFS for distributed storage. Moreover, proprietary tools

such as Power BI [164] and Tableau [165] are used for data visualization, providing interactive dashboards that present analysis results to end users. Finally, eSPY Carting Systems [226] is a real-time tracking system designed to anticipate potential faults in products and services, thereby improving the reliability of goods transport and delivery. The project inte- grates several big-data technologies, including Apache Spark and Hadoop for data processing and storage, Apache Kafka for data ingestion, and Apache Hive for metadata management.

Although additional studies in this area exist, such as [110] and [227], they often do not explicitly address big-data challenges or discuss the data tools employed. This indicates an opportunity for further exploration and documentation of DataOps applications across the entire supply-chain spectrum.

6) Emergency Management: In this area, we found a total of 10 cases, distributed between traffic event detection and response (e.g., traffic, congestion, etc.) [82], [189], [229], [230], [231] and public health safety logistic control and planning [119], [232], [233], [234], [235]. Regarding the former family, we mention the work by Alomari et al. [82], which integrates Apache Spark with ML algorithms for the automatic detection of road traffic events from tweets in Saudi Arabia. The framework SEHAA [234] combines Apache Spark to collect and process huge quantities of data from Twitter with ML models to assist healthcare services in Saudi Arabia. The ADMSv2 project [189], also seen among trans- portation management systems case studies, further supports emergency management operations efficiently by continuously ingesting and processing vast amounts of data using Apache data operations solutions. For public health safety logistic control and planning, we note several studies [119], [232], [233], [234], [235]. We also highlight the case study by Noori and Mehra [235], which combines Apache Spark, Twitter data, and ML models for timely fire emergency detection. In these applications, the use of DataOps solutions such as Apache Spark is justified by the large scale of data collected (from social media or IoT devices) and the requirement for fast processing to enable rapid decision-making.

7) Remarks: Overall, the reviewed case studies indicate that DataOps pipelines address recurring data management challenges across ITS&L domains, although the relative importance of these challenges differs across application areas. Vehicle Control Systems and Traffic Management are primar- ily driven by the need to ingest and process high-volume, high-velocity data streams generated by vehicles, sensors, and road infrastructure. Transportation Management Systems and Freight and Supply Chain Logistics require stronger support for data integration, as they combine geographically dis- tributed and heterogeneous sources, including transit records, operational logs, seaport data, supply-chain events, and user- generated information. Traveler Information and Experience Systems place greater emphasis on continuous data integra- tion and service responsiveness, since recommendations and travel information must be continuously updated from multiple external sources. Emergency Management applications place the strongest emphasis on low-latency processing, as timely event detection and response depend on the rapid analysis of social media, IoT, and traffic-related data. Across these

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 13

Fig. 3. MLOps abstract lifecycle.

domains, Apache-based technologies such as Kafka, Spark, and Hadoop recur frequently, confirming their practical rele- vance for scalable ingestion, storage, stream processing, batch analytics, metadata management, and resource orchestration in real-world DataOps deployments.

V. MLOPS

AI, particularly in the form of ML, is a major catalyst for innovation in products and services within ITS&L appli- cations. ML empowers the formulation of descriptive [236], diagnostic [236], predictive [236], and prescriptive analytics [236], enabling the monitoring, improvement, and innovation of new products and services [237]. The landscape of ML can be broadly divided into two major categories: Shallow ML and Deep ML. Shallow ML can handle unstructured, tabular data with small to medium cardinality, leveraging effective manual feature extraction and ML model design based on domain knowledge [238]. In contrast, Deep ML is mandatory when dealing with medium to huge cardinality unstructured data, such as images, graphs, and natural language [239], [240]. It enables the direct learning of well-suited represen- tations from the raw data [241], [242]. The building phase is commonly referred to as the training phase, while the usage of the model is termed the test phase. The training phase can be computationally demanding, especially for deep ML, often necessitating specific hardware accelerators (i.e., GPUs). In contrast, the test phase is typically less computationally demanding, even with substantial data quantities.

While ML can be conceptually viewed as a specific block within the DataOps pipeline, its unique characteristics necessitate a dedicated pipeline known as Machine Learn- ing Operations (MLOps) [243], [244]. This distinction arises from several reasons: (i) continuous update requirements, (ii) management of design complexity, (iii) management of deployment complexity, and (iv) monitoring challenges. MLOps, by adapting and extending DevOps principles for ML, facilitates the retrieval of training data, exploration, feature extraction, model training, testing, release, deployment, monitoring, and feedback processes with a high degree of automation (see Figure 3). MLOps offers a promising response to these challenges in the ITS&L sector. It integrates practices and technologies to streamline the entire ML lifecycle, from problem data collection to model training and testing, ensuring efficient processing and continuous model availability in the

deployment service [43]. In the end-to-end view presented in Figure 1, this section corresponds to the MLOps layer, which builds on reliable DataOps pipelines to manage the development, deployment, monitoring, and updating of ML models in operational ITS&L services. This layer transforms reusable data assets into AI-enabled decisions and services, while providing the mechanisms required to monitor model behavior and adapt models as data distributions and opera- tional conditions evolve.

In the following, we start by revisiting the underlying motivations for embracing MLOps, as detailed in Section V-A. Following that, we describe the various stages encompassed within an MLOps pipeline, as outlined in Section V-B. Subse- quently, we examine the principal open-source (Section V-C) and proprietary MLOps tools (Section V-D). A comparative analysis of these tools is presented in Section V-E, offering practical insights and implications. The section then delves into the case studies of the ITS&L domain that implement MLOps technologies, presented in Section V-F. This section concludes with the discussion of a specific branch of the MLOps called LLMOps, namely MLOps for large (multimodal) language models, which surely represent one of the highest impact and promising technologies of the last decade. In particular, we will deepen the necessity of this branch of MLOps and present some related case studies inside the ITS&L.

A. Motivations

The adoption of MLOps practices has been driven by four fundamental requirements.

1) Continuous Update: Data usually dynamically evolve in the ITS&L industry, demanding a continuous model update to address this data shift and provide accurate real-time decision-making [244], [245], [246]. Consider, for example, a scenario where a logistics company utilizes an ML model for route optimization. The dynamic nature of traffic conditions, road closures, and unforeseen events requires the model to continuously adapt to these changes to incorporate the latest information, ensuring that the recommended routes remain efficient and accurate [247]. Moreover, ML technology is also a fast-evolving field where new, more effective, or efficient models (e.g., convolutions or transformers) are developed every year, requiring, even in slow changes in the data, to advance the ML technologies continuously [248]. Without continuous model updates, the system may rely on outdated data or ML architectures, leading to sub-optimal route rec- ommendations and potentially impacting the timely delivery of goods [38], [247]. However, model update involves a sequence of non-trivial operations, such as tuning or retrain- ing, model versioning, functionality testing, deployment, log monitoring, etc. If performed manually, these operations could slow down the production process, potentially causing delays in bringing the upgraded service online [243], [245]. MLOps facilitates the implementation of a responsiveness and automatic ML pipeline that processes and integrates novel knowledge arising from data collected over time [43], [246] and novel ML models developed by researchers [249].

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

14 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

2) Design Complexity: The deployment of ML mod- els introduces intricacies that surpass those encountered in deploying traditional software modules. Unlike conventional software, ML systems combine input signals, creating entan- glements that challenge the isolation of improvements [250], a fundamental principle in software testing and DevOps method- ologies. A change in the input distribution, training algorithm, data processing, or any aspect of the ML development can lead to several changes to the whole pipeline [250]. Additionally, ML is distinguished by numerous free parameters, known as hyperparameters, which pose a challenge as these cannot be pre-set and require meticulous tuning whenever environmental changes occur. Furthermore, the ML technology landscape, as outlined in the 2023 ML AI Data Landscape Report,3 is characterized by substantial fragmentation in terms of both code quantity and quality. Managing this diversity presents a challenge, as a single ML project may require dozens of different libraries, each with varying levels of reliability [250]. Some libraries may become discontinued or unmaintained, requiring replacement or patching, while others may transition from open to closed-source, demanding careful monitoring of licenses and possible library changes to maintain the project’s reliance on open-source tools. In summary, the dynamic nature of ML software modules and the landscape demands contin- uous monitoring and adaptation to keep pace with evolving technology. This includes managing the interaction between traditional and ML-based software modules, tracking licenses, and being prepared to switch to alternatives. MLOps naively provides support for the solution of these problems.

3) Model Monitoring: Accurately monitoring ML-based model operations surely leads to several advantages, but, compared to traditional software, it raises many different criticalities [250], [251], [252]. Monitoring ML models implies monitoring two main aspects: input data and ML model results [244]. Input data may contain outliers or drifts, e.g., with respect to the training data, requiring detection (in the case of outliers) or retraining the models (in the case of drifts). Second, tracking ML model results and behavior provides valuable insights into how well the model aligns with the current data distribution and operational context [251]. How- ever, ML models are usually designed to optimize one or more metrics. For these reasons, multiple metrics describing the average behavior of the model need to be exploited simultaneously [48], [252]. These metrics can be technical (e.g., accuracy, robustness, energy, etc.) but they may be also ethical (e.g., explainability, fairness, privacy, etc.) [48]. This requirement is becoming more and more vital also in view of the recent AI-related regulations (e.g., AI Act [35], [36], [37]). Even if multiple metrics are continuously monitored, security issues may still arise due to the vulnerability of ML models that need to be readily patched. Finally, ML models are often non-deterministic both in the training (the training data may produce different ML models) and in the test phase (the same test data may produce different ML model outputs). MLOps can significantly contribute to the ML model performance

3https://www.lxahub.com/stories/key-takeaways-from-the-2023-ml-ai-data- landscape-report

evaluation, enabling proactive interventions and adjustments to ensure alignment with multiple technical and ethical metrics.

4) Complexity of ML Deployment: Two main factors influ- ence the complexity of deploying ML models. The first is reproducibility [250], and the second is resource allocation [250]. Reproducibility, defined as the ability to repeat experi- ments and obtain the same (or at least similar) results [253], is essential to increase transparency [253] and build user confidence in ML applications and their components [254]. However, the non-deterministic nature of many ML models and the need for continuous updates undermine reproducibility [250]. Resources allocation, instead, deals with the problem of retraining ML models and deploying them automatically while ensuring the scalability of the entire system and main- taining service availability for users. During the training phase, the system must manage fewer jobs, potentially requiring the adoption of on-demand distributed computing solutions [251]. These jobs typically involve training or updating a few hundred models and demand significant computational power. Conversely, during the testing phase, the system must handle a large volume of requests, often proportional to the number of users multiplied by the frequency of requests for the service. While these individual requests may have relatively modest computational requirements, the system must efficiently process and respond to many without compromis- ing service quality or responsiveness. MLOps supports both reproducibility and traceability throughout the entire ML life cycle together with an instrument able to fully automatize the deployment both for training and test phases.

B. MLOps Lifecycle

The MLOps lifecycle defines an iterative workflow for the continuous refinement, adaptation, monitoring, and improvement of ML processes to meet evolving application requirements and challenges [43], [249], [255]. Hence, a comprehensive MLOps process is depicted in Figure 3 to embody DevOps principles within the ML domain. This proposed framework distills the MLOps lifecycle into eight well-defined stages: data collection, data engineering, training, model testing, release, deployment, monitoring, and feedback.

1) Data Collection: The MLOps pipeline starts with the data collection tailored to meet specific application require- ments. Due to the large scale of data commonly employed, particularly in training Deep ML models, integrating DataOps solutions becomes fundamental to streamlining the entire process. Specifically, after the data is stored within a DataOps pipeline, it can be seamlessly retrieved for ML model training and test phases. This stage aligns with the Plan stage in DevOps [39].

2) Data Engineering: Data engineering includes data cleaning and feature preparation, often identified as among the most time-consuming activities in the workflow [256]. While data cleaning, i.e., improving data quality, can be part of both DataOps or MLOps, feature engineering is primarily con- cerned with MLOps. In fact, feature engineering deals with the problem of transforming raw data into meaningful information that can be handled by ML algorithms. For tabular or mildly unstructured data, the feature engineering phase is commonly

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 15

time-consuming as it is manually designed by domain experts or ML practitioners [256]. The resulting features are then fed directly to shallow ML algorithms. For unstructured data, these features cannot be manually designed and must be learned directly from the data (e.g., using convolutions or attention mechanisms). However, while the latter allows to remove the human intervention, making the process more automatic, it requires huge amount of data to be performed. Finally, data and the corresponding extracted features are split into multiple splits, typically clarified as training and validation, used during training (see Model Training in MLOps) and validation (see Model Validation in MLOps), respectively. This stage aligns with the Code phase in DevOps [39], where developers write and integrate code for new features or improvements.

3) Model Training: Model training includes both model engineering and model fitting. The former involves defin- ing and identifying, at higher level, relevant ML algorithms (e.g., XGBoost [257] or Deep Neural Networks [242]) with the associated possible architectures (e.g., convolution [258] or attentions [259]) and possible set of hyperparameters (e.g., number of weak models [257] or size of the convolu- tion kernels [258]) [243]. In other words, model engineering defines the set of possible ML algorithm configurations to be trained and subsequently evaluated to choose the best options (see Model Validation in MLOps). This phase is mainly guided by experience, data type, business and user requirements, and computational resources. Experience, i.e., human in the loop, plays a central role because the sheer number of choices in the ML algorithms makes a thorough evaluation of each one impractical. Therefore, past expe- rience becomes invaluable in guiding the decision-making process. The data type determines whether simpler and shal- lower ML approaches or more complex and deeper ML techniques are appropriate. Businesses and users provide perspectives on the potential commercial value, considering costs and benefits and users’ expectations on the quality of service provided by the ML model. Lastly, factors such as available computing power and infrastructure influence decisions, such as using pre-trained models or adopting end- to-end training approaches, narrowing down the available options [251].

Given a specific model engineered in the previous phase, the model fitting phase focuses on optimizing its parameters using the provided training data samples. This optimization process involves adjusting the model’s parameters to minimize a predefined loss function, which measures the discrepancy between the model’s predictions and the actual targets in the training dataset [260]. This process involves iterative updates to the model’s parameters using gradient-based optimiza- tion algorithms [260]. Furthermore, the choice of the model architecture and the optimization algorithm–together with its hyperparameters like learning rate and momentum–can signif- icantly impact the convergence and effectiveness of the fitting process. Shallow ML models, such as ridge or logistic regres- sion, typically lead to convex optimization problems with efficient convergence to global minima. In contrast, Deep ML models navigate non-convex optimization landscapes where local minima are found during training.

This stage aligns with the Build phase in DevOps [39], where developers refine ML models to meet the demands of the operational environment.

4) Model Validation: Model validation deals with the remaining choices left to be made during the Model Training phase of MLOps. Specifically, during the model training phase, candidate models are engineered and trained to reduce the error on the training dataset. The model validation phase focuses on selecting the one that provides the best compro- mise in terms of multiple metrics (e.g., technical, ethical, or business) on a validation dataset to avoid data-snooping [261]. However, the challenge lies in finding the proper set of metrics for the specific application and striking a balance among them, as they often conflict with each other. Tech- nical metrics aim to achieve optimal results with minimal resource utilization, while legislative frameworks like the AI-Act mandate adherence to specific ethical requirements (i.e., explainability, fairness, and robustness) to mitigate both intended and unintended harm. Additionally, business metrics aim to optimize user satisfaction while maximizing company gains. Finally, training and model evaluation are closely interconnected; if model evaluation produces poor results, model training must be adjusted accordingly. Conversely, if the model evaluation produces excellent results (e.g., performance that is much above the one requested by the application), the model complexity may be reduced (e.g., from deep to shallow model) to decrease the computational requirements and then increase the business profit. The validation phase of the MLOps pipeline aligns with the test phase of DevOps [39]. Here, developers integrate MLOps tools and techniques to validate the development and correctness of the models that meet desirable technical, ethical, and business requirements.

5) Release, Deploy, Monitoring, and Feedback: The final stages of the MLOps pipeline, similar to DevOps and DataOps, deal with operational and automated activities related to ML model release, deployment, monitoring, and feedback. During the release phase, the focus is on ensuring the consistency of the model and verifying its functionality for distribution, i.e., verifying that software dependencies are aligned with the deployment environment. Afterward, the model is deployed in the production environment and automatically scaled to ensure system availability. Then, monitoring mechanisms are put in place to monitor the performance of the deployed model in real time, using a subset, the same, or a superset of the technical, ethical, and business metrics defined in the Model Validation, and collect feedback [262]. This allows operators to check for abrupt deviation in real-time operativity of the model behavior due to changes in the data or in how we evaluate the model [244], [262]. Finally, the feedback loop in MLOps involves gathering information from the monitoring phase, facilitating continuous improvement and refinement over time while ensuring the ML model effectiveness and reliability [244].

C. Open-Source Solutions

This section, similar to Section IV-B for DataOps, reviews the most popular open-source tools that offer free access to the implementation of the MLOps pipeline, making them

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

16 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

cost-effective options for project development. These tools allow developers to build and customize their ML workflows while enabling them to integrate other third-party libraries (e.g., PyTorch, Keras, etc.). Furthermore, given their open- source nature, they benefit from the support of the open-source community, documentation, and discussion groups. It is worth noting that a single tool often only implements some of the components of the MLOps pipeline. Rather, developers usually combine several tools to obtain a complete pipeline.

1) Metaflow: Developed by Netflix, Metaflow [263] is an efficient tool to implement the MLOps pipeline in data-intensive applications. Specifically, it facilitates feature engineering, model building, training, validation, and deploy- ment for large-scale ML projects demanding scalability and productivity. With over 8 thousand stars on GitHub as of January 2025, it offers a user-friendly interface and the adapt- ability to meet the requirements of both prototyping and production stages. Metaflow excels in integrating with various on-demand cloud computing services (e.g., Amazon AWS [264]).

2) ZenML: ZenML [265] is the main MetaFlow alternative with over 4 thousand stars on GitHub as of January 2025. The peculiarity of the ZenML pipelines is in their structured approach, which treats each step of the ML workflow as an individual and reusable component. This design philoso- phy promotes reproducibility and consistency across diverse environments. Additionally, ZenML integrates advanced fea- tures for experiment tracking, data versioning, and pipeline monitoring with well-known third-party tools (e.g., Weights and Biases and MLFlow Tracking). These integrated tools empower developers to efficiently track experiments, version and iterate on datasets, evaluate model performance, reproduce and save models, monitor ML runtime behavior, and perform various other tasks, all from a unified dashboard interface.

3) BentoML: BentoML [266], introduced in 2019 and claiming over 7 thousand stars in GitHub as of January 2025, is an open-source tool meticulously conceived to simplify and accelerate the release and deployment of ML models, not implementing the entire MLOps pipeline. This tool offers compatibility with several ML frameworks, such as Tensor- Flow and PyTorch, as well as seamless integration capabilities with DataOps systems (e.g., Apache Airflow [162]). Ben- toML guarantees efficient scalability and monitoring of model deployments, providing a robust infrastructure for real-time model management and monitoring. Furthermore, it provides a user-friendly dashboard that simplifies the management and monitoring of ML models, providing an intuitive interface for users. Finally, it supports various deployment methods, including Docker [158], Kubernetes [159] with Yatai and inte- gration with various on-demand cloud services (e.g., Amazon AWS) through Bentoctl. BentoML is distinguished by several key features that integrate tightly with other MLOps tools (e.g., ZenML, MetaFlow), improving and accelerating large- scale deployment operations.

4) Kubeflow: Kubeflow [267] is an open-source tool, claim- ing over 14 thousand stars in GitHub as of January 2025, for integrating ML capabilities within the Kubernetes ecosystem. It leverages the robust Kubernetes infrastructure to manage

intricate ML workflows such as training on distributed nodes. Compared to BentoML, which offers versatility among vari- ous ML frameworks and integration with DataOps systems, Kubeflow specifically targets Kubernetes environments and deployment operations, prioritizing large-scale system repro- ducibility. By leveraging Kubeflow, users can orchestrate sophisticated ML pipelines, efficiently manage experiments, and seamlessly deploy and serve models in a Kubernetes environment. This approach improves scalability and resource management and promotes consistency and reproducibility of ML workflows, aligning with industry standards and best prac- tices. Furthermore, Kubeflow’s emphasis on integration with Kubernetes ensures compatibility and seamless deployment across different on-demand cloud services, further increasing the attractiveness for projects seeking to leverage Kubernetes for their ML operations.

5) Apollo: Apollo [268], developed by Baidu and reporting more than 24 thousand GitHub stars as of January 2025, is an open-source autonomous-driving platform that provides an integrated operational environment for deploying vehicle control applications. Within the MLOps lifecycle, Apollo primarily supports model development, simulation-based val- idation, deployment, and runtime execution for safety-critical ITS&L pipelines. Unlike general-purpose MLOps tools, Apollo is tailored to vehicle control applications, providing ITS&L practitioners with a domain-specific operational stack for autonomous-driving systems.

D. Proprietary Solutions

Proprietary MLOps solutions implement all the MLOps steps depicted in Figure 3, providing an end-to-end frame- work for model development and deployment. These MLOps services are commonly provided by cloud vendors [245] who offer on-demand computational resources and data and model storage, facilitating the training and versioning of ML models. Such a comprehensive suite of services simplifies the entire ML lifecycle, from development to deployment, providing organizations with the infrastructure and tools needed to deploy and scale ML solutions effectively [245].

1) VertexAI: VertexAI [269] is a proprietary platform designed by Google to facilitate the training, deployment, and customization of ML models and ML-based applications. The platform offers various options for data preparation, model training, validation, deployment, and monitoring. One key advantage of VertexAI is its simplicity in developing ML projects with user-friendly, accessible, and complete dash- boards (it can be used as a zero-code platform). VertexAI, for example, integrates AutoML, a service that enables ML model training on tabular, image, text, or video data without the need for coding or data preparation. Alternatively, custom training provides complete control over the training process, including using ML frameworks and hyperparameter tuning options. After ML model deployment, VertexAI offers end-to- end MLOps tools to automate and scale projects throughout its lifecycle. These tools run on fully managed infrastructure that can be customized based on performance needs and budget availability. The platform also offers services for managing

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 17

model versions and monitoring model quality and trustworthi- ness in production environments.

2) IBM Watson Studio: IBM Watson Studio [270] sim- plifies the development, training, and deployment of ML models, supporting multiple data sources. Akin to VertexAI, Watson Studio integrates AutoML capabilities, automating model pipeline creation, data pre-processing, model selection, and performance evaluation with new data. Users can effi- ciently manage models throughout their lifecycle by utilizing advanced capabilities such as AutoML and model monitoring. Combining open-source frameworks like PyTorch, Tensor- Flow, and scikit-learn with IBM tools, Watson Studio supports visual and code-based data science through Jupyter Notebook, JupyterLab, and command-line interfaces in Python, R, and Scala. Two aspects of MLOps development distinguish Watson Studio. First, it enables deployment on IBM cloud services (as for VertexAI) or on-premise. The latter becomes particularly interesting for companies that retain data and model privacy. Secondly, it provides reliable native metrics to monitor data accuracy, correctness, and drift, culminating in risk manage- ment scores to meet regulatory compliance and other risk objectives effectively.

3) Microsoft Azure Machine Learning and Amazon Sage Maker: Microsoft Azure Machine Learning [271] and Ama- zon SageMaker [272], developed by Microsoft and Amazon, respectively, are solid alternatives to Google VertexAI. Both platforms facilitate the efficient and user-friendly develop- ment of large-scale MLOps pipelines by supporting pop- ular programming languages (e.g., Python) and libraries (e.g., PyTorch). Azure ML features seamless integration with popular third-party tools such as BigQuery and Spark for efficient data processing and analysis. On the other hand, Amazon SageMaker seamlessly interacts with Amazon Elastic MapReduce to efficiently manage the data used in the MLOps pipeline. Like IBM Watson Studio, Azure ML enables on- premise usage of their services to promote data and model privacy if users or companies require it.

4) MATLAB and Simulink: Developed by MathWorks Inc., MATLAB [273] and Simulink [274] are two systems that enable users and companies to build an end-to-end pipeline of MLOps within their cloud architecture. The AutoML service has also been integrated to automatically find the best ML model and optimal hyperparameters for a given training data set. The AutoML component uses MATLAB Parallel Server to accelerate this process by simultaneously training and eval- uating multiple models. In particular, the MATLAB MLOps system integrates drift detection applications that compare the values of newly observed data with those of the model’s training set. In production, it can read the observed data in near real-time from an Apache Kafka stream, detect anomalies in the new data, and automatically create a new ML model optimized for the cleaned data.

E. Solutions Comparison

The selection between open-source and proprietary MLOps tools necessitates careful consideration of various factors.

To facilitate comparison across the reviewed tools, Table III summarizes open-source and proprietary MLOps solutions in

terms of lifecycle coverage, AutoML support, deployment model, end-to-end support, scalability, and main limitations.

Proprietary tools allow developers to construct end-to- end MLOps pipelines tailored to meet specific application requirements while offering flexibility in deployment options, particularly in hardware configuration. Leveraging features such as auto-scaling and containerization, these tools facilitate seamless scalability of ML models, vital for accommodating the fluctuating workloads inherent in ITS&L. Furthermore, most proprietary MLOps solutions boast user-friendly (or ever zero-code) interfaces and advanced features like AutoML, simplifying development processes, particularly for non-ML experts. Despite offering comprehensive support and main- tenance, proprietary solutions may incur higher costs and provide limited deployment options, especially for on-premise (or hybrid) scenarios. Moreover, they may not provide exhaus- tive monitoring of system performance or behavior. For instance, IBM Watson Studio integrates trustworthy artificial intelligence (TAI) monitoring and risk management capabili- ties that competitors do not natively implement. At the same time, however, the measures implemented within IBM Watson Studio are not exhaustive, leaving room for additional unmon- itored ethical considerations (e.g., robustness and privacy). In contrast, open-source MLOps tools benefit from extensive support from the open-source community and provide devel- opers with more flexibility in developing and customizing the application, optimizing the deployment and maintenance processes, and customizing the monitoring process (e.g., with custom technical, ethical, and business metrics) while reducing ML infrastructure costs. These tools can be integrated with one another to provide several utilities for easy deployment, including integration with proprietary cloud services or private servers, and allow the integration of external monitoring tools to ensure the correctness of ML model operations. How- ever, open-source solutions may pose challenges related to maintenance and longevity, as some libraries may face dis- continuation or lack of maintenance, necessitating developers to replace or patch them over time. Additionally, transitioning from open to closed-source software could require adjustments to maintain reliance on open-source tools.

F. Case Studies

This section presents real case studies in ITS&L that have implemented MLOps solutions on the 6 areas of ITS&L presented in Section III. However, in contrast to the extensive literature around the relevance of DataOps in ITS&L appli- cations, there is a noticeable lack of practical relevance and case studies of MLOps in this field. While AI has garnered significant interest in the ITS&L domain, many studies pri- oritize the predictive performance of integrated models over the computational efficiency during development and post- deployment phases [275]. For instance, the ADMSv2 project [189] employs ML models for traffic data analysis but lacks MLOps practices. Although it includes DataOps components to scale data operations effectively, it relies on classic training and deployment methods, lacking support for automatic scal- ing, thereby limiting its ability to manage increasing workloads dynamically. Similarly, Alomari et al. [82] and the SEHAA

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

18 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

TABLE III COMPARISON OF REPRESENTATIVE MLOPS TOOLS

framework in [234] integrate DataOps for data processing but do not incorporate MLOps techniques, highlighting a gap in fully automated and scalable ML model lifecycle management. In conclusion, compared to DataOps, there are significantly fewer research papers detailing the customization of ML pipelines. This gap is further confirmed by a recent literature review on MLOps in data science projects [276]. In the next paragraphs, we present the few case studies retrieved for each area in the last 10 years, highlighting complementary examples that demonstrate particularly complex ML management chal- lenges. From their analysis, we discover that the integration of both DataOps and MLOps in ITS&L literature remains limited, with only a few noteworthy examples addressing these com- bined practices, including the development by Conlan et al. [277] of a real-time traffic management system in the UK, the system by Pineda-Jaramillo and Viti [246] for delay prediction in European freight rail operations, and the framework by Mishra and Palanisamy [278] for urban air vehicle logistics. This underlines an ongoing challenge and potential area for future research, discussed further in Section VII.

1) Vehicle Control Systems: In this area, we found a total of 5 case studies that integrate MLOps practices into automotive applications [279], [280], [281], [282], [283]. Ferreira and Fernandes [279] explore the challenges encountered at Bosch Car Multimedia in developing DataOps and MLOps processes for autonomous driving systems, emphasizing the need for

automation in data management, training, and monitoring steps to manage large daily data inflows effectively and updates on the AI components. In their work the authors provide concrete implementation guidelines for the integration of DataOps and MLOps tools cases for the development of an efficient end-to-end pipeline. Similarly, Hartmannsgruber et al. [280] and Steffen et al. [281] showcase the transformative potential of continuous integration and delivery coming from MLOps frameworks in autonomous driving systems. Com- plementary to these efforts, Weber et al. [282] investigate MLOps maturity levels in automotive ML projects, identifying contextual factors such as data quality and tool suitability that influence adoption. Lastly, Socha et al. [283] address robustness challenges in vehicle control systems by integrating MLOps monitoring techniques to handle out-of-distribution data effectively, emphasizing the importance of resilience in real-world applications.

2) Traffic Management: In this area, we found a total of 3 case studies, all related to traffic forecasting and planning [277], [284], [285]. Angarita-Zapata et al. [284] explore the use of automated machine learning (AutoML) [286], a promis- ing way to automate development, training, and validation of the ML pipeline, for continuous and automated mainte- nance and monitoring of ML models in traffic forecasting applications. However, the implementation lacks full MLOps integration, as it does not address the need for automatic

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 19

and scalable deployment solutions, namely the Ops part of MLOps, that can dynamically adapt to increasing workloads or changing traffic conditions. On the other hand, Conlan et al. [277] introduce Foresight, an end-to-end DataOps, and MLOps system designed for managing traffic and road networks in the West Midlands region of the United Kingdom, covering major urban areas like Birmingham and Coventry, using real-time spatiotemporal forecasting. Foresight utilizes Amazon services (e.g., Amazon S3 [171], Amazon Kinesis [172]) to set up a pipeline capable of re-training traffic models periodically, such as once per week, to continuously incorporate new traffic data. The same project has later been extended in a more cost-effective inference architecture, namely Foresight Plus [285], which seamlessly utilizes Amazon SageMake on- demand services to handle workloads that have idle periods between traffic spurts and can tolerate cold starts (e.g., to accommodate peak traffic).

3) Transportation Management Systems: In this area, we found a total of 3 case studies, related to the railway domain [48] and public transportation [287], [288]. Zeller et al. [48] engineer an abstract MLOps framework to support a trustworthy development of ML models (see also Section VI for additional details) to ensure that, when deployed on transportation systems, they adhere to principles of reliability and safety. However, it does not propose a specific case study or suggestion on the tools to integrate, but instead provides a detailed conceptual architecture and emphasizes the need for continuous delivery, verification, and safety assessment within a safe MLOps framework. Similarly, Sengul et al. [287] present a conceptual end-to-end DataOps and MLOps-based architecture to support public transportation management in cities, but without testing it in practice on realistic data. However, complementary to [48], Sengul et al. [287] present a list of tools that can be integrated to develop their architecture (e.g., Apache Airflow for data management and Kubernetes to manage ML deployment). Lastly, Guzhva et al. [288] propose an MLOps pipeline to determine the precise location of a tram within an urban environment (i.e., tram positioning) to provide accurate feedback to end-users. Their pipeline has been validated in practice through multiple tests on two tram routes, continuously collecting and integrating images captured by onboard cameras.

4) Traveler Information and Experience Systems: The only relevant case study under this domain we identified is by Urias and Rossi [289], who explore the use of MLOps in smart city applications, focusing on smart parking and smart tourism, among other urban services. While Urias and Rossi [289] offer valuable insights into the use of MLOps for smart city applications for travelers, their work remains largely conceptual. Although they provide a thorough framework, there is still an opportunity for practical exploration and real- world implementation of these design principles on actual data to demonstrate the framework’s effectiveness.

5) Freight and Supply Chain Logistics: In this area, we identified 7 case studies spanning railway freight manage- ment [246], logistics in manufacturing production [290], [291], maritime logistics [292], inventory management and sales forecasting [293], e-commerce recommendation and

delivery [294], and large-scale delivery services [295]. Pineda- Jaramillo and Viti [246] focus on MLOps for freight rail operations by developing a tool for real-time delay pre- diction in freight trains. Their system automates the entire DataOps and MLOps pipeline using tools such as Apache Airflow and Kubeflow to manage railway trips across mul- tiple European countries and reduce disruptions in freight rail logistics. Although the study focuses on Luxembourg’s freight rail network, the proposed methodology can be adapted to other freight rail systems, highlighting the scalability of MLOps in real-world logistics applications. Faubel and Schmid [290] investigate MLOps integration for logistics manage- ment in three industrial production companies, including automotive manufacturers, showing that both DataOps and MLOps are essential for automating operational decision- making and improving efficiency in logistics workflows. The tools adopted by these companies include Kubernetes, Apache Spark, MLflow, and AWS and Azure solutions. Cha et al. [291] further emphasize the role of Kubeflow in supporting intelligent logistics for smart factories, improving manufac- turing productivity through the automation of ML pipelines. Morariu et al. [292] examine the maritime industry and stress the need for real-time decision-making systems in complex maritime environments. However, their study mainly addresses real-time data-processing challenges and does not provide a detailed practical implementation of MLOps for scalable and automated model lifecycle management.

On the supply-chain and delivery side, Jana [294] propose an AWS-based MLOps pipeline for e-commerce companies aiming to improve customer experience and optimize oper- ations, such as product recommendation, in real time. This work highlights the role of MLOps in enabling continuous model training and deployment on real data. Khan and Ahmed [293] present a survey on the optimization of retail oper- ations in China through MLOps for inventory management and sales forecasting. The study focuses on high-volume data and the challenges arising in this context, but does not describe the technical implementation of the MLOps pipeline in detail. Finally, Uber’s Michelangelo platform [295] exemplifies large-scale MLOps for delivery services such as Uber Eats, automating and scaling model deployment across Uber’s applications and substantially reducing the operational complexity of managing ML workflows in large-scale delivery services.

6) Emergency Management: In this area, we identified only 2 case studies, presented by Peramo et al. [296] and the ML4Floods project [297], both related to flood detec- tion and mapping for damage prevention, public safety, and emergency response during flood events. Specifically, Per- amo et al. [296] integrates Weights and Biases to support the development and monitoring of an ML-based flood detec- tion system using traffic images. ML4Floods [297], in turn, provides an end-to-end MLOps-oriented pipeline for flood- extent segmentation from optical satellite imagery, covering data ingestion and preprocessing, model training, evaluation, inference, deployment, post-processing, and flood-map visu- alization. The system operationalizes flood mapping under conditions commonly encountered during disasters, such as

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

20 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

cloud coverage, limited post-event observations, and the need to rapidly generate flood products for damage assessment. For example, the associated flood-mapping framework is designed to produce flood-extent maps without human intervention, can operate on both Sentinel-2 and Landsat 8/9 imagery, and was demonstrated on major flood events in Pakistan and Australia, where the generated flood maps were intersected with population, building, and cropland layers to support impact assessment [297].

Finally, although it does not implement a specific case study, the work of Remil et al. [298] is worth mentioning, as it presents a comprehensive review of ML solutions for incident management, although without supporting explicit MLOps solutions. Their work provides technical guidelines for implementing MLOps architectures to automate the detection and resolution of incidents in critical systems. We also identi- fied recent ML-based flood detection systems, such as [299], which combine YOLO-based detection [300] with real-time alerting mechanisms. However, these works mainly focus on model development and application-level response rather than explicit MLOps lifecycle automation.

7) Remarks: Overall, the reviewed case studies indicate that MLOps adoption in ITS&L remains considerably less mature than DataOps adoption, with only a limited number of studies addressing the full ML lifecycle beyond model training and evaluation. Across domains, these works reveal a common need for more systematic support for automated retraining, while each application area introduces additional operational requirements.

Vehicle Control Systems and Transportation Management Systems place particular emphasis on safety-aware continuous verification and monitoring, since deployed models must oper- ate under changing road, traffic, and environmental conditions. Traffic Management applications primarily highlight the value of periodic retraining and elastic inference infrastructure, which allow forecasting models to adapt to evolving traffic patterns and variable workloads.

Freight and Supply Chain Logistics provide some of the most complete examples of MLOps adoption, where tools such as Airflow and Kubeflow, together with cloud-based services, are used to automate training, deployment, and monitoring across geographically distributed logistics networks.

Traveler Information and Experience Systems and Emer- gency Management remain less explored from an MLOps perspective, with existing works often limited to concep- tual frameworks or application-level ML systems rather than mature lifecycle automation.

Finally, current MLOps case studies in ITS&L highlight a clear prioritization of isolated model development and predic- tive performance, with limited evidence on how ML-based systems are retrained, versioned, deployed, monitored, and updated after deployment.

G. The Recent LLMOps Development

Large Language Models (LLMs) have emerged in 2019 [301] as powerful Deep ML models, based on the transformer architecture, for language-related tasks (e.g., text classification

and generation), significantly outperforming their predeces- sors. LLMs have then evolved beyond text processing alone to be able to handle diverse data modalities, including images [302], graphs [303], and other structured and unstructured data types [304]. Today, LLMs are instrumental in extracting meaningful insights from free-form text [305] and multimodal data [304], utilizing pre-trained LLM embeddings [306]. They facilitate autonomous problem-solving through ChatBots [307] and assist operators in providing domain-specific answers with Retrieval Augmented Generation [308]. Consequently, organizations are increasingly incorporating LLMs into their services [309]. For example, the Melbourne Airport has devel- oped a ChatBot based on LLMs that provides information regarding real-time flight updates, shop searches, and parking information [309], [310]. However, while LLMs may appear as specific ML models within MLOps, they present distinct challenges, paving the way for Large Language Model Oper- ations (LLMOps) [311], [312].

LLMOps is treated as a specialized extension of MLOps for LLM-based systems, where additional operational con- cerns arise from prompt engineering, retrieval augmentation, fine-tuning or adaptation of pre-trained models, guardrails, hallucination monitoring, token and cost management, and the integration of human feedback.

For instance, in LLM-based text generation, ensuring that the output aligns with specific requirements, such as avoiding harm [313], necessitates the implementation of mechanisms to enforce generation safety and security. For this purpose, the control and monitoring mechanisms, such as guardrail [314], need to be integrated and deployed in the system. Furthermore, in most LLMs-based applications, it is pretty uncommon to train an LLM from scratch. This process is mainly accessible to the big techs (e.g., OpenAI, Google, and Mistral) who have both the huge data sources and computational power needed for this purpose [315], [316]. What is commonly employed, instead, are the pre-trained LLMs [317], [318], often referred to as foundation models [319], [320].

Note indeed that the term Foundation Model Operations (FMOps) [321], [322] can be understood as a broader gen- eralization of LLMOps to foundation models beyond text, including multimodal and generative models such as text- to-image, text-to-video, and speech-to-text systems [312]. However, since FMOps and LLMOps share many operational challenges, and since LLMOps is the most consolidated term in the literature, in this survey we use LLMOps as the main reference while noting that several considerations also extend to broader foundation-model deployments.

In the following, we describe how to align the MLOps lifecycle (i.e., from the data collection until the feedback steps) of Figure 3 toward LLMOps.

1) Data Collection: Due to the use of pre-trained foun- dation models, LLMOps requires data diversity and repre- sentatives more than volume [323]. Traditional MLOps often generate ML models from scratch, and consequently, large volumes of labeled data are required [324]. LLMOps, focusing on adjusting the foundation model to the specific task through model adaptation techniques, require small vertical and high- quality datasets [323], [325]. For this reason, these data must

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 21

be carefully curated to ensure that they are representative of the problem at hand.

Concretely, for ITS&L services, this shifts the empha- sis from collecting more data to selecting reliable domain evidence, such as updated traffic conditions, operational con- straints, regulations, and service information, that can safely support LLM-based decisions and recommendations.

2) Feature Engineering: Similarly to MLOps, when dealing with Deep ML, in LLMOps the feature engineering phase is performed by the trainable or pretrained foundational models (e.g., BERT [306]), which generate effective representations directly from raw data avoiding or strongly reducing the need for human intervention [326]. This can be done at the cost of the high computational requirement using these large models [301]. While foundation models often demonstrate considerable performance on their own, some applications require greater accuracy (e.g., legal [327] or medical [328] decisions) or better alignment with domain-specific metrics, such as ethical or cultural [327] considerations. In these cases, two feature engineering approaches can be integrated to further improve their effectiveness. The first one, prompt engineering [329], modifies the input (prompt) to enable the adaptation of models to specific tasks, further enhancing their performance and versatility across diverse applications leveraging the fact that a better input usually generates a better response in LLMs [329], [330]. The second one involves enriching the original input (prompt) by integrating external knowledge sources [331]. Techniques such as Retrieval Augmented Generation (RAG) pipelines [332] or Knowledge Graphs [333] enrich the original input with additional context-specific information aug- menting the model’s understanding and reasoning capabilities [331].

In LLMOps, feature engineering is therefore not limited to transforming raw data into model features; it also includes enriching the context provided to the model at inference time. In RAG-based systems, for example, relevant documents or data sources are retrieved when a user query is received and incorporated into the prompt to ground the model response.

3) Model Training: The training process of LLMs is chal- lenging and might be conceptualized into two (possibly three) modules. The first one encompasses one or multiple large Deep ML models designed to embed the raw data (e.g., text, images, graphs, and audio) into a representation vector (or embedding) [260]. Subsequently, this representation is utilized by a second module, called the subtask module, for classification [334], clustering [335], or generation [310] purposes. When the subtask generation module is utilized, an additional control module is introduced for refining [318] or safeguarding [336] the generated response.

Analogously to MLOps, model training in LLMOps involves both model engineering and model fitting. However, unlike classical MLOps, LLMOps in downstream deployments rarely focuses on full pre-training [311], [312]. Instead, model engineering typically concerns the selection of a suitable pre- trained foundation model, the definition of instruction-tuning or parameter-efficient fine-tuning strategies, and the configu- ration of safety alignment mechanisms and guardrails. When a subtask module is used, model engineering also includes

selecting the corresponding model type and hyperparameters. Finally, the model fitting phase focuses on adapting the selected components using the available task-specific data.

4) Model Validation: The model validation of LLMOps is conceptually very similar to the one of MLOps with one exception: a strong focus on the validation metrics for the generative tasks. When LLMs are used for classification or clustering subtasks, they can be viewed as an incremental evolution of classical ML techniques, similar to enhanced pre-trained models. However, when employed for generative purposes, LLMs have the potential to produce harmful or unethical content, whether intentionally or unintentionally.

For this reason, relying solely on metrics commonly used in classical ML is insufficient. To comprehensively evaluate the performance of the final LLM-based tool, a combination of quantitative and qualitative metrics is necessary, striking a balance between automation and human intervention. Quanti- tative metrics (e.g., ROUGE [337], BERTScore [338], BLEU [339], and many others [32]) are task-specific benchmarks for delivering automated quantitative insights into LLM perfor- mance. Human evaluators provide further insights into the quality, coherence, and relevance of generated content by identifying subtleties that automated metrics may overlook, such as appropriateness in context, grammatical correctness, and adherence to stylistic conventions.

5) Release, Deploy, Monitoring, and Feedback: Deploy- ment in LLMOps requires specialized considerations due to the nature of LLMs, which demand significantly more com- putational resources than previous ML models integrated in MLOps pipelines. While both MLOps and LLMOps involve managing computational resources, LLMOps require them throughout the training and deployment phases. In MLOps, computational resources are primarily focused on training tasks, where large-scale datasets and complex neural network architectures demand substantial computing power for model training. However, once the model is trained, deployment often involves less intensive computational requirements, shifting the focus to serving predictions efficiently and reliably. Con- versely, LLMOps faces distinct challenges related to training and deployment phases. LLMs involve performing more cal- culations and exhibit large memory footprints during training, fine-tuning, and inference, necessitating specialized infrastruc- ture and optimization techniques to handle their computational demands effectively. Specifically, once trained, LLM inference requires processing input sequences through multiple neural network layers, affecting inference times.

Regarding model validation, monitoring in LLMOps is a multifaceted process that tracks various quantitative metrics and human evaluations. During the monitoring phase, the focus is on detecting potential biases, ethical issues, or unintended consequences in the outputs generated by LLMs to ensure that LLMs operate ethically and effectively in diverse contexts [340], [341]. Finally, the feedback stage plays a more relevant role in refining and optimizing the performance of LLMs. The data collected from this stage can be further utilized to refine LLMs through interactions with human feedback [342] (e.g., explicit labels, ratings, or human corrections in response

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

22 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

to the model’s outputs), leading to continuous improvement and adaptation to real-world interactions.

6) Tools: Beyond the general-purpose MLOps platforms discussed in Sections V-C and V-D, the LLMOps landscape includes a growing set of dedicated tools designed to address the specific operational requirements of foundation-model deployments. Hugging Face [343], with more than 160 thou- sand GitHub stars as of June 2026, has become a widely adopted hub of the open-source LLM ecosystem, offering model and dataset repositories, the Transformers library for model loading and fine-tuning, and parameter-efficient adap- tation methods such as PEFT. These resources support several stages of the LLMOps lifecycle, including data collection, feature engineering, model adaptation, and deployment. For inference serving, vLLM [344], with more than 60 thousand GitHub stars as of June 2026, provides a high-throughput serv- ing engine with continuous batching, improving GPU memory utilization and inference efficiency. At the application layer, LangChain [345], with more than 130 thousand GitHub stars as of June 2026, supports the development of LLM-powered applications and their interaction with external tools, data sources, and memory. Finally, tools such as Deepchecks [346] and benchmarks such as HarmBench [347] support continuous validation, monitoring, and safety evaluation, complementing the TAI principles of robustness and trustworthiness discussed in Section VI. Collectively, these tools reflect a rapidly expanding and globally distributed LLMOps ecosystem that complements the general-purpose MLOps platforms surveyed in this work.

H. Emerging AgentOps for Agentic AI Systems

The discussion above focused on LLM-based applications in which the model is invoked to generate, classify, retrieve, or summarize information. However, recent AI deployments are increasingly moving toward agentic systems [348], [349], namely systems in which an AI component is not limited to producing a single response, but can pursue a goal through multiple intermediate steps. In this context, an agent can be understood as an AI-based system that observes an input or environment, decides which action to take next, and may call external tools, retrieve information, store or use memory, interact with users, or coordinate with other agents to complete a task [350]. For example, a conventional LLM-based applica- tion may answer a travel question, summarize a traffic report, or retrieve information about a transport service. By contrast, an agentic system may autonomously retrieve live traffic data, compare alternative routes, call booking or fleet-management tools, coordinate with other agents, and propose operational actions.

This shift motivates an additional operational layer, known as Agent Operations (AgentOps) [351], [352], [353]. Com- pared with LLMOps, which focuses on managing LLM-based applications, AgentOps focuses on the runtime behavior of agentic workflows. While MLOps addresses the lifecycle of ML models and LLMOps specializes this lifecycle for LLM-based applications, AgentOps is more system-oriented. It monitors and governs agentic workflows composed of models,

prompts, tools, memory, external data sources, and envi- ronment interactions. Therefore, AgentOps does not replace MLOps or LLMOps; rather, it extends them in terms of functionalities, responsibilities, and risks.

From a functional perspective, AgentOps shifts the monitor- ing target from the model output alone to the full execution trajectory of the agent, including prompts, intermediate reason- ing steps, retrieved information, tool calls, memory updates, interactions with external systems, and final actions. From a responsibility perspective, this means that developers and operators must govern not only the underlying model, but also the surrounding system that decides when to call tools, which information to use, and which actions to execute. This broader operational scope also increases the potential impact of failures, since an incorrect action may propagate across tools, services, users, or other agents. As a result, agentic systems introduce additional risks, including incorrect tool calls, unstable planning, memory failures, prompt injection, cascading errors across agents, and emergent behavior in multi-agent coordination [354].

For this reason, AgentOps introduces operational mecha- nisms beyond LLMOps to observe, control, diagnose, and recover the behavior of agentic systems during execution. These mechanisms aim to make agent workflows traceable, verify that actions remain consistent with operational and safety policies, identify the origin of failures, and support cor- rective interventions when the system behaves unexpectedly. They remain aligned with the broader MLOps lifecycle, but operate at a higher level of abstraction, where the monitored object is not only the underlying model but the complete agentic system built around it.

We argue that AgentOps will be highly relevant for future ITS&L deployments. At the same time, AgentOps is still an emerging and rapidly evolving area. Unlike DataOps, MLOps, and, to a lesser extent, LLMOps, it has not yet reached the same level of methodological maturity, standardization, or systematic adoption in the scientific literature. Accordingly, this survey does not aim to provide a complete treatment of AgentOps. Rather, we introduce it as a recent development likely to influence future trustworthy ITS&L systems, and refer the reader to dedicated surveys and technical reports for a broader discussion of agentic AI operations [351], [352], [353].

VI. TAI

In various decision-making domains, including ITS&L [59], [62], [63], healthcare [355], and policing, the widespread integration of AI has raised significant technical and ethical challenges and concerns [356]. This is due to the increas- ing complexity of AI-based systems, which, while achieving higher performance in specific metrics (e.g., accuracy and computational requirements), pose many technical challenges (i.e., the so-called technical debt [250]) and raise ethical concerns from society at large [356].

In simpler words, when designing technical instruments, optimizing single metrics may lead to problems with other hidden or implicit metrics. For example, focusing solely on predictive accuracy (i.e., correlation) without considering

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 23

model adherence to physical principles (i.e., causality) risks shortcut learning and spurious correlations. This lack of con- sideration for underlying principles can result in models that perform well on specific tasks but fail in real-world scenarios where causal relationships are crucial. Using historical data for predictive accuracy in social phenomena (e.g., hiring) can perpetuate societal biases (e.g., on gender and race). Overem- phasis on removing biases might result in hallucinations or misrepresentations. Prioritizing predictive accuracy over robustness can expose models to malicious users, potentially leading to undeserved benefits or social issues.

These concerns were highlighted by the record number of AI-related incidents reported in 2023, marking a significant reckoning for the field4 and driven partly by the introduction of the AI-Act in Europe [35], [36], [37]. Gartner estimates that by 2025, approximately 30% of all AI-based products will require a trusted AI framework [357]. Recent studies also indicate that 86% of users are likely to trust and remain loyal to companies supporting ethical AI principles [357], [358]. For this reason, researchers have approached this problem from different per- spectives, identifying two main research directions [359]. The first one studies how to design systems that can follow or learn technical principles and moral notions and autonomously behave accordingly. Works like the Comparative Moral Turing Test [360] or Ethical Turing Test [361] assess the morality of automated systems’ choices. Due to the strong connection between philosophical, ethical, and technical problems, this research branch remains relatively unexplored. The second direction focuses on how humans should design and develop AI-based systems to minimize possible harm from poor design, inappropriate application, or misuse [36], [357]. Our work falls within this category. In this context, the best solution that researchers have found is designing an increasing number of metrics that properly characterize the behavior of AI-based systems, considering all aspects needed for the particular application [362]. This approach aligns well with the DataOps and MLOps (including LLMOps) frameworks discussed in the previous sections. Both DataOps and MLOps include steps like “Analysis & Visualization” and “Monitor” for DataOps (see Figure 2) and “Model Validation” and “Monitor” for MLOps (see Figure 3), where these metrics can be integrated during both design (“Analysis & Visualization” for DataOps and “Model Validation” for MLOps) and operations (“Monitor” for both DataOps and MLOps). In the end-to-end view presented in Figure 1, this section corresponds to the TAI layer, which cuts across both the DataOps and MLOps lifecycles. Rather than serving as a final assessment step, TAI provides the metrics and mitigation strategies needed to evaluate data quality, model behavior, and deployed services in terms of sustainability, robustness, explainability, fairness, and privacy. In the following, we address the importance of developing new technical and ethical metrics to monitor AI-based systems and, in case of misbehavior, techniques to mitigate problems towards more TAI. These issues are especially relevant in the ITS&L domain, where the combination of technical, economic, legal, and human-related factors and requirements

4https://incidentdatabase.ai

makes trustworthy AI crucial for the widespread adoption of AI.

In the following sections, we focus on the main metrics that characterize the development of more TAI-based sys- tems for ITS&L: sustainability (Section VI-A), robustness (Section VI-B), explainability (Section VI-C), fairness (Sec- tion VI-D), and privacy (Section VI-E). Note that some metrics can be both technical and ethical. For instance, sustainability is a technical metric because it concerns the efficiency of ML models, but it is also an ethical requirement. Similarly, robustness prevents AI models from being manipulated, serv- ing as both a technical and ethical requirement. Note also that metrics can be combined, such as fairness and robust- ness as in [363], generating secondary yet important metrics. Finally, the metrics discussed in this section are those that, from the authors’ perspective, are among the most important, widespread, and relevant for ITS&L, although other important metrics can also be considered, such as the ability of a model to be formally verified [364]. For additional metrics, please refer to the following surveys [51], [357], [365]. The section then concludes with case studies in ITS&L (Section VI-F).

A. Sustainability

The two main phases of ML training and inference present significant challenges regarding their economic costs and associated environmental impacts [366], [367].

Training, in particular, requires collecting large amounts of data and optimizing (or fine-tuning) a substantial number of parameters, which increases the computational demands and energy consumption of both the training and inference phases [367]. To mitigate these challenges, one approach involves optimizing data usage by leveraging techniques such as data selection [368], [369], or transfer learning from pre- trained models [370], [371]. Another strategy is to modify the model’s architecture to reduce the number of parameters [372], [373] or their precision [374], [375]. Additionally, selecting appropriate training optimizers can speed up the training process, and using custom hardware (e.g., GPUs, TPUs) can reduce training time or enhance efficiency using the same amount of energy [376], [377]. However, high-end hardware often remains necessary, which may be cost-prohibitive for smaller enterprises or beyond the technical skills of larger ones in ITS&L [378]. Consequently, organizations typically rely on Machine Learning as a Service (MLaaS) providers to outsource the training process, thereby reducing complexity and costs [378], [379]. Nonetheless, adopting these strategies necessitates trust in third-party entities responsible for training the models [26].

The inference phase poses fewer computational challenges than the training phase, as it is generally computationally efficient. Even complex models (e.g., LLMs) may take only a few seconds to make a prediction. However, in the ITS&L domain, the sheer volume of required predictions (e.g., for all delivery vehicles) and limitations in available hardware (e.g., computational requirements or battery life) can pose significant challenges. As a result, even small savings in the computational cost of individual predictions can substantially impact sustainability, prediction time, and costs [3]. In other

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

24 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

cases, the use of hardware accelerators may be impractical [376], [377].

1) Metrics and Tools: Given the growing call for the development and incorporation of new evaluative metrics that capture a comprehensive range of factors to ensure sustainable AI, various metrics have been developed to monitor high-level factors (e.g., costs, carbon emissions, power consumption) influencing the environmental footprint of training or operating AI models. Costs indicate the financial resources required for training and operating AI models, including hardware, energy consumption, and personnel expenses. The carbon emissions metric measures the number of greenhouse gases emitted during the training and operation of AI models. To measure and manage these metrics effectively, several tools have been developed. CodeCarbon [380] and CarbonTracker [381] are popular tools for measuring carbon emissions and footprint. Eco2AI [382] accumulates statistics about power consumption and CO2 emissions during code execution, while Zeus [383] focuses on deep learning energy measurement and optimiza- tion. For monitoring power consumption, Scaphandre [384] and PowerJoular [385] are widely used; they provide metrics on electrical power consumption across multiple platforms and processes. Complementary to these high-level metrics, low- level factors (e.g., number of epochs to reach convergence, representation precision, number of weights to be trained) can be tuned to further decrease the computational complexity and power consumption of AI. All these metrics and tools can be integrated together within MLOps pipelines into the “Monitoring” phase to balance the economic and environmen- tal footprint of training or operating AI models with other performance metrics, such as accuracy.

B. Robustness

As ML-based systems are increasingly deployed in critical applications (e.g., autonomous driving, traffic management, predictive maintenance), their robustness and resilience against unforeseen errors, misuse, and malicious users become of paramount importance. In autonomous driving, for example, vehicles need to be extremely reliable in the face of both intentional and unintentional changes in the surrounding envi- ronment. However, ML models have demonstrated inherent vulnerabilities to carefully crafted attacks, including data poi- soning [26], [365], adversarial examples [49], model stealing [386], membership inference [387], and energy-latency attacks [388]. These attacks can influence both training (e.g., poison- ing) and inference (e.g., adversarial examples) phases of an ML pipeline. Moreover, attacks can target different aspects of the model’s functionality and security, including availability [389], making the ML models unavailable for use at critical times or making them inaccessible to legitimate users, integrity [389], leading to incorrect or misleading predictions, and privacy [389], resulting in data breaches or leakage of confi- dential data (see [26], [49] for details). Therefore, exhaustive data monitoring, cleaning, and anomaly removal, together with preventive security testing of AI systems, become fundamental for safeguarding the business and ensuring compliance with relevant regulatory standards and guidelines. Implementing these measures helps ensure that AI models maintain their

reliability, accuracy, and security throughout their lifecycle. However, the complexity of testing and measuring the reliabil- ity of ML-based systems increases with the complexity of the model itself and its intrinsic explainability (see Section VI-C). Explainable models are, by definition, much easier to test, while deep ML models are significantly harder to evaluate. Moreover, there is often an asymmetry between the ease of testing and attacking; in some cases, testing can be hard while attacking is simple (and vice versa), highlighting the inherent advantage attackers may have over defenders [390].

1) Metrics and Tools: Multiple robust testing procedures have been developed to proactively assess the prerequisites for robustness and compliance before deploying ML systems in critical applications. To check the model’s robustness, widely adopted measures include the Attack Success Rate (ASR), security (or robustness) evaluation curves, and accuracy under distribution shift. The ASR quantifies the percentage of adver- sarial examples or poisoning attacks that successfully deceive the model into making incorrect predictions. Security (or robustness) evaluation curves plot the model’s performance (e.g., accuracy) against different attack strengths or pertur- bation levels, offering insights into its vulnerability across various scenarios. The security evaluation curves are also used to compute the optimality score [391], which measures the effectiveness of the attack used to evaluate the adversarial robustness of the model under inspection. The higher the optimality score, the better the attack algorithm is at assessing the model’s robustness. Accuracy under distribution shift evaluates how well a model performs when the distribution of the input data shifts from the training distribution [392]. In other words, it measures the model’s ability to generalize to new, unseen data distributions, which is crucial for real- world deployment where data may vary over time or across different environments. Lastly, the energy consumption ratio [388], [393] is the measure monitored when mounting attacks targeting the energy-latency performance of a model. These metrics together evaluate whether ML systems exhibit unpre- dictable behavior, deviating from established safety protocols and potentially leading to accidents. Among the tools and resources integrating them, we find FoolBox [394], SecML [395], CleverHans [396], and Art [397], which serve as benchmarks for assessing the robustness of ML models in the presence of malicious users. The algorithm proposed in [388] and [393] can be utilized to assess the resilience of AI systems against recent attacks designed to increase power consumption. Additionally, the SHIFT dataset [392] has been introduced to investigate performance degradation in perception systems and assess the generality of ML models. However, it is worth mentioning that most of these tools and datasets target the image vision domain, leaving the need for standardized benchmarks for other input domains (e.g., graphs and tabular data) open. Furthermore, to date, robustness is mostly evaluated empirically, with gradient-based [398] or gradient-free attacks [399]. Nevertheless, empirical evaluations have been proven to fail in some cases due to the inherent complexity of the attack optimization process, giving a false sense of security [400]. To address these common challenges, recent work has proposed standardized evaluation

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 25

frameworks [391], [401], [402], practical recommendation guidelines [391], [403], [404], and debugging tools [405]. These efforts aim to enhance the reliability and robustness of ML systems across various domains, ensuring their safe and effective deployment in critical applications. Alternatively, formal verification tools have been developed to theoreti- cally guarantee the robustness of ML models [406], [407], [408], [409]. However, the robustness verification problem is NP-complete [406], [410], and these tools often suffer from scalability issues [411], making them mainly applicable to smaller models.

C. Explainability

Explainable AI (XAI) involves elucidating or revealing the decision-making mechanisms of ML models, allowing users to understand how inputs and outputs have been related, thereby understanding why AI models make specific decisions [412]. In other words, explainability aims at pro- viding model/outcome explanations for black-box ML models (e.g., deep networks), encouraging trust in their users [362], [413]. Trust is essential in ITS&L as the decisions may have an impact on individuals or customers. For example, explainable route planning or automatic vehicle location [414] may improve dispatcher comfort and service level for users. However, defining explainability within a single, universally accepted notion is challenging, as many proposed definitions offer partial clarity. Many researchers argue that the ongoing research on XAI lacks a formal framework [415], [416], [417]. A consequence of this lack is the high fragmentation of the field [412] even if recently a lot of efforts have been spent to rationalize and systematize the field [412], [418], [419], [420].

In this work, we follow the taxonomy developed in [421], which distinguishes XAI between models that are interpretable by design (transparent) and those explained by external XAI techniques, called post-hoc methods, which focus on explain- ing the entire model (global explanations) or single (or a group of) predictions (local explanations). Transparent models are inherently explainable algorithms, allowing users to com- prehend the model’s processes to produce input data outputs [422], [423], [424]. The level of transparency can vary signif- icantly. Fully transparent models, such as linear or rule-based models, provide clear insights into their decision-making pro- cesses, making them easy for both humans and mathematical tools to inspect and query. On the other hand, less transparent models, like Random Forests [425] or Kolmogorov-Arnold Networks [426], are more complex and less interpretable. When a model is not sufficiently transparent for the application or its inspection is too complex for humans, post-hoc methods need to be applied. These methods help to approximate and explain the non-transparent models, balancing the need for accuracy and interoperability.

Post-hoc techniques are further divided into global and local explanations. Global explanations provide a global overview of the model’s decision-making process [51]. They aim to explain the general patterns, trends, and feature importance that the model has learned. For example, they may approximate the entire model to offer a general understanding of what has been learned from the data and how the model performs predictions

[427], or use perturbation techniques (e.g., permutation test) to evaluate the feature importance for the model [428]. Local explanations provide insights into why a model made a specific prediction for a single (or a group of) instance(s). An example is LIME [429], which offers insights into how a black- box model arrived at a particular prediction by generating a simpler, more interpretable model near that prediction. Lastly, techniques such as SHAP [430] can support both local and global explanations, offering flexible tools to interpret model decisions across different contexts.

1) Metrics and Tools: In XAI, various metrics can be utilized to assess the effectiveness and reliability of model explanations [431], [432]. Among them, we find quantitative metrics to evaluate the quality, efficacy, and reliability of explanations in XAI. Accuracy is a fundamental metric that compares explanations against a reference to ascertain their alignment with the model’s behavior. Fidelity (or Faithfulness) is associated with how well the explanation approximates the prediction of the black box model. It is essential in post-hoc methods using surrogate models like LIME [429] to assess how well the surrogate model approximates the black-box model’s behavior locally. Fidelity complements Correctness, which evaluates the alignment between the surro- gate model’s predictions and the original model’s predictions for the same input data. Consistency evaluates the stability of explanations across instances or datasets, where statistical measures like variance or standard deviation are utilized to ensure reliability under varying conditions. It evaluates whether functionally equivalent models produce similar expla- nations. Stability (or Robustness) assesses the resilience of explanations to input variations, scrutinizing changes induced by small (natural or adversarial) perturbations to the input data. Certainty (or Uncertainty) quantifies the confidence level conveyed by explanations, employing metrics like entropy or confidence intervals to gauge reliability. Lastly, Importance elucidates the significance of features in explanations, using techniques such as feature importance scores or permutation importance to identify key elements driving model decisions. Complementary, qualitative metrics use language, narratives, and contextual details to explain concepts, phenomena, or processes, providing insight into a model’s behavior. The goal is to make the model’s decisions understandable to humans by describing the rationale behind predictions in a comprehensible way [432].

Dedicated tools and frameworks have emerged to evalu- ate and improve model explainability and compliance with transparency requirements. For instance, ktrain simplifies the building and training of explainable models through an inter- face to Keras, supporting both text and image classification tasks. PyTorch leverages Captum for model interpretability, offering techniques such as smoothGrad and integrated gra- dients to understand the model behavior. AI Explainability 360 is an open-source library that provides explainable AI techniques for tabular, text-based, image, and time series data. Arize AI provides an ML observability framework for model monitoring and assessment, aiding in diagnosing model out- puts and detecting validation checks. InterpretML, developed by Microsoft, facilitates the training of interpretable models

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

26 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

and explaining black-box systems, offering a suite of methods for model explanation and comparison.

D. Fairness

Algorithmic Fairness is an emerging field that aims to ensure that the results of models are not unfairly influenced by sensitive attributes, thus avoiding perpetuating historical discrimination (e.g., based on ethnicity or gender) [27], [433]. Discrimination occurs when models treat differently similar individuals or groups of individuals just based on a sensitive, and not influent for the task, feature [362]. Discrimination can be explicit (i.e., the model explicitly leverages the sensitive attribute to make decisions) or implicit (i.e., even if the sensitive attribute is unknown, the model possibly leverages a correlation to that in the data) [51]. An example of an unfair model might be one that unintentionally prioritizes hiring people of a specific gender for a specific job [434]. Simi- larly, regarding ITS&L, an algorithm might favor delivery in specific, more lucrative areas or clients [90]. Fairness notions in AI can be broadly divided into two families: individual and group fairness definition [435]. Individual notions of fairness ensure that similar individuals receive similar results, such as ensuring that two people with the same skills are equally likely to be hired, regardless of race or gender [27]. Group fairness aims to treat different demographic groups statistically equally, for example, by ensuring that the acceptance rates of financial loans are similar for white and black-skinned individuals [27].

Algorithmic discrimination is mainly due to two causes: data and model biases. In fact, ML models are trained on historical data that are affected by societal biases (e.g., jobs usually performed by a certain gender or ethnicity). Moreover, the concept of fairness varies over time (what is considered fair today could not be fair in a few years) and space (what is considered fair in a country could not be fair in another). As a result, data may be affected by historical, representation, mea- surement, evaluation, temporal, etc. biases [433]. Moreover, AI models are currently mostly based on ML and then statistical machines that tend to exacerbate biases, disregard minorities, and find spurious correlation [436]. Then, the combination of data and model biases can lead learning algorithms to either exaggerate existing biases or introduce new ones, resulting in discrimination [437].

Three main approaches exist to address fairness in AI: pre-processing, in-processing, and post-processing techniques. Pre-processing techniques [27] involve cleaning the data to remove biases before training the model. This approach is helpful in making out-of-the-box ML algorithms fairer without altering the algorithms themselves. In-processing techniques [27] directly incorporate fairness constraints into the learning phase, ensuring fairness is embedded within the model’s structure. Post-processing techniques [27] adjust the outputs of already trained models to ensure fairness. This approach is advantageous when retraining or fine-tuning complex models is impractical. Each approach has advantages and applications, and often, a combination of these methods is employed to ensure comprehensive fairness in AI systems [27].

1) Metrics and Tools: In a recent work [438], more than 20 different notions and measurements of fairness have been

reviewed, categorizing them into group and individual fairness notions. Regarding the notion of group fairness, Demographic Parity, Equal Opportunity, and Equal Odds are surely the most used ones [439]. Demographic Parity checks that the positive outcome rate should be similar across different groups. Equality of Opportunity checks that the subset of individuals in each subgroup who should receive a positive outcome actually have an equal chance of being predicted to receive that outcome. Equal Odds measure the prediction quality (e.g., accuracy) difference between the different subgroups. Individual fairness, in contrast, delves into the concept of treating similar individuals similarly, advocating for decisions that are independent of sensitive attributes. In this direction, metrics like Fairness Through Unawareness [439], Fairness Through Awareness [440], Counterfactual Fairness [441], and Path-specific Counterfactual Fairness [442] are the most known ones and search to measure why and how the decision would have changed by changing the sensitive attribute of the individual.

Because of the importance of the subject, various develop- ment environments and libraries for algorithmic fairness have been developed. These tools include techniques for training, monitoring, and adjusting the fairness of such models. Among them, Fairlearn [443], an open-source initiative, focuses on reducing biases in ML models to promote fairness and trans- parency. Developed by Microsoft Research, Fairlearn provides a suite of algorithms and visualization tools designed to help users understand and mitigate biases in their AI models. The toolkit includes techniques for fairness assessment, mitigation, and visualization, enabling data scientists and developers to analyze and address fairness issues in their models. AIF360 [444], another valuable tool, offers a Python toolkit dedicated to detecting and mitigating bias in AI models. Developed by the AI Fairness 360 team at IBM Research, AIF360 provides a comprehensive set of algorithms and metrics for assessing and mitigating bias across various stages of the machine learning pipeline. Lastly, Fairness Indicators [445] is a tool designed to help TensorFlow developers assess model performance across different data slices, identify biases, and understand the impact of model decisions on various demographic groups.

E. Privacy

Privacy focuses on securing and protecting the data of individuals or groups from being disclosed or violated [51]. Privacy considerations within AI encompass safeguarding both training and test data (e.g., patient data in healthcare) [28], [446] and the proprietary information inherent in the models themselves (e.g., model architecture and weights) [28], [446]. There is an inherent tension between providing valuable ser- vices and exposing sensitive information; for example, location data or travel patterns can improve travel time or comfort but may also lead to potential breaches in user privacy [447]. Such breaches violate individual privacy rights and expose users to potential exploitation by malicious entities seeking to misuse or manipulate this data for nefarious purposes. On the other hand, protecting the privacy of the model becomes essential for companies striving to maintain a competitive edge in the market. Malicious users can stage multiple attacks to

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 27

leak information about both data and models. The major ones include: (i) reconstruction [448], which aims to reconstruct raw data from pre-processed ones (e.g., after the feature engineering phase or anonymization); (ii) model inversion [449], which reconstructs the inputs of ML models that have generated a particular output; (iii) membership inference [387], used to determine if a data record was used to train an ML model; and (iv) model discovery [386], which reconstructs the ML model architecture and weights.

Four major families of privacy-preserving ML exists: anonymization, perturbation, cryptographic techniques (CTs), and distributed protocols [28], [446]. Each method must con- sider three aspects: its ability to preserve privacy (i.e., the protection of individual or group information), its utility (i.e., the ability to extract meaningful information), and whether it requires a trusted curator to supervise and enforce the privacy-preserving ML method. Anonymization methods [450], [451] aim to protect the privacy of data subjects by obscuring identifying information within a dataset while preserving data utility. However, these methods often compro- mise utility to ensure confidentiality and are not suitable for preserving model privacy [362]. Perturbation methods [452], [453] exploit noise to corrupt the data, the ML algorithm, or the learned model to obscure sensitive information while allowing meaningful analysis. Even in this case, there is also a trade-off between privacy and utility: increasing resilience against attacks or privacy breaches requires injecting more noise, which subsequently reduces utility. Furthermore, both anonymization and perturbation techniques assume the exis- tence of a trusted data curator. When such a trusted entity is unavailable, cryptographic techniques and distributed pro- tocols can be employed [362]. CT approaches [454], [455] consist of performing computations on encrypted data to protect both data and model privacy (i.e., privacy is preserved with no utility loss) at the cost of significant computational overhead. Distributed protocols [456], [457] involve multiple participants, each with private data, computing aggregated results while keeping their inputs secret. Similar to CTs, these protocols preserve privacy and utility but at the cost of high computational and information exchange overhead. To miti- gate this, distributed protocols are often relaxed to federated methods, where only aggregated information is exchanged [458]. Nevertheless, privacy can still be compromised when local parameters, such as updating gradients, are shared [459]. Thus, these protocols are often combined with perturbation and cryptographic methods to enhance privacy [362].

1) Metrics and Tools: The metrics employed in the context of privacy-preserving ML are mainly three: privacy-leakage metrics, data utility metrics, and computation metrics [28], [460]. Privacy-leakage metrics [461], [462] measure the secu- rity of data assessing how much sensitive information can be inferred from transformed data or model after apply- ing privacy-preserving methods. While some privacy-leakage metrics can be theoretically computed, they are generally evaluated empirically by executing known attacks and calcu- lating the resulting privacy leakage [387]. Data utility metrics [463], [464] quantify the loss of information or utility due to privacy-preserving techniques. These metrics are generally

assessed by comparing a model’s performance (e.g., accuracy) on the original data versus the privacy-preserved data. The aim is to determine how much the privacy-preserving trans- formation degrades the model’s effectiveness. Computation metrics [465], [466] focus on the computational overhead (e.g., pre-processing, training, and inference execution time, or memory requirements) introduced by the privacy-preserving techniques.

Several popular tools and frameworks have been devel- oped, each addressing different aspects of privacy-preserving ML techniques. ARX [467] is an open-source library sup- porting data anonymization techniques (e.g., k-anonymity [451] and l-anonomity [450]). Differential privacy tools, such as Google’s Differential Privacy and TensorFlow Privacy, implement ε-differentially private algorithms [468], [469] and provide optimizers for training models with differential pri- vacy, ensuring that individual data points remain protected during training. Homomorphic encryption [470] is another key approach, with tools like Concrete-ML [471] and Microsoft SEAL [472] simplifying fully homomorphic encryption (FHE) for secure computations on encrypted data. Federated learning [473] frameworks such as FATE (Federated AI Technology Enabler) [474], FedML [475], and Flower [476] enable dis- tributed learning processes, facilitating collaborative data use while maintaining privacy. Confidential computing solutions like BastionLab enhance secure data science collaboration using confidential computing, access controls, and differential privacy techniques. Multi-Party Computation (MPC) frame- works, including PySyft [477] and Rosetta [478], integrate secure, private deep learning into popular ML libraries like PyTorch and TensorFlow, separating private data from model training. Combined privacy solutions like OpenFL [479] and Substra [480] offer comprehensive frameworks for secure, traceable, and collaborative machine learning. These tools collectively provide robust mechanisms for implementing privacy-preserving machine learning, ensuring data and model security throughout the AI lifecycle.

F. Case Studies

This section presents real case studies in ITS&L that have implemented TAI solutions to align their applications with regulatory and ethical principles [36], [37]. At first glance, our review reveals that most papers address only one or a few aspects of trustworthiness, focusing on its relevance within specific application domains and proposing algorithmic or system solutions to meet these needs. However, no existing works take a holistic approach to TAI principles (as also noted in other research outside ITS&L [362]). Moreover, very few studies provide detailed information on the verification tools used, often relying on custom implementations. The use of standardized testing and verification tools provided by libraries-where quality control is typically stringent or supported by a large community-remains largely unexplored. Lastly, only a few articles address the relevance and integration of trustworthiness principles within MLOps pipelines. This highlights a gap not only in the literature on the integration of DataOps and MLOps (as noted in Section V and further elaborated in Section VII) but also in the scientific exploration

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

28 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

of their alignment with tools designed to develop and monitor systems under trustworthiness guidelines. In the following paragraphs, we present the case studies retrieved for each ITS&L area over the past 10 years, covering trustworthiness principles and highlighting complementary examples that par- ticularly demonstrate the need for integrating them in real applications.

1) Vehicle Control Systems: This area includes 21 case studies focusing on privacy, XAI, robustness and sustain- ability for intelligent driving assistance systems [447], [481], [482], [483], [484], [485], [486], [487], [488], [489], [490], [491], [492], [493], [494], [495], [496], [497], [498], [499], [500], [501]. Djenouri et al. [481] develop a blockchain- based framework, SISIV, to ensure data transmission privacy among vehicles and enable accurate traffic forecasting using distributed data. They validate their approach on four standard benchmark datasets for connected vehicles. Sucasas et al. [447] design an authentication scheme for vehicle commu- nication systems that preserves data privacy while requiring a trusted authority only once. Similarly, Jiang et al. [482] create secure data-sharing schemas to protect privacy in-vehicle networks, focusing on sustainability and privacy-preserving mechanisms to enhance trust and security. Bylykbashi et al. [483] optimize vehicle communication for addressing traffic congestion, achieving fast and cost-efficient data transfer to improve road safety and passenger comfort. Vemoori [484] explore XAI’s role in enhancing decision-making robustness for autonomous vehicles, testing computational intelligence frameworks. While they identify vulnerabilities to adversarial examples, they do not assess real-world performance. Extend- ing this work, further studies by Vemoori [486], [487], [488] investigate XAI applications in vehicle control systems. These studies emphasize interpretability in tasks such as obstacle detection, route planning, and system transparency, aiming to build trust and reliability in autonomous driving. On the robustness front, Berghoff et al. [489] propose a systematic method for testing the reliability of AI-powered traffic sign recognition systems in autonomous vehicles under varying ambient conditions. Tested on the on the German Traffic Sign Recognition Benchmark. Unal et al. [490] assess model robustness by generating challenging inputs that intentionally maximize prediction uncertainty. Deng et al. [491] present a systematic approach to evaluate the robustness of four autonomous driving models, testing them against five adver- sarial attack methods and four defense techniques using the Udacity dataset of real-world road images. Xu et al. [502] introduce a graph-based approach to select more resilient models in automated machine learning pipelines, focusing on improving reliability of models during their development. He et al. [492] propose a defense-aware reinforcement learning technique to enhance the safety and robustness of autonomous vehicle policies, ensuring they can withstand adversarial attacks. Lastly, there is a growing focus on improving the sustainability and efficiency of AI components in autonomous driving systems, as highlighted by several case studies [493], [494], [495], [496], [497], [498], [499], [500], [501]. These works explore multiple techniques to reduce computational demands while preserving model performance. For instance,

Ke et al. [496], Mahaur et al. [497], and Pei and Li [493] apply pruning methods to remove redundant model param- eters, improving efficiency without compromising accuracy. Liu et al. [501] further reduces model complexity using multi- modal tensor decomposition to compress the model lowering both parameter counts and computational costs. Alternatively, works such as [494], [495], [498], [499], and [500], focus on designing smaller, lightweight network architectures that main- tain strong performance through optimized structural design.

2) Traffic Management: In this area, we identified a total of 10 case studies focusing on traffic forecasting, route plan- ning, and traffic control, accentuating interpretability, privacy, robustness, and sustainability principles [38], [503], [504], [505], [506], [507], [508], [509], [510], [511]. Ou et al. [503] propose an interpretable traffic flow forecasting framework utilizing ensemble learning and tree-based algorithms. The framework is validated on traffic datasets collected from arterial roads, expressways, and freeways in China and the USA, demonstrating its flexibility across diverse scenarios. Liu et al. [38] address route planning in large-scale road networks with a privacy-preserving, graph-based solution. Their approach ensures that cloud servers cannot deduce real user trajectories, safeguarding data privacy while providing efficient route recommendations. Wollenstein-Betech et al. [505] present an interpretability tool for traffic light control systems that use deep neural networks. By analyzing his- torical state-action pairs, the tool offers actionable insights into system behavior, helping to reason about and improve the controller’s decision-making. Raja et al. [506] develop an intuitive and privacy-preserving traffic light control system for autonomous vehicles. Their system minimizes privacy risks associated with data collection, ensuring secure and efficient traffic flow management. Ying et al. [507] design Priva- cySignal, a privacy-preserving traffic signal control system for intelligent transportation. The system enables vehicles to share data with roadside units for signal optimization with- out exposing sensitive information, maintaining user privacy while enhancing traffic efficiency. On the robustness frontier, Tan et al. [508] propose a robust training framework for rein- forcement learning (RL) agents in noisy traffic environments. Evaluating their model with the SUMO traffic simulator [512], the authors demonstrate improved policy robustness under varying levels of data noise, highlighting the framework’s potential for real-world urban mobility scenarios. Lastly, we observe multiple case studies related to the sustainability and need for efficient ML models in production, especially in traffic control applications. Carvalho et al. [509] propose a lightweight design strategy to decrease the execution time of the vehicle identification model they develed and tested on Brazilian vehicle roads. Chen et al. [510] and Dai et al. [511], in a similar manner, develop lightweight models to accelerate the detection of traffic flow under the premise of high precision on real-word traffic scenes in the UA-DETRAC dataset [513].

3) Transportation Management Systems: In this area, we identified a total of 6 case studies addressing privacy- preserving frameworks, explainable predictions, and inte- grative approaches to manage public and safety-critical transportation systems [48], [93], [514], [515], [516], [517].

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 29

Zeller et al. [514] emphasize safe MLOps practices for transportation systems, focusing on aligning workflows with safety regulations to ensure secure ML deployment. While the study offers detailed safety guidelines, it does not extend to other trustworthiness principles or propose an integrated MLOps framework. Expanding on this, Zeller et al. [48] introduce a conceptual MLOps framework for trustworthy ML in transportation, addressing principles like reliability, safety, and continuous verification. However, the study lacks practical implementations or specific tools to demonstrate its applicabil- ity. The BIGSEA project [93] introduces a privacy-preserving DataOps framework for public transportation systems by leveraging k-anonymity techniques [451] to protect user data during processing. The approach integrates privacy-conscious practices into DataOps workflows, enhancing data trustworthi- ness in transportation management. Monje et al. [515] develop a deep learning-based explainable AI model for bus passenger forecasting, using extensive historical data from Madrid’s municipal transport system. Their model combines prediction accuracy with linguistic interpretability, offering actionable insights for decision-makers. Sobrie and Verschelde [516] design a support system to provide explainable predictions and personalized recommendations for railway control room operators. The final system has been successfully tested for its effectiveness in real-time operations at Infrabel’s railway control rooms in Belgium. Finally, Sariyer et al. [517] present a XAI-driven model to predict and interpret public trans- portation usage rates, incorporating data from buses, ferries, and railways. Tested on an open dataset from Izmir City, Turkey, the model identifies key features like passenger types and special calendar days, demonstrating its versatility in analyzing and forecasting multimodal transportation usage trends.

4) Traveler Information and Experience Systems: In this area, we identified a total of 4 case studies, primarily focusing on addressing user privacy and sustainability concerns when leveraging data for tourism support [518], [519], [520], [521], [522]. Indeed, the tourism industry faces complex regulatory and privacy challenges, which can limit the application of ML to customer data [523]. Shao et al. [518] utilize explainable AI to help stakeholders understand why regions like Shanghai achieve the most effective integration between cultural her- itage resources and the tourism industry, providing actionable insights to address disparities in other regions. Munawar and Piantanakulchai [519] propose a federated learning framework for autonomous taxi demand forecasting, enabling collabora- tive model training across regions while preserving passenger privacy. Using a real-world dataset from users in Bangkok, the authors demonstrate that the framework can provide a scalable and privacy-preserving solution for smart city transportation systems. Similarly, Cai et al. [520] employ federated learning to develop a personalized travel recommendation system for cultural tourism attractions in Fujian-Taiwan. Regarding sus- tainability, Zhuang et al. [521] address the resource constraints of edge devices in parking surveillance systems. They propose a quantized model deployed on edge devices to reduce com- putational complexity while maintaining accuracy, minimizing data transmission latency and central server load.

5) Freight and Supply Chain Logistics: In this area, we identified 12 case studies addressing different dimensions of trustworthiness across the supply chain, including security and privacy in fleet and freight management [524], [525], [526], [527], [528], fairness in delivery operations [114], [529], [530], and sustainability through lightweight model design [522], [531], [532], [533].

On the privacy and security frontier, Akram et al. [524] evaluate security, privacy, and safety issues for both dynamic and static fleets of logistics drones, highlighting the challenges of ensuring robust protections in drone-based operations, a rapidly expanding area of logistics management. Ekat- pure [525] examine data-security and privacy challenges in AI-powered fleet management systems, emphasizing the need for robust privacy-preserving mechanisms as fleets increasingly rely on AI for decision-making and operational efficiency. Belletti and Bayen [526] propose privacy-preserving mechanisms for fleet management in Mobility-as-a-Service systems, while Yao et al. [527] apply ε-differential privacy techniques to protect sensitive data in logistics operations, including cargo activities. Their work illustrates how dif- ferential privacy can safeguard carrier data while allowing intelligent logistics systems to operate efficiently and comply with data-protection regulations. Di Capua et al. [528] extend privacy considerations to warehousing logistics by exploring privacy-preserving federated learning, enabling multiple stake- holders to collaboratively train ML models while keeping their data decentralized and private.

On the fairness frontier, Lu et al. [529] introduce fairness- aware delivery systems that optimize delivery processes while ensuring that delivery times and costs are distributed equi- tably between service providers and customers. Chen et al. [114] adopt a similar perspective in same-day delivery sys- tems, using reinforcement learning to balance fairness and efficiency while reducing inequalities in service experience. Gupta et al. [530] address fairness in food-delivery platforms, with a particular focus on the income distribution of gig workers. By analyzing data from real-world food-delivery platforms in India, they propose FairFoody, a system designed to ensure fairer income distribution among delivery agents while preserving timely deliveries, achieving up to a tenfold improvement in equitable pay over baseline strategies with limited impact on customer satisfaction.

On the sustainability frontier, Ye et al. [531] propose a lightweight model to reduce prediction time for timely obstacle detection on railway lines, addressing a major safety threat to train operations. Similarly, Hussain and Hill [532] and Zhang et al. [533] propose lightweight ML models to reduce the computational footprint of edge-deployed systems for fast automated object detection in smart warehouses, while Li and He [522] introduce a lightweight model for predicting liner handling volumes at container terminals, balancing model complexity and predictive accuracy. These works emphasize the practical need for lightweight and sustainable models, which are often deployed on edge devices with limited com- putational resources across the supply chain.

6) Emergency Management: In this area, we found 2 case studies focusing on fairness in resource allocation and

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

30 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

emergency response [121], [437]. Elzayn et al. [437] explore fairness in allocation problems, applying their allocation and learning algorithms to the publicly available Philadelphia Crime Incidents dataset. Their approach demonstrates how fair algorithms can effectively address societal challenges by ensuring equitable resource distribution in emergency sce- narios. While the study provides a practical application of fairness principles, it does not consider other aspects of trustworthiness, such as robustness or privacy. Building on this, Aringhieri et al. [121] extend fairness considerations to ambulance routing in post-disaster management. They incor- porate fairness constraints into their routing algorithms to ensure equitable access to emergency services during crises. The proposed methodology is tested on synthetically generated data, highlighting its potential for practical deployment in real- world disaster scenarios.

7) Remarks: Overall, the reviewed case studies indicate that TAI research in ITS&L remains more fragmented than holistic, with most contributions addressing one or a few trustworthiness principles rather than their joint integration into operational pipelines. Across domains, the studies reveal a shared need for more systematic support to verify, monitor, and document trustworthiness properties, while also showing that each application area introduces additional domain-specific requirements.

Vehicle Control Systems constitute the most active area of TAI research, with substantial attention devoted to pri- vacy, robustness, explainability, and sustainability. In this domain, the main challenge is to integrate these principles into safety-critical ML lifecycles that must operate under changing road conditions, distribution shifts, and adversarial perturba- tions. Traffic Management applications primarily highlight the need to balance privacy, robustness, and sustainability, since algorithmic decisions directly influence road efficiency, con- gestion, environmental impact, and users’ mobility experience. Transportation Management Systems emphasize explainabil- ity, privacy preservation, and safety compliance, especially in public-facing and regulated settings where data-driven deci- sions must remain transparent, auditable, and aligned with operational constraints. Traveler Information and Experience Systems mainly focus on privacy-preserving and sustainable ML solutions, reflecting both the sensitivity of user mobility and tourism data and the need for efficient deployment on resource-constrained devices. Freight and Supply Chain Logis- tics provide one of the broadest views of TAI requirements, spanning privacy and security in fleet operations, fairness in delivery services, and sustainability in edge-deployed mod- els. Finally, Emergency Management remains comparatively underrepresented in the TAI literature; however, the available studies clearly show the importance of fairness in resource allocation and emergency response, where algorithmic deci- sions can directly affect vulnerable populations and high-stakes public-safety outcomes.

VII. OPEN PROBLEMS AND FUTURE PERSPECTIVES

In the previous sections, we examined the integration of DataOps, MLOps, and TAI principles in the context of ITS&L. We showed that, together, these perspectives support the

development and operation of trustworthy AI-enabled services by addressing complementary aspects of the data, model, and governance lifecycles. Building on this integrated perspective, we now synthesize the main insights, identify open challenges, and outline future research directions in this domain. To assist readers, Table IV summarizes the case studies presented in Sections IV-F, V-F, and VI-F. Notably, this table offers a visual overview of the current state of MLOps, DataOps, and TAI integration in ITS&L, thereby underscoring important unresolved challenges and suggesting avenues for further investigation.

1) Lack of a Unified Framework for DataOps, MLOps, and TAI Integration:

A major challenge in the AI lifecycle for ITS&L is the lack of a single framework that effectively integrates DataOps, MLOps, and TAI principles. As shown in Table IV, no exist- ing approach comprehensively addresses all three domains within a unified solution; most efforts focus on these practices either separately or in pairs. For instance, several DataOps frameworks for large-scale applications, including those from Amini et al. [20] and Gillani and Niaz [148], primarily address data automation but overlook facets such as data quality, security, defenses against data poisoning attacks [26], and fairness considerations [534]. Other projects, such as the privacy-preserving DataOps BIGSEA [93] initiative and the end-to-end pipelines by Pineda-Jaramillo and Viti [246] and Conlan et al. [277], incorporate at least two of these domains and have demonstrated promising results in real- world large-scale environments. However, such integrative approaches remain the exception rather than the norm. A truly comprehensive framework that unites DataOps, MLOps, and TAI principles is still missing, which presents a valu- able opportunity for advancing the efficiency, reliability, and trustworthiness of Data and AI systems. Although Mishra and Palanisamy [278] propose a conceptual model combin- ing DataOps, MLOps, and safety practices, their work lacks concrete implementation guidelines, further underscoring the need for robust, unified solutions in this space.

2) Limited Adoption to MLOps and Emerging Needs for LLMOps in ITS&L:

Compared to DataOps, which is now widely adopted for large-scale data pipelines (as evidenced by the higher number of case studies), the use of MLOps in ITS&L remains com- paratively limited. The absence of standardized practices often leads to custom ML solutions that fail to scale effectively or ensure long-term reliability [276]. For example, Alomari et al. [82] and Alotaibi et al. [234] combine DataOps solutions with ML algorithms but do not rely on MLOps principles.

This gap is exacerbated by the growing adoption of LLMs, which deliver strong performance but introduce greater com- plexity in lifecycle governance, making LLMOps practices increasingly important. Nevertheless, there is limited evidence of publicly documented LLMOps implementations within ITS&L. Consequently, the underutilization of MLOps and LLMOps offers a compelling avenue for future research, where

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 31

TABLE IV

CASE STUDIES SUMMARY FOR DATAOPS, MLOPS, AND TAI

systematic integration of these practices could enhance real- time data processing, model management, and scalability in ITS&L contexts.

3) Unequal and Non-Holistic Attention to TAI Principles:

The adoption of TAI principles in ITS&L applications is often imbalanced, with some metrics receiving far more attention than others. Privacy considerations, driven by regu- lations such as GDPR [535] and the EU AI Act [35], [36], [37], frequently take precedence, prompting the development of privacy-preserving technologies (e.g., data anonymization and consent management) [447], [481]. Sustainability has also gained substantial traction in recent years, particularly in response to green-transition policies [536], fueling efforts to reduce the computational overhead of increasingly large models for deployment on resource-constrained edge systems

[493], [494], [496]. Robustness, while supported by numerous case studies, tends to be explored primarily in the context of natural data corruption (e.g., sensor failures), with limited focus on adversarial attacks that deliberately exploit model vulnerabilities [26]. Meanwhile, fairness and explainability dimensions have received comparatively less scrutiny, despite their growing relevance under evolving regulatory frameworks such as the AI Act [36]. Moreover, TAI principles are often tackled individually, with research typically optimizing one metric in isolation without assessing its broader impact on the others. However, regulatory requirements call for a compre- hensive approach that addresses TAI dimensions holistically. Consequently, the deployment of ML systems must concur- rently ensure compliance with all relevant TAI principles, underscoring the need for novel methodologies and tools that seamlessly integrate these considerations throughout the AI lifecycle. Currently, fairness and privacy emerge as recurring

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

32 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

gaps across multiple sectors, underscoring the need for more balanced and comprehensive research efforts.

4) Unbalanced Coverage of Case Studies Across Areas:

The analysis of case studies in ITS&L from Table IV highlights an uneven distribution of research focus and explo- ration. While most work concentrates on traffic management and vehicle control, significant gaps exist in domains such as emergency management and warehousing and delivery. This imbalance suggests that several high-impact areas remain underexplored, despite clear opportunities for DataOps, MLOps, and TAI principles to orchestrate, deploy, and main- tain AI-driven solutions. In fields like emergency management, strict confidentiality constraints often prevent data from being shared openly. As a result, large-scale empirical validations and detailed methodological publications become difficult to pursue [537]. Organizations may be unwilling to release sen- sitive datasets out of concern for confidentiality breaches and regulatory noncompliance. Nevertheless, these very domains, particularly critical infrastructures and public services, stand to benefit from DataOps, MLOps pipelines, and TAI prin- ciples, which could enhance robustness, reproducibility, and trustworthiness in current systems. In conclusion, the cur- rent distribution of case studies in ITS&L exhibits a clear imbalance in research emphasis. While high-profile sectors receive greater scrutiny and benefit from more advanced AI implementations, other equally or even more impactful domains garner insufficient attention. Bridging these gaps requires collaborative efforts among academia, industry, and policymakers to foster transparent data-sharing initiatives, robust methodological frameworks, and TAI standards that can be responsibly applied across every dimension of intelligent transportation and logistics.

VIII. CONCLUSION

In this paper, we examine the role of DataOps, MLOps, and TAI in modern ITS&L, highlighting how these com- plementary paradigms support the development of reliable, scalable, and trustworthy AI-enabled services. Through the analysis of tools, methodologies, and real-world case studies, we show that DataOps provides the foundation for manag- ing large-scale, heterogeneous data pipelines, while MLOps enables the systematic development, deployment, monitoring, and maintenance of ML models in production environments. Together, these capabilities are fundamental for addressing critical operational challenges in ITS&L applications.

Beyond these technical foundations, we emphasize the growing need to build TAI-oriented solutions that ensure such systems remain robust, explainable, fair, sustainable, and privacy-preserving throughout their lifecycle. Our review underscores that trustworthiness extends beyond model per- formance, requiring additional metrics and dimensions to be continuously considered. In this context, adopting DataOps and MLOps is not merely a matter of operational convenience, but a strategic requirement for ensuring trustworthiness in AI-driven ITS&L. However, our analysis reveals that, while DataOps solutions are reaching a relatively mature stage of

adoption, the systematic adoption of MLOps remains more limited, and the integration of DataOps, MLOps, and TAI within a unified framework is still uncommon in real-world deployments.

Looking ahead, the rapid evolution of AI creates new oppor- tunities for ITS&L across its diverse application domains. In particular, the growing adoption of LLMs, generative AI, and multimodal systems is extending traditional MLOps practices toward LLMOps, while recent advances in autonomous and agentic systems are motivating the development of AgentOps. These developments substantially increase the capabilities of AI-enabled services, facilitating more advanced interaction, reasoning, planning, and autonomous decision support. How- ever, they also increase the technical debt associated with AI systems by introducing additional layers of models, tools, data sources, and software components that must be developed, monitored, maintained, and governed throughout their lifecy- cle. As a result, ensuring trustworthiness becomes increasingly challenging as AI systems grow in complexity. In this context, DataOps, MLOps, LLMOps, and future AgentOps frame- works are expected to play a fundamental role in automating operational processes and enabling organizations to manage increasingly sophisticated AI systems at scale. More broadly, the evolution of AI in ITS&L is likely to shift the focus from developing isolated models toward governing complete data- and AI-driven ecosystems, with future services expected to integrate increasingly complex combinations of data pipelines, AI models, external services, autonomous agents, and human decision-makers across the entire operational lifecycle.

Future progress will therefore depend not only on improving model capabilities, but also on developing methodologies, tools, and standards that ensure these systems remain reli- able, transparent, sustainable, and aligned with trustworthiness requirements throughout their lifecycle. From this perspective, tighter integration of DataOps, MLOps, and TAI principles represents a promising direction for supporting the next gen- eration of intelligent transportation and logistics services. We hope this work provides a useful reference for researchers, practitioners, and decision-makers, helping them navigate the evolving landscape of data and AI in ITS&L while offering a foundation for future research and trustworthy AI adoption.

REFERENCES

[1] L. S. Iyer, “AI enabled applications towards intelligent transportation,” Transp. Eng., vol. 5, Sep. 2021, Art. no. 100083.

[2] A. I. Torre-Bastida, J. Del Ser, I. Laña, M. Ilardia, M. N. Bilbao, and S. Campos-Cordobés, “Big data for transportation and mobility: Recent advances, trends and challenges,” IET Intell. Transp. Syst., vol. 12, no. 8, pp. 742–755, Oct. 2018.

[3] X. Yin, G. Wu, J. Wei, Y. Shen, H. Qi, and B. Yin, “Deep learning on traffic prediction: Methods, analysis, and future directions,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 6, pp. 4927–4943, Jun. 2022.

[4] R. Sahal, S. H. Alsamhi, K. N. Brown, D. O’Shea, C. McCarthy, and M. Guizani, “Blockchain-empowered digital twins collaboration: Smart transportation use case,” Machines, vol. 9, no. 9, p. 193, Sep. 2021.

[5] A. V. Barenji, W. M. Wang, Z. Li, and D. A. Guerra-Zubiaga, “Intelligent e-commerce logistics platform using hybrid agent based approach,” Transp. Res. E, Logistics Transp. Rev., vol. 126, pp. 15–31, Jun. 2019.

[6] J. Caird, “In-vehicle intelligent transportation systems,” Transp. Aging Soc., vol. 236, pp. 1–15, Oct. 2004.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 33

[7] A. Bhimani and L. Willcocks, “Digitisation, ”Big Data’ and the trans- formation of accounting information,” Accounting Bus. Res., vol. 44, no. 4, pp. 469–490, Jul. 2014.

[8] X.-Y. Chen and Z.-G. Jin, “Research on key technology and applica- tions for Internet of Things,” Phys. Proc., vol. 33, pp. 561–566, Jun. 2012.

[9] G. M. D’silva, A. Khan, Gaurav, and S. Bari, “Real-time processing of IoT events with historic data using apache kafka and apache spark with dashing framework,” in Proc. 2nd IEEE Int. Conf. Recent Trends Elec- tron., Inf. Commun. Technol. (RTEICT), May 2017, pp. 1804–1809.

[10] A. K. Haghighat, V. Ravichandra-Mouli, P. Chakraborty, Y. Esfandiari, S. Arabi, and A. Sharma, “Applications of deep learning in intelligent transportation systems,” J. Big Data Anal. Transp., vol. 2, pp. 115–145, Aug. 2020.

[11] M. Sathyanarayana and K. Akshay Reddy, “Deep learning on traffic prediction methods analysis and future directions,” Int. J. Adv. Res. Sci., Commun. Technol., vol. 23, no. 6, pp. 4927–4943, 2021.

[12] A. Ait Ouallane, A. Bakali, A. Bahnasse, S. Broumi, and M. Talea, “Fusion of engineering insights and emerging trends: Intelligent urban traffic management system,” Inf. Fusion, vol. 88, pp. 218–248, Dec. 2022.

[13] K. N. Qureshi and A. H. Abdullah, “A survey on intelligent trans- portation systems,” Middle-East J. Scientific Res., vol. 15, no. 5, pp. 629–642, 2013.

[14] N. Boysen, S. Fedtke, and S. Schwerdfeger, “Last-mile delivery con- cepts: A survey from an operational research perspective,” OR Spectr., vol. 43, no. 1, pp. 1–58, Mar. 2021.

[15] A. Sładkowski and W. Pamuła, Intelligent Transportation Systems- problems and Perspectives. Cham, Switzerland: Springer, 2016.

[16] H. Atwal, Practical DataOps: Delivering Agile Data Science At Scale. Cham, Switzerland: Springer, 2019.

[17] R. Dautov and S. Distefano, “Quantifying volume, velocity, and variety to support (Big) data-intensive application development,” in Proc. IEEE Int. Conf. Big Data (Big Data), Dec. 2017, pp. 2843–2852.

[18] J. Barrachina et al., “CAOVA: A car accident ontology for VANETs,” in Proc. IEEE Wireless Commun. Netw. Conf. (WCNC), Apr. 2012, pp. 1864–1869.

[19] L. Zhu, F. R. Yu, Y. Wang, B. Ning, and T. Tang, “Big data analytics in intelligent transportation systems: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 1, pp. 383–398, Jan. 2019.

[20] S. Amini, I. Gerostathopoulos, and C. Prehofer, “Big data analytics architecture for real-time traffic control,” in Proc. 5th IEEE Int. Conf. Models Technol. Intell. Transp. Syst. (MT-ITS), Jun. 2017, pp. 710–715.

[21] J. P. Bharadiya, “Machine learning and ai in business intelligence: Trends and opportunities,” Int. J. Comput., vol. 48, no. 1, pp. 123–134, 2023.

[22] J. D. Novaković, A. Veljović, S. S. Ilić, Ž. Papić, and M. Tomović, “Evaluation of classification models in machine learning,” Theory Appl. Math. & Comput. Sci., vol. 7, no. 1, p. 39, 2017.

[23] S. Yan et al., “Positive-congruent training: Towards regression-free model updates,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recog- nit. (CVPR), Jun. 2021, pp. 14294–14303.

[24] R. Spring and A. Shrivastava, “Scalable and sustainable deep learning via randomized hashing,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2017, pp. 445–454.

[25] F. Sperrle et al., “A survey of human-centered evaluations in human- centered machine learning,” Comput. Graph. Forum, vol. 40, no. 3, pp. 543–568, Jun. 2021.

[26] A. E. Cinà et al., “Wild patterns reloaded: A survey of machine learning security against training data poisoning,” ACM Comput. Surv., vol. 55, no. 13s, pp. 1–39, Dec. 2023.

[27] S. Caton and C. Haas, “Fairness in machine learning: A survey,” ACM Comput. Surv., vol. 56, no. 7, pp. 1–38, Jul. 2024.

[28] B. Liu, M. Ding, S. Shaham, W. Rahayu, F. Farokhi, and Z. Lin, “When machine learning meets privacy: A survey and outlook,” ACM Comput. Surv., vol. 54, no. 2, pp. 1–36, Mar. 2022.

[29] A. Chowdhury, G. Karmakar, J. Kamruzzaman, and S. Islam, “Trustworthiness of self-driving vehicles for intelligent transportation systems in industry applications,” IEEE Trans. Ind. Informat., vol. 17, no. 2, pp. 961–970, Feb. 2021.

[30] H. Tao et al., “TrustData: Trustworthy and secured data collection for event detection in industrial cyber-physical system,” IEEE Trans. Ind. Informat., vol. 16, no. 5, pp. 3311–3321, May 2020.

[31] F. Fui-Hoon Nah, R. Zheng, J. Cai, K. Siau, and L. Chen, “Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration,” J. Inf. Technol. Case Appl. Res., vol. 25, no. 3, pp. 277–304, Jul. 2023.

[32] Y. Chang et al., “A survey on evaluation of large language models,” ACM Trans. Intell. Syst. Technol., vol. 15, no. 3, pp. 1–45, 2024.

[33] C. Cui et al., “A survey on multimodal large language models for autonomous driving,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis., Jun. 2024, pp. 958–979.

[34] H. Sha et al., “LanguageMPC: Large language models as decision makers for autonomous driving,” 2023, arXiv:2310.03026.

[35] Artificial Intelligence Act: Council and Parliament Strike a Deal on the First Rules for AI in the World. Accessed: Feb. 14, 2025. [Online]. Available: https://www.consilium.europa.eu/en/press/press-releases/2023/12/09/ artificial-intelligence-act-council-and-parliament-strike-a-deal-on-the- first-worldwide-rules-for-ai/

[36] S. Nativi and S. De Nigris, “AI Standardisation landscape: State of play and link to the EC proposal for an AI regulatory framework,” ICT Standardisation Observatory and Support Facility in Europe, Luxembourg, Tech. Rep., 2021.

[37] Proposal for a Regulation Laying Down Harmonised Rules on Artificial Intelligence and Amending Certain Union Legislative Acts. Accessed: Dec. 15, 2023. [Online]. Available: https://eur-lex.europa.eu/legal- content/EN/TXT/?uri=celex3A52021PC0206

[38] Q. Liu, P. Hou, G. Wang, T. Peng, and S. Zhang, “Intelligent route planning on large road networks with efficiency and privacy,” J. Par- allel Distrib. Comput., vol. 133, pp. 93–106, Nov. 2019.

[39] A. Sen, “DevOps, DevSecOps, AIOPS- paradigms to IT operations,” in Lecture Notes in Electrical Engineering, 2021, pp. 211–221.

[40] H. Khalajzadeh, M. Abdelrazek, J. Grundy, J. Hosking, and Q. He, “A survey of current end-user data analytics tool support,” in Proc. IEEE Int. Congr. Big Data (BigData Congr.), Jul. 2018, pp. 41–48.

[41] M. Rodriguez, L. J. P. de Araújo, and M. Mazzara, “Good practices for the adoption of DataOps in the software industry,” J. Phys., Conf. Ser., vol. 1694, no. 1, Dec. 2020, Art. no. 012032.

[42] M. Treveil et al., Introducing MLOps. Sebastopol, CA, USA: O’Reilly Media, 2020.

[43] M. Testi et al., “MLOps: A taxonomy and a methodology,” IEEE Access, vol. 10, pp. 63606–63618, 2022.

[44] A. R. Munappy, D. I. Mattos, J. Bosch, H. H. Olsson, and A. Dakkak, “From ad-hoc data analytics to DataOps,” in Proc. Int. Conf. Softw. Syst. Processes, Jun. 2020, pp. 165–174.

[45] M. Chowdhury, K. Dey, and A. Apon, Data Analytics for Intelligent Transportation Systems. Amsterdam, The Netherlands: Elsevier, 2024.

[46] S. D. Das and P. K. Bala, “What drives MLOps adoption? An analysis using the TOE framework,” J. Decis. Syst., vol. 33, no. 3, pp. 376–412, Jul. 2024.

[47] S. Mäkinen, H. Skogström, E. Laaksonen, and T. Mikkonen, “Who needs MLOps: What data scientists seek to accomplish and how can MLOps help?” in Proc. IEEE/ACM 1st Workshop AI Eng.-Softw. Eng. AI (WAIN), May 2021, pp. 109–112.

[48] M. Zeller, T. Waschulzik, R. Schmid, and C. Bahlmann, “Toward a safe MLOps process for the continuous development and safety assurance of ML-based systems in the railway domain,” AI Ethics, vol. 4, no. 1, pp. 123–130, Feb. 2024.

[49] B. Biggio and F. Roli, “Wild patterns: Ten years after the rise of adversarial machine learning,” Pattern Recognit., vol. 84, pp. 317–331, Dec. 2018.

[50] B. Li et al., “Trustworthy AI: From principles to practices,” ACM Comput. Surv., vol. 55, no. 9, pp. 1–46, 2022.

[51] L. Oneto et al., “Towards learning trustworthily, automatically, and with guarantees on graphs: An overview,” Neurocomputing, vol. 493, pp. 217–243, Jul. 2022.

[52] B. Chander, C. John, L. Warrier, and K. Gopalakrishnan, “Toward trustworthy artificial intelligence (TAI) in the context of explainability and robustness,” ACM Comput. Surv., vol. 57, no. 6, pp. 1–49, Jun. 2025.

[53] A. S. Albahri et al., “A systematic review of trustworthy artificial intelligence applications in natural disasters,” Comput. Electr. Eng., vol. 118, Sep. 2024, Art. no. 109409.

[54] W. Wei and L. Liu, “Trustworthy distributed AI systems: Robust- ness, privacy, and governance,” ACM Comput. Surv., vol. 57, no. 6, pp. 1–42, Jun. 2025.

[55] T. Bailoni and M. Dragoni, “A review on trustworthiness of digital assistants for personal healthcare,” ACM Comput. Surv., vol. 57, no. 7, pp. 1–34, Jul. 2025.

[56] E. I. Vlahogianni, “Computational intelligence and optimization for transportation big data: Challenges and opportunities,” in Engineering and Applied Sciences Optimization: Dedicated to the Memory of Pro- fessor MG Karlaftis. Cham, Switzerland: Springer, 2015, pp. 107–128.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

34 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[57] I. Lana, J. Del Ser, M. Velez, and E. I. Vlahogianni, “Road traffic forecasting: Recent advances and new challenges,” IEEE Intell. Transp. Syst. Mag., vol. 10, no. 2, pp. 93–109, Feb. 2018.

[58] S. Hireche and A. Dennai, “Machine learning techniques for road traffic automatic incident detection systems: A review,” in Smart Energy Empowerment in Smart and Resilient Cities: Renewable Energy for Smart and Sustainable Cities. Cham, Switzerland: Springer, 2020, pp. 60–69.

[59] N. Bešinović et al., “Artificial intelligence in railway transport: Taxon- omy, regulations, and applications,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 14011–14024, Sep. 2022.

[60] Z. Zhou, Z. Yang, Y. Zhang, Y. Huang, H. Chen, and Z. Yu, “A comprehensive study of speed prediction in transportation system: From vehicle to traffic,” iScience, vol. 25, no. 3, pp. 1–29, Mar. 2022.

[61] B. B. Elallid, N. Benamar, A. S. Hafid, T. Rachidi, and N. Mrani, “A comprehensive survey on the application of deep and reinforcement learning approaches in autonomous driving,” J. King Saud Univ.- Comput. Inf. Sci., vol. 34, no. 9, pp. 7366–7390, Oct. 2022.

[62] R. Guo, M. Vallati, Y. Wang, H. Zhang, Y. Chen, and F.-Y. Wang, “Sustainability opportunities and ethical challenges of AI-enabled connected autonomous vehicles routing in urban areas,” IEEE Trans. Intell. Vehicles, vol. 9, no. 1, pp. 55–58, Jan. 2024.

[63] L. Gaur and B. M. Sahoo, Explainable Artificial Intelligence for Intelligent Transportation Systems - Ethics and Applications. Cham, Switzerland: Springer, 2022.

[64] S. Ahmad, I. Shakeel, S. Mehfuz, and J. Ahmad, “Deep learning models for cloud, edge, fog, and IoT computing paradigms: Survey, recent advances, and future directions,” Comput. Sci. Rev., vol. 49, Aug. 2023, Art. no. 100568.

[65] H. Li et al., “A survey on graph neural networks in intelligent transportation systems,” 2024, arXiv:2401.00713.

[66] R. A. Khalil, Z. Safelnasr, N. Yemane, M. Kedir, A. Shafiqurrah- man, and N. Saeed, “Advanced learning technologies for intelligent transportation systems: Prospects and challenges,” IEEE Open J. Veh. Technol., vol. 5, pp. 397–427, 2024.

[67] D. Mahmud et al., “Integrating LLMs with ITS: Recent advances, potentials, challenges, and future directions,” IEEE Trans. Intell. Transp. Syst., vol. 26, no. 5, pp. 5674–5709, May 2025.

[68] H. Yan and Y. Li, “Generative AI for intelligent transportation sys- tems: Road transportation perspective,” ACM Comput. Surveys, vol. 57, no. 12, pp. 1–45, Dec. 2025.

[69] D. A. Tamburri, “Sustainable MLOps: Trends and challenges,” in Proc. 22nd Int. Symp. Symbolic Numeric Algorithms Scientific Comput. (SYNASC), Sep. 2020, pp. 17–23.

[70] E. Black, R. Naidu, R. Ghani, K. Rodolfa, D. Ho, and H. Heidari, “Toward operationalizing pipeline-aware ML fairness: A research agenda for developing practical guidelines and tools,” in Proc. Equity Access Algorithms, Mech., Optim., Oct. 2023, pp. 1–11.

[71] B. Eken, S. Pallewatta, N. K. Tran, A. Tosun, and M. A. Babar, “A multivocal review of mlops practices, challenges and open issues,” 2024, arXiv:2406.09737.

[72] M. Zarour, H. Alzabut, and K. T. Al-Sarayreh, “MLOps best practices, challenges and maturity models: A systematic literature review,” Inf. Softw. Technol., vol. 183, Jul. 2025, Art. no. 107733.

[73] F. Bayram and B. S. Ahmed, “Towards trustworthy machine learning in production: An overview of the robustness in MLOps approach,” ACM Comput. Surv., vol. 57, no. 5, pp. 1–35, May 2025.

[74] M. Alam, J. Ferreira, and J. Fonseca, “Introduction to intelligent transportation systems,” in Intelligent Transportation Systems: Depend- able Vehicular Communications for Improved Road Safety. Cham, Switzerland: Springer, 2016.

[75] J. Zhang, F.-Y. Wang, K. Wang, W.-H. Lin, X. Xu, and C. Chen, “Data- driven intelligent transportation systems: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 4, pp. 1624–1639, Dec. 2011.

[76] M. V. Martı́nez, I. Del Campo, J. Echanobe, and K. Basterretxea, “Driving behavior signals and machine learning: A personalized driver assistance system,” in Proc. IEEE 18th Int. Conf. Intell. Transp. Syst., Sep. 2015, pp. 2933–2940.

[77] W. Tong, A. Hussain, W. X. Bo, and S. Maharjan, “Artificial intel- ligence for vehicle-to-everything: A survey,” IEEE Access, vol. 7, pp. 10823–10843, 2019.

[78] S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A survey of deep learning techniques for autonomous driving,” J. Field Robot., vol. 37, no. 3, pp. 362–386, Apr. 2020.

[79] S. A. Ferguson, “The effectiveness of electronic stability control in reducing real-world crashes: A literature review,” Traffic Injury Prevention, vol. 8, no. 4, pp. 329–338, Oct. 2007.

[80] R. Singh et al., “Highway 4.0: Digitalization of highways for vulnerable road safety development with intelligent IoT sensors and machine learning,” Saf. Sci., vol. 143, Nov. 2021, Art. no. 105407.

[81] O. Avatefipour and F. Sadry, “Traffic management system using IoT technology–A comparative review,” in Proc. IEEE Int. Conf. Elec- tro/Inf. Technol. (EIT), May 2018, pp. 1041–1047.

[82] E. Alomari, R. Mehmood, and I. Katib, “Road traffic event detection using Twitter data, machine learning, and apache spark,” in Proc. IEEE SmartWorld, Ubiquitous Intell. & Comput., Adv. & Trusted Comput., Scalable Comput. & Commun., Internet People Smart City Innov., 2019, pp. 1888–1895.

[83] C. Lombardi, L. Picado-Santos, and A. M. Annaswamy, “Model-based dynamic toll pricing: An overview,” Appl. Sci., vol. 11, no. 11, p. 4778, May 2021.

[84] G. Guerreiro, P. Figueiras, R. Silva, R. Costa, and R. Jardim-Goncalves, “An architecture for big data processing on intelligent transportation systems. An application scenario on highway traffic flows,” in Proc. IEEE 8th Int. Conf. Intell. Syst. (IS), Sep. 2016, pp. 65–72.

[85] A. A. Zhilenkov and I. R. Epifantsev, “Problems of a trajectory planning in autonomous navigation systems based on technical vision and AI,” in Proc. IEEE Conf. Russian Young Researchers Electr. Electron. Eng. (EIConRus), Jan. 2018, pp. 1032–1035.

[86] A. Degas et al., “A survey on artificial intelligence (AI) and eXplain- able AI in air traffic management: Current trends and development with future research trajectory,” Appl. Sci., vol. 12, no. 3, p. 1295, Jan. 2022.

[87] E. K. Nti, S. J. Cobbina, E. E. Attafuah, E. Opoku, and M. A. Gyan, “Environmental sustainability technologies in biodiversity, energy, transportation and water management using artificial intelligence: A systematic review,” Sustain. Futures, vol. 4, Mar. 2022, Art. no. 100068.

[88] R. Tang et al., “A literature review of artificial intelligence applications in railway systems,” Transp. Res. C, Emerg. Technol., vol. 140, 2022, Art. no. 103679.

[89] A. M. de Souza, C. A. Brennand, R. S. Yokoyama, E. A. Donato, E. R. Madeira, and L. A. Villas, “Traffic management systems: A clas- sification, review, challenges, and future perspectives,” Int. J. Distrib. Sensor Netw., vol. 13, no. 4, Apr. 2017, Art. no. 155014771668361.

[90] Loginext. Accessed: Feb. 15, 2025. [Online]. Available: https://www.loginextsolutions.com/blog/logistics-route-optimization- software-bias-fix-solution/

[91] A. A. Musa, S. I. Malami, F. Alanazi, W. Ounaies, M. Alshammari, and S. I. Haruna, “Sustainable traffic management for smart cities using Internet-of-Things-oriented intelligent transportation systems (ITS): Challenges and recommendations,” Sustainability, vol. 15, no. 13, p. 9859, Jun. 2023.

[92] S. Fiore et al., “An integrated big and fast data analytics platform for smart urban transportation management,” IEEE Access, vol. 7, pp. 117652–117677, 2019.

[93] A. S. Alic et al., “BIGSEA: A big data analytics platform for public transportation information,” Future Gener. Comput. Syst., vol. 96, pp. 243–269, Jul. 2019.

[94] D. Ushakov, E. Dudukalov, L. Shmatko, and K. Shatila, “Artificial intelligence as a factor of public transportations system development,” Transp. Res. Proc., vol. 63, pp. 2401–2408, Jun. 2022.

[95] M. Saleem, S. Abbas, T. M. Ghazal, M. Adnan Khan, N. Sahawneh, and M. Ahmad, “Smart cities: Fusion-based intelligent traffic conges- tion control system for vehicular networks using machine learning techniques,” Egyptian Informat. J., vol. 23, no. 3, pp. 417–426, Sep. 2022.

[96] B. Qi, A. Costin, and M. Jia, “A framework with efficient extraction and analysis of Twitter data for evaluating public opinions on transportation services,” Travel Behav. Soc., vol. 21, pp. 10–23, Oct. 2020.

[97] T. Ruiz, L. Mars, R. Arroyo, and A. Serna, “Social networks, big data and transport planning,” Transp. Res. Proc., vol. 18, pp. 446–452, Feb. 2016.

[98] J. C. Cepeda-Pacheco and M. C. Domingo, “Deep learning and Internet of Things for tourist attraction recommendations in smart cities,” Neural Comput. Appl., vol. 34, no. 10, pp. 7691–7709, May 2022.

[99] K. A. Fararni, F. Nafis, B. Aghoutane, A. Yahyaouy, J. Riffi, and A. Sabri, “Hybrid recommender system for tourism based on big data and AI: A conceptual framework,” Big Data Mining Anal., vol. 4, no. 1, pp. 47–55, Mar. 2021.

[100] D. Levinson, “The value of advanced traveler information systems for route choice,” Transp. Res. C, Emerg. Technol., vol. 11, no. 1, pp. 75–87, Feb. 2003.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 35

[101] D. Wang, J. Zhang, W. Cao, J. Li, and Y. Zheng, “When will you arrive? Estimating travel time based on deep neural networks,” in Proc. AAAI Conf. Artif. Intell., 2018, pp. 2500–2507.

[102] C. Siripanpornchana, S. Panichpapiboon, and P. Chaovalit, “Travel- time prediction with deep learning,” in Proc. IEEE Region 10 Conf. (TENCON), Nov. 2016, pp. 1859–1862.

[103] D. Chi, C. Tang, and C. Yin, “Design and implementation of hotel big data analysis platform based on Hadoop and spark,” J. Phys., Conf. Ser., vol. 2010, no. 1, Sep. 2021, Art. no. 012079.

[104] S. P. R. Asaithambi, R. Venkatraman, and S. Venkatraman, “A thematic travel recommendation system using an augmented big data analytical model,” Technologies, vol. 11, no. 1, p. 28, Feb. 2023.

[105] J. Gao, P. Peng, F. Lu, C. Claramunt, and Y. Xu, “Towards travel recommendation interpretability: Disentangling tourist decision-making process via knowledge graph,” Inf. Process. Manage., vol. 60, no. 4, Jul. 2023, Art. no. 103369.

[106] K. Tsolaki, T. Vafeiadis, A. Nizamis, D. Ioannidis, and D. Tzovaras, “Utilizing machine learning on freight transportation and logistics applications: A review,” ICT Exp., vol. 9, no. 3, pp. 284–295, Jun. 2023.

[107] R. de la Torre, C. G. Corlu, J. Faulin, B. S. Onggo, and A. A. Juan, “Simulation, optimization, and machine learning in sustainable trans- portation systems: Models and applications,” Sustainability, vol. 13, no. 3, p. 1551, Feb. 2021.

[108] D. Sarabia-Jacome, C. E. Palau, M. Esteve, and F. Boronat, “Seaport data space for improving logistic maritime operations,” IEEE Access, vol. 8, pp. 4372–4382, 2020.

[109] A. P. Pandian, “Artificial intelligence application in smart warehousing environment for automated logistics,” J. Artif. Intell., vol. 1, no. 2, pp. 63–72, 2019.

[110] C. K. M. Lee, Y. Lv, K. K. H. Ng, W. Ho, and K. L. Choy, “Design and application of Internet of Things-based warehouse management system for smart logistics,” Int. J. Prod. Res., vol. 56, no. 8, pp. 2753–2768, Apr. 2018.

[111] T. Borgi, N. Zoghlami, M. Abed, and M. S. Naceur, “Big data for operational efficiency of transport and logistics: A review,” in Proc. 6th IEEE Int. Conf. Adv. Logistics Transp. (ICALT), Jul. 2017, pp. 113–120.

[112] G. Marchet, M. Melacini, and S. Perotti, “Environmental sustainability in logistics and freight transportation: A literature review and research agenda,” J. Manuf. Technol. Manage., vol. 25, no. 6, pp. 775–811, Jul. 2014.

[113] W. Lyu et al., “Towards fair workload assessment via homogeneous order grouping in last-mile delivery,” in Proc. 31st ACM Int. Conf. Inf. Knowl. Manage., Oct. 2022, pp. 3361–3370.

[114] X. Chen, T. Wang, B. W. Thomas, and M. W. Ulmer, “Same-day delivery with fair customer service,” Eur. J. Oper. Res., vol. 308, no. 2, pp. 738–751, Jul. 2023.

[115] M. Thibaud, H. Chi, W. Zhou, and S. Piramuthu, “Internet of Things (IoT) in high-risk environment, health and safety (EHS) industries: A comprehensive review,” Decis. Support Syst., vol. 108, pp. 79–95, Apr. 2018.

[116] D. Huang, S. Wang, and Z. Liu, “A systematic review of prediction methods for emergency management,” Int. J. Disaster Risk Reduction, vol. 62, Aug. 2021, Art. no. 102412.

[117] Y. Jiang and Y. Yuan, “Emergency logistics in a large-scale disaster context: Achievements and challenges,” Int. J. Environ. Res. Public Health, vol. 16, no. 5, p. 779, Mar. 2019.

[118] R. R. Arinta and E. Andi W. R., “Natural disaster application on big data and machine learning: A review,” in Proc. 4th Int. Conf. Inf. Technol., Inf. Syst. Electr. Eng. (ICITISEE), Nov. 2019, pp. 249–254.

[119] E. L. Huamanı́, A. Mantari, and A. Roman-Gonzalez, “Machine learning techniques to visualize and predict terrorist attacks worldwide using the global terrorism database,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 4, pp. 562–570, 2020.

[120] R. Garriga et al., “Machine learning model to predict mental health crises from electronic health records,” Nature Med., vol. 28, no. 6, pp. 1240–1248, Jun. 2022.

[121] R. Aringhieri, S. Bigharaz, D. Duma, and A. Guastalla, “Fairness in ambulance routing for post disaster management,” Central Eur. J. Oper. Res., vol. 30, no. 1, pp. 1–23, Mar. 2022.

[122] T. Alexakis, N. Peppes, K. Demestichas, and E. Adamopoulou, “A distributed big data analytics architecture for vehicle sensor data,” Sensors, vol. 23, no. 1, p. 357, Dec. 2022.

[123] X. Wang, H. Schuster, R. Borrison, and B. Klöpper, “Technical debt management in industrial ML–state of practice and management model proposal,” in Proc. IEEE 21st Int. Conf. Ind. Informat. (INDIN), Jul. 2023, pp. 1–9.

[124] C. Hogan, “‘Creating a data-driven enterprise with DataOps: Insights from facebook, uber, LinkedIn, Twitter, and eBay‘ by ashish thusoo and joydeep sen sarma,” Appl. Marketing Anal., Peer-Rev. J., vol. 3, no. 4, pp. 374–375, 2017.

[125] J. Ereth, “Dataops-towards a definition,” LWDA, vol. 2191, pp. 104–112, Aug. 2018.

[126] M. Wang, H. Shan, R. Lu, R. Zhang, X. Shen, and F. Bai, “Real- time path planning based on hybrid-VANET-enhanced transportation system,” IEEE Trans. Veh. Technol., vol. 64, no. 5, pp. 1664–1678, May 2015.

[127] G. Nagpal, G. K. Bishnoi, H. S. Dhami, and A. Vijayvargia, “Use of data analytics to increase the efficiency of last mile logistics for ecommerce deliveries,” in Handbook of Research on Engineering, Business, and Healthcare Applications of Data Science and Analytics. London, U.K.: IGI Global Scientific Publishing, 2021.

[128] T. Borgi, N. Zoghlami, and M. Abed, “Big data for transport and logistics: A review,” in Proc. Int. Conf. Adv. Syst. Electric Technol. (IC ASET), Jan. 2017, pp. 44–49.

[129] F. Zhou, Q. Yang, T. Zhong, D. Chen, and N. Zhang, “Variational graph neural networks for road traffic prediction in intelligent transportation systems,” IEEE Trans. Ind. Informat., vol. 17, no. 4, pp. 2802–2812, Apr. 2021.

[130] A. Gupta, H. P. Gupta, B. Biswas, and T. Dutta, “An early classi- fication approach for multivariate time series of on-vehicle sensors in transportation,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 12, pp. 5316–5327, Dec. 2020.

[131] Z. Ji, I. Ganchev, M. O’Droma, L. Zhao, and X. Zhang, “A cloud- based car parking middleware for IoT-based smart cities: Design and implementation,” Sensors, vol. 14, no. 12, pp. 22372–22393, Nov. 2014.

[132] H. Parineh, M. Sarvi, and S. A. Bagloee, “Acoustic sensors and audio signal processing in intelligent transportation systems: A survey,” IEEE Trans. Intell. Vehicles, vol. 10, no. 8, pp. 4171–4190, Aug. 2025.

[133] Zulkarnain and T. D. Putri, “Intelligent transportation systems (ITS): A systematic review using a natural language processing (NLP) approach,” Heliyon, vol. 7, no. 12, Dec. 2021, Art. no. e08615.

[134] J. Warren and N. Marz, Big Data: Principles and Best Practices of Scalable Realtime Data Systems. New York, NY, USA: Simon and Schuster, 2015.

[135] V. C. Storey and I.-Y. Song, “Big data technologies and management: What conceptual modeling can do,” Data Knowl. Eng., vol. 108, pp. 50–67, Mar. 2017.

[136] K. Guo et al., “Optimized graph convolution recurrent neural network for traffic prediction,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 2, pp. 1138–1149, Feb. 2021.

[137] J. Ye et al., “Cyber–physical security of powertrain systems in modern electric vehicles: Vulnerabilities, challenges, and future visions,” IEEE J. Emerg. Sel. Topics Power Electron., vol. 9, no. 4, pp. 4639–4657, Aug. 2021.

[138] J. Holmström, A. Kostis, E. Galariotis, D. Roubaud, and C. Zopouni- dis, “Stalled data flows in digital innovation networks: Underlying mechanisms and the role of related variety,” Ind. Marketing Manage., vol. 121, pp. 16–26, Aug. 2024.

[139] C. Larman and V. R. Basili, “Iterative and incremental developments. A brief history,” Computer, vol. 36, no. 6, pp. 47–56, Jun. 2003.

[140] A. Thusoo and J. Sarma, Creating a Data-Driven Enterprise With DataOps. Sebastopol, CA, USA: O’Reilly Media, 2017.

[141] J. L. Harrington, Relational Database Design and Implementation. San Mateo, CA, USA: Morgan Kaufmann, 2016.

[142] K. Krishnan, Data Warehousing in the Age of Big Data. San Mateo, CA, USA: Morgan Kaufmann, 2013.

[143] F. Nargesian, E. Zhu, R. J. Miller, K. Q. Pu, and P. C. Arocena, “Data lake management: Challenges and opportunities,” VLDB Endowment, Tech. Rep. 12, 2019.

[144] M. Zaharia, A. Ghodsi, R. Xin, and M. Armbrust, “Lakehouse: A new generation of open platforms that unify data warehousing and advanced analytics,” in Proc. Innov. Data Syst. Res., 2021, pp. 1–8.

[145] A. A. Harby and F. Zulkernine, “From data warehouse to lakehouse: A comparative review,” in Proc. IEEE Int. Conf. Big Data (Big Data), Dec. 2022, pp. 389–395.

[146] V. Grover, R. H. L. Chiang, T.-P. Liang, and D. Zhang, “Creating strate- gic business value from big data analytics: A research framework,” J. Manage. Inf. Syst., vol. 35, no. 2, pp. 388–423, Apr. 2018.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

36 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[147] A. Boulanger, “Open-source versus proprietary software: Is one more reliable and secure than the other?” IBM Syst. J., vol. 44, no. 2, pp. 239–248, 2005.

[148] M. Gillani and H. A. Niaz, “Machine learning based data collection protocol for intelligent transport systems: A real-time implementa- tion on Dublin m50, Ireland,” Complex Intell. Syst., vol. 10, no. 2, pp. 1879–1897, Apr. 2024.

[149] N. Garg, Apache Kafka. Birmingham, U.K.: Packt Publishing, 2013. [150] Apache Kafka. Accessed: Dec. 15, 2023. [Online]. Available: https://

kafka.apache.org/

[151] D. Borthakur, “The Hadoop distributed file system: Architecture and design,” Hadoop Project Website, vol. 11, p. 21, Mar. 2007.

[152] Apache Hadoop. Accessed: Dec. 15, 2023. [Online]. Available: https:// hadoop.apache.org/

[153] M. Zaharia, R. S. Xin, P. Wendell, and T. Das, “Apache spark: A unified engine for big data processing,” Commun. ACM, vol. 59, no. 11, pp. 56–65, 2016.

[154] Apache Spark. Accessed: Dec. 15, 2023. [Online]. Available: https:// spark.apache.org/

[155] P. Carbone, A. Katsifodimos, S. Ewen, V. Markl, S. Haridi, and K. Tzoumas, “Apache flink: Stream and batch processing in a single engine,” Bull. Tech. Committee Data Eng., vol. 38, no. 4, pp. 28–38, 2015.

[156] Apache Flink. Accessed: Dec. 15, 2023. [Online]. Available: https:// flink.apache.org/

[157] Dbt Labs. Accessed: Feb. 2, 2025. [Online]. Available: https:// www.getdbt.com/

[158] Docker. Accessed: Feb. 2, 2025. [Online]. Available: https:// www.docker.com/

[159] Kubernetes. Accessed: Feb. 5, 2025. [Online]. Available: https:// kubernetes.io/

[160] Y. D. Dessalk, N. Nikolov, M. Matskin, A. Soylu, and D. Roman, “Scalable execution of big data workflows using software containers,” in Proc. 12th Int. Conf. Manage. Digit. EcoSystems, Nov. 2020, pp. 76–83.

[161] Prometheus. Accessed: Feb. 5, 2025. [Online]. Available: https:// prometheus.io/

[162] Apache Airflow. Accessed: Feb. 5, 2025. [Online]. Available: https:// airflow.apache.org/

[163] Jenkins. Accessed: Feb. 5, 2025. [Online]. Available: https:// www.jenkins.io/

[164] Microsoft Power BI. Accessed: Dec. 11, 2023. [Online]. Available: https://www.microsoft.com/en-us/power-platform/products/power-bi

[165] Tableau. Accessed: Dec. 11, 2023. [Online]. Available: https:// www.tableau.com/

[166] Git. Accessed: Feb. 5, 2025. [Online]. Available: https://git-scm.com/

[167] Apache Subversion. Accessed: Feb. 5, 2025. [Online]. Available: https://subversion.apache.org/

[168] J. P. Caulkins, G. Feichtinger, D. Grass, R. F. Hartl, P. M. Kort, and A. Seidl, “When to make proprietary software open source,” J. Econ. Dyn. Control, vol. 37, no. 6, pp. 1182–1194, Jun. 2013.

[169] Google Cloud Dataflow. Accessed: Dec. 15, 2023. [Online]. Available: https://cloud.google.com/dataflow?hl=en

[170] Amazon EMR. Accessed: Dec. 15, 2023. [Online]. Available: https:// aws.amazon.com/emr/

[171] Amazon S3. Accessed: Dec. 15, 2023. [Online]. Available: https:// aws.amazon.com/s3/

[172] Amazon Kinesis. Accessed: Dec. 15, 2024. [Online]. Available: https:// aws.amazon.com/kinesis/

[173] Azure Data Factory. Accessed: Dec. 15, 2023. [Online]. Available: https://azure.microsoft.com/en-us/products/data-factory

[174] J. Jaisingh, E. W. K. See-To, and K. Y. Tam, “The impact of open source software on the strategic choices of firms developing proprietary software,” J. Manage. Inf. Syst., vol. 25, no. 3, pp. 241–276, Dec. 2008.

[175] R. Sen, “A strategic analysis of competition between open source and proprietary software,” J. Manage. Inf. Syst., vol. 24, no. 1, pp. 233–257, Jul. 2007.

[176] L. Lin, “Impact of user skills and network effects on the competition between open source and proprietary software,” Electron. Commerce Res. Appl., vol. 7, no. 1, pp. 68–81, Mar. 2008.

[177] T. Kilamo, I. Hammouda, T. Mikkonen, and T. Aaltonen, “From proprietary to open source—Growing an open source ecosystem,” J. Syst. Softw., vol. 85, no. 7, pp. 1467–1478, Jul. 2012.

[178] S. Kahveci, B. Alkan, M. H. Ahmad, B. Ahmad, and R. Harrison, “An end-to-end big data analytics platform for IoT-enabled smart factories: A case study of battery module assembly system for electric vehicles,” J. Manuf. Syst., vol. 63, pp. 214–223, Apr. 2022.

[179] R. Dintén, S. Garcı́a, and M. Zorrilla, “Fleet management systems in logistics 4.0 era: A real time distributed and scalable architectural proposal,” Proc. Comput. Sci., vol. 217, pp. 806–815, Jan. 2023.

[180] M. Zhang, T. Wo, T. Xie, X. Lin, and Y. Liu, “Carstream: An industrial system of big data processing for Internet-of-Vehicles,” Proc. VLDB Endowment, vol. 10, no. 12, pp. 1766–1777, 2017.

[181] W. Xu et al., “Supporting large scale connected vehicle data analysis using HIVE,” in Proc. IEEE Int. Conf. Big Data (Big Data), Dec. 2016, pp. 2296–2304.

[182] Y. C. Nair, P. V. Neethu, V. K. Menon, and K. P. Soman, “Real time vehicular data analytics utilising bigdata platforms and cost effective ECU networks,” Indian J. Sci. Technol., vol. 9, no. 30, pp. 1–5, Aug. 2016.

[183] I. Sbai and S. Krichen, “A real-time decision support system for big data analytic: A case of dynamic vehicle routing problems,” Proc. Comput. Sci., vol. 176, pp. 938–947, Oct. 2020.

[184] I. Triguero, G. P. Figueredo, M. Mesgarpour, J. M. Garibaldi, and R. I. John, “Vehicle incident hot spots identification: An approach for big data,” in Proc. IEEE Trustcom/BigDataSE/ICESS, Aug. 2017, pp. 901–908.

[185] O. F. Ademola, S. Misra, and A. Agrawal, “Improving real-time intelligent transportation systems in predicting road accident,” in Proc. Int. Conf. Recent Innov. Comput., 2022, pp. 225–239.

[186] A. Yoo, S. Shin, J. Lee, and C. Moon, “Implementation of a sensor big data processing system for autonomous vehicles in the C-ITS environment,” Appl. Sci., vol. 10, no. 21, p. 7858, Nov. 2020.

[187] E. Priyanka, “Real-time performance analysis of multiple parameters of automotive sensor’s CAN data to predict vehicle driving efficiency,” Int. J. Comput. Digit. Syst., vol. 11, no. 1, pp. 1337–1357, Apr. 2022.

[188] R. O. Sinnott, L. Morandini, and S. Wu, “SMASH: A cloud-based architecture for big data processing and visualization of traffic data,” in Proc. IEEE Int. Conf. Data Sci. Data Intensive Syst., Dec. 2015, pp. 53–60.

[189] C. Anastasiou, J. Lin, C. He, Y.-Y. Chiang, and C. Shahabi, “ADMSv2: A modern architecture for transportation data management and analysis,” in Proc. 2nd ACM SIGSPATIAL Int. Workshop Adv. Resilient Intell. Cities, Nov. 2019, pp. 25–28.

[190] P. Figueiras, G. Guerreiro, R. Silva, R. Costa, and R. Jardim-Gonçalves, “Data processing and harmonization for intelligent transportation sys- tems: An application scenario on highway traffic flows,” in Learning Systems: From Theory to Practice. Cham, Switzerland: Springer, 2018, pp. 281–301.

[191] A. Daaif, O. Bouattane, M. Youssfi, and S. M. Snineh, “An efficient traffic monitoring model using a stream processing platform based on smart highways events generator,” in Lecture Notes In Real-Time Intelligent Systems. Cham, Switzerland: Springer, 2019.

[192] H. Khazaei, R. Veleda, M. Litoiu, and A. Tizghadam, “Realtime big data analytics for event detection in highways,” in Proc. IEEE World Forum Internet Things, 2016, pp. 472–477.

[193] F.-H. Tseng, J.-H. Hsueh, C.-W. Tseng, Y.-T. Yang, H.-C. Chao, and L.-D. Chou, “Congestion prediction with big data for real-time highway traffic,” IEEE Access, vol. 6, pp. 57311–57323, 2018.

[194] H. Khalajzadeh, A. J. Simmons, M. Abdelrazek, J. Grundy, J. Hosking, and Q. He, “An end-to-end model-based approach to support big data analytics development,” J. Comput. Lang., vol. 58, Jun. 2020, Art. no. 100964.

[195] A. Sundareswaran and L. K., “Real-time vehicle traffic prediction in apache spark using ensemble learning for deep neural networks,” Int. J. Intell. Inf. Technol., vol. 16, no. 4, pp. 19–36, Oct. 2020.

[196] J. Yu, Z. Fu, and M. Sarwat, “Dissecting GeoSparkSim: A scalable microscopic road network traffic simulator in apache spark,” Distrib. Parallel Databases, vol. 38, no. 4, pp. 963–994, Dec. 2020.

[197] Z. Fu, J. Yu, and M. Sarwat, “Demonstrating GeoSparkSim: A scalable microscopic road network traffic simulator based on apache spark,” in Proc. 16th Int. Symp. Spatial Temporal Databases, Aug. 2019, pp. 186–189.

[198] M. Babar and F. Arif, “Real-time data processing scheme using big data analytics in Internet of Things based smart transportation environment,” J. Ambient Intell. Humanized Comput., vol. 10, no. 10, pp. 4167–4177, Oct. 2019.

[199] W. Tärneberg, V. Chandrasekaran, and M. Humphrey, “Experiences creating a framework for smart traffic control using AWS IoT,” in Proc. IEEE/ACM 9th Int. Conf. Utility Cloud Comput. (UCC), Dec. 2016, pp. 63–69.

[200] M. M. Rathore, A. Ahmad, A. Paul, and U. K. Thikshaja, “Exploiting real-time big data to empower smart transportation using big graphs,” in Proc. IEEE Region 10 Symp. (TENSYMP), May 2016, pp. 135–139.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 37

[201] C. Castellanos, B. Perez, and D. Correal, “Smart transportation: A reference architecture for big data analytics,” in Smart Cities: A Data Analytics Perspective. Cham, Switzerland: Springer, 2021.

[202] G. Vonitsanos, T. Panagiotakopoulos, A. Kanavos, and A. Tsakalidis, “Forecasting air flight delays and enabling smart airport services in apache spark,” in Proc. IFIP Int. Conf. Artif. Intell. Appl. Innov., 2021, pp. 407–417.

[203] J. Chen, Y. Cao, and D. Sun, “Modeling, optimization, and operation of large-scale air traffic flow management on spark,” J. Aerosp. Inf. Syst., vol. 14, no. 9, pp. 504–516, Sep. 2017.

[204] S. Ji and Q. Sun, “Low-carbon planning and design in B&R logistics service: A case study of an e-commerce big data platform in China,” Sustainability, vol. 9, no. 11, p. 2052, Nov. 2017.

[205] C. K. Leung, P. Braun, C. S. H. Hoi, J. Souza, and A. Cuzzocrea, “Urban analytics of big transportation data for supporting smart cities,” in Proc. Big Data Anal. Knowl. Discovery, 2019, pp. 24–33.

[206] M. M. Rathore, S. Attique Shah, A. Awad, D. Shukla, S. Vimal, and A. Paul, “A cyber-physical system and graph-based approach for transportation management in smart cities,” Sustainability, vol. 13, no. 14, p. 7606, Jul. 2021.

[207] H. Antunes, P. Figueiras, R. Costa, J. Teixeira, and R. Jardim- Gonçalves, “Big data analytics for extracting mobility patterns in a large urban center,” in Proc. ICIST Int. Conf. Inf. Soc. Techology, 2019, pp. 124–129.

[208] M. Luckner, M. Grzenda, R. Kunicki, and J. Legierski, “IoT archi- tecture for urban data-centric services and applications,” ACM Trans. Internet Technol., vol. 20, no. 3, pp. 1–30, Aug. 2020.

[209] L. Oneto et al., “Train delay prediction systems: A big data analytics perspective,” Big Data Res., vol. 11, pp. 54–64, Mar. 2018.

[210] M.-T. Vo, T.-V. Tran, D.-T. Pham, and T.-H. Do, “A practical real-time flight delay prediction system using big data technology,” in Proc. IEEE Int. Conf. Commun., Netw. Satell. (COMNETSAT), Nov. 2022, pp. 160–167.

[211] J. Mulerikkal, S. Thandassery, D. M. Dixon K, V. Rejathalal, and B. Ayyappan, “JP-DAP: An intelligent data analytics platform for metro rail transport systems,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 7, pp. 9146–9156, Jul. 2022.

[212] N. T. Lapatta, “Ecotourism recommendations based on sentiments using skyline query and apache-spark,” J. Social Sci., vol. 3, no. 3, pp. 534–546, May 2022.

[213] R. Ambolkar, A. Bhagat, B. Buga, and S. Gharat, “Hotel recommen- dation system using advanced efficiency and accuracy with modified BERT technique,” in Proc. 2nd Int. Conf. Artif. Intell. Smart Energy (ICAIS), Feb. 2022, pp. 1747–1752.

[214] M. Malkawi and R. Alhajj, “Real-time web-based international flight tickets recommendation system via apache spark,” in Proc. IEEE 24th Int. Conf. Inf. Reuse Integr. Data Sci. (IRI), Aug. 2023, pp. 279–282.

[215] V. Bellandi, P. Ceravolo, E. Damiani, and E. Tacchini, “Designing a recommender system for touristic activities in a big data as a service platform,” in Innovations in Big Data Mining and Embedded Knowledge. Cham, Switzerland: Springer, 2019.

[216] A. Petrozziello and I. Jordanov, “Data analytics for online travelling recommendation system: A case study,” in Proc. Model., Identificat. Control, Jordan, 2017, pp. 1–7.

[217] P. Suanpang and P. Jamjuntr, “The integration of a big data framework and a mobile application on the iOS platform to support smart tourism,” Int. J. Mach. Learn. Comput., vol. 10, no. 6, pp. 714–722, Dec. 2020.

[218] R. S. Kumar, C. Rani, and P. G. Kumar, “Design of smart logistics transportation system using MapReduce intelligent water drops algo- rithm in Hadoop environment,” Int. J. Logistics Syst. Manage., vol. 31, no. 2, pp. 249–266, 2018.

[219] S. Hadavi, S. Verlinde, W. Verbeke, C. Macharis, and T. Guns, “Monitoring urban-freight transport based on GPS trajectories of heavy-goods vehicles,” IEEE Trans. Intell. Transp. Syst., vol. 20, no. 10, pp. 3747–3758, Oct. 2019.

[220] N. Dimokas, D. Margaritis, M. Gaetani, and A. Favenza, “A cloud- based big data architecture for an intelligent green truck,” in Mobility- as-a-Service Systems. Cham, Switzerland: Springer, 2021.

[221] A. Belfadel et al., “A conceptual digital twin framework for city logistics,” Comput., Environ. Urban Syst., vol. 103, Jul. 2023, Art. no. 101989.

[222] S. Selvaraj, H. Kim, and E. Choi, “Offline-to-online service and big data analysis for end-to-end freight management system,” J. Inf. Process. Syst., vol. 16, no. 2, p. 377, 2020.

[223] N. Provatas et al., “SELIS BDA: Big data analytics for the logistics domain,” in Proc. IEEE Int. Conf. Big Data (Big Data), Dec. 2020, pp. 2416–2425.

[224] N. Silva et al., “Advancing logistics 4.0 with the implementation of a big data warehouse: A demonstration case for the automotive industry,” Electronics, vol. 10, no. 18, p. 2221, Sep. 2021.

[225] S. Ruan et al., “Filling delivery time automatically based on couriers’ trajectories,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 2, pp. 1528–1540, Feb. 2023.

[226] D. Sugathan, M. Revanth, V. P. Kumar, S. Desai, and L. Keshava, “ESPY carting systems: The real -time tracking mechanism architecture,” in Proc. Int. Conf. Design Innov. 3Cs Compute Com- municate Control (ICDI3C), Jun. 2021, pp. 184–190.

[227] H. Hong, X. Li, D. He, Y. Zhang, and M. Wang, “Crowdsourcing incentives for multi-hop urban parcel delivery network,” IEEE Access, vol. 7, pp. 26268–26277, 2019.

[228] M. A. Mhand, A. Boulmakoul, and H. Badir, “Scalable and distributed architecture based on apache spark streaming and PROM6 for process- ing RoRo terminals logs,” in Proc. New Challenges Data Sci., Acts 2nd Conf. Moroccan Classification Soc., Mar. 2019, pp. 1–4.

[229] K. R. Pandhare and M. A. Shah, “Real time road traffic event detection using Twitter and spark,” in Proc. Int. Conf. Inventive Commun. Comput. Technol. (ICICCT), Mar. 2017, pp. 445–449.

[230] E. Alomari, I. Katib, A. Albeshri, T. Yigitcanlar, and R. Mehmood, “Iktishaf+: A big data tool with automatic labeling for road traffic social sensing and event detection using distributed machine learning,” Sensors, vol. 21, no. 9, p. 2993, Apr. 2021.

[231] A. Seal and A. Mukherjee, “Real time accident prediction and related congestion control using spark streaming in an AWS EMR cluster,” in Proc. SoutheastCon, Apr. 2019, pp. 1–7.

[232] S. V. Kovalchuk, E. Krotov, P. A. Smirnov, D. A. Nasonov, and A. N. Yakovlev, “Distributed data-driven platform for urgent decision making in cardiological ambulance control,” Future Gener. Comput. Syst., vol. 79, pp. 144–154, Feb. 2018.

[233] M. M. Rathore, A. Ahmad, A. Paul, J. Wan, and D. Zhang, “Real-time medical emergency response system: Exploiting IoT and big data for public health,” J. Med. Syst., vol. 40, no. 12, pp. 1–10, Dec. 2016.

[234] S. Alotaibi, R. Mehmood, I. Katib, O. Rana, and A. Albeshri, “Sehaa: A big data analytics tool for healthcare symptoms and diseases detection using Twitter, apache spark, and machine learning,” Appl. Sci., vol. 10, no. 4, p. 1398, Feb. 2020.

[235] M. A. R. Noori and R. Mehra, “Fire emergency detection from Twitter using supervised principal,” in Proc. IEEE 15th Int. Conf. Ind. Inf. Syst. (ICIIS), Nov. 2020, pp. 403–408.

[236] D. Roy, R. Srivastava, M. Jat, and M. S. Karaca, “A complete overview of analytics techniques: Descriptive, predictive, and prescriptive,” in Decision Intelligence Analytics and the Implementation of Strategic Bus. Cham, Switzerland: Springer, 2022.

[237] M. G. Kibria, K. Nguyen, G. P. Villardi, O. Zhao, K. Ishizu, and F. Kojima, “Big data analytics, machine learning, and artificial intel- ligence in next-generation wireless networks,” IEEE Access, vol. 6, pp. 32328–32338, 2018.

[238] Y. Gorishniy, I. Rubachev, V. Khrulkov, and A. Babenko, “Revisiting deep learning models for tabular data,” in Proc. Neural Inf. Process. Syst., vol. 34, 2021, pp. 18932–18943.

[239] S. Cao, W. Lu, and Q. Xu, “Deep neural networks for learning graph representations,” in Proc. AAAI Conf. Artif. Intell., 2016, pp. 1145–1152.

[240] Q. Li et al., “A survey on text classification: From shallow to deep learning,” 2020, arXiv:2008.00364.

[241] F. Shaheen, B. Verma, and M. Asafuddoula, “Impact of automatic feature extraction in deep learning architecture,” in Proc. Int. Conf. Digit. Image Comput., Techn. Appl. (DICTA), Nov. 2016, pp. 1–8.

[242] M. M. Najafabadi, F. Villanustre, T. M. Khoshgoftaar, N. Seliya, R. Wald, and E. Muharemagic, “Deep learning applications and chal- lenges in big data analytics,” J. Big Data, vol. 2, no. 1, pp. 1–21, Dec. 2015.

[243] L. Faubel, K. Schmid, and H. Eichelberger, “MLOps challenges in Industry 4.0,” Social Netw. Comput. Sci., vol. 4, no. 6, p. 828, Oct. 2023.

[244] G. Symeonidis, E. Nerantzis, A. Kazakis, and G. A. Papakostas, “MLOps–definitions, tools and challenges,” in Proc. IEEE 12th Annu. Comput. Commun. Workshop Conf. (CCWC), Jan. 2022, pp. 0453–0460.

[245] E. Raj, D. Buffoni, M. Westerlund, and K. Ahola, “Edge MLOps: An automation framework for AIoT applications,” in Proc. IEEE Int. Conf. Cloud Eng. (IC2E), Oct. 2021, pp. 191–200.

[246] J. Pineda-Jaramillo and F. Viti, “MLOps in freight rail operations,” Eng. Appl. Artif. Intell., vol. 123, Aug. 2023, Art. no. 106222.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

38 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[247] S. Koh et al., “Real-time deep reinforcement learning based vehicle navigation,” Appl. Soft Comput., vol. 96, Nov. 2020, Art. no. 106694.

[248] G. Menghani, “Efficient deep learning: A survey on making deep learning models smaller, faster, and better,” ACM Comput. Surveys, vol. 55, no. 12, pp. 1–37, Dec. 2023.

[249] D. Kreuzberger, N. Kühl, and S. Hirschl, “Machine learning opera- tions (MLOps): Overview, definition, and architecture,” IEEE Access, vol. 11, pp. 31866–31879, 2023.

[250] D. Sculley et al., “Hidden technical debt in machine learning systems,” in Proc. Neural Inf. Process. Syst., 2015, pp. 2503–2511.

[251] L. Cardoso Silva et al., “Benchmarking machine learning solutions in production,” in Proc. 19th IEEE Int. Conf. Mach. Learn. Appl. (ICMLA), Dec. 2020, pp. 626–633.

[252] A. Banerjee, C.-C. Chen, C.-C. Hung, X. Huang, Y. Wang, and R. Chevesaran, “Challenges and experiences with MLOps for perfor- mance diagnostics in hybrid-cloud enterprise software deployments,” in Proc. USENIX Conf. Oper. Mach. Learn., 2020, pp. 1–2.

[253] B. Haibe-Kains, G. A. Adam, A. Hosny, and F. Khodakarami, “Transparency and reproducibility in artificial intelligence,” Nature, vol. 586, no. 7829, pp. E14–E16, 2020.

[254] D. Adkins et al., “Method cards for prescriptive machine-learning transparency,” in Proc. 1st Int. Conf. AI Engineering: Softw. Eng. AI, May 2022, pp. 90–100.

[255] S. Alla, S. K. Adari, S. Alla, and S. K. Adari, “What is MLOps?” in Beginning MLOps With MLFlow: Deploy Models in AWS SageMaker, Google Cloud, and Microsoft Azure. Cham, Switzerland: Springer, 2021.

[256] D. Chicco, L. Oneto, and E. Tavazzi, “Eleven quick tips for data clean- ing and feature engineering,” PLOS Comput. Biol., vol. 18, no. 12, Dec. 2022, Art. no. e1010718.

[257] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Aug. 2016, pp. 785–794.

[258] Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: Analysis, applications, and prospects,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 12, pp. 6999–7019, Dec. 2022.

[259] A. Vaswani et al., “Attention is all you need,” in Proc. Neural Inf. Process. Syst., 2017, pp. 1–11.

[260] S. Shalev-Shwartz and S. Ben-David, Understanding Machine Learn- ing: From Theory to Algorithms. Cambridge, U.K.: Cambridge Univ. Press, 2014.

[261] H. White, “A reality check for data snooping,” Econometrica, vol. 68, no. 5, pp. 1097–1126, Sep. 2000.

[262] C. Zhang and P. Patras, “Long-term mobile traffic forecasting using deep spatio-temporal neural networks,” in Proc. 18th ACM Int. Symp. Mobile Ad Hoc Netw. Comput., Jun. 2018, pp. 231–240.

[263] Metaflow. Accessed: Feb. 5, 2025. [Online]. Available: https:// metaflow.org/

[264] Amazon Web Services (AWS). Accessed: Feb. 5, 2025. [Online]. Avail- able: https://aws.amazon.com/?nc1=h ls

[265] ZenML. Accessed: Feb. 5, 2025. [Online]. Available: https:// www.zenml.io/

[266] BentoML. Accessed: Feb. 5, 2025. [Online]. Available: https:// www.bentoml.com/

[267] Kubeflow. Accessed: Feb. 5, 2025. [Online]. Available: https:// www.kubeflow.org/

[268] (2025). Apollo: An Open Autonomous Driving Platform. Accessed: Jun. 15, 2026. [Online]. Available: https://github.com/ApolloAuto/apollo

[269] Vertex AI Platform. Accessed: Feb. 5, 2025. [Online]. Available: https:// cloud.google.com/vertex-ai

[270] IBM Watson Studio. Accessed: Feb. 5, 2025. [Online]. Available: https://www.ibm.com/it-it/products/watson-studio

[271] Azure Machine Learning. Accessed: Feb. 5, 2025. [Online]. Available: https://azure.microsoft.com/en-us/products/machine-learning

[272] Aws Sagemaker. Accessed: Oct. 23, 2024. [Online]. Available: https:// aws.amazon.com/it/sagemaker/

[273] MATLAB. Accessed: Feb. 5, 2025. [Online]. Available: https:// ch.mathworks.com/products/MATLAB.html

[274] Simulink. Accessed: Feb. 5, 2025. [Online]. Available: https:// ch.mathworks.com/products/simulink.html

[275] K. Jin, J. Wi, E. Lee, S. Kang, S. Kim, and Y. Kim, “TrafficBERT: Pre-trained model with large-scale data for long-range traffic flow forecasting,” Expert Syst. Appl., vol. 186, Dec. 2021, Art. no. 115738.

[276] C. Haertel, D. Staegemann, C. Daase, M. Pohl, A. Nahhas, and K. Turowski, “MLOps in data science projects: A review,” in Proc. IEEE Int. Conf. Big Data (BigData), Dec. 2023, pp. 2396–2404.

[277] C. Conlan, J. Oakley, G. V. Demirci, A. Sfyridis, and H. Ferhatos- manoglu, “Real-time spatio-temporal forecasting with dynamic urban event and vehicle-level flow information,” in Proc. CEUR Workshop, vol. 3379, 2023, pp. 1–11.

[278] S. Mishra and P. Palanisamy, “Autonomous advanced aerial mobility—An end-to-end autonomy framework for UAVs and beyond,” IEEE Access, vol. 11, pp. 136318–136349, 2023.

[279] A. L. Ferreira and J. M. Fernandes, “Mlops for developing machine- learning-enhanced automotive applications,” IEEE Softw., vol. 42, no. 1, pp. 34–41, Jan. 2024.

[280] A. Hartmannsgruber et al., “Improving 3D object detection for autonomous driving-a case study of data-driven development,” in Proc. Int. ATZ Conf., 2024, pp. 73–87.

[281] B. Steffen, J. Zohren, U. Pazarci, F. Kullmann, and H. Weißenfels, “Continuous integration of neural networks in autonomous systems,” in Proc. Int. Conf. Eng. Comput.-Based Syst., 2023.

[282] M. Weber, J. Schniertshauer, L. Przybilla, A. Hein, J. Weking, and H. Krcmar, “Streamlining the operation of AI systems: Examining MLOps maturity at an automotive firm,” in Proc. Annu. Hawaii Int. Conf. Syst. Sci., 2024, pp. 3771–3780.

[283] K. Socha, M. Borg, and J. Henriksson, “SMIRK: A machine learning- based pedestrian automatic emergency braking system with a complete safety case,” Softw. Impacts, vol. 13, Aug. 2022, Art. no. 100352.

[284] J. S. Angarita-Zapata, A. D. Masegosa, and I. Triguero, “General- purpose automated machine learning for transportation: A case study of auto-sklearn for traffic forecasting,” in Proc. Int. Conf. Inf. Process. Manage. Uncertainty Knowl.-Based Syst., 2020, pp. 728–744.

[285] J. Oakley, C. Conlan, G. V. Demirci, A. Sfyridis, and H. Ferhatos- manoglu, “Foresight plus: Scalable serverless real-time spatio-temporal traffic forecasting,” GeoInformatica, vol. 28, no. 4, pp. 649–677, 2023.

[286] S. K. Karmaker, M. M. Hassan, M. J. Smith, L. Xu, C. Zhai, and K. Veeramachaneni, “AutoML to date and beyond: Challenges and opportunities,” ACM Comput. Surv., vol. 54, no. 8, pp. 1–36, Nov. 2022.

[287] M. K. Sengul, C. Tarhan, and V. Tecim, “Application of intelligent transportation system data using big data technologies,” in Proc. Innov. Intell. Syst. Appl. Conf. (ASYU), Sep. 2022, pp. 1–6.

[288] N. S. Guzhva, B. Ali, K. S. Bakulev, R. N. Sadekov, and A. V. Sholokhov, “Evaluating the accuracy of tram positioning system in high-rise building environment using data from visual geoinforma- tion systems,” in Proc. 30th Saint Petersburg Int. Conf. Integr. Navig. Syst. (ICINS), May 2023, pp. 1–5.

[289] I. B. Urias and R. Rossi, “Mlops and microservices frameworks in the perspective of smart cities,” EAI Endorsed Trans. Smart Cities, vol. 7, no. 3, pp. 1–11, 2023.

[290] L. Faubel and K. Schmid, “MLOps: A multiple case study in Industry 4.0,” 2024, arXiv:2407.09107.

[291] J. Cha et al., “Development of mlops platform based on power source analysis for considering manufacturing environment changes in real- time processes,” in Proc. Int. Conf. Human-Comput. Interact., 2023, pp. 224–236.

[292] A.-R. Morariu, T. Ahmad, B. Iancu, J. Poikonen, and J. Björkqvist, “Analysing MLOps and its applicability in the maritime domain through a systematic mapping study,” in Proc. IEEE 7th Int. Conf. Ind. Cyber-Physical Syst. (ICPS), May 2024, pp. 1–8.

[293] A. Khan and A. Ahmed, “Optimizing retail operations, inventory management and sales forecasting with big data and AI in China,” Emerg. Trends Mach. Intell. Big Data, vol. 16, no. 1, pp. 18–37, 2024.

[294] A. K. Jana, “Framework for automated machine learning workflows: Building end-to-end MLOps tools for scalable systems on AWS,” J. Artif. Intell., Mach. Learn. Data Sci., vol. 1, no. 3, pp. 575–579, Sep. 2023.

[295] Meet Michelangelo: Uber’s Machine Learning Platform. Accessed: Oct. 13, 2024. [Online]. Available: https://www.uber.com/en-IT/blog/ michelangelo-machine-learning-platform/

[296] E. J. Piedad et al., “Intelligent flood detection using traffic surveillance images based on convolutional neural network and image parsing,” in Proc. IEEE Int. Conf. Comput. (ICOCO), Nov. 2022, pp. 220–225.

[297] E. Portalés-Julià, G. Mateo-Garcı́a, C. Purcell, and L. Gómez-Chova, “Global flood extent segmentation in optical satellite images,” Sci. Rep., vol. 13, no. 1, p. 20316, Nov. 2023.

[298] Y. Remil, A. Bendimerad, R. Mathonat, and M. Kaytoue, “AIOps solutions for incident management: Technical guidelines and a com- prehensive literature review,” 2024, arXiv:2404.01363.

[299] G. Shankar, M. K. Geetha, and P. Ezhumalai, “Disaster management systems: Utilizing YOLOv9 for precise monitoring of river flood flow levels using video surveillance,” Social Netw. Comput. Sci., vol. 6, no. 3, p. 288, Mar. 2025.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 39

[300] Z.-Q. Zhao, P. Zheng, S.-T. Xu, and X. Wu, “Object detection with deep learning: A review,” IEEE Trans. Neural Netw. Learn. Syst., vol. 30, no. 11, pp. 3212–3232, Nov. 2019.

[301] W. X. Zhao et al., “A survey of large language models,” 2023, arXiv:2303.18223.

[302] X. Zhou, M. Liu, B. L. Zagar, E. Yurtsever, and A. C. Knoll, “Vision language models in autonomous driving and intelligent transportation systems,” 2023, arXiv:2310.14414.

[303] S. M. Moghadas, Y. Lyu, B. Cornelis, A. Alahi, and A. Munteanu, “Strada-LLM: Graph LLM for traffic prediction,” 2024, arXiv:2410.20856.

[304] D. Le, A. Yunusoglu, K. Tiwari, M. Isik, and I. Dikmen, “Multimodal LLM for intelligent transportation systems,” 2024, arXiv:2412.11683.

[305] M. Movahedi and J. Choi, “The crossroads of LLM and traffic control: A study on large language models in adaptive traffic signal control,” IEEE Trans. Intell. Transp. Syst., vol. 26, no. 2, pp. 1701–1716, Feb. 2025.

[306] J. D. Kenton, M.-W. Devlin, L. Chang, and K. Toutanova, “BERT: Pre- training of deep bidirectional transformers for language understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186.

[307] B. Wang, Z. Cai, M. M. Karim, C. Liu, and Y. Wang, “Traffic perfor- mance GPT (TP-GPT): Real-time data informed intelligent chatbot for transportation surveillance and management,” 2024, arXiv:2405.03076.

[308] J. Yang, X. Dai, Y. Lv, L. Kovács, and F.-Y. Wang, “Transrag for parallel transportation: Toward reliable and trustworthy transportation systems via retrieval-augmented generation,” Frontiers Inf. Technol. & Electron. Eng., vol. 26, pp. 1–7, Jan. 2024.

[309] S. Wandelt, C. Zheng, S. Wang, Y. Liu, and X. Sun, “Large language models for intelligent transportation: A review of the state of the art and challenges,” Appl. Sci., vol. 14, no. 17, p. 7455, Aug. 2024.

[310] Melbourne Airport - Flight Information, Shopping & Parking. Accessed: Feb. 5, 2025. [Online]. Available: https:// www.melbourneairport.com.au

[311] J. Diaz-De-Arcaya, J. López-De-Armentia, R. Miñón, I. L. Ojanguren, and A. I. Torre-Bastida, “Large language model operations (LLMOps): Definition, challenges, and lifecycle management,” in Proc. 9th Int. Conf. Smart Sustain. Technol. (SpliTech), Jun. 2024, pp. 1–4.

[312] M. Sinha, S. Menon, and R. Sagar, “LLMOps: Definitions, framework and best practices,” in Proc. Int. Conf. Electr., Comput. Energy Technol. (ICECET, Jul. 2024, pp. 1–6.

[313] L. Weidinger et al., “Ethical and social risks of harm from language models,” 2021, arXiv:2112.04359.

[314] T. Rebedea, R. Dinu, M. Sreedhar, C. Parisien, and J. Cohen, “Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails,” 2023, arXiv:2310.10501.

[315] D. Narayanan et al., “Efficient large-scale language model training on GPU clusters using megatron-LM,” in Proc. SC21: Int. Conf. High Perform. Comput., Netw., Storage Anal., Nov. 2021, pp. 1–14.

[316] H. Naveed et al., “A comprehensive overview of large language models,” 2023, arXiv:2307.06435.

[317] E. J. Hu et al., “LoRA: Low-rank adaptation of large language models,” 2021, arXiv:2106.09685.

[318] B. Min et al., “Recent advances in natural language processing via large pre-trained language models: A survey,” ACM Comput. Surveys, vol. 56, no. 2, pp. 1–40, Feb. 2024.

[319] C. Zhou et al., “A comprehensive survey on pretrained foundation models: A history from BERT to ChatGPT,” Int. J. Mach. Learn. Cybern., vol. 16, no. 12, pp. 1–65, Dec. 2025.

[320] S. Yang, O. Nachum, Y. Du, J. Wei, P. Abbeel, and D. Schuurmans, “Foundation models for decision making: Problems, methods, and opportunities,” 2023, arXiv:2303.04129.

[321] C. K. Tantithamthavorn, F. Palomba, F. Khomh, and J. J. Chua, “MLOps, LLMOps, FMOps, and beyond,” IEEE Softw., vol. 42, no. 1, pp. 26–32, Jan. 2025.

[322] A. Bhattacharjee, “Overview of GenAIOPS,” in A Practical Guide to Generative AI Using Amazon Bedrock, 2025, pp. 351–368.

[323] C. Shi, P. Liang, Y. Wu, T. Zhan, and Z. Jin, “Maximizing user experience with llmops-driven personalized recommendation systems,” 2024, arXiv:2404.00903.

[324] M. M. John, H. H. Olsson, and J. Bosch, “Towards MLOps: A framework and maturity model,” in Proc. 47th Euromicro Conf. Softw. Eng. Adv. Appl. (SEAA), Sep. 2021, pp. 1–8.

[325] R. Shan and T. Shan, “Enterprise LLMOps: Advancing large language models operations practice,” in Proc. IEEE Cloud Summit, Jun. 2024, pp. 143–148.

[326] D. Luo, C. Feng, Y. Nong, and Y. Shen, “AUTOM3L: An automated multimodal machine learning framework with large language models,” in Proc. ACM Int. Conf. Multimedia, 2024, pp. 8586–8594.

[327] I. Cheong, K. Xia, K. J. K. Feng, Q. Z. Chen, and A. X. Zhang, “(A)I am not a lawyer, but...: Engaging legal experts towards respon- sible LLM policies for legal advice,” in Proc. ACM Conf. Fairness Accountability Transparency, Jun. 2024, pp. 2454–2469.

[328] S. Goyal et al., “HealAI: A healthcare LLM for effective medical documentation,” in Proc. 17th ACM Int. Conf. Web Search Data Mining, Mar. 2024, pp. 1167–1168.

[329] B. Chen, Z. Zhang, N. Langrené, and S. Zhu, “Unleashing the potential of prompt engineering for large language models,” 2023, arXiv:2310.14735.

[330] J. White et al., “A prompt pattern catalog to enhance prompt engineer- ing with ChatGPT,” 2023, arXiv:2302.11382.

[331] Q. Dong et al., “A survey on in-context learning,” 2022, arXiv:2301.00234.

[332] W. Fan et al., “A survey on RAG meeting LLMs: Towards retrieval- augmented large language models,” in Proc. 30th ACM SIGKDD Conf. Knowl. Discovery Data Mining, Aug. 2024, pp. 6491–6501.

[333] J. Z. Pan et al., “Large language models and knowledge graphs: Opportunities and challenges,” 2023, arXiv:2308.06374.

[334] K. Ruan, X. Wang, and X. Di, “From Twitter to reasoner: Understand mobility travel modes and sentiment using large language models,” 2024, arXiv:2411.02666.

[335] P. Bhandari, A. Anastasopoulos, and D. Pfoser, “Urban mobility assess- ment using LLMs,” in Proc. 32nd ACM Int. Conf. Adv. Geographic Inf. Syst., Oct. 2024, pp. 67–79.

[336] Y. Dong et al., “Building guardrails for large language models,” 2024, arXiv:2402.01822.

[337] C.-Y. Lin and E. Hovy, “Automatic evaluation of summaries using N-gram co-occurrence statistics,” in Proc. Conf. North Amer. Chapter Assoc. Comput. Linguistics Human Lang. Technol.-NAACL, vol. 1, 2003, pp. 71–78.

[338] T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTScore: Evaluating text generation with BERT,” 2019, arXiv:1904.09675.

[339] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” in Proc. 40th Annu. Meeting Assoc. Comput. Linguistics, 2001, p. 311.

[340] Y. Chen et al., “Hallucination detection: Robustly discerning reliable answers in large language models,” in Proc. 32nd ACM Int. Conf. Inf. Knowl. Manage., Oct. 2023, pp. 245–255.

[341] Y. Liu et al., “Efficient detection of toxic prompts in large language models,” in Proc. 39th IEEE/ACM Int. Conf. Automated Softw. Eng., Oct. 2024, pp. 455–467.

[342] L. Ouyang et al., “Training language models to follow instructions with human feedback,” in Proc. Adv. Neural Inf. Process. Syst., 2022, pp. 27730–27744.

[343] Hugging Face. Accessed: Oct. 16, 2026. [Online]. Available: https:// huggingface.co/

[344] W. Kwon et al., “Efficient memory management for large language model serving with PagedAttention,” in Proc. 29th Symp. Operating Syst. Princ., Oct. 2023, pp. 611–626.

[345] LangChain. Accessed: Jun. 16, 2026. [Online]. Available: https:// www.langchain.com/

[346] Deepchecks. Accessed: Jun. 16, 2026. [Online]. Available: https:// github.com/deepchecks/deepchecks

[347] M. Mazeika et al., “HarmBench: A standardized evaluation frame- work for automated red teaming and robust refusal,” 2024, arXiv:2402.04249.

[348] D. B. Acharya, K. Kuppan, and B. Divya, “Agentic AI: Autonomous intelligence for complex goals—A comprehensive survey,” IEEE Access, vol. 13, pp. 18912–18936, 2025.

[349] A. K. Pati, “Agentic AI: A comprehensive survey of technolo- gies, applications, and societal implications,” IEEE Access, vol. 13, pp. 151824–151837, 2025.

[350] R. Sapkota, K. I. Roumeliotis, and M. Karkee, “AI agents vs. Agentic AI: A conceptual taxonomy, applications and challenges,” Inf. Fusion, vol. 126, Feb. 2026, Art. no. 103599.

[351] Z. Wang et al., “A survey on AgentOps: Categorization, challenges, and future directions,” 2025, arXiv:2508.02121.

[352] L. Dong, Q. Lu, and L. Zhu, “AgentOps: Enabling observability of LLM agents,” 2024, arXiv:2411.05285.

[353] Y. Liu et al., “Agent design pattern catalogue: A collection of archi- tectural patterns for foundation model based agents,” J. Syst. Softw., vol. 220, Feb. 2025, Art. no. 112278.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

40 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[354] H. Su et al., “A survey on autonomy-induced security risks in large model-based agents,” IEEE Trans. Pattern Anal. Mach. Intell., early access, Apr. 28, 2026, doi: 10.1109/TPAMI.2026.3688650.

[355] P. Rajpurkar, E. Chen, O. Banerjee, and E. J. Topol, “AI in health and medicine,” Nature Med., vol. 28, no. 1, pp. 31–38, 2022.

[356] S. Du and C. Xie, “Paradoxes of artificial intelligence in consumer markets: Ethical challenges and opportunities,” J. Bus. Res., vol. 129, pp. 961–974, May 2021.

[357] D. Kaur, S. Uslu, K. J. Rittichier, and A. Durresi, “Trustworthy artificial intelligence: A review,” ACM Comput. Surv., vol. 55, no. 2, pp. 1–38, 2022.

[358] Trust Barometer. Accessed: Feb. 15, 2025. [Online]. Available: https://www.edelman.com/sites/g/files/aatuss191/files/2019-02/ 2019 Edelman Trust Barometer Global Report.pdf

[359] A. F. Winfield, K. Michael, J. Pitt, and V. Evers, “Machine ethics: The design and governance of ethical AI and autonomous systems [Scanning the Issue],” Proc. IEEE, vol. 107, no. 3, pp. 509–517, Mar. 2019.

[360] C. Allen, G. Varner, and J. Zinser, “Prolegomena to any future artificial moral agent,” J. Experim. Theor. Artif. Intell., vol. 12, no. 3, pp. 251–261, Jul. 2000.

[361] M. Anderson and S. L. Anderson, “Geneth: A general ethical dilemma analyzer,” Paladyn, J. Behav. Robot., vol. 12, no. 3, pp. 251–261, 2018.

[362] L. Oneto, S. Ridella, and D. Anguita, “Towards algorithms and models that we can trust: A theoretical perspective,” Neurocomputing, vol. 592, Aug. 2024, Art. no. 127798.

[363] D. Franco, L. Oneto, and D. Anguita, “Mitigating robustness bias: Theoretical results and empirical evidences,” in Proc. Eur. Symp. Artif. Neural Netw., Comput. Intell. Mach. Learn., 2023, pp. 187–192.

[364] X. Huang, M. Kwiatkowska, S. Wang, and M. Wu, “Safety verification of deep neural networks,” in Proc. Int. Conf. Comput. Aided Verifica- tion, 2017, pp. 3–29.

[365] A. E. Cinà, K. Grosse, A. Demontis, B. Biggio, F. Roli, and M. Pelillo, “Machine learning security against data poisoning: Are we there yet?” Computer, vol. 57, no. 3, pp. 26–34, Mar. 2024.

[366] E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy con- siderations for modern deep learning research,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 13693–13696.

[367] E. Garcı̂a-Martı́n, C. F. Rodrigues, G. D. Riley, and H. Grahn, “Estimation of energy consumption in machine learning,” J. Parallel Distrib. Comput., vol. 134, pp. 75–88, Dec. 2019.

[368] V. Kaushal, R. Iyer, S. Kothawade, R. Mahadev, K. Doctor, and G. Ramakrishnan, “Learning from less data: A unified data subset selection and active learning framework for computer vision,” in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), Jan. 2019, pp. 1289–1299.

[369] H.-L. Wei and S. Billings, “Feature subset selection and ranking for data dimensionality reduction,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 162–166, Jan. 2007.

[370] S. Zhou, H. Xu, G. Zhang, T. Ma, and Y. Yang, “Leveraging deep convolutional neural networks pre-trained on autonomous driving data for vehicle detection from roadside LiDAR data,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 22367–22377, Nov. 2022.

[371] T. Liu, S. Alibhai, J. Wang, Q. Liu, X. He, and C. Wu, “Exploring transfer learning to reduce training overhead of HPC data in machine learning,” in Proc. IEEE Int. Conf. Netw., Archit. Storage (NAS), Aug. 2019, pp. 1–7.

[372] X. Dong, S. Chen, and S. Pan, “Learning to prune deep neural networks via layer-wise optimal brain surgeon,” in Proc. Neural Inf. Process. Syst., 2017, pp. 4857–4867.

[373] H. Cheng, M. Zhang, and J. Q. Shi, “A survey on deep neural network pruning: Taxonomy, comparison, analysis, and recommendations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 12, pp. 10558–10578, Dec. 2024.

[374] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Quantized neural networks: Training neural networks with low preci- sion weights and activations,” J. Mach. Learn. Res., vol. 18, no. 187, pp. 1–30, 2018.

[375] B. Rokh, A. Azarpeyvand, and A. Khanteymoori, “A comprehen- sive survey on model quantization for deep neural networks in image classification,” ACM Trans. Intell. Syst. Technol., vol. 14, no. 6, pp. 1–50, Dec. 2023.

[376] N. P. Jouppi et al., “In-datacenter performance analysis of a tensor processing unit,” in Proc. ACM/IEEE 44th Annu. Int. Symp. Comput. Archit. (ISCA), Jun. 2017, pp. 1–12.

[377] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient processing of deep neural networks: A tutorial and survey,” Proc. IEEE, vol. 105, no. 12, pp. 2295–2329, Dec. 2017.

[378] M. Ribeiro, K. Grolinger, and M. A. M. Capretz, “MLaaS: Machine learning as a service,” in Proc. IEEE 14th Int. Conf. Mach. Learn. Appl. (ICMLA), Dec. 2015, pp. 896–902.

[379] Y. Yao, Z. Xiao, B. Wang, B. Viswanath, H. Zheng, and B. Y. Zhao, “Complexity vs. Performance: Empirical analysis of machine learning as a service,” in Proc. Internet Meas. Conf., Nov. 2017, pp. 384–397.

[380] CodeCarbon. Accessed: Feb. 15, 2025. [Online]. Available: https:// github.com/mlco2/codecarbon

[381] L. F. W. Anthony, B. Kanding, and R. Selvan, “Carbontracker: Tracking and predicting the carbon footprint of training deep learning models,” 2020, arXiv:2007.03051.

[382] S. A. Budennyy et al., “Eco2AI: Carbon emissions tracking of machine learning models as the first step towards sustainable AI,” Doklady Math., vol. 106, no. S1, pp. S118–S128, Dec. 2022.

[383] J. You, J.-W. Chung, and M. Chowdhury, “Zeus: Understanding and optimizing GPU energy consumption of DNN training,” in Proc. USENIX NSDI, 2023, pp. 119–139.

[384] Scaphandre. Accessed: Feb. 6, 2025. [Online]. Available: https:// hubblo-org.github.io/scaphandre-documentation/index.html

[385] A. Noureddine, “PowerJoular and JoularJX: Multi-platform software power monitoring tools,” in Proc. 18th Int. Conf. Intell. Environments (IE), Jun. 2022, pp. 1–4.

[386] T. Orekondy, B. Schiele, and M. Fritz, “Knockoff nets: Stealing functionality of black-box models,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2019, pp. 4949–4958.

[387] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in Proc. IEEE Symp. Secur. Privacy (SP), May 2017, pp. 3–18.

[388] A. E. Cinà, A. Demontis, B. Biggio, F. Roli, and M. Pelillo, “Energy- latency attacks via sponge poisoning,” Inf. Sci., vol. 702, Jun. 2025, Art. no. 121905.

[389] M. Barreno, B. Nelson, A. D. Joseph, and J. D. Tygar, “The security of machine learning,” Mach. Learn., vol. 81, pp. 121–148, May 2010.

[390] S. Marro and M. Lombardi, “Computational asymmetries in robust classification,” in Proc. Int. Conf. Mach. Learn., 2023, pp. 24082–24138.

[391] A. E. Cinà et al., “Attackbench: Evaluating gradient-based attacks for adversarial examples,” in Proc. AAAI Conf. Artif. Intell., 2025, pp. 2600–2608.

[392] T. Sun et al., “SHIFT: A synthetic driving dataset for continuous multi- task domain adaptation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2022, pp. 21339–21350.

[393] I. Shumailov, Y. Zhao, D. Bates, N. Papernot, R. Mullins, and R. Ander- son, “Sponge examples: Energy-latency attacks on neural networks,” in Proc. IEEE Eur. Symp. Secur. Privacy (EuroS&P), Sep. 2021, pp. 212–231.

[394] J. Rauber, R. Zimmermann, M. Bethge, and W. Brendel, “Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX,” J. Open Source Softw., vol. 5, no. 53, p. 2607, Sep. 2020.

[395] M. Melis, A. Demontis, M. Pintor, A. Sotgiu, and B. Biggio, “secml: A Python library for secure and explainable machine learning,” 2019, arXiv:1912.10013.

[396] N. Papernot et al., “Technical report on the cleverhans v2.1.0 adver- sarial examples library,” 2018, arXiv:1610.00768.

[397] Adversarial Robustness Toolbox. Accessed: Feb. 15, 2025. [Online]. Available: https://github.com/Trusted-AI/adversarial-robustness- toolbox

[398] A. Chakraborty, M. Alam, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay, “Adversarial attacks and defences: A survey,” 2018, arXiv:1810.00069.

[399] Y. Zhu, Y. Zhao, Z. Hu, T. Luo, and L. He, “A review of black-box adversarial attacks on image classification,” Neurocomputing, vol. 610, Dec. 2024, Art. no. 128512.

[400] A. Athalye, N. Carlini, and D. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” in Proc. Int. Conf. Mach. Learn., 2018, pp. 274–283.

[401] P. Chao et al., “JailbreakBench: An open robustness benchmark for jailbreaking large language models,” 2024, arXiv:2404.01318.

[402] F. Croce et al., “Robustbench: A standardized adversarial robustness benchmark,” in Proc. Conf. Neural Inf. Process. Syst. Datasets Bench- marks Track, 2021, pp. 1–17.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 41

[403] Y. Li et al., “Towards robustness of deep neural networks via regularization,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Oct. 2021, pp. 7476–7485.

[404] F. Tramer, N. Carlini, W. Brendel, and A. Madry, “On adaptive attacks to adversarial example defenses,” in Proc. Neural Inf. Process. Syst., 2020, pp. 1–14.

[405] M. Pintor et al., “Indicators of attack failure: Debugging and improv- ing optimization of adversarial examples,” in Proc. Adv. Neural Inf. Process. Syst., 2022, pp. 23063–23076.

[406] A. Kantchelian, J. D. Tygar, and A. Joseph, “Evasion and hardening of tree ensemble classifiers,” in Proc. Int. Conf. Mach. Learn., 2016, pp. 2387–2396.

[407] D. Guidotti, F. Leofante, L. Pulina, and A. Tacchella, “Verification of neural networks: Enhancing scalability through pruning,” 2020, arXiv:2003.07636.

[408] K. Jia and M. Rinard, “Efficient exact verification of binarized neural networks,” in Proc. Neural Inf. Process. Syst., 2020, pp. 1782–1795.

[409] F. Ranzato and M. Zanella, “Abstract interpretation of decision tree ensemble classifiers,” in Proc. AAAI Conf. Artif. Intell., 2020, pp. 5478–5486.

[410] Y. Wang, H. Zhang, H. Chen, D. Boning, and C.-J. Hsieh, “On LP-norm robustness of ensemble decision stumps and trees,” in Proc. Int. Conf. Mach. Learn., 2020, pp. 10104–10114.

[411] S. Calzavara, L. Cazzaro, G. E. Pibiri, and N. Prezza, “Verifiable learning for robust tree ensembles,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., Nov. 2023, pp. 1850–1864.

[412] S. Ali et al., “Explainable artificial intelligence (XAI): What we know and what is left to attain trustworthy artificial intelligence,” Inf. Fusion, vol. 99, Nov. 2023, Art. no. 101805.

[413] S. Repetto, I. Maljkovic, M. Lotto, A. E. Cinà, S. Vascon, and F. Roli, “Evaluating the robustness of explainable AI in medical image recog- nition under natural and adversarial data corruption,” Mach. Learn., vol. 115, no. 1, p. 4, Jan. 2026.

[414] L. Gaur and B. M. Sahoo, Explainable Artificial Intelligence for Intelligent Transportation Systems: Ethics and Applications. Cham, Switzerland: Springer, 2022.

[415] A. Páez, “The pragmatic turn in explainable artificial intelligence (XAI),” Minds Mach., vol. 29, no. 3, pp. 441–459, Sep. 2019.

[416] Z. C. Lipton, “The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery,” Queue, vol. 16, no. 3, pp. 31–57, Jun. 2018.

[417] Y. Yao, “Explanatory pluralism in explainable AI,” in Proc. Mach. Learn. Knowl. Extraction, 2021, pp. 275–292.

[418] R. Dwivedi et al., “Explainable AI (XAI): Core ideas, techniques, and solutions,” ACM Comput. Surv., vol. 55, no. 9, pp. 1–33, Sep. 2023.

[419] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi, “A survey of methods for explaining black box models,” ACM Comput. Surveys, vol. 51, no. 5, pp. 1–42, Sep. 2019.

[420] M. Mattioli, A. E. Cinà, and M. Pelillo, “Understanding xai through the Philosopher’s lens: A historical perspective,” in Proc. Eur. Conf. Artif. Intell., 2024, pp. 987–994.

[421] A. Barredo Arrieta et al., “Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,” Inf. Fusion, vol. 58, pp. 82–115, Jun. 2020.

[422] R. Tibshirani, “Regression shrinkage and selection via the lasso,” J. Roy. Stat. Soc. Ser. B, Stat. Methodol., vol. 58, no. 1, pp. 267–288, Jan. 1996.

[423] C.-C.-J. Kuo, M. Zhang, S. Li, J. Duan, and Y. Chen, “Interpretable convolutional neural networks via feedforward design,” J. Vis. Com- mun. Image Represent., vol. 60, pp. 346–359, Apr. 2019.

[424] K. Kawaguchi, “Deep learning without poor local minima,” in Proc. Neural Inf. Process. Syst., 2016.

[425] A. Cutler, D. R. Cutler, and J. R. Stevens, “Random forests,” in Ensem- ble Machine Learning: Methods and Applications. Cham, Switzerland: Springer, 2012.

[426] Z. Liu et al., “KAN: Kolmogorov-Arnold networks,” 2024, arXiv:2404.19756.

[427] E. Soares, P. P. Angelov, B. Costa, M. P. G. Castro, S. Nageshrao, and D. Filev, “Explaining deep learning models through rule-based approximation and visualization,” IEEE Trans. Fuzzy Syst., vol. 29, no. 8, pp. 2399–2407, Aug. 2021.

[428] M. Robnik-šikonja and M. Bohanec, “Perturbation-based explana- tions of prediction models,” in Human and Machine Learning: Visible, Explainable, Trustworthy and Transparent. Cham, Switzerland: Springer, 2018.

[429] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why should I trust you?’ Explaining the predictions of any classifier,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2016, pp. 1135–1144.

[430] S. M. Lundberg and S.-I. Lee, “A unified approach to interpret- ing model predictions,” in Proc. Neural Inf. Process. Syst., 2017, pp. 4765–4774.

[431] D. V. Carvalho, E. M. Pereira, and J. S. Cardoso, “Machine learning interpretability: A survey on methods and metrics,” Electronics, vol. 8, no. 8, p. 832, Jul. 2019.

[432] M. T. Hosain, J. R. Jim, M. F. Mridha, and M. M. Kabir, “Explainable AI approaches in deep learning: Advancements, applications and challenges,” Comput. Electr. Eng., vol. 117, Jul. 2024, Art. no. 109246.

[433] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” ACM Comput. Surv., vol. 54, no. 6, pp. 1–35, Jul. 2022.

[434] Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women. Accessed: Feb. 15, 2025. [Online]. Available: https:// www.reuters.com/article/idUSKCN1MK0AG/

[435] A. Castelnovo, R. Crupi, G. Greco, D. Regoli, I. G. Penco, and A. C. Cosentini, “A clarification of the nuances in the fairness metrics landscape,” Sci. Rep., vol. 12, no. 1, p. 4209, Mar. 2022.

[436] W. Ye, G. Zheng, X. Cao, Y. Ma, and A. Zhang, “Spurious correlations in machine learning: A survey,” 2024, arXiv:2402.12715.

[437] H. Elzayn et al., “Fair algorithms for learning in allocation problems,” in Proc. Conf. Fairness, Accountability, Transparency, Jan. 2019, pp. 170–179.

[438] M. T. Islam, A. Fariha, A. Meliou, and B. Salimi, “Through the data management lens: Experimental analysis and evaluation of fair classification,” in Proc. Int. Conf. Manage. Data, Jun. 2022, pp. 232–246.

[439] S. Barocas, M. Hardt, and A. Narayanan, Fairness and Machine Learning: Limitations and Opportunities. Cambridge, MA, USA: MIT Press, 2023.

[440] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in Proc. 3rd Innov. Theor. Comput. Sci. Conf., Jan. 2012, pp. 214–226.

[441] M. J. Kusner, J. Loftus, C. Russell, and R. Silva, “Counterfactual fairness,” in Proc. Neural Inf. Process. Syst., 2017, pp. 4069–4079.

[442] S. Chiappa, “Path-specific counterfactual fairness,” in Proc. AAAI Conf. Artif. Intell., 2019, pp. 7801–7808.

[443] H. Weerts, M. Dudı́k, R. Edgar, A. Jalali, R. Lutz, and M. Madaio, “Fairlearn: Assessing and improving fairness of ai systems,” J. Mach. Learn. Res., vol. 24, no. 257, pp. 1–8, 2023.

[444] R. K. E. Bellamy et al., “AI fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias,” IBM J. Res. Develop., vol. 63, nos. 4–5, pp. 1–15, Jul. 2019.

[445] Fairness Indicators. Accessed: Feb. 6, 2025. [Online]. Available: https://www.tensorflow.org/tfx/guide/fairness indicators

[446] M. Al-Rubaie and J. M. Chang, “Privacy-preserving machine learning: Threats and solutions,” IEEE Secur. Privacy, vol. 17, no. 2, pp. 49–58, Mar. 2019.

[447] V. Sucasas, G. Mantas, F. B. Saghezchi, A. Radwan, and J. Rodriguez, “An autonomous privacy-preserving authentication scheme for intel- ligent transportation systems,” Comput. Secur., vol. 60, pp. 193–205, Jul. 2016.

[448] M. Al-Rubaie and J. M. Chang, “Reconstruction attacks against mobile- based continuous authentication systems in the cloud,” IEEE Trans. Inf. Forensics Security, vol. 11, no. 12, pp. 2648–2663, Dec. 2016.

[449] M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proc. 22nd ACM SIGSAC Conf. Comput. Commun. Secur., Oct. 2015, pp. 1322–1333.

[450] A. Machanavajjhala, D. Kifer, J. Gehrke, and M. Venkitasubramaniam, “L-diversity: Privacy beyond k-anonymity,” ACM Trans. Knowl. Dis- covery Data, vol. 1, no. 1, p. 3, 2007.

[451] L. Sweeney, “k-anonymity: A model for protecting privacy,” Int. J. Uncertainty, Fuzziness Knowl.-Based Syst., vol. 10, no. 5, pp. 557–570, 2002.

[452] M. Abadi et al., “Deep learning with differential privacy,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2016, pp. 308–318.

[453] P. C. Mahawaga Arachchige, P. Bertok, I. Khalil, D. Liu, S. Camtepe, and M. Atiquzzaman, “Local differential privacy for deep learning,” IEEE Internet Things J., vol. 7, no. 7, pp. 5827–5842, Jul. 2020.

[454] L. T. Phong, Y. Aono, T. Hayashi, L. Wang, and S. Moriai, “Privacy- preserving deep learning via additively homomorphic encryption,” IEEE Trans. Inf. Forensics Security, vol. 13, no. 5, pp. 1333–1345, May 2018.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

42 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

[455] H. Fang and Q. Qian, “Privacy preserving machine learning with homomorphic encryption and federated learning,” Future Internet, vol. 13, no. 4, p. 94, Apr. 2021.

[456] F. Sattler, K.-R. Müller, and W. Samek, “Clustered federated learn- ing: Model-agnostic distributed multitask optimization under privacy constraints,” IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 8, pp. 3710–3722, Aug. 2021.

[457] R. Hu, Y. Guo, H. Li, Q. Pei, and Y. Gong, “Personalized federated learning with differential privacy,” IEEE Internet Things J., vol. 7, no. 10, pp. 9530–9539, Oct. 2020.

[458] J. Liu et al., “From distributed machine learning to federated learning: A survey,” Knowl. Inf. Syst., vol. 64, no. 4, pp. 885–917, Apr. 2022.

[459] L. Lyu et al., “Privacy and robustness in federated learning: Attacks and defenses,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 7, pp. 8726–8746, Jul. 2024.

[460] S. Supritha, S. Sushmitha, D. Asghar, K. Mohan, and S. Basavaraju, “Privacy-preserving data mining: Methods, metrics and applications,” Perspect. Commun., Embedded-Syst. Signal-Process.-PiCES, vol. 2, no. 9, pp. 218–222, 2018.

[461] R. Agrawal and R. Srikant, “Privacy-preserving data mining,” in Proc. ACM SIGMOD Int. Conf. Manage. Data, 2000, pp. 439–450.

[462] D. Agrawal and C. C. Aggarwal, “On the design and quantification of privacy preserving data mining algorithms,” in Proc. 20th ACM SIGMOD-SIGACT-SIGART Symp. Princ. Database Syst., May 2001, pp. 247–255.

[463] S. Fletcher and M. Z. Islam, “Measuring information quality for privacy preserving data mining,” Int. J. Comput. Theory Eng., vol. 7, no. 1, pp. 21–28, Feb. 2014.

[464] M. S. Alvim, M. E. Andrés, K. Chatzikokolakis, P. Degano, and C. Palamidessi, “Differential privacy: On the trade-off between utility and information leakage,” in Proc. Formal Aspects Secur. Trust, 8th Int. Workshop, 2012, pp. 39–54.

[465] P. Mohassel and Y. Zhang, “SecureML: A system for scalable privacy- preserving machine learning,” in Proc. IEEE Symp. Secur. Privacy (SP), May 2017, pp. 19–38.

[466] S. Tan, B. Knott, Y. Tian, and D. J. Wu, “CryptGPU: Fast privacy- preserving machine learning on the GPU,” in Proc. IEEE Symp. Secur. Privacy (SP), May 2021, pp. 1021–1038.

[467] F. Prasser, J. Eicher, H. Spengler, R. Bild, and K. A. Kuhn, “Flexible data anonymization using ARX—Current status and challenges ahead,” Softw., Pract. Exper., vol. 50, no. 7, pp. 1277–1304, Jul. 2020.

[468] R. J. Wilson, C. Y. Zhang, W. Lam, D. Desfontaines, D. Simmons- Marengo, and B. Gipson, “Differentially private SQL with bounded user contribution,” 2019, arXiv:1909.01917.

[469] TensorFlow Privacy. Accessed: Feb. 6, 2025. [Online]. Available: https://www.tensorflow.org/responsible ai/privacy/guide

[470] J. Li, X. Kuang, S. Lin, X. Ma, and Y. Tang, “Privacy preservation for machine learning training and classification based on homomorphic encryption schemes,” Inf. Sci., vol. 526, pp. 166–179, Jul. 2020.

[471] (2022). Concrete ML: A Privacy-Preserving Machine Learning Library Using Fully Homomorphic Encryption for Data Scientists. [Online]. Available: https://github.com/zama-ai/concrete-ml

[472] (2025). Microsoft SEAL. [Online]. Available: https://github.com/ Microsoft/SEAL

[473] C. Zhang, Y. Xie, H. Bai, B. Yu, W. Li, and Y. Gao, “A survey on federated learning,” Knowl.-Based Syst., vol. 216, Mar. 2021, Art. no. 106775.

[474] (2025). FATE. [Online]. Available: https://fate.readthedocs.io/en/latest/ [475] (2025). FedML. [Online]. Available: https://github.com/FedML-AI/

FedML?tab=readme-ov-file [476] D. J. Beutel et al., “Flower: A friendly federated learning research

framework,” 2020, arXiv:2007.14390. [477] (2025). PySyft. [Online]. Available: https://docs.openmined.org/en/

latest/quick-install.html [478] (2025). Rosetta: A Privacy-Preserving Framework Based on Ten-

sorflow. [Online]. Available: https://github.com/LatticeX-Foundation/ Rosetta

[479] P. Foley et al., “OpenFL: The open federated learning library,” Phys. Med. Biol., vol. 67, no. 21, Nov. 2022, Art. no. 214001.

[480] (2025). Substra. [Online]. Available: https://docs.substra.org/en/stable/

[481] Y. Djenouri, A. Belhadi, D. Djenouri, G. Srivastava, and J. C.-W. Lin, “A secure intelligent system for Internet of Vehicles: Case study on traffic forecasting,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 11, pp. 13218–13227, Nov. 2023.

[482] B. Jiang, G. Huang, T. Wang, J. Gui, and X. Zhu, “Trust based energy efficient data collection with unmanned aerial vehicle in edge network,” Trans. Emerg. Telecommun. Technol., vol. 33, no. 6, p. 3942, Jun. 2022.

[483] K. Bylykbashi, D. Elmazi, K. Matsuo, M. Ikeda, and L. Barolli, “Effect of security and trustworthiness for a fuzzy cluster manage- ment system in VANETs,” Cognit. Syst. Res., vol. 55, pp. 153–163, Jun. 2019.

[484] V. Vemoori, “Towards secure and trustworthy autonomous vehicles: Leveraging distributed ledger technology for secure communication and exploring explainable artificial intelligence for robust decision- making and comprehensive testing,” J. Sci. Technol., vol. 1, no. 1, pp. 130–170, 2020.

[485] V. Vemoori, “Towards safe and equitable autonomous mobility: A multi-layered framework integrating advanced safety protocols, data-informed road infrastructure, and explainable ai for transparent decision-making in self-driving vehicles,” Human-Comput. Interact. Perspect., vol. 2, no. 2, pp. 10–41, 2022.

[486] H. Mankodiya, D. Jadav, R. Gupta, S. Tanwar, W.-C. Hong, and R. Sharma, “OD-XAI: Explainable AI-based semantic object detection for autonomous vehicles,” Appl. Sci., vol. 12, no. 11, p. 5310, May 2022.

[487] H. Mankodiya, M. S. Obaidat, R. Gupta, and S. Tanwar, “XAI-AV: Explainable artificial intelligence for trust management in autonomous vehicles,” in Proc. Int. Conf. Commun., Comput., Cybersecurity, Infor- mat. (CCCI), Oct. 2021, pp. 1–5.

[488] A. V. S. Madhav and A. K. Tyagi, “Explainable artificial intelli- gence (XAI): Connecting artificial decision-making and human trust in autonomous vehicles,” in Proc. Int. Conf. Comput., Commun., Cyber- Secur., 2022, pp. 123–136.

[489] C. Berghoff, P. Bielik, M. Neu, P. Tsankov, and A. Von Twickel, “Robustness testing of AI systems: A case study for traffic sign recognition,” in Proc. Artif. Intell. Appl. Innov., 2021, pp. 256–267.

[490] D. Unal, F. O. Catak, M. T. Houkan, M. Mudassir, and M. Hammoudeh, “Towards robust autonomous driving systems through adversarial test set generation,” ISA Trans., vol. 132, pp. 69–79, Jan. 2023.

[491] Y. Deng, X. Zheng, T. Zhang, C. Chen, G. Lou, and M. Kim, “An analysis of adversarial attacks and defenses on autonomous driving models,” in Proc. IEEE Int. Conf. Pervasive Comput. Commun. (Per- Com), Mar. 2020, pp. 1–10.

[492] X. He, W. Huang, and C. Lv, “Trustworthy autonomous driving via defense-aware robust reinforcement learning against worst-case observational perturbations,” Transp. Res. C, Emerg. Technol., vol. 163, Jun. 2024, Art. no. 104632.

[493] J. Pei and W. Li, “Dual model pruning enables efficient federated learning in intelligent transportation systems,” IEEE Trans. Intell. Transp. Syst., vol. 26, no. 10, pp. 17292–17303, Oct. 2025.

[494] S. Ahmad, S. Malik, D.-H. Park, and D. Kim, “Design of lightweight driver-assistance system for safe driving in electric vehicles,” Sensors, vol. 19, no. 21, p. 4761, Nov. 2019.

[495] K.-F. Lee et al., “An intelligent driving assistance system based on lightweight deep learning models,” IEEE Access, vol. 10, pp. 111888–111900, 2022.

[496] R. Ke et al., “Lightweight edge intelligence empowered near-crash detection towards real-time vehicle event logging,” IEEE Trans. Intell. Vehicles, vol. 8, no. 4, pp. 2737–2747, Apr. 2023.

[497] B. Mahaur, K. K. Mishra, and A. Kumar, “An improved lightweight small object detection framework applied to real-time autonomous driving,” Expert Syst. Appl., vol. 234, Dec. 2023, Art. no. 121036.

[498] L. Chen, Q. Ding, Q. Zou, Z. Chen, and L. Li, “DenseLightNet: A light-weight vehicle detection network for autonomous driving,” IEEE Trans. Ind. Electron., vol. 67, no. 12, pp. 10600–10609, Dec. 2020.

[499] M. A.-M. Khan et al., “LLDNet: A lightweight lane detection approach for autonomous cars using deep learning,” Sensors, vol. 22, no. 15, p. 5595, Jul. 2022.

[500] T. Liang, H. Bao, W. Pan, and F. Pan, “ALODAD: An anchor-free lightweight object detector for autonomous driving,” IEEE Access, vol. 10, pp. 40701–40714, 2022.

[501] D. Liu, L. T. Yang, R. Zhao, J. Wang, and X. Xie, “Lightweight tensor deep computation model with its application in intelligent transportation systems,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 2678–2687, Mar. 2022.

[502] P. Xu et al., “Adversarial robustness in graph-based neural architecture search for edge AI transportation systems,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 8, pp. 8465–8474, Aug. 2023.

[503] J. Ou, J. Li, C. Wang, Y. Wang, and Q. Nie, “Building trust for traffic flow forecasting components in intelligent transportation systems via interpretable ensemble learning,” Digit. Transp. Saf., vol. 3, no. 3, pp. 126–143, 2024.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

CINÀ et al.: TRUSTWORTHY DATA- AND ML-Ops FOR INTELLIGENT TRANSPORTATION SYSTEMS AND LOGISTICS 43

[504] Y. Xie, N. Pongsakornsathien, A. Gardi, and R. Sabatini, “Explanation of machine-learning solutions in air-traffic management,” Aerospace, vol. 8, no. 8, p. 224, Aug. 2021.

[505] S. Wollenstein-Betech, C. Muise, C. G. Cassandras, I. Ch. Paschalidis, and Y. Khazaeni, “Explainability of intelligent transportation systems using knowledge compilation: A traffic light controller case,” in Proc. IEEE 23rd Int. Conf. Intell. Transp. Syst. (ITSC), Sep. 2020, pp. 1–6.

[506] G. Raja et al., “Intuitive and privacy-preserving traffic light control system for autonomous vehicles,” IEEE Internet Things J., vol. 12, no. 6, pp. 6563–6572, Mar. 2025.

[507] Z. Ying, S. Cao, X. Liu, Z. Ma, J. Ma, and R. H. Deng, “PrivacySignal: Privacy-preserving traffic signal control for intelligent transportation system,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 16290–16303, Sep. 2022.

[508] K. L. Tan, A. Sharma, and S. Sarkar, “Robust deep reinforcement learning for traffic signal control,” J. Big Data Analytics Transp., vol. 2, no. 3, pp. 263–274, Dec. 2020.

[509] R. Carvalho Barbosa, M. Shoaib Ayub, R. Lopes Rosa, D. Zegarra Rodrı́guez, and L. Wuttisittikulkij, “Lightweight PVIDNet: A priority vehicles detection network model based on deep learning for intelligent traffic lights,” Sensors, vol. 20, no. 21, p. 6218, Oct. 2020.

[510] C. Chen, B. Liu, S. Wan, P. Qiao, and Q. Pei, “An edge traffic flow detection scheme based on deep learning in an intelligent trans- portation system,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 3, pp. 1840–1852, Mar. 2021.

[511] Y. Dai et al., “Making you only look once faster: Toward real-time intelligent transportation detection,” IEEE Intell. Transp. Syst. Mag., vol. 15, no. 3, pp. 8–25, May 2023.

[512] D. Krajzewicz, G. Hertkorn, C. Rössel, and P. Wagner, “Sumo (sim- ulation of urban mobility)-an open-source traffic simulation,” in Proc. Middle East Symp. Simulation Modeling, 2002, pp. 1–5.

[513] L. Wen et al., “UA-DETRAC: A new benchmark and protocol for multi-object detection and tracking,” Comput. Vis. Image Understand., vol. 193, Apr. 2020, Art. no. 102907.

[514] M. Zeller, T. Waschulzik, M. Rothfelder, and C. Klein, “Safety assur- ance of a driverless regional train -insight in the safe.trAIn project,” in Proc. IEEE 34th Int. Symp. Softw. Rel. Eng. Workshops (ISSREW), Oct. 2023, pp. 41–42.

[515] L. Monje, R. A. Carrasco, C. Rosado, and M. Sánchez-Montañés, “Deep learning XAI for bus passenger forecasting: A use case in Spain,” Mathematics, vol. 10, no. 9, p. 1428, Apr. 2022.

[516] L. Sobrie and M. Verschelde, “Real-time decision support for human–machine interaction in digital railway control rooms,” Decis. Support Syst., vol. 181, Jun. 2024, Art. no. 114216.

[517] G. Sariyer, S. K. Mangla, M. E. Sozen, G. Li, and Y. Kazancoglu, “Leveraging explainable artificial intelligence in understanding pub- lic transportation usage rates for sustainable development,” Omega, vol. 127, Sep. 2024, Art. no. 103105.

[518] D. Shao, K. Zoh, and Y. Xie, “The spatial differentiation mechanism of intangible cultural heritage and its integration with tourism develop- ment based on explainable machine learning and coupled coordination models: A case study of the jiang-zhe-hu in China,” Heritage Sci., vol. 12, no. 1, pp. 1–22, Nov. 2024.

[519] A. Munawar and M. Piantanakulchai, “A collaborative privacy- preserving approach for passenger demand forecasting of autonomous taxis empowered by federated learning in smart cities,” Sci. Rep., vol. 14, no. 1, p. 2046, Jan. 2024.

[520] Y. Cai, H. Gao, J. Liao, X. Li, Y. Xu, and J. Xiong, “A personalized recommendation model based on collaborative filtering and federated learning for cultural tourism attractions in fujian-taiwan,” in Proc. Int. Conf. Softw. Syst. Eng. (ICoSSE), Apr. 2023, pp. 69–77.

[521] Y. Zhuang, Z. Pu, H. Yang, and Y. Wang, “Edge–artificial intelligence- powered parking surveillance with quantized neural networks,” IEEE Intell. Transp. Syst. Mag., vol. 14, no. 6, pp. 107–121, Nov. 2022.

[522] B. Li and Y. He, “A feature-extraction-based lightweight convolutional and recurrent neural networks adaptive computing model for container terminal liner handling volume forecasting,” Discrete Dyn. Nature Soc., vol. 2021, no. 1, Aug. 2021, Art. no. 6721564.

[523] J. C. S. Núñez, J. A. Gómez-Pulido, and R. R. Ramı́rez, “Machine learning applied to tourism: A systematic review,” WIREs Data Mining Knowl. Discovery, vol. 14, no. 5, p. 1549, Sep. 2024.

[524] R. N. Akram et al., “Security, privacy and safety evaluation of dynamic and static fleets of drones,” in Proc. IEEE/AIAA 36th Digit. Avionics Syst. Conf. (DASC), Sep. 2017, pp. 1–12.

[525] R. Ekatpure, “Artificial intelligence-driven solutions for intelligent fleet management in automotive engineering: Advanced models, tech- niques, and real-world applications,” J. Artif. Intell. Res., vol. 3, no. 1, pp. 71–112, 2023.

[526] F. Belletti and A. M. Bayen, “Privacy-preserving MaaS fleet management,” Transp. Res. Proc., vol. 23, pp. 1000–1024, Sep. 2017.

[527] Z. Yao et al., “Sensitive data privacy protection of carrier in intelligent logistics system,” Symmetry, vol. 16, no. 1, p. 68, Jan. 2024.

[528] M. D. Capua, A. Ciaramella, and A. De Prisco, “Machine learning and computer vision for the automation of processes in advanced logistics: The integrated logistic platform (ILP) 4.0,” Proc. Comput. Sci., vol. 217, pp. 326–338, Jan. 2023.

[529] Z. Lu, R. Cui, T. Sun, and L. Wu, “The value of last-mile delivery in online retail,” SSRN Electron. J., p. 65, 2023.

[530] A. Gupta, R. Yadav, A. Nair, A. Chakraborty, S. Ranu, and A. Bagchi, “Fairfoody: Bringing in fairness in food delivery,” in Proc. AAAI Conf. Artif. Intell., 2022, pp. 11900–11907.

[531] T. Ye, Z. Zhao, S. Wang, F. Zhou, and X. Gao, “A stable lightweight and adaptive feature enhanced convolution neural network for efficient railway transit object detection,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 10, pp. 17952–17965, Oct. 2022.

[532] M. Hussain and R. Hill, “Custom lightweight convolutional neural network architecture for automated detection of damaged pallet rack- ing in warehousing & distribution centers,” IEEE Access, vol. 11, pp. 58879–58889, 2023.

[533] L. Zhang, X. Zhang, and M. Liu, “Lightweight convolutional neural network for fast visual perception of storage location status in stereo warehouse,” J. Intell. Manuf., vol. 36, no. 5, pp. 1–21, Jun. 2025.

[534] D. Clifford and J. Ausloos, “Data protection and the role of fairness,” in Yearbook of European Law, vol. 37. USA: Oxford Univ. Press, 2018, pp. 130–187.

[535] N. Azam, L. Michala, S. Ansari, and N. B. Truong, “Data privacy threat modelling for autonomous systems: A survey from the GDPR’s perspective,” IEEE Trans. Big Data, vol. 9, no. 2, pp. 388–414, Apr. 2023.

[536] C. Fetting, “The European green deal,” ESDN Office, Vienna, Tech. Rep. 1, 2020.

[537] A. Larsson and C. Große, “Data use and data needs in critical infrastructure risk analysis,” J. Risk Res., vol. 26, no. 5, pp. 524–546, May 2023.

Antonio Emanuele Cinà (Member, IEEE) received the Ph.D. degree (cum laude) in computer science from the Ca’ Foscari University of Venice in 2023, defending a thesis on the vulnerabilities and emerg- ing risks arising from the malicious use of training data in AI. He is currently an Assistant Professor with the University of Genoa, Italy. His research interests encompass all aspects of AI system security and the study of their trustworthiness, with primary expertise in training (poisoning) and inference-time (evasion) attacks. He has also been investigating the

capabilities of AI for cybercrime detection and prevention.

Giovanni Scodeller (Member, IEEE) received the master’s and Ph.D. degrees in computer science from the Ca’ Foscari University of Venice in 2023. He is currently a Ph.D. Student with the University of Genoa. He is a member of the sAIfer Lab, a joint laboratory between the University of Genoa and the University of Cagliari, focused on machine learning security. His research interests include uncertainty quantification for machine learning and machine learning security.

Cecilia Caterina Pasquale (Member, IEEE) received the bachelor’s degree in civil engineering from the Polytechnic University of Turin, Turin, Italy, in 2009, and the master’s degree in transport and logistics engineering and the Ph.D. degree in monitoring of systems and environmental risk man- agement from the University of Genoa, Genoa, Italy, in 2012 and 2016, respectively. She is currently an Assistant Professor with the University of Genoa. Her research interests include modeling, optimiza- tion, and control methods applied to the field of

transportation systems. She is the Chair of the Technical Committee on Planning and Control of Transportation Networks of the IEEE Intelligent Transportation Systems Society. She is an Associate Editor of Control Engineering Practice.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

44 IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

Silvia Siri (Senior Member, IEEE) received the Ph.D. degree in information and communication technologies from the University of Genoa, Genoa, Italy, in 2006. She is currently an Associate Pro- fessor of automatic control with the University of Genoa. She is the co-author of more than 150 papers in international journals, chapters, and conference proceedings. Her research interests include the mod- eling and control of freeway traffic systems, the optimal management of electric mobility in smart cities, and the optimization and planning of freight

logistics. From 2019 to 2021, she served as a member of the Board of Governors for the IEEE Intelligent Transportation Systems Society. Since 2021, she has been the Chair of the IEEE ITSS Italian Chapter. She is a Senior Editor of IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS and an Associate Editor of IEEE CONTROL SYSTEMS LETTERS.

Davide Anguita (Senior Member, IEEE) received the Laurea degree in electronic engineering and the Ph.D. degree in computer science and electronic engineering from the University of Genoa, Genoa, Italy, in 1989 and 1993, respectively. After work- ing as a Research Associate with the International Computer Science Institute, Berkeley, CA, USA, on special-purpose processors for neurocomputing, he returned to the University of Genoa. He is currently a Full Professor of computer engineering with the Department of Informatics, Bioengineering,

Robotics, and Systems Engineering (DIBRIS). His current research focuses on the theory and application of kernel methods and artificial neural networks.

Fabio Roli (Fellow, IEEE) is currently a Full Pro- fessor of computer engineering with the University of Genoa and the University of Cagliari, Italy, and Founding Director of the Pattern Recognition and Applications Laboratory, University of Cagliari. He has been doing research on the design of pattern recognition and machine learning systems for thirty years. He has provided seminal contributions to the fields of multiple classifier systems and adversarial machine learning. He has played a leading role in the establishment and advancement of these research

themes. He has been appointed as a fellow of the International Association for Pattern Recognition. He was a recipient of the Pierre Devijver Award for his contributions to statistical pattern recognition.

Simona Sacone (Senior Member, IEEE) is currently a Professor of automatic control with the Depart- ment of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa, Genoa, Italy, where she teaches systems theory, identifi- cation and estimation techniques, and sustainable systems modeling, and where she also acts as the Coordinator of the Ph.D. degree course on systems engineering. She has authored and co-authored more than 150 papers published in international journals, books, and conference proceedings. Her research

interests include the optimization and control of complex physical processes by means of discrete event and hybrid modeling, with control approaches mainly focusing on freeway traffic control and logistic networks planning. From 2019 to 2024, she was a member of the Board of Governors of the IEEE Intelligent Transportation Systems Society. She is the Editor-in-Chief of IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS.

Luca Oneto (Senior Member, IEEE) was born in Rapallo, Italy, in 1986. He received the B.Sc. and M.Sc. degrees in electronic engineering and the Ph.D. degree in computer engineering from the University of Genoa in 2008, 2010, and 2014, respectively. From 2014 to 2016, he worked as a Post-Doctoral Researcher in computer engineering with the University of Genoa, where he worked as an Assistant Professor from 2016 to 2019. He co- founded company ZenaByte s.r.l. in 2018. In 2019, he became an Associate Professor of computer sci-

ence with the University of Pisa and from 2019 to 2024, he held the position of an Associate Professor of computer engineering with the University of Genoa. Currently, he is a Full Professor of computer engineering with the University of Genoa. He has been the Coordinator and local responsible in numerous industrial, H2020, and Horizon Europe projects. His primary research interests include statistical learning theory and trustworthy AI. Additionally, he focuses on data science, utilizing and improving cutting-edge machine learning and AI algorithms to tackle real-world problems. He has received prestigious recognitions, including the Amazon AWS Machine Learning Award and the Somalvico Award for the Best Young AI Researcher in Italy.

Open Access funding provided by ‘Universita degli studi di Genova’ within the CRUI CARE Agreement

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.