1
Thesis Research Proposal
Using Machine Learning Techniques for Software Fault Prediction in Development and User
Environments
Name
Institution
Course
Tutor
Date
2
Introduction
One of the most critical customer-oriented perspectives of software quality is reliability,
which refers to the probability of failure-free functionality of a system for a definite period in a
specified environment. As automation gained wider use across different sectors, the demand for
software products equally grew exponentially. However, the majority of customers and
developers paid extra attention to validation and verification, which ensured functional and
business requirements were met. Currently, a lot has changed in the software market. Software
demand is still high, but the number of developers has also grown tenfold. Therefore, clients are
inclined to look for other things beyond meeting the usual user requirements, which has come to
issues such as reliability, scalability, maintainability, flexibility, and portability aspects. Focusing
on reliability alone, it is startling how the demand landscape is quickly evolving around it. This
has forced the industry to create a mini-industry, christened Software Reliability Engineering,
with the main objective of looking into issues of software reliability at large.
Software Reliability Engineering's overarching goal is to offer measurements for
planning and regulating the resources used in a software development process. At this stage in
the development process, the expenses of Verification and Validation to uncover remaining
flaws are no longer justified by their impacts (Pandey, Mishra, & Tripathi, 2021). In this context,
fault prediction (FP) measurements can assist in studying the development process as a whole
and determining when to halt testing and release the product (Khalil, Eldash, Kumar, &
Bayoumi, 2020). Different development models provide different fault prediction scenarios. The
waterfall model presents the most cost-effective approach to handling faults since it ensures the
finalization of each phase before moving to the next. On the other hand, Evolutionary Life Cycle
3
Model or Iterative Development Model presents critically precarious outcomes when handling
faults because different phases of the project are executed concurrently.
The study assesses the performance of 5 classifiers and proposes an automated technique
that uses machine learning to solve software problems within a system. The study employed
three PROMISE Software Engineering Repository data sets (CM1, PC1, and JM1) with 22
characteristics, including Halstead, McCabe, as well as other additional metrics comprising fault
information (Angelopoulos et al., 2019). We normalized our chosen data in order to allocate
associated columns, as well as investigated data imbalance concerns using several computational
approaches like PCA, SMOTE, and Resample (Choudhary et al,. 2018). As a result, this research
gives a comparative examination of five machine learning algorithms for predicting software
faults inside a program. The objective of this thesis proposal is to look at how a fault prediction
technique can be introduced into present Evolutionary Life Cycle Model processes and at the
user environment in order to limit the potential damages that undetected fault can cause to a
business. This pertains to the general viability on the one hand and the cost advantages of fault
prediction on the other (Sobie, Freitas, & Nicolai, 2018). It is vital to point out that
this thesis' scope is assumed to be acceptable for a master's qualification.
Problem Statement
The current fault prediction techniques widely used in the software engineering industry
fall short of flagging errors in the evolutionary life cycle model development environment as
well as the strictly iterative model development environment and the user environment. This
thesis proposes a software fault prediction development model that leverages machine learning
techniques to predict faults in all large systems at development and use levels. The proposal
comes with three or more important solutions for developers and others for users: reducing the
4
cost of validation and verification in large systems, cutting down the time spent in conducting
unit testing as well as the integrated testing, and finally ensuring that the fault detection in the
large development software is no longer phase based in iterative and evolutionary life cycle
approaches but global. For users, the cost of on-and-off repairs is reduced, and disruption to
critical business processes is also significantly limited.
The prediction of software faults has long been regarded as a crucial concern for the tech
sector and software experts. Traditional approaches needed prior expertise with defects or a
defective module for identifying software flaws within a program or an application. Currently,
the industry is leveraging automated software fault recovery models to effectively detect and
repair software glitches. However, a proper solution to fault detection during the development
process is yet to be found. Also, the current solutions are either historical data-dependent or
algorithms which have certain known limitations. The most affected development models are
those that utilize iteration. In already deployed systems, the proposed solution allows the
software to function more efficiently, reducing problems and expenses as it runs parallel to the
system.
The significance of the proposed solution lies in the understanding of the difference in
software fault prediction in the traditional waterfall development method and evolutionary life
cycle model. The waterfall model has a specific temporal sequence, with one step having to be
completed before the next phase begins. Such an arrangement makes the procedure easier to
manage. Aside from this basic model, various alternative process models have emerged over the
last few decades. The primary cause of this issue is that fundamental and traditional models are
unlikely to be appropriate in industrial situations, where processes are significantly more
intricate and harder to regulate and plan.
5
This is not the case when it comes to the evolutionary life cycle development model,
which is a hybrid of iterative and incremental techniques of software development life cycle and
involves the division of the development process plus deliverables into manageable chunks that
are delivered in phases to the customers to boost their confidence levels. Although suitable due
to user feedback and other factors, the sub-division of the software complicates the fault
prediction efforts since the software only becomes a complete entity at the integration level. In
other words, fault detection mechanisms can only be run at the chunk level but not the system as
a whole. The problem may be even greater in strictly iterative development environments.
The paper recommends a software fault prediction utilizing a machine-learning approach
that is effective for both incremental and iterative development methods. Despite performing its
primary function, the solution runs parallel to ongoing processes in its environment of use and
hence does not impede the efficiency of the software. However, the proposed solution has a way
of predicting the possibility of fault exists in the disjointed system as long as the factors forming
the key business and function requirements form a predictable pattern.
Research Questions
The following research questions are addressed in this study:
RQ 1 How do the models under study perform in predicting the fault inflow evolutionary life
cycle development models and user environments?
RQ 2 How do the models under study perform with regard to short-term predictions?
Literature Review
6
Organizations frequently base their growth on specific processes that include a number
of activities, as illustrated by Royce's (1970) waterfall model. As Tumar, Hassouneh, Turabieh,
& Thaher, (2020) puts it, the waterfall model has a specific temporal sequence, with one step
having to be completed prior to the beginning of the subsequent phase. Such an arrangement
makes the procedure easier to manage. Aside from this basic model, various alternative process
models have emerged over the last few decades. The primary cause of this issue is that
fundamental and conventional models are unlikely to be appropriate in industrial situations,
where processes are significantly more intricate and harder to regulate and plan (Kumar, Misra,
& Rath, 2017). For example, needs modifications in software projects are common during
advanced stages or at the tail-end stages of development. This, in turn, necessitates equivalent
modifications in design and execution, as well as a unit or module testing and integration testing
that incorporates all modules making up a program. Iterative techniques address these issues and
attempt to incorporate modifications. As a result, they replicate aspects of traditional
development procedures throughout the development lifecycle, producing complete, deliverable
software with each iteration (Arshad, Riaz, Jiao, & Murthy, 2018). The Rational Unified Process
(RUP) is one example of such an iterative technique (Malatesta, & Lamb, 2018).
In the mid-1990s, plan-driven systems such as RUP were criticized for being overly
cumbersome, and agile methodologies emerged (Malatesta, & Lamb, 2018). These approaches
are based on innovation, experience, face-to-face communication, and quick consumer
satisfaction. Nonetheless, because neither the plan-driven nor the agile methods can be used in
isolation, the real iterative operations are a hybrid of the two systems (Rathore, & Kumar, 2019).
Fault prediction employs particular models and approaches during the software development
process as well as during industrial products used to provide accurate metrics. Fault Prediction
7
Models and Methods (FPMM), in particular, depended on statistical approaches. These
approaches are further classified as parametric statistics and nonparametric statistics. The
distinction is that parametric statistics presume that the selected samples are uniformly
distributed. Nonparametric models, on the other hand, do not make this assumption (Thota,
Shajin, & Rajesh, 2020).
A number of statistical-based FPMMs approaches have previously been proposed. (Van
Casteren, 2017), for instance, the examination of numerous models incorporating parametric
statistics. Such models mainly differed in the assumptions each makes about the original dataset
(Purba, & Sondang, 2022). Nevertheless, the majority of these FPMMs are not necessarily
acceptable for use in the industry for a variety of motives: First, there is little information
available early in iterative processes, making it difficult to give trustworthy forecasts.
Furthermore, in many circumstances, certain FPMMs cannot be used because the relevant data is
either unavailable or too expensive to obtain. Furthermore, practitioners may be hesitant to
employ FPMMs if they are too difficult to use (Van Casteren, 2017).
Finally, the majority of extant FPMMs are aimed at fundamental agile methodologies. As
initially indicated, the process in a typical incremental environment is that development teams
finalize each phase at a time before proceeding to the next. In other words, all processes
pertaining to a phase are often completed before jumping to the succeeding phase, and this
includes verification and validation. (Pacella et al., 2017) indicates that in the event of an
existence of an error, the team can easily take apart the affected phase and retrieve the error.
(García-Martínez, González-Acolt, & Díaz-Flores, 2022) further observes that the cost of
detecting an error in this development method is incredibly low since there is the certainty that
the previous phase was error-free. As a result, their somehow rigid assumptions make matching
8
the frameworks to an iterative software development process difficult. The difficulty, according
to (Chevance et al., 2020), is the models' exclusive reliance on relevant data, including size and
sophistication indicators. The author, therefore, advocates for the use of unified data from all
aspects influencing software development, including the process of development itself, the
design and the difficulties, and the defect detection method (Gowtham et al., 2017).
In general, there is a little study concerning fault prediction to date. Staron and Meding
provide the only example that was discovered (Van Casteren, 2017). The authors suggest an
approach for fault prediction three weeks in advance using project indicators and current fault
data in their paper. This strategy is then contrasted with other estimating techniques. One of their
conclusions is that the authors' prediction approach is more efficient than expert guesses, for
example, since the prediction results are more accurate and easier to produce.
Earlier fault prediction research utilized a broader classification algorithm. The findings
of these investigations revealed that the algorithm's abilities were restricted, casting doubt on its
reliability in predicting software faults (Turabieh, Mafarja, & Li, 2019). The prediction accuracy of
fault-prediction algorithms was discovered to be significantly lower, ranging from 70% to 85%,
with a high rate of misclassification (Ji, & Wang, 2017). Since the software fault prediction
landscape continues to shift towards increased accuracy and less dependence on historical data, a
machine learning-supported approach remains the way to go (obie, Freitas, & Nicolai, 2018).
Fault Prediction Models and Methods (FPMM) that depend on statistical approaches as well as
the fault-prediction algorithms, which, a number of studies established, are significantly lower in
accuracy, ranging from 70% to 85%, with a high rate of misclassification, can no longer be the
answer needed by users struggling to master fault’s pattern of occurrence in large, sophisticated
system programs.
9
Software fault localization and ease of maintenance are defined as the capability of a
software application or its modules to be modified to rectify faults, increase efficiency as well as
throughput or software and system testing, system development methodologies, or adapt to a
different platform (Kumar, Misra, & Rath, 2017). A software fault prediction model that uses
machine learning has an in-built predictive analytical capability that enables it to establish fault
trends from detailed system performance characteristics that no other model can currently match.
This helps businesses to assist in reducing the total maintenance effort, time, and cost of a
software project (Angelopoulos et al., 2019). To ensure the quality of excellent software, it must
be dependable, with a lower frequency of faults occurring throughout program run time
(Choudhary et al,. 2018). As a result, the categorization of errors in software modules has a
significant influence on the software development process. However, the real-world scenario
would be difficult because when a developer updates his program within an application and it is
tied to other modules, the chances of encountering failure get high.
On the other hand, the amount of literature related to software fault proneness is growing
by the day in response to the growing need for automated services (Turabieh, Mafarja, & Li,
2019). Ruchita Malhotra (Malatesta, & Lamb, 2018) presented research comparing ANN, SVM,
DT, CCN, GMDH, and GEP for predicting software defect modules. The author obtained three
PROMISE NASA project data for the Repository for Software Engineering. Created the
performance evaluation. In this investigation, DT outperformed the others in terms of accuracy.
The research and classifiers employed 21 CK measurements, McCabe measurements, and as well
as Halstead measurements (Choudhary et al,. 2018) created a flaw. The KPWE model is a
predictive framework. It also included two approaches, namely Weighted Extreme Learning
Machine (WELM) and Principal Kernel Component Analysis (KPCA).
10
The 44 Software Project data relied on the KPWE, which is at the center of this thesis
proposal, further indicating that it is indeed an excellent approach to the baseline techniques in
the majority of situations. (Kumar, Misra, & Rath, 2017) conducted research comparing SVM,
RF, Regression Tree, and C4.5 (DT) for predicting software fault modules. The results from the
experiment demonstrated that among classifiers, RF presented the best accuracy. (Angelopoulos
et al., 2019), on the other hand, created a new feature classifying approach and feature subset
selection technique to test how effective the feature selection for cross-project defect prediction
(CPDP) can be. The experiment revealed that CPDP feature selection techniques could increase
software fault analysis performance, especially in a usable state but not in during development.
(Kumar, Misra, & Rath, 2017) said that the software program may learn user expectations from
semantic contexts across various apps.
Several researchers are passionately tracking the developments taking place in the field of
what is currently referred to as Software Reliability Engineering, particularly the progress of AI
and Machine Learning enabled fault prediction in large industrial software systems (Thota,
Shajin, & Rajesh, 2020). Liu, Fang, Dong, & Xu, (2021), on the other hand, presented case-based
reasoning (CBR) approaches that offered software with abilities to predict fault in software
systems developed in the iterative environment, which appear to be extremely suited for
decentralized software applications. According to research (Angelopoulos et al., 2019), the
techniques described in the article "A New Way to Find Bugs in Self-Driving AI Could Save
Lives" is another critical milestone in fault prediction. According to two studies, machine
learning-based techniques outperform statistical-based models for system program or application
fault prediction in all development environments, i.e., the traditional phase-based waterfall
technique or the iterative agile methodologies (Liu, Fang, Dong, & Xu, 2021). Because of their
11
dependability, consistency, and verifiability, open-source datasets are critical (Choudhary et al,.
2018).
Angelopoulos et al., (2019) investigates a critical issue; that software fault categorization
is crucial in real-time; or else, the cost and effort of discovering bugs hidden in a system software
or cloud application are bound to rapidly increase. The author further suggests that fault
prediction techniques are now making a lot of sense in user environments as opposed to
development environments since the latter is just for purposes of knowing that the product that
has been built is valid and passes the verification test, but main work is at the user's environment
where there is the effect of "wear and tear" that is likely to invite more and more defects that can
significantly impact the performance of the system. This is further corroborated by (Malatesta, &
Lamb, 2018), which indicates that a lot of large software systems introduce faults during use as a
result of varying use-knowledge among primary users and other factors. This motivates the
creation of automated fault prediction models for software fault prediction, which can foresee
software problems during the time of use. If software problems are found prior to software
release, the developer may quickly assign and rectify those defect modules.
The absence of appropriate performance assessment methods to assess the capabilities of
fault prediction models is a major challenge in software fault prediction (Choudhary et al,. 2018).
Another source of worry is the uneven distribution of defects in software fault databases, which
may result in biased learning (Thota, Shajin, & Rajesh, 2020). Furthermore, several concerns,
such as the selection of software metrics to be used in fault prediction models, the influence of
context on prediction performance, the cost-effectiveness of fault prediction models, and fault
density prediction, require additional exploration. A variety of software project dataset
repositories, such as PROMISE Data Repository and NASA Metrics Data Program1, have
12
recently been publicly available hence helping fill the gap. The availability of these archives has
stimulated further research and the development of new applications. As a result, a review of the
state-of-the-art in this field may be beneficial to the scientific community.
The most prevalent use case among researchers as well as experts in the software industry
currently is software failure prediction using machine learning techniques (Liu, Fang, Dong, &
Xu, 2021). The journey to this stage has been consistent, and the apparent shift from statistical-
based models is informed by the complex nature of predicting and detecting a fault in a large
software application. Modern machine learning algorithms are employed to identify faulty
modules in software programs for study and to provide effective remedies for users (Tong, Liu,
& Wang, 2018). Different researchers have employed a number of prominent machine learning
classifiers recommended in the most current systematic literature review (Choudhary et al,.
2018). All of the chosen classification approaches are applied to several real-world application
datasets relevant to software defect prediction. However, (Turabieh, Mafarja, & Li, 2019) advises
that it is important to examine certain qualities in regard to data quality, which cannot be
checked in terms of accuracy. As a result, cutting-edge (Van Casteren, 2017) machine learning
algorithms have been used to the fault datasets to improve prediction by removing redundant
features using a variety of techniques for feature selection and imbalance to balance data
methods.
Conclusion
The paper proposes an accurate machine learning-enabled fault prediction that can be
effective in incremental and iterative development environments as well as in the user
environment. The study establishes that a number of existing fault prediction models have certain
limitations in the evolutionary life cycle development model environment and industrial user
13
environment. For instance, the Fault Prediction Models and Methods (FPMM) are based on
statistical approaches and utilize historical data to issue predictions on faults. Additionally, there
is the fault-prediction algorithms which a number of studies established as significantly lower in
accuracy, ranging from 70% to 85%, with a high rate of misclassification, can no longer be the
answer needed by users struggling to master fault pattern of occurrence in large, sophisticated
system programs. Also revealed in the study is that the evolutionary life cycle development
model and iterative development models present a lot of challenges to a number of existing fault
prediction models as opposed to waterfall development, which provides a global view of the
development process. Finally, fault prediction techniques are now making a lot of sense in user
environments as opposed to development environments since the latter is just for the purposes of
knowing that the product that has been built is valid and passes the verification test, but the main
work is at the user's environment where there is the effect of "wear and tear" that is likely to
invite more and more defects that can significantly impact the performance of the system.
14
References
Angelopoulos, A., Michailidis, E. T., Nomikos, N., Trakadas, P., Hatziefremidis, A., Voliotis, S.,
& Zahariadis, T. (2019). Tackling faults in the industry 4.0 era—a survey of machine-
learning solutions and key aspects. Sensors, 20(1), 109.
Arshad, A., Riaz, S., Jiao, L., & Murthy, A. (2018). Semi-supervised deep fuzzy c-mean
clustering for software fault prediction. IEEE Access, 6, 25675-25685.
Chevance, M., Kruijssen, J. D., Hygate, A. P., Schruba, A., Longmore, S. N., Groves, B., ... &
Utomo, D. (2020). The lifecycle of molecular clouds in nearby star-forming disc galaxies.
Monthly Notices of the Royal Astronomical Society, 493(2), 2872-2909.
Choudhary, G. R., Kumar, S., Kumar, K., Mishra, A., & Catal, C. (2018). Empirical analysis of
change metrics for software fault prediction. Computers & Electrical Engineering, 67, 15-
24.
García-Martínez, B., González-Acolt, R., & Díaz-Flores, M. (2022). Robotics and Automation:
Evolutionary Approach Based on Their Life-Cycle. Environmental, Social and
Governance Approach for Competitiveness, 89.
Gowtham, V., Manoj, Y., Pooventhiran, G., Praveen, A., Shivaram, R., & Kathiresan, A. (2017).
Evolutionary models in software engineering. International Journal of New Technology
and Research, 3(5), 30-33.
Ji, W., & Wang, L. (2017). Big data analytics based fault prediction for shop floor scheduling.
Journal of Manufacturing Systems, 43, 187-194.
15
Khalil, K., Eldash, O., Kumar, A., & Bayoumi, M. (2020). Machine learning-based approach for
hardware faults prediction. IEEE Transactions on Circuits and Systems I: Regular Papers,
67(11), 3880-3892.
Kumar, L., Misra, S., & Rath, S. K. (2017). An empirical analysis of the effectiveness of
software metrics and fault prediction model for identifying faulty classes. Computer
standards & interfaces, 53, 1-32.
Liu, M., Fang, S., Dong, H., & Xu, C. (2021). Review of digital twin about concepts,
technologies, and industrial applications. Journal of Manufacturing Systems, 58, 346-361.
Malatesta, L. C., & Lamb, M. P. (2018). Formation of waterfalls by intermittent burial of active
faults. GSA Bulletin, 130(3-4), 522-536.
Malatesta, L. C., & Lamb, M. P. (2018). Formation of waterfalls by intermittent burial of active
faults. GSA Bulletin, 130(3-4), 522-536.
Pacella, D., Ponticorvo, M., Gigliotta, O., & Miglino, O. (2017). Basic emotions and adaptation.
A computational and evolutionary model. PLoS one, 12(11), e0187463.
Pandey, S. K., Mishra, R. B., & Tripathi, A. K. (2021). Machine learning based methods for
software fault prediction: A survey. Expert Systems with Applications, 172, 114595.
Purba, R. A., & Sondang, S. (2022). Design and Build Monitoring System for Pregnant Mothers
and Newborns using the Waterfall Model. INTENSIF: Jurnal Ilmiah Penelitian dan
Penerapan Teknologi Sistem Informasi, 6(1), 29-42.
Rathore, S. S., & Kumar, S. (2019). A study on software fault prediction techniques. Artificial
Intelligence Review, 51(2), 255-327.
16
Sobie, C., Freitas, C., & Nicolai, M. (2018). Simulation-driven machine learning: Bearing fault
classification. Mechanical Systems and Signal Processing, 99, 403-419.
Thota, M. K., Shajin, F. H., & Rajesh, P. (2020). Survey on software defect prediction
techniques. International Journal of Applied Science and Engineering, 17(4), 331-344.
Tong, H., Liu, B., & Wang, S. (2018). Software defect prediction using stacked denoising
autoencoders and two-stage ensemble learning. Information and Software Technology,
96, 94-111.
Tumar, I., Hassouneh, Y., Turabieh, H., & Thaher, T. (2020). Enhanced binary moth flame
optimization as a feature selection algorithm to predict software fault prediction. IEEE
Access, 8, 8041-8055.
Turabieh, H., Mafarja, M., & Li, X. (2019). Iterated feature selection algorithms with layered
recurrent neural network for software fault prediction. Expert systems with applications,
122, 27-42.
Van Casteren, W. (2017). The Waterfall Model and the Agile Methodologies: A comparison by
project characteristics. Research Gate, 2, 1-6.
Van Casteren, W. (2017). The Waterfall Model and the Agile Methodologies: A comparison by
project characteristics. Research Gate, 2, 1-6.
17
