Surname 1
Homework 10
Arizona State University
CSE 230 - Computer Organization and Assembly Language Programming
October 20, 2020
Homework 10: Understanding the Role of Hardware Counters and Profiling Tools in
System Performance Optimization
1. Explain what is meant when we say that a program or system is memory bound.
What other types of bindings have we discussed?
A memory-bound program or system is a program or system whose performance is limited to
the memory access speed and is not limited to the CPU processing speed. In such a system,
the processor is often idle waiting for data transfer from the main memory instead of
instruction execution. This is because memory access times are much slower than CPU
speeds and thus a bottleneck is created (Null and Lobur 312). Hence, the processor's
efficiency is lowered even when it is of high capacity, if it is continually waiting for data
from memory.
In the majority of computer systems, data moves through different levels of memory that
consist of registers, several cache levels, and main memory. Every level has different speeds
and sizes, with the registers being the fastest and the smallest, and the main memory being
the slowest and the largest. When the CPU wants data not present in the cache, which is
called a cache miss, it has to get the data from the main memory, thus causing a delay.
Frequent cache misses lead to instruction pipeline stalling and performance degradation,
Surname 2
thereby making the system memory bound (Null and Lobur 315). The issue is prevalent in
applications that process huge amounts of data like artificial intelligence, scientific modeling,
and image processing as the data cannot be entirely stored in the faster cache memory.
Arithmetic intensity is a term often used to illustrate the extent of computations performed
for every single unit of data fetched from memory. A program exhibiting a low arithmetic
intensity is one that does minimal calculations for each memory access, and such programs
are generally limited by memory. On the other hand, those with high arithmetic intensity, i.e.,
that execute many calculations for each memory access, are termed CPU-bound as their
speed relies mostly on the processor rather than the memory latency (Null and Lobur 319).
Knowledge of this proportion allows developers to figure out computational vs. memory
performance bottlenecks and so make the right call on performance enhancements.
There are quite a few ways to get the performance of memory-bound systems up. One of the
ways is to enhance data locality which, simply put, refers to data structures where data items
accessed together in time are also stored together in memory. This has an impact on cache
utilization thereby lowering the cache miss rate (Null and Lobur 320). Prefetching is another
abundant technique by which the system anticipates the next data to be used and places it in
cache ahead of the CPU's request. Apart from that, programmers may also transition to a
caching-friendly algorithm by, for example, partitioning the big datasets into small chunks so
that each can be comfortably loaded in memory at a time. On the same note, the memory
latency could be mitigated by the hardware solutions such as upgrading to larger cache,
installing faster memory, or enabling multiple memory channels.
Apart from memory binding, it is possible for systems to suffer from different kinds of
bindings. A CPU-bound system is one where the processor's speed is the main factor that
Surname 3
limits the performance, rather than memory or input/output speed. Thus, such systems lead to
optimization attempts to crank up computation through faster processors, improved
algorithms, or parallel execution (Null and Lobur 322). On the flip side, I/O-bound systems
struggle with sluggishness due to input and output operations, examples of which may be
reading from disks and communication with external devices. To enhance I/O performance,
one may need to resort to using swift storages, buffering techniques, or concurrent
input/output operations.
Network-bound and storage-bound systems are also included in the discussion. Network-
bound systems are those that are limited by the speed or the capacity of the data that can be
transmitted over the network, and this is typical of cloud computing and distributed
applications. Consequently, reducing network latency and data transfer optimization protocol
are the main ways of improving the situation (Null and Lobur 325). Storage-bound systems
that may also be regarded as a type of I/O-bound systems are the ones that have limited
performance due to slow disk access times. The problem can be solved by using solid-state
drives, caching, or efficient file management.
A memory-bound system is one where the performance is limited by the rate at which data
can be transferred between memory and the processor. The issue arises because the increase
in CPU speed has gone way beyond the improvements in memory speed, hence the term
memory wall is often used. It is very important to understand whether a system is memory-
bound, CPU-bound, or I/O, network, or storage-bound in order to locate performance
bottlenecks and decide on the proper optimization techniques (Null and Lobur 327). By
making computation and memory performance levels equal, computer designers and
programmers can improve the total system efficiency to a great extent.
Surname 4
2. What does Amdahl’s Law tell us about performance optimization?
Amdahl's Law is the first theory to come to mind when talking about hierarchy in computer
architecture, which shows the limits of applying such a method halfway through the system.
It reveals the fact that the overall program speedup depends not only on how much a
particular portion is improved but also on how significant that portion is to the total
execution time (Null and Lobur 330). Simply put, the proportional performance gain of the
newly optimized fraction of the program is limited by the unoptimized part of the program.
The said principle was first raised by Gene Amdahl in 1967 to indicate that performance
optimization has a marginal effect. The law is best represented in the form of a formula:
Speedup = 1 / [(1 - P) + (P / S)], where P is the part of the program that can be improved, and
S is the speedup of that part. To illustrate, if 40% of a program is optimized to run ten times
faster, then the overall speedup would be approximately 1.56 times only. It thus provides
evidence that big changes in one part of a system are largely ineffectual if the biggest chunk
of the system is left intact (Null and Lobur 332).
Amdahl's Law points out the stay relevance of parallel computing in the context of the
performance issue that be solved by how tasks could be subdivided among several
processors. What it reveals is that the part of a program which has to be executed
sequentially limits the total gain of the parallelization. The effects of adding more processors
are less and less significant as further processors are added because it is necessary to
synchronize, communicate, and execute the sequential portion of the code (Null and Lobur
334).
3. Which of the means is useful for comparing rates?
Surname 5
When you are comparing different rates, the harmonic mean should be your choice of
measure. Contrary to arithmetic or geometric means, the harmonic mean returns the correct
average when the quantities are in the form of rates, e.g., speed, throughput, or efficiency
(Null and Lobur 336). The reason for this is that it goes back to the time or share related to
each rate, thus it does not allow the fastest or the highest to determine the overall result to a
great extent.
The harmonic mean is the inverse of the average of the inverses: H = n / (1/x₁ + 1/x₂ + ... +
1/xₙ), here x₁, x₂, xₙ are the individual rates. So, if the data is being processed at 100 MB/s for
50% of the time and at 200 MB/s for the rest, the overall rate will not be the arithmetic mean
of 150 MB/s, but the harmonic mean which will be 133.3 MB/s. Hence, the true average rate
over time has been represented (Null and Lobur 337).
Harmonic mean in the performance evaluation helps to elude the deception that arises from
wrongly averaged rates. It is the most valuable representative of total system performance, in
particular, when the unit is subjected to variable speeds or network transfer rates.
4. For what kinds of data is the arithmetic mean inappropriate?
The mean value, most notably called the “average,” is one of the most extensively used
statisticians of central tendency. Its value is derived by adding together all data values and
dividing the sum by the number of observations. However, the mean is not appropriate in
every circumstance. The mean may be confusing or completely wrong when it is used on
non-linear data, data which is categorical, is highly skewed, or is given as rates or ratios
(Null and Lobur 338). Hence, proper understanding of when it is not appropriate to use the
Surname 6
mean is a prerequisite for correct data interpretation and subsequent decision-making not
only in computer performance evaluation but also in other areas of analysis.
Firstly, the use of mean is inconvenient when dealing with categorical data, those data which
represent the properties of the objects and are of qualitative nature. Examples of such data
may be processor type, color, or operating system. These groups of data have no logical way
of being summed or averaged for they have no numerical relationships between them. In this
case, the mode (the category that occurs most frequently) or the median (in case of ordinal
data) serve as better central tendency measures (Null and Lobur 340). Applying the mean to
categorical data would provide results of a mathematical operation that do not represent
anything because the categories do not have a mathematical distance between them.
Secondly, the mean does not reflect the central value of skewed data distributions correctly.
In situations where a dataset has a major concentration of values on one side and just a few
wholefarthing on the other, the arithmetic mean is very much influenced by these extreme
values. For example, in performance testing, if the speed of program execution is mostly
around one second and only a few executions take ten seconds due to system delays, the
arithmetic mean will show an average time that is longer than the actual typical time. The
median is therefore more indicative of such data as it is not influenced by the extreme values
and it is thus a better measure of central tendency (Null and Lobur 341). This explains why
the median is mostly used in the reporting of response times, latency, or income where
outliers frequently occur.
Thirdly, the arithmetic mean is not a good measure when rates, ratios, or percentages are
involved. Rates like speed, throughput, or efficiency are in their very nature reciprocal, and
hence the arithmetic mean does not quite reflect their average behavior. Consider the case of
Surname 7
network transfer rates or processing speeds if they are averaged using the arithmetic mean.
The result, in this case, would be an untruthful one because the arithmetic mean does not take
into consideration the proportion of time during which each rate was attained. The harmonic
mean is the appropriate mean in that it weights each rate's contribution according to time or
occurrence correctly (Null and Lobur 342). The difference becomes extremely crucial in
cases such as the analysis of computer performance when a wrong average of instruction
execution rates or data transfer speeds may end up leading to incorrect measures of system
efficiency.
Moreover, the arithmetic mean is visually deceptive for logarithmic or exponential data. The
geometric mean is a better choice when dealing with values that vary multiplicatively, such
as growth rates or compression ratios. The arithmetic mean model presupposes that the
values are additive, but exponential growth is a multiplicative pattern. As an illustration, if a
program’s performance is increased by 10 percent in one year and 20 percent the following
year, it is the geometric mean that gives the right average rate of improvement, not the
arithmetic mean (Null and Lobur 344).
The arithmetic mean is not a good measure for categorical, skewed, rate-based, and
exponential data as it does not accurately reflect the true central tendency or proportional
relationships in such cases. When dealing with computer performance data, it is extremely
important to choose the right type of mean that corresponds to the nature of the dataset.
Incorrect usage of measures can lead to misrepresentations of system efficiency, response
time, or overall performance. The arithmetic mean is at its best when dealing with symmetric
numerical data with uniform variance; however, analysts have to resort to the median,
harmonic mean, or geometric mean when dealing with specialized datasets. Familiarity with
Surname 8
these differences guarantees more precise and insightful interpretations of performance
results (Null and Lobur 345).
5. Give a definition for optimum performance.
Optimum performance is when a computer operating system runs at the highest level of
efficiency or effectiveness under certain conditions. It is the perfect combination of speed,
resource usage and even accuracy, whereby the system is able to carry out operations with
minimal delay and maximum throughput without any wastage of resources (Null and Lobur
348). The process of achieving optimum performance is all about tweaking the hardware and
software to work together smoothly, thereby removing bottlenecks and ensuring that all parts
like the CPU, memory, and I/O devices are functioning in harmony. But very often, the ideal
optimum performance is just a theoretical one because systems in real life have to make
compromises between speed, cost, and energy consumption. Thus, performance optimization
is a gradual process of getting closer to this ideal by fixing the most critical areas of the
system's operation and at the same time ensuring that stability and efficiency are retained
(Null and Lobur 349).
6. What is a price-performance ratio? What makes it hard to apply?
Price-performance ratio is a ratio, which is employed to measure the cost-effectiveness of a
computer system or component. It shows the level of performance that is received relative to
the cost and in most cases the cost is divided by the price of the system or vice versa (Null
and Lobur 350). Simply, it assists in the determination of the level of computing power or
efficiency a user has paid. The ratio comes in handy when comparing systems of similar
Surname 9
capabilities and varying prices, e.g. processors, units of memory or storage devices. A lower
price-to-performance ratio is a good sign, that is, more performance with less expenditure.
System designers, consumers, and organizations usually use the price-performance ratio to
make purchases. As an example, considering two processors, one can provide 20 percent
higher performance at 10 percent higher price, and it would make it a better deal. Data
centers and business, on the same note, employ this ratio to equate investment of their
computing resources to anticipated work load or productivity benefits (Null and Lobur 351).
It enables level comparisons to be made between different architectures, vendors, or
configurations to enable decision-makers to determine what system they can use most
efficiently with regard to fund utilization.
Nonetheless, it is not easy to apply price-performance ratio as there are a number of factors.
To begin with, performance measurement is situational and complicated. The level of
performance of a system depends on the nature of workload, application behavior and
optimization methods. As an example, a processor can be very good at floating-point
calculations and bad at data-intensive tasks. In this way, it is rather difficult to identify one
and only "performance" value that can reflect all the use cases (Null and Lobur 352).
Second, it is not that easy always to price. The cost of hardware is liable to market
conditions, change in technologies and economies of scale. Also, the cost of a system is a
sum of the cost of hardware plus software, maintenance, and energy consumption. These
indirect cost incurring costs are usually very hard to determine, but they greatly influence the
actual price-performance ratio.
Surname 10
Third, multi-dimensional trade-offs have been simplified by the ratio. There are numerous
components of performance such as: speed, memory capacity, scalability, reliability, power
efficiency, and so on, that cannot be artificially converted into a single numerical metric. To
optimize one area of performance, it gets sacrificed, and it is difficult to ascribe only one
meaningful ratio (Null and Lobur 353).
Although the ratio between price and the performance of the computing systems is a useful
tool used in the comparison of the computing systems, it has a limitation in application due to
the complexity of formulation of constant performance measures and variable prices
structure, and due to complexity of current day computing loads. The real analysis should go
beyond this ratio and examine the situation in the larger scope of the requirements of the
system, the overall ownership costs, and the intended application.
7. What is the shortcoming of using MIPS or FLOPS as a measure of system
throughput?
Two more classic measures of computer performance and system throughput are MIPS
(millions of instructions per second) and FLOPS (betas of floating-point operations per
second). MIPS is an indicator of the ability of a processor to execute machine-level code at
one second, whereas FLOPS is an indicator of the ability of a processor to perform floating-
point calculations at one second (Null and Lobur 355). Though they are fast numerical
measure of computation capacity, there are serious flaws in their applications in measuring
the overall system throughput. The two do not consider variations in the type of instruction,
the nature of work and the structure of the system, which results in misleading or incomplete
outcomes.
Surname 11
The primary weakness of MIPS as a performance measure quality is that it lacks an accurate
representation of the work being done by a computer system. Different instructions do not
complete in the same time and they do not do the same quality of calculation. Here are some
examples; a processor can run many simpler instructions very quickly but cannot run more
complicated tasks, which has a high MIPS rating which is not proportional to actual
performance. According to Null and Lobur, there are two systems whose MIPS values are the
same but vastly different in actual ability to perform based upon the instruction sets and
workload type of the basic programs (357). That is, MIPS does not measure computational
productivity as it should have been but the speed of executing instructions.
The other significant constraint of the MIPS is that it relies too much on the processor
instruction set architecture (ISA). The various instruction lengths, complexities and execution
cycles are used in different architectures. An example is a Reduced Instruction Set Computer
(RISC) processor that can take many simple instructions to do a task that would otherwise
take one complex instruction to do on a Complex Instruction Set Computer (CISC).
Consequently, the RISC could be registered to give a higher value of MIPS though the
systems take the same time to accomplish the same job or even take longer (Null and Lobur
358). This structural flexibility renders it almost infeasible to compare the systems of
differing renditions with the use of alone MIPS.
Equally, FLOPS, although better at quantifying scientific or mathematical work, is also
limited. Floating-point performance only is measured by Florida and as such does not count
the integer operations, data movement, or input/output activity that usually are controlling in
real-world applications. As an example, a processor can be very fast at a controlled
benchmark running a floating-point program, but very slow at executing general-purpose
Surname 12
workloads with logic and memory access (branching) (Null and Lobur 359). By doing so,
FLOPS is capable of overstating performance capability, especially to systems that are
optimized to perform floating-point-based, rather than mixed-purpose, computing.
In addition, both MIPS and FLOPS do not take into account system-level issues that
influence throughput including: memory hierarchy, cache performance, I/O speed, and
interconnect latency. Throughput is a rate of how much valuable work a system can
accomplish during a certain period which is not only related to straightforward rates of
instruction or execution of the operations. As an example, a processor which scores highly in
an independent benchmark on a single processor may suffer bottlenecks during interaction
with big datasets, because of slow memory access or limited bandwidth. When this happens,
Microchips with high theoretical performance performance as indicated by MIPS or FLOPS,
do not transfer to high real-world performance (Null and Lobur 360).
The other weakness is that both MIPS and FLOPS are machine and workload dependent
measures. These can change based on the compiler optimization, program makeup and the
benchmark that is employed to measure the performance. Because of this, very different
values of MIPS or FLOPS scores can be obtained on the same computer system, depending
on the test conditions. This slackness makes them not helpful at target comparisons across
platforms or applications.
Also, these measures stimulate the investigation of only raw speed without the consideration
of the efficiency of the entire system. The designers and customers could excessively attach
importance to the processors which have large values of MIPS or FLOPS without looking
into the aspects like power-expenses, reliability, and of course affordability. A system, which
Surname 13
processes a greater number of instructions in an instant, yet using more power or causing a
higher latency, might not be the best candidate to use in practice (Null and Lobur 361).
Historically MIPS and FLOPS are significant performance measures but they are not
sufficiently important in determining overall system throughput. MIPS is faulty, as it counts
execution of instructions but not useful work and different architectures and mixes of
instructions will give different values. FLOPS however, does not need to count the overall
computational context but just the floating-point operations. The two overlook the system
level factors such as, speed of memory, input/output performance and parallelism. The
contemporary performance measurement is thus based on more holistic methods like SPEC
and TPC which measure actual workloads and induce a more realistic view of the overall
efficiency (Null and Lobur 362).
8. How is the Dhrystone benchmark different from Whetstone and Linpack?
Compared to the Whetstone and Linpack, the Dhrystone benchmark is different in the kind of
workload it is measuring and the kind of workloads it is measuring. Although Whetstone and
Linpack are dealing with the floating-point computing and scientific computing performance,
Dhrystone is designed to check the integer and string-processing performance performances
that are much closer to the general business and system applications (Null and Lobur 364).
Started with Reinhold Weicker in 1984, it was an artificial programming benchmark that
avoided using heavy mathematical processing and instead used synthetic programming, such
as, though not limited to, procedure calls, assignment statements, control statements, and text
manipulation.
Surname 14
The Whetstone benchmark instead was written earlier to test the performance of systems
arithmetic (floating-point) in floating-point arithmetic, especially in engineering and
scientific applications. It measures the performance of a computer in performing
trigonometric functions, logarithms, and array operations and gives the performance in
Millions of Whetstone Instructions Per Second (MWIPS). Jack Dongarra invented a slightly
more specialized version, Linpack. It uses the solution of dense system of linear equations as
a metric of the floating-point performance of a system, a task found in numerical and
scientific computing. The results of Linpack are reported in Floating-Point Operations Per
Second (FLOPS) and can now be considered a standard by which supercomputers can be
ranked (Null and Lobur 365).
Hence, their primary distinction is their area of application. Dhrystone attempts to measure
general-purpose performance, particularly integer arithmetic and control flow performance,
characteristic of the business and system program. Whetstone and Linpack, in their turn, are
interested in floating-point performance on the basis of scientific workloads. The simplicity
and portability of dhrystone made it one of the tools most commonly used to measure
microprocessors and embedded systems, although Linpack is the most commonly used
metric of high-performance computing.
9. What are the deficiencies in the Whetstone, Dhrystone, and Linpack benchmarks
that are addressed by the SPEC CPU benchmarks?
The first of this was the Whetstone, Dhrystone and Linpack benchmarks which were created
to test the performance of a computer but each has grievous flaws that restrain its quality and
representativeness. To fill in these deficiencies, the SPEC CPU benchmarks (or Standard
Performance Evaluation Corporation CPU benchmarks) have been developed to offer a more
Surname 15
detailed, standardized, and realistic measure of the performance of a processor and a
complete system. SPEC tests are more workload-oriented and balanced, and they can be used
to compare systems on a more equal platform (Null and Lobur 367).
Whetstone, Dhrystone, and Linpack have a significant weakness in that they are synthetic
benchmarks. They run simplified, artificial sequence codes, which make an attempt to
simulate some kind of work (instead of running actual application programs). This means
that their results do not always represent the real performance of systems executing real user
applications. Dhrystone was used as an example: integer and string manipulation are
measured, but a representation of modern computing tasks such as access patterns in
memory, use of cache/caches and parallel execution are not considered. Whetstone and
Linpack have the same defect, putting a lot of emphasis on floating-point calculations,
though the range of operations that are performed by modern systems has much more variety.
SPEC CPU benchmarks have solved this problem by employing real world source code
based on engineering, scientific, and business. This is used to make sure that the performance
reflects the performance of systems under realistic conditions (Null and Lobur 368).
The other weakness is the limited scope of previous benchmarks. Whetstone is mostly used
to gauge the benchmark of floating-point, Dhrystone is used to gauge the benchmark of
integer and Linpack is created to solve the linear algebra problems. Both thus only test a
single axis of system capability, ignoring system interactions between CPU, memory
optimization and compiler optimization. Conversely, the SPEC CPU algorithm is separated
into two broad subtopics, including SPECint (which measures integer performance) and
SPECfp (which measures floating-point performance). This segregation enables the ability to
test particular areas but remain able to carry out a larger analysis of what a system is capable
Surname 16
of. Both the processor and memory subsystems of SPEC are also emphasized, which makes it
a more comprehensive variable of computational performance (Null and Lobur 369).
Another issue with previous benchmarks consists of portability and standardization.
Whetstone, Dhrystone and Linpack could give different results whenever they are compiled
or optimized. As an example, the compiler optimizations might be inflating benchmark
scores without having any effect on the actual performance. The solution for this issue in
SPEC is associated with tight run policies and methodologies. All benchmarks should be
compiled, run and reported in a controlled and consistent manner and they should be
manipulated minimally and fair comparisons among vendors. This standardization is one of
the major factors of why the results of SPEC are highly accepted in the computer industry
(Null and Lobur 370).
The other weakness is related to the development of the hardware architectures. The previous
benchmarks were made many decades ago when a single core processor and sparse memory
hierarchies ruled. They fail to consider the current design issues like deep cache hierarchies,
multiple cores, superscalar pipelines, or branch prediction. The SPEC CPU benchmarks,
however are regularly revised to encompass types of current technology trends and
workloads. They are crafted with the more contemporary characteristics of parallelism at
instruction level, efficiency on the cache, and effectiveness in compiler optimization,
therefore, are more applicable in the assessment of present architecture (Null and Lobur 371).
10. Explain the term benchmarking.
Benchmarking is a procedure of gauging and assessing the functionality of a machine system
or its constituents in accordance with a collection of uniform examinations or programs.
Surname 17
Benchmarks, these tests are performed to mimic certain workloads or real world applications
in order to gauge the level of hardware and software efficacy in performing certain tasks
which include computation, transferring data and accessing data to the memory. The above is
intended to be achieved through the benchmarking of objective and comparable data on
performance that can be used to guide system selection, optimize, and design decisions. It
assists in establishing the strengths and weaknesses in the aspects such as processor speed,
memory bandwidth and input/output processes. Null and Lobur state that benchmarking is a
key tool that can be used to compare various systems and also establish the validity of a
given architectural improvement as well as speculating on the performance under real
workloads (373). In current computers, benchmarks are as simple as the instruction tests and
tend to be elaborate such as SPEC which measures system efficiency in general.
11. How is the focus of the TPC different from SPEC?
In both the Standard Performance Evaluation Corporation (SPEC) and the Transaction
Processing Performance Council (TPC), the benchmarks are industry-standards, but their
focus is vastly different. SPEC has put a primary focus on tracking CPU and computational
performance through work loads that are based on scientific, engineering and other general-
purpose computing workloads. Its benchmarks include SPECint and SPECfp that measure
the efficiency of a processor-memory system in performing integer and floating-point
computations in controlled conditions. Conversely, TPC benchmarks are concentrated on the
performance of database and transaction processing and determine the ability of complete
system (hardware, operating system, and database management software) to process large
amount of parallel transactions and queries (Null and Lobur 374).
Surname 18
Whereas SPEC benchmarks check the efficiency of the processor, the TPC benchmarks look
at system-level throughput, response time and cost-performance ratios in business application
scenarios. The results of TPC are in terms of transactions per minute or per second which
represents commercial workload performance. Therefore, SPEC focuses on the
computational benchmarking, whereas TPC focuses on data processing, data reliability, and
scalability in the applications based on databases where the information flow is used on an
enterprise level (Null and Lobur 375).
12. Explain delayed branching.
Delayed branching is an operation in processor hardware logic that helps make instruction
pipelines more efficient by reducing the performance lag that often occurs when branching
instructions are used. In a pipelined processor, the execution path can be temporarily
unknown if a branching decision is waiting to be made as a result of which the processor is
not able to fetch the next instruction to be executed. This issue is solved by delayed
branching whereby the pipeline is permitted to execute one or more instructions placed
immediately after the branching instruction before the branching instruction is actually
executed (Null and Lobur 212).
Thus, the very next instruction after the branch which is termed as the delay slot is executed
in any case, whether the branch is carried out or not. This prevents the loss of cycles with
stalls or flushes and keeps the pipeline functioning smoothly. To give an example, an
independent instruction can follow a conditional branch instruction in which case the
independent instruction can fill the delay slot and be executed while the branch condition is
still under consideration (Null and Lobur 213). This method is efficient in terms of processor
cycles utilization.
Surname 19
The role of compilers in the implementation of delayed branching is very significant as they
are the ones who figure out the most appropriate instructions that can be put into the delay
slot. Normally, the selected instructions should be those which are not dependent on the
branch condition so as to prevent any unexpected side effects. Nevertheless, there are times
when the suitable instructions cannot be found and a no-operation (NOP) instruction has to
be placed in the position thus reducing the performance improvement (Null and Lobur 214).
Delayed branching was an important feature in the first RISC architectures such as MIPS that
aimed for simplicity and uniform instruction timing. On the other hand, contemporary CPUs
make use of advanced branch prediction and speculative execution strategies rather than
having delay slots in order to accomplish the same objectives. However, delayed branching is
still a significant theoretical framework in learning how early pipelined architectures handled
control flow effectively (Null and Lobur 215).
13. What is branch prediction? What is it used for?
Branch prediction is a technique in modern computer architectures that helps in making the
instruction pipeline more efficient. It works by predicting the result of branch instructions
even before they are actually done. Conditional branch instructions are the ones that change
the flow of the program. They may be if-then-else statements or loops. Without prediction,
the processor would have to wait for the branch condition to be checked before it can fetch
the next instruction. This waiting would cause pipeline stalls and thus the performance would
be low. Branch prediction, therefore, enables the processor to decide in advance the
execution path and thus continue fetching and executing instructions that follow the
anticipated path (Null and Lobur 216).
Surname 20
Static and dynamic are the two main types of branch prediction. Static branch prediction
makes the decision based on certain rules or compiler hints that are already known. For
instance, it can be assumed that there will be a branch in a loop (backward branch) while
there will be no branch in a conditional statement (forward branch). The method is
straightforward but it can hardly achieve more than 50% correct predictions when a program
is complex. A dynamic branch prediction system utilizes the hardware features which inspect
the results of the previous branches to forecast the upcoming ones. To give an example, the
branch history table records past decisions and then uses them to make more accurate
predictions (Null and Lobur 217).
The main goal of branch prediction is to keep the pipelines fed with instructions in
processors that have pipelined and superscalar capabilities. If the processor manages to
predict the right direction for the branch, then the pipeline will be kept full and the delays
arising from control hazards will be very few. Today, a processor that finds its prediction
incorrect can handle such a situation by flushing the instructions that are wrong and
recovering quickly. The higher the prediction accuracy is, the more the performance is bound
to be. This holds especially true for the architectures that have deep pipelines as well as for
the applications in which branching is frequent (Null and Lobur 218).
14. Give three examples of pipeline hazards.
Pipeline hazards are basically situations where the next instruction cannot be carried out in
the processor pipeline at the same time as its predecessor even though it should be the next
clock cycle. They are the kind of factors that disrupt the smooth flow of instructions and lead
to lower performance efficiency. The three main classes of pipeline hazards are data hazards,
Surname 21
control hazards, and structural hazards, which, respectively, affect the pipeline in different
ways (Null and Lobur 220).
Data hazards refer to situations when a program needs to use the output of another program
that is still in the process of execution. For instance, if one program requires a certain value
that is yet to be computed by another, the pipeline will have to wait until the value is
computed. This delay is called read-after-write (RAW) dependency. Data hazards can be
solved by employing mechanisms such as data forwarding and pipeline stalls, which make it
possible for the dependent instruction to receive the data from some intermediate pipeline
stage or to wait till the data is out (Null and Lobur 221).
Control hazard is the other name for branch hazard. It results from an execution of branch or
jump instructions changing the program’s control flow. After the branch, the processor may
have several options of instructions to fetch but may not be sure which one is correct because
the decision to take the branch has not yet been made. As a result of this unpredictability,
pipeline stalls or incorrect instruction fetching are possible. By either predicting or
scheduling around possible branch stalls, branch prediction and delayed branching
techniques serve as a solution to prophylactic control hazards (Null and Lobur 222).
Structural hazards, the last in line, occur when several instructions want to use the same
hardware resource at the same time. For example, if a single memory unit is used both for
fetching instruction and accessing data, a conflict is going to arise. Usually, in modern
architectures, solutions to structural hazards are to either duplicate the affected hardware
units or reschedule the instructions so as not to have overlap (Null and Lobur 223).
15. Define the terms loop fusion, loop fission, loop peeling, and loop interchange.
Surname 22
In the world of computer architecture and optimizing compilers, numerous strategies are put
forward to have more efficient loop executions in programs. Since loops frequently have a
large impact on a program's runtime as they are executed multiple times, the optimization of
loops may lead to a substantial increase in the program's speed and a better use of the
system's resources. The paper "High Performance Embedded Architecture" by Null and
Lobur explains that these four techniques: loop fusion, loop fission, loop peeling, and loop
interchange (Null and Lobur 225) are the most prominent. One can achieve better instruction
scheduling, cache utilization, and parallel processing by employing these techniques.
Loop fusion is an optimization technique which merges two, or more, loops that are side by
side and run over the same data range into one loop. Loop fusion aims at removing the
overhead associated with a loop, such as initialization, condition checking, and branching,
and at improving cache performance by accessing related data all within a single pass. To put
it another way, two loops that individually execute the same operations on an array can be
combined into one to reduce the number of iterations. As a result, the technique promotes
spatial locality, and at the same time, it can lessen instruction pipeline stalls (Null and Lobur
226).
Loop fission or loop splitting is a technique that is reverse to loop fusion. It breaks down a
loop into two or more segments that each performs the work of the original loop on a subset
of the data. In a situation where a big loop compromise cache performance, loop fission can
be applied to fix the problem and can also make the parts of the loop that are being worked
on able to run in parallel on different processors. For example, separating computations that
use different memory sections can eliminate cache conflicts. Apart from that, the method can
Surname 23
be used to identify operations that are independent of each other and thus become more
easily schedulable (Null and Lobur 227).
Loop peeling and loop interchange are two more methods that change the way a loop is
executed for optimization. Loop peeling takes out one or more iterations from the start or the
end of a loop to deal with the special cases separately, thus, the main loop can be executed
with the simplified conditions. It speeds up the execution when the boundary conditions
make it more difficult. Conversely, loop interchange means changing the nesting order of
two loops in a multi-dimensional loop structure. The aim is to improve data locality and
memory access patterns, for instance, in matrix or array computations. By interchanging
loops, compilers can make sure that the consecutive memory elements are accessed more
efficiently, thus, less cache misses occur, and the computation becomes faster (Null and
Lobur 228).
16. According to queuing theory, what is the critical disk utilization percentage?
Based on the queuing theory, the threshold for the critical utilization of a disk is around 70
percent. In other words, it is said that when a disk is occupied with more than 70% of the
time, the average waiting time for requests increases substantially, hence the performance is
lowered. Queuing theory is the one that models and explains the system resources, like disks,
which work with certain loads and give back solutions for different request rates. When
utilization gets very close to 100%, the length of the queue and the time of the response
increase very fast, almost exponentially, rather than in a linear way (Null and Lobur 230).
Around 70% utilization, the disk is still performing well, efficiently managing both
throughput and responsiveness. Any further requests will have to wait for a longer time in the
Surname 24
queue, thus, the latency goes up, and the system slows down. This limit is significant,
especially, in systems that are heavily dependent on disk access like databases and file
servers. Consequently, the reduction of disk utilization below this critical level guarantees
uninterrupted performance and the avoidance of bottlenecks that have the potential to flow
into the whole computing system (Null and Lobur 231).
17. What is the risk involved in using the SSTF disk scheduling algorithm?
The Shortest Seek Time First (SSTF) disk scheduling algorithm chooses the disk I/O request
that minimally moves the read/write head from its current position. By doing this, the
average seek time is lessened in comparison to a simpler method like First-Come, First-
Served (FCFS), which results in better disk performance. Nevertheless, in spite of its
productivity, the SSTF has a considerable downside which is the risk of starvation or
indefinite postponement that can cause unfairness and dissatisfaction of the system's response
(Null and Lobur 232).
Starvation is a phenomenon that arises because in SSTF the requests that are closest to the
current head position are most of the time the ones that get priority, thus those that are farther
away are left out. Suppose, for instance, that new requests keep coming up in the vicinity of
the head’s current location; consequently, the algorithm may be constantly servicing them
and therefore postponing or even ignoring completely the distant requests. After a certain
period, this can make certain processes to wait for an unlimited period of time, which is
particularly the case in systems with heavy and uneven workloads. Due to this risk, the use of
SSTF is limited to situations where safety and prediction of response times are not a concern,
such as in real-time operating systems (Null and Lobur 233).
Surname 25
Moreover, the SSTF algorithm has an inherent risk of uneven wear and head thrashing when
the cause of the access is a localized pattern. Since the algorithm keeps servicing the nearby
tracks, it may become excessively moving within a narrow region of the disk. Apart from an
increase in mechanical wear, this can also be the reason for high latency of those requests
that are in other regions. Furthermore, the performance of SSTF is largely influenced by the
location of the requests; when requests are dispersed randomly, the algorithm's efficiency
may be on a par with FCFS (Null and Lobur 234).
To mitigate these risks, more advanced scheduling algorithms such as SCAN (also known as
the elevator algorithm) and C-SCAN (Circular SCAN) were developed. These algorithms
move the disk arm in a systematic direction, servicing requests in order, and then reversing or
looping back to the start. This ensures that every request is eventually handled, eliminating
the starvation problem while maintaining reasonable seek times. In summary, while SSTF
can improve efficiency in lightly loaded systems, its tendency to cause starvation and uneven
service makes it unreliable for workloads requiring fairness and consistency (Null and Lobur
235).
18. How is LOOK different from SCAN?
The LOOK disk scheduling algorithm is a change of the SCAN algorithm. It is also called
the "elevator" algorithm that moves the disk arm in one direction, servicing all pending
requests until it reaches the end of the disk, then reverses direction. The main difference
between the two is how far the disk arm goes. The SCAN algorithm, the arm moves all the
way to the end of the disk, even if there are no pending requests at the farthest track. The
LOOK algorithm "looks ahead" to find the last request in each direction and only goes as far
as it serves that one before turning - around (Null and Lobur 236). This makes LOOK more
Surname 26
efficient as it reduces unnecessary arm movements which in turn lowers average seek time
and increases disk throughput.
By not going to the physical end of the disk if it is not required, LOOK lessens mechanical
wear and saves time, particularly in systems with uneven request distribution. For instance, if
all requests are concentrated in the middle tracks, SCAN would still move the arm to the
outermost track, thus wasting movement, whereas LOOK would reverse direction after the
last relevant request. This flexibility enables LOOK to perform better in real workloads and
at the same time, be fair and not cause starvation. To sum up, the LOOK algorithm serves as
a reminder of the SCAN method by cleverly cutting down disk arm motion to actual request
locations, thus enhancing efficiency without giving up the service order (Null and Lobur
237).
19. What is disk prefetching? What are its advantages and disadvantages?
Disk prefetching or read-ahead is a method that enhances the performance of the system. It is
a technique, which is usually employed by the OS and the storage controllers, in order to
minimize the waiting time of I/O operations. Prefetching, the primary concept, implies that
very few applications do not have predictable access patterns. Especially when the
applications read files or sequential data, their future requests for data can be anticipated and
the system can then put the data in the cache or main memory which results in making the
waiting time very short if not non-existent (Null and Lobur 238).
Disk operations most often suffer from slowdowns due to the fact that after a disk request,
there are several mechanical delays like seek time (the time it takes to move the read/write
head to the correct track) and rotational latency (the time waiting for the correct sector to
Surname 27
rotate under the head). The idea behind prefetching is that it performs its reads ahead of time
and thus, the performance bottlenecks resulting from waiting for the disk to complete its
operations cannot be observed. For example, if a program is sequentially reading a file, the
operating system can predict that later blocks will be required and thus it will fetch them in
advance. So, when it comes to the next read request, memory will be able to provide the data
and not the disk, thus speeding up the whole process and also increasing the system's
throughput extensively (Null and Lobur 239).
Prefetching may be driven by hardware or by software. In hardware-based prefetching, the
disk controller internally detects sequential access patterns and preloads additional blocks in
its cache. In software-based prefetching, which is carried out by the operating system, file
system logic and I/O scheduling algorithms are used to make decisions regarding the time
and data for prefetching. Most of the current technologies are using both of these approaches
together. Thus, when performing operations on large multimedia files, or when running
database queries, the OS may perform one large contiguous block request for data, and at the
same time, the disk controller can do even more reading beyond the requested range to
optimize the operation (Null and Lobur 240).
Disk prefetching advantages are very many and quite powerful especially for such systems as
those having high read-to-write ratios or workloads mostly consisting of sequential access
patterns. Reduced I/O latency is the major advantage. The data being already in memory
when the application requests it, from the program's point of view, read operations are almost
instantaneous. Consequently, application performance is elevated and system responsiveness
becomes smoother, this is particularly evident in the cases of such activities as file loading,
video playback, or sequential data analysis (Null and Lobur 241).
Surname 28
Additionally, an advantage that disk throughput is enhanced. By fetching data in bigger,
sequential chunks rather than in several small random ones, the prefetching drastically cuts
down on the mechanical movements of the disk arm, thus the reduction in seek times and
rotational delays. In this manner, the disk is able to perform at a higher efficiency, producing
greater sustained transfer rates. Moreover, prefetching is instrumented to hide disk latency by
engaging in computation and I/O operations concurrently; whereas the CPU is busy with one
data batch, the disk subsystem can be fetching the next one in the background thereby
ensuring the continuous availability of data. Such interleaving increases the total system
capacity and makes sure that the CPU as well as the disk do not remain in a state of idle for a
long time unnecessarily (Null and Lobur 242).
In addition to that, prefetching improves the user experience of such apps that depend on
uninterrupted data streaming or interactive file access. For example, media players and video
editing software are resources that maily rely on the timeliness of data retrieval to avoid
buffering or playback interruptions. Prefetching guarantees that the data is there in advance
thus it completely prevents the lag phenomena. In the case of database systems, prefetching
can be a great factor in the enhancement of query performance by preloading index pages or
data blocks that are sufficiently relevant before the query execution engine requesting them.
Correspondingly, in operating systems, the use of boot times can be made better via the
method of prefetching frequently accessed files during the startup period (Null and Lobur
243).
Disk prefetching has several drawbacks and risks besides advantages that may lower the
effectiveness of disk prefetching or in some scenarios, even cause performance degeneration.
The most important drawback is a wrong prediction. Prefetching works on the principle that
Surname 29
accessing patterns can be predicted, but in cases of irregular and random access behavior, the
principle is violated most of the time. If the system prefetches the data that it does not use,
which is called prefetch pollution, it wastes the resource of memory and disk bandwidth.
Such an unnecessary operation can lead to cache displacement when the data that is useful is
removed from memory to give space to the unused prefetched blocks (Null and Lobur 244).
Another point is that the system overhead may become larger as well. Prefetching needs
additional I/O operations, data management, and memory allocation. In cases where the
system is heavily loaded, or there are a lot of random reads, the overhead from speculative
prefetching can worsen the situation by using disk bandwidth which should have been used
for actual I/O requests. That can lead to longer waiting times and lessened performance,
particularly in a multitasking environment where several processes are competing for disk
access (Null and Lobur 245).
Disk prefetching is also a reason for energy inefficiency, especially in the case of mobile or
embedded devices. Doing the fetch of the unnecessary data keeps the disk spinning longer,
and it requires more power, which results in the reduction of battery life. Likewise, in solid-
state drives (SSDs), which already have low access latency, aggressive prefetching may offer
almost no performance improvement while increasing write amplification and shortening the
storage device's life. So, prefetching algorithms need to be precisely adjusted depending on
the properties of the hardware beneath and the nature of the workload (Null and Lobur 246).
Interaction with caching policies also poses a challenge to prefetching. Prefetching may
hinder the cache replacement algorithms of the operating system, for example, Least
Recently Used (LRU). When huge amounts of prefetched data are sent to the cache, they
may expel other data which users have accessed frequently thus causing cache misses to
Surname 30
occur more times later. This issue is especially serious for systems which have limited
memory or when the working set size is larger than the cache available. As a matter of fact, a
server preforming multiple applications, one process's aggressive prefetching could be using
cache resources too much and cause other processes' performance to be lowered (Null and
Lobur 247).
On top of that, prefetching is almost non-existent, and even an adversary, in the case of
random-access workloads such as database transaction processing or file systems with non-
sequential data layouts. The reason for that is predicting the next block to be used is hard or
unfeasible thus prefetching becomes inefficient. Quite simply, in an online transaction
processing (OLTP) system, user queries may access data in unpredictable orders, so
prefetching may be obtaining irrelevant blocks thus wasting I/O bandwidth with no benefits
accruing. In these kinds of environments, it is usually advantageous to use adaptive caching
or demand-driven access strategies rather than speculative prefetching (Null and Lobur 248).
To balance those trade-offs, current operating systems and storage controllers have put in
place adaptive prefetching algorithms. These algorithms are capable of monitoring access
patterns on the fly and then changing prefetching behavior according to the degree of
sequentiality or randomness which has been noticed. To illustrate, the system may opt to
widen the prefetch window if it detects sequential access but on the contrary, it may cut
down prefetching or totally switch it off if random access is involved. Moreover, the
employment of context-aware prefetching might be another option wherein application
behavior or file metadata can be used for making even more precise predictions. As an
example, Windows operating system has SuperFetch which it uses to learn user habits and
thus preload files that are most likely to be accessed shortly (Null and Lobur 249).
Surname 31
Disk prefetching is notably one of the most effective ways to upgrade the system. It often
leads to a drop in I/O latency, an increase in disk throughput as well as a continuous data
flow from storage to memory. Most of its benefits can be realized in sequential or predictable
access pattern workload, e.g. media streaming, large file transfers, or database scans.
Nevertheless, it is not a perfect solution. The case of an inefficient or overdone prefetching is
a resource wastage, power consumption rise, and performance decrease in random-access
workloads. Hence, the success of prefetching is largely dependent on the precision of
prediction and the awareness of the adaptation of the workload. Today, systems are always
improving prefetching methods by employing dynamic algorithms and hybrid hardware-
software coordination to achieve the best level of responsiveness with efficiency (Null and
Lobur 250).
20. What are the advantages and disadvantages of caching disk writes?
Caching disk writes is an example of a common system performance optimization whereby
data to be stored on disk is held temporarily in a faster memory area, usually RAM or a
specially allocated cache buffer, before the actual disk write is performed. This method's goal
is to upgrade the whole input/output work-hands process by cutting shortening the time
needed as well as making the write operation to the user or application seem to be done
immediately (Null and Lobur 252). As the main memory is accessed much quicker than a
mechanical disk, caching disk writes could dramatically improve the performance that a user
sees and the system's responsiveness. Even though this scheme provides a lot of advantages,
it also comes with some inherent hazards and drawbacks that have to be taken care of by
proper system design and policy implementation.
Surname 32
The most important benefit of caching disk writes is the drastic improvement of write
performance. Writing data straight to disk is a process that entails mechanical changes, e.g.,
the movement of the read/write head and waiting for the proper sector to pass under the head.
The times for the former operation known as seek time and for the latter as rotational latency
together making the whole process particularly slow are the reasons why disk writes are
relatively slow. The system through the cache memory which stores the write operations
temporarily can thus signal the completion of a write request straight away even if the data
has not yet been physically written to the disk (Null and Lobur 253). This frees the CPU and
applications to continue with their work thus increasing throughput and decreasing the
system's idle time.
Another very important advantage is write coalescing. In cases where there are multiple
small write operations devoted to adjacent or related disk sectors, the cache can merge them
into one single larger write operation before actually writing the data to the disk. This is done
for the purpose of reducing the number of separate I/O operations, thus also minimizing the
mechanical overhead that is associated with several disk accesses. Such a procedure is
extremely useful in multitasking environments where, at the same time, various applications
can heavily and concurrently stimulate the storage with small write requests. Write caching
makes such requests behave thus efficiency getting better and storage getting less fragmented
(Null and Lobur 254).
Another example is how disk scheduling can accomplish this by grouping and prioritizing the
writes that are contiguous and combining them into a single write operation, thus reducing
the disk rotation times. The gains it realizes from this are often multiplied in the server,
Surname 33
database, and environment types situation where application speed and responsiveness are
heavily reliant on I/O performance (Null and Lobur 255).
There is no doubt that a further advantage of the technology is enhancing user experience and
system responsiveness. Users perceive that data is saved rapidly and that the system runs
smoothly as applications can continue their work immediately even after data has been
written to the cache. The responsiveness observed in multitasking operating systems
contributes to the system's stability and speed which are merely speed under heavy load
conditions. For example, when a user is editing a document or transferring files, there is a
possibility that the system may not be lagging because the data has been cached quickly, and
the slower disk operations are happening in the background (Null and Lobur 256).
Nevertheless, the technique has been if not obstructive, not-so-good, the most notable of
which is the problem of data loss. The cause is that cached data is stored in volatile memory,
which is lost in the event of power failure, crash, or hardware malfunction that occurs before
the data is permanently saved to the disk. In such instances, the operating system may
indicate that the data has been successfully written even though the data has not been
transferred to the storage device. The problem is very serious in particular systems that deal
with sensitive or irrevocable data like financial transactions, database records, configuration
files (Null and Lobur 257).
To remove the risk of data loss, some systems utilize write-through caching whereby data is
written to the cache and the disk at the same time. This method, although it guarantees data
safety, gives up a large part of the performance boost that comes with caching. On the other
hand, write-back caching postpones disk writes until some other time, hence, it provides
better performance but it is less reliable. In case of power failure, write-back cache usually
Surname 34
has to be supported by some non-volatile memory (like battery-backed or flash-based cache)
to retain the data. Thus, a lot of storage controllers and solid-state drives (SSDs) nowadays
are equipped with non-volatile write caches that help them keep the pending writes safe in
case of an abrupt system shutdown (Null and Lobur 258).
Another drawback is the possibility of cache inconsistency. In a system having several levels
of cache, like the one in an operating system, a disk controller, and a storage device, the data
in the cache may become different. Suppose the OS cache and the hardware cache store
different versions of the same data block. In that case, synchronization issues may occur
resulting in corrupted files or inconsistent states. This problem becomes worse in distributed
or networked systems where writing cache across nodes needs to be done in such a way that
data coherence is not violated and stale reads are prevented (Null and Lobur 259).
Moreover, the use of write caching may cause performance unpredictability in some
instances. When the cache is full, the system is required to flush some data to the disk in
order to make space available and, thus, it can suddenly drop its performance or increase its
disk activity. This is called cache flushing and, in this case, it can be very harmful to real-
time applications or time-sensitive operations, where the consistency of performance is more
important than the maximum throughput. Besides, if the cache is inefficiently managed, it
can lead to cache thrashing, a situation in which the cache is frequently updated with data
that is still not issued to the disk in an efficient manner (Null and Lobur 260).
Energy consumption is one more factor to be considered. If disk writes are cached, power
consumption may be increased because operations to maintain cache coherence and a flush in
the background consume CPU and memory resources. In portable devices like laptops and
Surname 35
tablets, quite an intensive write caching may result in batteries discharging faster, though the
impact varies with each system setup and load (Null and Lobur 261).
By caching disk writes a great deal of system performance can be achieved which makes the
technique a very powerful one. It substantially lowers the write latency, increases throughput,
and provides a more fluid multitasking experience. It achieves these benefits through
mechanisms such as write coalescing, scheduling optimization, and deferred writes.
Nevertheless, the method has somewhat of a risk potential, especially in terms of data
integrity and dependability. Unexpected power outages, cache inconsistencies, and erratic
flushing can cause data loss or corruption if the situation is not handled well. Contemporary
devices solve these problems by means of different methods like write-through caching, non-
volatile caches, and intelligent cache management policies. In the end, the choice to write
cache usage and the degree of aggressiveness in which to carry it out is a function of the
trade-off between performance and reliability requirements in a particular computing
environment (Null and Lobur 262).
21. How does virtual memory management affect overall system performance and
process efficiency?
Virtual memory management refers to the collection of hardware and software methods that
enable a computer system to provide an application with the illusion that it has a large block
of memory, even when the actual cache of the system is limited. The way virtual memory
works is by performing the required data operations through main memory and secondary
storage, typically a hard disk or SSD device to look like a bigger memory space. Besides, it is
a user-friendly programming model that supports multitasking, and it also makes it possible
to run huge applications which are capable of loading the physical memory beyond the
Surname 36
available space. Yet, although designing virtual memory in this way gives the system more
liberty and less stability issues, the layout and administration of it still have a major impact
on the system's and the processor’s performance (Null and Lobur 264).
Virtual memory management considerably affects system performance through one of its
main features - paging, the operation wherein the memory is cut into equal-sized pages.
Along with data, the pointer to its storage location comes as well, and if the data is not there
a page fault is raised, which consequently means an operating system has to load the
necessary page from secondary storage. This operation, which is referred to as page
swapping, results in a relay to heavy metal, in this case, it is very slow compared to memory,
thus significant time is lost for every page swap. If the number of page faults keeps rising,
which is situation goes by the name of thrashing, then the CPU most of the time is occupied
with connecting to data locations on disks rather than running user instructions. Thrashing
reduces system efficiency to a horrible extent and has the potential to bring down system
speed drastically (Null and Lobur 265).
Efficient page replacement strategies are crucial to recovering such performance losses. Least
Recently Used (LRU), First-In-First-Out (FIFO), and Optimal Replacement are algorithms
that decide which memory pages should be discarded when new ones have to be loaded.
Inappropriate or wrongly set page replacement policies may cause unnecessary swapping,
which in turn leads to the system’s effective throughput being wasted. Besides that, the size
of the page still affects the performance: on one hand, smaller pages lessen internal
fragmentation but increase the overhead due to more page tables; on the other hand, bigger
pages decrease the overhead, however, some memory may be 'wasted' in the case of partially
Surname 37
filled pages. Finding a balance between these trade-offs is paramount to achieving the best
performance (Null and Lobur 266).
Moreover, virtual memory management contributes to the overall efficiency of the process
by allowing process isolation and memory protection. Every process has its own virtual
address space, thus one process cannot interfere with another’s data. Apart from the fact that
this isolation elevates the system’s reliability and security, it also makes it possible for a
number of applications to be run concurrently without any concern for data corruption.
Additionally, through the use of memory in a flexible manner, the system is able to enhance
the utilization of its resources and be able to give the priority to those processes which are in
the state of activity. As a result, multitasking becomes fluent and the system’s general
responsiveness gets better, especially in situations where several users or applications are
sharing resources (Null and Lobur 267).
Translation lookaside buffer (TLB) management is another very important factor that
influences the performance of the system. The TLB is a cache that keeps the most recent
virtual-to-physical address translations. Since the page tables can be anywhere in the main
memory and thus accessing them can be very slow, the TLB is the one that makes memory
translation very fast. On the other hand, if the TLB is too small or if the TLB is inefficiently
managed, this may result in frequent TLB misses and in that case, more memory has to be
accessed and the performance is slowed down. That is why it is very important to TLB to be
equipped with a proper size and to have the replacement policy optimized because it is the
key to efficient virtual memory in large address space systems or systems having a high
process switching rate (Null and Lobur 268).
Surname 38
Virtual memory management also serves the purpose of memory sharing and copy-on-write,
which are two mechanisms that primarily facilitate multitasking systems. Suppose multiple
processes access the same shared library; in this case, the virtual memory system enables
these processes to use the same physical memory pages rather than creating new ones. In the
same way, with the copy-on-write method, the moment when duplication of data takes place
is only when a process modifies the data thereby saving memory and CPU time. Such
methods not only lessen the redundant data storage but also make multitasking more scalable
and resource-efficient (Null and Lobur 269).
On the other hand, virtual memory has some overhead in terms of hardware and software
even though it has many advantages. For maintenance of page tables, address translation, and
page fault handling, CPU cycles must be spent. If there is not enough RAM in a system and
the system is reliant on disk-based virtual memory, the performance will be drastically
lowered due to slow data transfers. Hence, the efficiency of virtual memory is conditioned on
factors such as the size of the physical memory, characteristics of the workload, and the
memory management algorithms of the operating system. In order to minimize the paging
delays, high-performance systems utilize a large amount of RAM and SSDs (Null and Lobur
270).
22. In what ways do compiler optimizations influence instruction-level parallelism and
execution time?
Compiler optimizations are one of the major contributors to program execution efficiency as
it basically maximizes instruction-level parallelism (ILP)— which is the processor's ability to
execute multiple instructions at the same time. Essentially, compilers of today examine the
source code and convert it into the optimized machine code that is most suitable for the
Surname 39
hardware pipeline architecture, cache hierarchy, and parallel execution units. With the help
of compiler optimizations, which is done by rearranging the instructions, reducing the
dependencies, and minimizing the stalls, the execution time is directly reduced and system
throughput is increased (Null and Lobur 272).
Instruction scheduling is one significant method through which a compiler can increase ILP.
The processor instruction set is reordered to reduce pipeline hazards such as data, control, or
structural dependencies that might occur. For instance, the compiler can move independent
instructions between the one that is waiting for the result of the previous and the previous
one, so that the pipeline is kept filled. Hence, fewer cycles are to be found where the
processor is idle and the continuous flow of instructions is guaranteed, which is indispensable
to superscalar and pipelined architectures. The concept of software pipelining is one of the
advanced techniques that goes beyond this idea by executing different iterations of the loop
at the same time, thus even more instructions can be processed in parallel (Null and Lobur
273).
One of the major optimizations also revolves around loop transformations namely loop
unrolling, loop fusion, and loop interchange. These methods restructure loops to have better
instruction scheduling and data locality. As an example, loop unrolling spreads a loop's body
by duplicating instructions, which cuts down on loop overhead and thus additional parallel
execution can be done. In the same manner, loop fusion merges loops that traverse the same
data set, which in turn shortens memory access latency and thus cache utilization is
improved. These techniques not only expose more ILP but also lower instruction count and
execution time through the reduction of redundant operations and the enhancement of data
reuse (Null and Lobur 274).
Surname 40
Compilers further support ILP through the use of register allocation and dead code
elimination. Proper register allocation is the main reason for the most frequently accessed
data to be kept in very fast CPU registers of the order of magnitude, while the slower
memory remains for less frequently accessed data. This in turn reduces memory latency and
instruction stalls. On the other hand, dead code elimination gets rid of unnecessary or
unreachable instructions, thus decreasing the total number of operations the processor must
execute. In concert, these measures facilitate instruction flow and provide faster execution
speed. Furthermore, branch prediction improvements mitigate control hazards partly by code
reorganization or the insertion of hints that help the processor to more accurately predict the
outcome of branches, thus lessening misprediction penalties (Null and Lobur 275).
23. Discuss how cache memory hierarchy design impacts CPU throughput and latency.
Cache memory hierarchy design is one of the most influential factors in determining a CPU’s
overall throughput and latency. Modern processors rely on multi-level cache systems,
typically designated as L1, L2, and L3 caches, to bridge the speed gap between the ultra-fast
CPU cores and much slower main memory. The purpose of this hierarchy is to store
frequently accessed data closer to the processor, reducing the average time required to
retrieve information and allowing the CPU to execute instructions at higher rates. An
efficiently designed cache hierarchy can significantly improve system performance by
minimizing memory access delays and keeping the processor pipelines active (Null and
Lobur 278).
The cache hierarchy operates on the principle of locality of reference, which includes both
temporal and spatial locality. Temporal locality suggests that recently accessed data is likely
to be used again soon, while spatial locality implies that data located near recently accessed
Surname 41
addresses will likely be needed shortly. L1 cache, the smallest and fastest, resides closest to
the CPU core and handles immediate data and instruction access. L2 and L3 caches are
progressively larger but slower, providing a layered system where data that is not found in
the higher-level cache can be retrieved from the next level. This multi-level structure ensures
a high cache hit rate, reducing the number of costly main memory accesses and improving
overall CPU throughput (Null and Lobur 279).
Cache memory hierarchy also directly impacts latency, the time it takes for the CPU to
access data. A well-designed cache hierarchy minimizes average memory access time by
balancing speed and size across levels. For instance, L1 cache can deliver data in just a few
nanoseconds, while accessing main memory may take hundreds of nanoseconds. If the cache
hit rate is high, the CPU experiences fewer delays waiting for data, enabling more
instructions to be executed per cycle. However, when cache misses occur, particularly in the
lower levels, the resulting stalls can significantly degrade performance. Therefore, effective
cache management strategies, such as prefetching and write-back policies, are essential to
maintaining low latency and high throughput (Null and Lobur 280).
Besides that, the performance and bandwidth of the cache hierarchy between CPU levels
depend on the organization and associativity of the caches as well. Set-associative caches
combine the advantages of speed and flexibility of a direct-mapped cache, while also
lowering the number of conflict misses that arise. Larger block sizes, for example, may
increase latency as more data than necessary will be fetched, however, they can make better
use of the spatial locality principle. The trade-off between cache size and access time (i.e.,
how long it takes to search the cache) is another matter that the designers have to figure out.
In order to support coherence and resource utilization, modern CPUs frequently make use of
Surname 42
shared L3 caches across multiple cores, which in turn promotes efficient data sharing in
multicore environments. This hierarchical coherence management, in turn, allows further
parallel performance improvement thus, ensuring throughput at the same level even under
heavy workloads (Null and Lobur 281).
Cache memory hierarchy design is the main factor that affects CPU performance. The
hierarchy is a balance between the aspects of speed, size, and efficiency of the different
cache levels. An efficiently organized hierarchy not only shortens the memory access time
but also raises data availability and CPU throughput by making sure that the majority of data
requests are fulfilled from fast cache memory rather than from slow main memory. By
perfecting cache organization, associativity, and prefetching techniques, contemporary
processors can reach higher levels of parallel execution and thereby reduce the number of
idle cycles, which in turn results in faster and more responsive computing performance (Null
and Lobur 282).
24. What are the main differences between static and dynamic performance analysis
techniques?
Static and dynamic performance analysis techniques signify two different sets of tools that
are usually employed to figure out and improve the performance level of computer systems.
The biggest distinction between these two forms of analysis lies in the time and mode in
which the inspection is carried out. Static performance analysis investigates the source code
of a program or its compiled binary without the execution of the program. Meanwhile,
dynamic performance analysis is all about a run-time review when a program is being
executed to record CPU usage, memory access patterns, execution time, and other real-time
performance data (Null and Lobur 285).
Surname 43
Static performance analysis is all about locating potential performance-related problems in
the code that later will be run. It throws out inefficiencies such as unnecessary calculations,
wrong loops, and excessive branching. Being that it does not require execution, static
analysis is a great tool for coming up with early-stage optimization and finding structural
defects that hamper efficiency. This method is mostly based on compiler analysis tools which
are used for instruction counts, control flow, and data dependencies when predicting
performance. Unfortunately, it is not possible for the static analyzer to consider runtime
behavior, e.g., dynamic memory allocation, input variability, or system load, that can affect
the real-world performance (Null and Lobur 286).
Conversely, dynamic performance analysis serves as an experiment that can only be true if
the observations can come from the actual hardware a program is running on. The data thus
obtained are like cache hit rates, instruction throughput, and I/O delays, showing how the
program cooperates or interacts with the system. The runtime performance bottlenecks and
difficulties caused by system contention or data-dependent operations can be easily
diagnosed by runtime profiling and simulation tools. The major merit of dynamic analysis is
that it actually reflects real system behavior under different workloads. However, it still has
its limitations as it brings execution overhead and at times, certain runtime situations cannot
be easily recreated (Null and Lobur 287).
25. How can parallel processing and multithreading improve performance, and what
limitations exist?
Parallel processing and multithreading are two major points of focus that bring about vast
changes in computer performance by allowing the execution of different tasks at the same
time. Essentially, they both come from one idea concurrency which is further exploited to
Surname 44
increase system throughput, reduce latency and enhance resource utilization. In short, parallel
processing is about handling large computational tasks by breaking them down into smaller
units that can be run simultaneously on different processors or cores, whereas multithreading
is about allowing a single process to carry out multiple operations concurrently within the
same application. These methods, when flawlessly carried out, can hugely shorten the time of
execution and increase the total system capacity, thus making the system more responsive
(Null and Lobur 290).
Essentially, parallel processing makes use of the power in different processors or cores to
handle independent computations at the same time. It, therefore, is very effective in situations
where a lot of data are involved or the computation is so heavy to handle such as scientific
simulations, large-scale data analysis, and 3D rendering. Parallel systems through their
multiple processors divide and conquer the work thus finishing the job much faster as
compared to the case of a single-core processor. In other words, the division of labor makes
sure that there is no idle CPU time and that the hardware is efficiently utilized. The gains that
come with parallelism are in most cases, shown through speedup which is a metric that
shows the rate at which a task is done in parallel compared to when it is done sequentially.
Nevertheless, the power of parallel processing is constrained to some extent by the factors
that are used to divide the work since there are some issues which are designed in such a way
that they have to be solved sequentially thus limiting the degree to which the performance
can be improved as it is stated in Amdahl’s Law (Null and Lobur 291).
Multithreading is a related term that denotes the ability of multiple threads within a single
process to share memory space and system resources. A thread, as a part of the whole, can
take care of any task, e.g., input/output operations or background computations. Thus,
Surname 45
asynchronously in a web browser, one thread may take care of user input while another
depicts images. The overlaps reduce the waiting time and make the system use faster. At
multicore systems, multithreading enabled true parallelism where threads physically run on
different cores thereby performance up-gradation results. In a single-core system, also there
is multithreading that the processor switches between threads during the cycles which are
idle because are waiting for data or I/O operations (Null and Lobur 292).
Still, a few limitations are faced by parallel processing and multithreading apart from their
benefits. The foremost challenge is synchronization overhead, a situation when different
threads or processors that access the same resources coordinate to prevent conflicts or data
corruption. The coordination leading to the delays and which reduce the total effectiveness of
the system is the cause behind this. Moreover, some tasks cannot be divided into smaller
tasks. Such programs as those heavily dependent on data or having sequential logic may gain
slightly from concurrency. Besides, the complexity of the software increases due to hard
parallel management; developers should tackle race conditions, deadlocks, and load
balancing among other problems thus ensuring stable performance (Null and Lobur 293).
Moreover, physical limitations pose a challenge to scalability. With an increase in the
number of processors, the time required for communication between cores may act as the
performance bottleneck. Memory bandwidth and cache coherence protocols are also factors
that limit performance as several cores demand access to the same shared data. Furthermore,
the gains coming from multithreading will be minimal if the operating system or application
is not designed for concurrent execution. Additional threads in such scenarios may affect the
performance negatively because of the context-switching overhead (Null and Lobur 294).
Surname 46
26. Explain how system bottlenecks are identified and mitigated in performance tuning.
System bottlenecks are scenarios where restrictions of conditions limit the overall
performance of a computer system by controlling the rate of tasks processed. The recognition
of such bottlenecks and their removal is a primary point of performance tuning which targets
the optimization of system efficiency and its response capabilities. Bottlenecks can be
ascribed to any source of the computing environment like the CPU, memory, disk storage, or
network and usually happen when a single part of the system is at or close to full capacity
while the rest is still underutilized. To eliminate these bottlenecks, one has to take a very
deliberate and thorough approach, which includes the steps of measurement, analysis, and
optimization (Null and Lobur 297).
One of the most significant parts in finding bottlenecks is the performance measurement
stage, where the system metrics are followed through profiling together with diagnostic tools.
In effect, the CPU utilization, the memory access, the disk I/O throughput, and the network
latency are only a few of the metrics that indicate areas of inefficiency. For example, a high
CPU utilization with scarce memory or disk activity might suggest that the processor is
overloaded, whereas long disk queues could indicate slow storage performance. The major
sources of data for quantitative system behavior are performance counters, event logs, and
benchmarking tools like SPEC or TPC. By looking at present performance metrics against
the baseline values, administrators have the authority to tell if a particular subsystem that has
been inefficiently utilized is underneath its optimal performance level (Null and Lobur 298).
After a bottleneck has been located, the following stage includes performance analysis that
makes use of the data accrued to single out the roots of the problem. It might be the case that
an inefficient algorithm, the occurrence of excessive context switching or the lack of
Surname 47
parallelism that are the reasons behind the CPU always running at its maximum capacity. On
the other hand, if the memory utilization is high, the reason for that might be insufficient
RAM or wrong memory management, which leads to paging. As for the bottlenecks related
to disks, they usually are the result of fragmented files, limited cache buffers, or inefficient
disk scheduling algorithms. Knowing the relationship between these parts is quite important
because once you improve one without solving the root problem, you might simply move the
bottleneck to another part of the system (Null and Lobur 299).
Mitigation strategies are dependent on the characteristics of the bottleneck. If the problem is
CPU-related, then one or all of these measures could be taken: optimizing the code,
increasing parallelism, or getting a newer/more powerful processor. Remedies for memory
bottlenecks include installing more RAM, increasing cache size, or upgrading virtual
memory management. Solutions for disk bottlenecks involve defragmentation, switching to
faster SSDs, implementing disk caching, or changing scheduling algorithms such as going
from FCFS to SSTF or LOOK for more efficient disk usage. Network bottlenecks, on the
other hand, can be solved by steps like raising bandwidth, trimming packet sizes, or using
load-balancing techniques to distribute the traffic uniformly (Null and Lobur 300).
Balancing trade-offs is an important part of performance tuning. As an example, the
expansion of cache memory may speed up the system but at the same time increase its cost;
similarly, if you turn on aggressive prefetching it may cut the latency but at the same time
increase memory usage. Hence, the presence of a continuous monitoring system along with
iterative refinement are the only ways leading to the optimal performance of a system. Tools
providing real-time analytics and predictive modeling give the freedom to system
Surname 48
administrators to be a step ahead of bottlenecks and hence they can take preventive measures
before system throughput gets decreased (Null and Lobur 301).
Finding and fixing bottlenecks in the system is an ongoing journey that requires accurate
measurement, analysis, and precise adjustment. By implementing performance metrics and
diagnostic instruments, system designers, and administrators, are able to pinpoint the exact
components that are the most performance-limiting, and then they can take the necessary
corrective steps. Well-done performance tuning not only eliminates the existing bottlenecks
but also leads to the overall stability, scalability, and efficiency of the system which in turn
makes sure that computing resources are utilized to their fullest for maximum productivity
(Null and Lobur 302).
27. What role do hardware counters and profiling tools play in performance
measurement and improvement?
Hardware counters and profiling tools are the main instruments that help in measuring and
optimizing the performance of a system. These tools give detailed information about the
usage of hardware and software resources which allows the engineers, system administrators,
and developers to effortlessly locate the sources of inefficiencies and thus make the
performance-enhancing decisions based on the given data. The tools by following the
programs' behavior during execution and also checking for the occurrence of even the
smallest system events help in a very instrumental way in resolving bottlenecks, increasing
the efficiency of the code, and hence being able to extract more from the hardware (Null and
Lobur 304).
Surname 49
Hardware performance counters are limited registers that are purposely added to modern
processors in order to keep track of certain hardware-related events for example CPU cycles,
cache hits and misses, branch predictions, as well as memory accesses. These counters work
at an extremely low level, giving users the ability to obtain very accurate data on the
performance of the system without the need for a heavy burden of the system. In fact, they
can determine the performance of the CPU pipeline with a given program or how often the
cache misses are causing the stalling. Accordingly, this quantitative data provides the means
for developers to single out the directions in which inefficient instructions, poor memory
access patterns, and suboptimal algorithm designs are hiding. Carrying out these code
changes, programmers can achieve a higher level of parallelism, thus enabling their codes to
run faster and more efficiently (Null and Lobur 305).
Profiling tools by themselves work more conceptually and give out the overall analysis of
system and application behavior. Profilers like gprof, Perf, Intel VTune, and OProfile gather
the performance data either by sampling or by instrumenting the program execution. They
also take the measurements of the metrics such as the time when the function is executed, the
space in memory which is being used, thread activity, and I/O operations. Profilers are the
ones that indicate the performance-critical parts of the code most of which are “hot spots”
and these are the parts that require the most resources. The developers then can use their
optimization skills in the most efficient way by doing it in these areas rather than the parts
that have the least performance impact (Null and Lobur 306).
Working in tandem, profiling utilities and hardware counters depict an all-around picture
concerning the performance of a system. Counters built into hardware give very detailed data
on events happening at the lowest level, whereas profiling tools provide means of
Surname 50
communication in terms of practical steps. This, in turn, allows the bridge between hardware
engineers and software developers, highlighting the scenarios in which the substandard
performance arises due to inefficient coding or lack of hardware support, or merely due to
processor design. The changes can also be confirmed as first, optimization, second, actual
performance gains, and third, the prevention of new inefficiencies because of the measures
which have been taken by these devices (Null and Lobur 307).
Hardware counters and profiling tools together are the major instruments that can measure
and improve system performance by showing how the hardware works. Besides pinpointing
precisely where the system should be improved, they make the entire process more direct, as
well as ensuring that the resources are being utilized correctly. Eventually, through constant
usage of these instruments, the process of perfecting system performance and reliability
becomes possible (Null and Lobur 308).
Surname 51
Works Cited
Null, Linda, and Julia Lobur. The Essentials of Computer Organization and Architecture. 4th
ed., Jones & Bartlett Publishers, 2014.
