Discussion

Performance Measures

Developed by UMUC Faculty

What are performance measures?

• Performance measures are quantifiable indicators that are used to determine how well an organization or business is achieving its objectives

• One use of performance measures is to determine how well an IT system is doing in meeting the expected outcomes

• IT Performance measures include: • Quantifiable benefits from using an IT system

• When those quantifiable benefits should be realized, and

• How the results will be determined

When should performance measures be developed? • Performance measures should be developed as part of the business

case that proposes, explains and justifies the implementation of an IT system

• They are derived from the benefits expected from the proposed system

• They strengthen and help to defend the system proposal

Why do we need performance measures?

• Performance measures demonstrate how the proposed system is expected to impact the business

• Performance measures show how, when, and how much the system can be expected to improve the business

• Performance measures tell the decision-makers and stakeholders • What quantifiable benefits they should receive by implementing the system

• When they should realize those quantifiable benefits

• How the results will be determined- what will be measured and where the data will come from

What should be included?

• Business improvements that stakeholders are interested in • Financial improvements

• Improved customer service

• Improved customer satisfaction

• Improved (faster, less expensive) business processes

• Support for strategic goals

• Ability to expand the business

System Performance Measures • Such things as

• Availability • Reliability • Response time • Usability

• Not included in the business benefit performance measures • Stakeholders are focused on business measures • Stakeholders assume system will be expected to perform well

• System Performance Measures are still important and may be included in the list of requirements and Service Level Agreements with vendors

Project Performance Measures

• Such things as • Schedule/time to implement

• Cost

• Functionality

• Quality

• Not included in the business benefit performance measures • Stakeholders will be focused on these during the development period

• Stakeholders assume the project will be well managed

• May be included in SLAs or contracts with service providers

System and Project Performance Measures

• Stakeholders are interested in how the system performs and how well the IT project is managed

• These should be monitored and managed by the IT managers

• Stakeholders should receive regular reports on project progress and system performance

• In general, neither of these are meaningful as business performance measures

• These types of measures do not provide stakeholders with information about how the system improves the business

Developing “SMART” Performance Measures

• Specific – measures should be specific, clear, concise

• Measurable – need a metric (a “thing” that can be measured) that makes sense and is available

• Achievable – if measures are not achievable, do not include them

• Relevant – must relate to the business and be an expected improvement from the system being implemented

• Time-Based – target timeframe needs to be established

Documenting Performance Measures

• Performance Measures are documented using the following: • Expected Improvement (e.g., increased sales)

• Metric – what will be used to measure progress (e.g., monthly sales totals)

• Source of Data – where the metric data will come from (e.g., monthly report)

• Baseline – the starting point (e.g., current monthly sales)

• Target – estimated improvement point (e.g., monthly sales after system implementation)

• Timeframe – when the target should be reached (e.g., six months after system implementation)

• Performance Measures are often documented in a table, as shown on the following slides

Table of Performance Measures

Problem or Expected Improvement

Metric – used to measure

Source of Data Baseline Target Time-frame

Decrease time to log in item for repair

This example uses a repair shop, for which the proposed system will reduce the time it takes to log an item in for repair. The expected improvement – decrease time to log in item for repair – is entered in the first column.

Table of Performance Measures

Problem or Expected Improvement

Metric – used to measure

Source of Data Baseline Target Time-frame

Decrease time to log in item for repair

Time to record item entry into the repair process

The metric – the “thing" that can be measured to determine if login time is being reduced – is the actual amount of time it takes to complete the task. In this example, we will be comparing the time it takes to manually get out the log book, write in all the information and write a receipt for the customer – to an automated process where the data is entered through a computer screen and a receipt is automatically printed by the system.

Table of Performance Measures

Problem or Expected Improvement

Metric – used to measure

Source of Data Baseline Target Time-frame

Decrease time to log in item for repair

Time to record item entry into the repair process

Task observation and stopwatch reading

Next, we have to determine where the data will come from to tell us how we are doing. In order to determine the amount of time needed to login an item, we will observe someone performing the task and measure the time with a stopwatch.

Table of Performance Measures

Problem or Expected Improvement

Metric – used to measure

Source of Data Baseline Target Time-frame

Decrease time to log in item for repair

Time to record item entry into the repair process

Task observation and stopwatch reading

Four minutes

The baseline column shows what the source of the data shows for the current way of doing business, prior to system implementation. In our example, it will be an observation with a stopwatch to see how long a manual login and receipt creation take. It shows four minutes for the current process.

Table of Performance Measures

Problem or Expected Improvement

Metric – used to measure

Source of Data Baseline Target Time-frame

Decrease time to log in item for repair

Time to record item entry into the repair process

Task observation and stopwatch reading

Four minutes Two minutes

The target shows what improvement is expected. We need to estimate how long the login and receipt generation process will take after the system is implemented. The estimate is that it will take about a minute for the clerk to enter the data into the system and about 30 seconds for a receipt to be printed. Adding a little slack time for slow data entry, we set the target for 2 minutes.

Table of Performance Measures

Problem or Expected Improvement

Metric – used to measure

Source of Data Baseline Target Time-frame

Decrease time to log in item for repair

Time to record item entry into the repair process

Task observation and stopwatch reading

Four minutes Two minutes Two weeks after implementation

The final column provides the timeframe by which the target should be attained. It could be immediately after the system is implemented, or in a longer time period. For this example, it is predicted that by the end of two weeks after system implementation, the clerk should be familiar enough with the system to login an item and print a receipt within the targeted two minute timeframe.

Using the Performance Measures

• Performance measures are not to be developed and forgotten

• Regular progress reports should be provided to the IT Governance Committee and other stakeholders to apprise them of progress toward meeting the goals

• Additional measures beyond those initially conceived may be added during the life cycle of the system

• Achievement of the performance measures forms part of the analysis of whether the system was a success or not, and should be considered when future system proposals are developed

In Summary

Performance Measures

• Help justify a system proposal

• Focus on business benefits

• Clearly show how the business will benefit by implementing the system being proposed

• Gain the support of decision-makers to implement the system

• Demonstrate achievement of the system objectives (or not)