IFSM DQ

Implementing a SaaS Solution

Differentiating Between Commercial Off‐the‐ Shelf Software (COTS) and SaaS Solutions

Up to this point, we have been using the term commercial off‐the‐shelf

(COTS) to include software‐as‐a‐service (SaaS) solutions. COTS is most‐

often used to refer to applications that are purchased and installed at the

user location, either on a personal computer or on a server for multiple

individuals to use. This includes such familiar purchased software as word

processing or spreadsheet applications. Some COTS solutions come with

vendor maintenance and updates, while others require an additional

payment to be made for an upgraded version. Once the organization

purchases a COTS solution, the vendor's involvement in the day‐to‐day

operation is nonexistent.

A SaaS solution, on the other hand, is usually leased or subscribed to by

the customer, and the software is owned by the vendor, runs on the

vendor's hardware, and is accessed via the internet as a "service."

Microsoft is now providing its office applications as a service via Office

365 for Business, which is provided as a subscription service rather than a

purchased download. In that instance, it becomes a SaaS application.

Even though it is a COTS product, the way it is delivered to the end user

via the internet, along with ongoing service and maintenance from the

vendor, makes it a SaaS solution. Other well‐known SaaS products are

Learning Resource

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

1 of 14 2/28/2021, 9:55 AM

SalesForce (customer relationships management system), Amazon Web

Services (eCommerce platform), and DocuSign (electronic signature

services). For SaaS solutions, the vendor is responsible for the day‐to‐day

operation of the system, for the ongoing operation and maintenance of

the system, for protecting the sensitive business data housed in the

system, for upgrading and enhancing the system, and for providing

training and support. Usually all that is required at the customer location

is an internet connection and end‐user devices to connect to the system.

Unique Considerations for Selecting a SaaS Solution

When a SaaS solution is being considered, a primary aspect is that the

relationship with the vendor is very different from a solution that is

hosted on‐site at the organization. A long‐term relationship is established

with the vendor beginning with the lease or subscription to the system.

The customer becomes reliant upon the vendor for all the services listed

above.

Since the system is not purchased (instead, the customers are "renting" or

"leasing" the software and services), the customer will make monthly or

annual payments for its use; these can either be a set amount or can

fluctuate depending on the actual use of the system. A Service Level

Agreement (SLA) is used to document the responsibilities and

commitments of the vendor and the customer. Most vendors of SaaS

solutions have an SLA already developed for their customers; this should

be studied thoroughly, and changes negotiated if necessary, prior to the

customer signing up for the services.

One big consideration is that the system is operated at the vendor's

location. It is much more likely that a vendor supporting multiple

customers can achieve a higher level of security for the system than an

individual organization. The vendor has the combined resources to hire

and retain security experts to manage the system, the hardware, the

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

2 of 14 2/28/2021, 9:55 AM

network, and the facility. Many SaaS vendors have implemented a

distributed system so that hardware, software, and databases are housed

at multiple locations; many vendors provide "hot backup" meaning that

the database is replicated elsewhere so that if one database or system is

unavailable, there is an automatic switch to the replicated database. SaaS

vendors also can afford to offer quick recovery at a much lower cost than

is available to an individual organization. They are also much more likely

to have physical security measures in place to protect the data center,

including fire suppression, surveillance, access security, and guards.

Since SaaS solutions depend on use of the internet to connect users to

the application or system, the following should also be considered:

the availability and speed of the internet connection;

protection of proprietary or personal information transmitted via the

internet; and

location of the system. Some government systems are required to be

hosted within the United States, and not overseas.

Identifying COTS/SaaS Solutions

Over the past decades, COTS and SaaS solutions have proven to be viable

models for acquiring software. SaaS is now a mature model that can be

relied upon if a vendor is selected based on a deliberate evaluation and

selection process. There are many sources for locating a vendor, including

technical journals, industry survey, vendor advertisements, advisory or

consultancy services, and even internet searches. An organization would

be wise to identify a few solutions that appear to meet their needs and

then conduct a detailed evaluation of each one. It is important to identify

solutions that align with achieving the business strategy, improve the

process(es), and meet the requirements.

Evaluating COTS/SaaS Solutions

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

3 of 14 2/28/2021, 9:55 AM

In evaluating a COTS or SaaS solution, four major factors are involved:

user requirements, system performance requirements (system quality and

security requirements), the vendor, and cost. The method for evaluating

each of these is discussed below.

Most SaaS vendors provide access to a "free" trial version of their system.

During the product evaluation period, the trial version can be used to

determine the basic functionality and performance of the system. This

version of the software is used for marketing purposes and may not

exactly represent how the software would function in a specific situation.

Some vendors may offer to provide access to a more robust version of

their system in order to allow further testing and evaluation. An

organization should try out the software for itself and not rely on vendor

demonstrations, which can be set up to appear to provide functionality

and ease of use that is actually not part of the system.

User Requirements

The first step in evaluating a COTS/SaaS solution is to address the user

requirements and answer the following questions:

How closely do the capabilities and functions of the solution meet

the requirements?

Conversely, are there a lot of extra "bells and whistles" that the

organization does not need or would not use, but add to the cost and

complexity of the system?

How closely does the application package fit the process used by the

organization? If the solution is implemented, would the organization

be able to use it for their process? Will the business process need to

change significantly, requiring additional training and other

organizational changes? Would the changes in the process used by

the vendor’s solution actually help improve the business process?

The more the business process has to adapt to the system, the less

likely the system is to be accepted by the users. If significant

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

4 of 14 2/28/2021, 9:55 AM

differences exist between the system and the process in use, and

major changes are required to the off‐the‐shelf system, the cost,

complexity, and risk may well outweigh any benefits of the COTS

solution. However, if the organization is seeking to improve its

business processes, COTS/SaaS solutions often implement optimized

business processes in the software, a benefit for the organizations

that implement them.

How much configuration or customization will be needed to put the

COTS system into use? Some COTS products allow or require

extensive configuration or customization in order to make the

system useful to any organization. Others require minimal

configuration to set the system up for use in a specific organization.

These activities are major determinants of initial cost and

implementation time, and add to the ongoing maintenance costs.

Configuration is functionality that can be created using built‐in

workflow tools and templates that come with the product.

Customization is functionality that is added to or replaces

functionality as provided by the vendor. There is no guarantee

that customizations will be compatible with future upgrades,

and they can be extremely costly to maintain over time.

How much and what data will need to be migrated to the new

application/system? How easily can that be accomplished and at

what cost? The organization likely has information that supports the

process for which a system is being sought, and that information will

most likely need to be imported into the new system. If the data is

already in electronic form (in a spreadsheet or database), the

migration of that data should be accommodated. However, if data is

in paper form, decisions will need to be made about how much of the

existing data is to be manually entered into the system, and in what

form it will be entered.

Migrating data into a new system can be very time consuming and costly,

so these are important considerations for the organization.

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

5 of 14 2/28/2021, 9:55 AM

System Performance Requirements

Next, the quality of the COTS/SaaS solution is evaluated answering the

following types of questions regarding the attributes of the system (which

are specified as system performance requirements):

Usability—Can new users quickly adapt to the software? How easy is

the system to use, and how is help provided for the users? Does the

vendor provide training? Is online help provided in the system? Is

user support provided (e.g., a help desk or documentation)?

Scalability—Can the system accommodate the anticipated number of

eventual users and/or records/transactions? Can it be scaled back if

there are actually fewer users or transactions?

Availability—Will the system be available for use when needed? If

there is any anticipated maintenance downtime, is that compatible

with the organization's needs?

Reliability—Does the system create and maintain the data correctly?

Maintainability—What is the vendor's approach to maintenance and

how often are updates applied? How quickly can corrections be

implemented?

Performance—Is the system able to meet response time

requirements? Is it able to handle the volume of the expected

workload (or number of transactions)?

Portability—Does the system run or operate on the types of end‐

user devices and operating systems that the organization uses or

anticipates using?

Interoperability—Is the system capable of exchanging data with any

required legacy (existing) system?

Security—What security protections are provided by the vendor?

What security steps are needed within the organization? How is the

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

6 of 14 2/28/2021, 9:55 AM

system protected from malicious or accidental actions? How will

users authenticate to the system and be authorized to perform

functions and/or access data? Does the system effectively prevent

unauthorized access and prevent unauthorized ability to change

data? How is data protected as it is transmitted and when it is

stored? Does the system keep a log of who logged in, when they

logged in, what information they accessed and what changes they

made? What data backup and recovery is provided by the vendor?

The answers to these questions will help determine whether the

system provides adequate security.

Vendor Ability

The vendor's ability to support the organization and provide the services

needed is a third area of consideration. The organization should do its due

diligence and consider the financial stability of the vendor and look at

such things as how long they have been in business, how robust their

customer support is, and their industry reputation. The number of paying

customers and the length of time they have been with the SaaS vendor is

a good indication of the quality of the software and the vendor's services.

In evaluating a SaaS vendor, it is a good idea to check with some of their

customers to learn about their experience with the SaaS. The

organization needs to ensure the vendor will be able support it for some

time to come. Keep in mind that once the organization signs up, the

expectation is that there will be a long‐term relationship—the

organization does not want to keep changing its SaaS software and

vendor, and the vendor will want to keep the organization as a long‐term

customer providing recurring revenue. At the end of the day, the

organization is responsible for the use of the system as it impacts their

employees and customers. Although the vendor owns and hosts the

system, the reputation of the organization can be at risk if issues arise

and are not properly addressed.

Total Cost of Ownership (TCO)

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

7 of 14 2/28/2021, 9:55 AM

The fourth area of consideration is the cost of the COTS/SaaS solution. In

determining how a system is to be acquired and/or which system is to be

acquired, the organization must consider the total cost of ownership

(TCO) of the solution. The TCO for each alternative can be estimated in

order to make comparisons. This concept is something we are very

familiar with when we are making a major purchase in our daily lives. In

general terms, the total cost of ownership (TCO) is the sum of all costs

associated with an acquisition that will accumulate over the life of the

asset. One of the personal acquisitions for which we use the TCO is the

purchase of a new car. Clearly, the purchase price is not the only

consideration. Today, automakers recognize the importance of the TCO to

their customers; in their advertising, they talk about gas mileage, resale

value, length of warranty, free servicing over some period of time, and

special financing terms.

The table below identifies the cost categories of an IT TCO. Although

there are several ways of categorizing and listing the costs, this list

contains some of the often overlooked and crucial costs that are

important to understand. The specifics of how the categories apply to a

SaaS solution are also provided.

Cost Categories of an IT TCO

Cost Categories Description

Costs as Applied to SaaS Solution

acquisition The costs of acquiring IT

assets: the lease, purchase, or

subscription cost of hardware

and software, including

research, travel, freight, and

tax; and/or the cost of

developing the software from

scratch.

Lease or subscription costs

for software and system

(SaaS vendor).

Purchase or lease of end‐

user hardware devices

(PCs, tablets, printers,

etc.).

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

8 of 14 2/28/2021, 9:55 AM

Cost Categories Description

Costs as Applied to SaaS Solution

communications The cost of all

communications, including

network costs, wiring, service

provider fees,

communications hardware,

and software.

Initial setup costs of

Internet Service Provider

(ISP) and ongoing monthly

charges.

security The costs of ensuring

security of the IT

infrastructure and data,

including security software,

usage monitoring, and facility

security costs.

Most security services

provided by vendor,

documented in the SLA.

End‐user policies and

device protection are the

responsibility of the

customer organization.

installation The costs of making IT assets

operational; could include

building modifications,

increased cooling

requirements, and increased

utility capacity at the

datacenter.

Responsibility of the

vendor.

configuration The costs associated with

COTS or SaaS software to set

it up to function correctly

within the organization; using

built‐in tools such as

workflow, report layout,

terminology and/or

organizational logo.

Costs to configure SaaS to

function for the

organization (e.g.,

workflow, reports,

terminology, logo).

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

9 of 14 2/28/2021, 9:55 AM

Cost Categories Description

Costs as Applied to SaaS Solution

customization The costs of making changes

to the COTS or SaaS

software that are unique to

the organization. The

ongoing cost of maintaining

these changes over time and

testing future upgrades must

be considered as well.

Costs to make changes to

the software for the

specific customer; may

cause additional cost for

maintenance.

testing The costs of preparing test

cases and using the system to

determine whether it is

functioning properly and

meets the requirements. Also

includes the costs of

recording deficiencies and re‐

testing when changes are

made.

Costs generally are limited

to the customer creating

and using test cases to

ensure the system works

as needed. This is very

different from using a

demonstration or “free

trial” system before

selection; it is testing the

actual system after it is

configured and is

operational for the

customer.

support The cost of keeping the

infrastructure functioning as

planned; could include a help

desk, hardware technicians,

telecommunications

specialists, programmers, and

maintenance support staff.

Most costs borne by

vendor. There may be an

additional charge for user

help‐desk support or

technical support, or it

may be included in the

monthly/annual fee.

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

10 of 14 2/28/2021, 9:55 AM

Cost Categories Description

Costs as Applied to SaaS Solution

maintenance The cost of keeping IT assets

current and in a condition

that can meet their planned

functions; includes updates

and enhancements as well as

fixes for problems; could

include maintenance

contracts, programmers, and

telecommunications

specialists.

These costs are borne by

the vendor. The customer

pays a monthly/annual fee

for ongoing service and

system maintenance.

coordination

costs

The costs related to keeping

the infrastructure tuned to

maintain optimal

performance when changes

to an infrastructure element

are required

These costs are borne by

the vendor.

disaster recovery The costs of ensuring

continued operation of the

infrastructure, including

maintenance of a current

plan, cost of backup sites and

equipment, costs of

emergency power, and costs

of practice exercises.

Most of these costs are

borne by the vendor (if the

vendor provides disaster

recovery services) since

the vendor is responsible

for its hardware, software

and internet access; but

the organization is

responsible for its own

infrastructure (end‐user

devices, internet access,

local power, etc.).

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

11 of 14 2/28/2021, 9:55 AM

Cost Categories Description

Costs as Applied to SaaS Solution

organizational

change

management

Any costs associated with

organizational changes

resulting from

implementation of the

system; includes such things

as consolidating

departments, establishing

new groups or

responsibilities, reorganizing

or reassigning personnel.

Always a customer cost.

data migration The costs of determining

what existing data (either in

electronic or paper form)

would need to be entered

into the system to get

started, and entering that

data.

The customer must bear

the cost of determining

what existing data

(electronic or paper) is to

be entered into the

system.

The cost of entering the

data is borne by the

customer; sometimes the

vendor is willing to assist

for a fee.

SaaS solutions generally offer many of these categories of service as part

of their initial fee and/or the ongoing maintenance fee. All must be taken

into consideration when developing the TCO.

Making the Selection

In the end, a cost‐benefit analysis can be used to determine which

solution best meets the needs of the organization. All four factors

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

12 of 14 2/28/2021, 9:55 AM

discussed above must be considered, with the organization determining

which of them is most important or which combination of the factors best

suits that organization, considering any specific needs, such as security of

highly sensitive data, particular functionality that must be present,

controlling costs, etc.

Implementing the System

Implementation of a COTS or a SaaS solution is a major project for the

organization. A system owner and a project team should be designated,

and best practices for IT project management should be employed. A

project plan for implementing a SaaS solution should include the

following steps:

Establish the vendor agreement, contract or SLA; a mechanism needs

to be put in place to give the organization access to the system,

identify responsibilities of the vendor and the customer, and lay out

initial and ongoing costs.

Acquire the end‐user hardware and telecommunications, if

necessary, and/or validate the capability of existing hardware and

telecommunications to access and use the system.

Configure the system for use in the organization; identify what needs

to be done to implement the organization’s desired workflow,

reports, terminology, logo, etc.; identify who will configure the

system and how it will be done, and whether there is any additional

cost.

Develop a plan for User Acceptance Testing (UAT), and test the

configured system to ensure requirements are met and that it is

functioning correctly, including use of any user support tools or

services provided. The UAT plan explains how each requirement will

be specifically tested to ensure it is working properly and the

requirement is met. For example, if the requirement is that the

system determine the customer’s city and state based on the zip

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

13 of 14 2/28/2021, 9:55 AM

code entered, then a zip code would be entered into the system and

the result would be checked to ensure the correct city and state

were provided.

Apprise the employees of what is taking place and why, and make

any organizational or process changes that are needed. Leaders of

the organization need to be involved as sponsors and coaches to

encourage system adoption and use, and they should employ change

management techniques to ensure a smooth transition.

Train administrative personnel in their role(s) for supporting the

system.

Conduct user training.

Migrate the data needed to operate the system; determine how this

will be done (electronically, manually, etc.), who will do it, how long it

will take, and what it will cost.

Oversee operations to ensure continued end‐user support and

system maintenance are performed by the vendor according to the

SLA; identify any need for support or maintenance by the

organization itself, such as hardware and software upgrade for end‐

user devices, a local help desk, etc.

Using a comprehensive project plan as laid out above will help ensure a

successful implementation and ongoing support for the new system.

© 2021 University of Maryland Global Campus

All links to external sites were verified at the time of publication. UMGC is not responsible for the

validity or integrity of information located at external sites.

Implementing a SaaS Solution https://leocontent.umgc.edu/content/umuc/tus/ifsm/ifsm300/2212/learnin...

14 of 14 2/28/2021, 9:55 AM