1
Chapter 1: Introduction
1.1. Overview
The concept of cloud computing refers to a form of computing which focuses on the
interaction of all the shared resources being interlinked on the Internet. In the last
decade, cloud computing has gained a lot of popularity with different entities relying
on its application to deploy its systems as a backup plan should their physical
facilities fail to function [1]. However, as an idea, cloud computing began in the 90’s
although it did not at the time bear the name “cloud computing” but was just known
as “the cloud”. Its application at the time revolved around webpages until the idea of
telecommuting became a necessity for large organizations. Its boom was attained in
the year 2006 when Amazon.com introduced their elastic compute clouds [1]. Today,
various computer-based companies have also followed suit in responding to the new
market, with new inventions being created every day to reshape and reinvent the
application of the trend in the market.
Deployment of data to the cloud, despite seeming easy and beneficial towards the
undertaking of the operations of a company and the storage of data, has faced major
challenges. The most notable ones arise from security concerns, since the existence
of clouds in the Internet exposes valuable data to espionage and hacking in accessing
the data, with attackers relying on the availability of data on the cloud and the
existence of the Internet to log onto these platforms.
Every day, these malicious individuals, organizations, or governments are finding
ways to bypass the security protocols available, making it possible to access the data.
Various contributions from researchers in organizations such as the following have
elaborated on how the cloud computing concepts need to be organized and deployed
[2][3][4][5][6]:
2
National Institute of Standards and Technology (NIST).
The European Union Agency for Network and Information Security (ENISA).
International Telecommunication Union (ITU).
International Electrotechnical Commission (IEC).
International Organization for Standardization (ISO).
Information Systems Audit and Control Association (ISACA).
Control Objectives for Information and Related Technologies (COBIT).
Generally, cloud computing relates to sharing on-demand scalability of extremely
reliable and available pooled computing resources, and protected access to metered
services from virtually anywhere, along with displacement of services and data
from outside to inside the organization [7]. A report by the International Trade
Administration published in 2016 [8] forecast the future powering of data resources
in 2020. The report stated that computing power and resources will be sold and
deployed by cloud company providers rather than by enterprise data centers,
especially trending toward platform and infrastructure cloud capability types.
Particularly, the infrastructure computing market has grown by 40 percent since
2011, and it is expected to reach an annual growth rate of 25 percent by 2019.
Moreover, it is predicted that the majority of virtual machines will be delivered by
infrastructure providers by that time [9].
From a security perspective, the trend assumes the growing understanding that cloud
providers will offer state-of-the-art security. For instance, innovations in the area of
security-related services that increase the trust in public clouds are of particular
interest [9]. However, a report provided by Cloud Passage shows that some security
barriers to cloud adoption currently exist [10].
3
Figure 1 Cloud Adoption Barriers [10]
Cloud security concerns still top the list of barriers to cloud adoption, and some of
these concerns continue to increase. For instance, “General security concerns (with
53% up from 45% in last year’s survey), legal and regulatory compliance concerns
(42% up from 29%), data loss and leakage risks (40% slightly down from 41%),
integration with existing IT environments (35% up from 29%) and lack of expertise
(26% up from 16%)” [10] have been seen.
Moreover, the increasing competition in the cloud industry has played a significant
role towards the progress that the field has experienced. Part of this progress is the
new approach that urges companies to pool their resources to find solutions as a
group and to address most of the challenges they have been facing individually. In
this regard, Google and Cisco have agreed to share some of the technological features
that are unique to each company to allow them to better respond to market needs and
to outdo Microsoft and Amazon, who are the major players. The new arrangement,
as Weinberger [11] explains, will see Google customize services for Cisco customers
while Cisco participants in the development of cloud computing needs such as
participation in the hybrid cloud.
From another point of view, Spitzer has portrayed the leaders Microsoft and Amazon
in a merger as they seek expansion plans for their Artificial intelligence venture,
which would allow developers to access open source software to advance their
systems, Alexa and Cortana. The pool of resources between these two companies
4
will significantly facilitate their capacity to explore different features that may need
to be addressed [12].
In our dissertation, the research focuses on the area of cloud security metrics. Each
organization needs to have a clear understanding of all layers of security. There are
shared concerns between those who run the cloud as providers and those who use the
cloud as customers. By the use of metrics, both cloud customers and cloud providers
will be able to have visibility of their performances, and by providing a common
language it will increase understanding of their services and help with future
improvement. It is very important for cloud customers and providers to clearly
specify who is responsible for what in order to address the issue of shared
responsibility in cloud security [13].
There are areas of security metrics and cloud computing which need to be organized
in such a way that allows their adoption from a cloud security metrics perspective.
In order to achieve that, we have proposed a taxonomy for cloud security metrics
which improves the understanding of the relationship between cloud computing and
security. At the same time, having a cloud security metrics taxonomy shows that
there are shared responsibilities between cloud providers and customers that need to
be identified. We also suggest a framework for determining cloud security
responsibility.
The dissertation consists of seven chapters.
Chapter 1 is an introduction to the research area of cloud computing and associated
risks and includes the problem statement and research questions. In chapter 2, we
review some background information about cloud computing security along with
work related to the area of security metrics, cloud security metrics, and shared
security responsibilities. In chapter 3, we explain the dissertation methodology. In
5
chapter 4, we describe the cloud security metrics in detail starting with earlier works
and then explaining the validation procedure. In chapter 5, we describe our cloud
security metrics responsibility framework in detail starting with earlier work and then
explaining the validation procedure. In chapter 6, we describe the guidelines for
allocating cloud shared responsibility and demonstrate them with a case study. In
chapter 7, we discuss the overall results, and answer the research questions, and
explain our future work.
1.2. Problem Statement
New standards and regulations have been published to organize the cloud computing
industry, especially in the area of cyber security [14] [15] [16] [17] [18]. These new
standards require the creation of a new measurement system that transfers the
existing information security metrics into the cloud environment. However, the
progress of this development needs to provide a clear distinction between the
conflicting responsibilities of cloud customers and those of the provider, with
consideration of the different deployment models and capability types
[15][16][17][18].
More efforts have to be made to maintain a secure cloud environment. For instance,
guidelines, standards, measurement, compliance etc..., are all areas where it is
necessary to clarify the different degrees of control between cloud providers and
cloud customer .
Cloud organizations need to have a better understanding of the relationship between
the key components of cloud computing and cyber security. There is a clear security
risk to organizations which move to the cloud where they turn over control of their
data and privacy to others.
6
The cloud shared responsibility conflict has been mentioned by the National Institute
of Standards and Technology (NIST) Standards (Special Publication 500-292) [18],
Payment Card Industry Security Standards Council [19], Cloud Standards Customer
Council CSCC [20], and International Organization for Standardization ISO/IEC
27017 [21], where they recommended that the cloud organization consider the split
of control between both parties and emphasize that there are shared responsibilities
in providing acceptable protections to the cloud-based systems. “Security is a shared
responsibility. Security controls, i.e., measures used to provide protections, need to
be analyzed to determine which party is in a better position to implement”[18].
This recommendation has been applied partly by several cloud providers, such as
Microsoft [22], Amazon [23], and Oracle [24], where they include a Shared
Responsibility Model section in their agreement. However, these agreements are
only superficial and general in their specifications, and they need more explanation
and more detailed work on their measurement. The movement toward cloud
computing brings with it the need to protect organizational data and privacy and to
meet government requirements, regulations, and standards.
1.3. Motivation
Cloud adoption has priorities, which must be identified and required to be among the
considerations for cloud computing communities. NIST has published ten
requirements as a roadmap for the adoption of cloud computing by federal agencies
and to support the private sector [25]. The 10th requirement on the NIST roadmap
states that the cloud industry needs to establish Cloud Service Metrics to be able to
measure cloud resources. In comparison to other measured resources, such as
electricity, light, or fuels, cloud computing measurements are relatively imprecise
[25].
7
In the related area of measuring security in cloud computing, there is an impact in
both directions, from a customer perspective and from a provider perspective. Since
“service providers have not defined and applied standardized units of measurement
that can be specified in Service-Level Agreements and interoperability exchanges”
[25], that motivates us to study the area of cloud security metrics to suggest a solution
to organize the measurement of the shared responsibility between customers and
provider.
NIST also published a special publication (Draft) covering the area of cloud
computing service metrics description. In making the decision to move to the cloud,
both cloud customers and providers must have measurable capabilities for access, for
instance - quality of service, availability, security, and reliability. Primarily, that
allows the cloud service customer the tools and opportunity to get informed choices
and to have a better understanding of the service being delivered [26].
There is a measurement gap in supplying the information needed for decision-
making in service agreements between provider and customers. It makes it difficult
for the cloud service customer to compare services in regard to monitoring the
security of the service. At the same time, it makes it difficult for the provider to prove
that the service is performing correctly.
This clearly shows that “It is critical to have the capacity to represent what needs to
be measured, how the measurement results are used, and how they impact business
and technical decisions” [26]. That motivates us to develop a framework that closes
this gap and provides a clear overview of how the responsibility will be shared
between both parties. Several cloud-specific security standards and regulations have
been published (ISO/IEC 27017 [21] ,COBIT [14] and NIST [2] [27] [28][18]),
including the variety of cloud deployment and capabilities, the types of cloud shared
responsibility, and the need for a measurement system that helps build a relationship
8
of trust in the enterprise of cloud computing between cloud customers and providers.
These need to be identified in a service level agreement SLA [29], and it is this need
which has motivated us to develop our proposed framework for determining cloud
security metrics responsibility to be used to increase trust by having clearly
identified responsibility in the SLA.
Luna, Ghani, Germanus and Suri state that the common research challenges to be
taken into consideration for developing the cloud security metrics framework is
having a cloud security metrics taxonomy [30]. They suggested an adaptation of
well-known taxonomies of information security metrics to “model the cloud’s unique
features. The taxonomy should be flexible enough (representing the cloud’s security,
privacy or risk) and able to cope with the cloud’s service oriented nature” [30]. This
was the motivation to create a taxonomy in order to classify these concepts in a way
that helps balance the responsibilities for metrics evaluation between the cloud
provider and the customer.
1.4. Research Questions
1. Will cloud adoption of existing information security metrics be more useful
by sharing responsibility between cloud customer and provider?
2. Should the proportion of responsibility for evaluating security metrics
between the customer and the provider vary for the different cloud
capabilities (Application, Platform, and Infrastructure)?
3. Should the proportion of metrics responsibility for evaluating security
metrics between the customer and the provider vary for the different metrics
categories (Management, Operational and Technical)?
4. Does the relationship of cloud computing and security need to be organized
to distinguish the shared responsibility of cloud security metrics?
9
1.5. Contribution
1.5.1. Develop a Cloud Security Metrics Taxonomy
In order to provide a unique novel framework and to help balance metrics
responsibility, we need a new classification that shows the nature of the relationships
between cloud computing and security metrics. We have developed a Cloud Security
Metrics Taxonomy that merges two components: Cloud Computing and Security
Metrics and adjust the requirements for both fields. The taxonomy organizes these
concepts to set a foundation that helps group metrics responsibility between the cloud
provider and customers based on their selections and can be used in a later stage of
the guideline and framework.
1.5.2. Create a Framework for Allocating Cloud Security Metrics
Responsibility
We create a Framework for Allocating Cloud Security Metrics Responsibility
between the customers and the provider. Shared Security responsibility on the cloud
has multiple impact factors that are defined in the taxonomy. The use of metrics in
the cloud helps in service selection, service agreement, and service verification [26].
Therefore, the framework’s goal is to help the decision maker in these purposes from
a security perspective.
1.5.3. Provide Guideline for Allocating Cloud Shared Responsibility
We provide a guideline to help to best implement our framework. The guideline
explains step by step how to implement existing metrics to the cloud and to allocate
responsibility between customer and provider. The guideline assists both cloud
customers and providers by supporting their decisions in selecting metrics and
allocating responsibility, thus avoiding any possible gap or overlap in security
responsibility.
10
Chapter 2: Literature Review
2.1. Background Information
2.1.1. Cloud Computing
For a clear understanding of cloud computing, it is necessary that we be aware that
there are different components and ways in which they are related to each other in
order to build a clear image of how the cloud functions.
Figure 2 Visual model of cloud computing definition – NIST [8][2]
The following terminology is widely used in the area of cloud computing, including
some terms commonly used in the general area of computer science, with others from
different scientific specialties, and some which are new and only used for the field
of cloud computing. These terms will be described briefly and will be explained later
with more detail.
11
Cloud Deployment Models: They are characterized by management and
organization of the use and access of cloud resources by different classes of
customers [31]. They are based on many needs including data classifications,
level of confidentially, and location of data storage. These deployment
models are the public cloud, the private cloud, the hybrid cloud, and the
community cloud.
Cloud Service Models: NIST defines a cloud service model by stating that
it “dictates an organization’s scope and control over the computational
environment, and characterizes a level of abstraction for its use” [31] between
the cloud provider and the cloud customer. The service models are Software
as service SaaS, Platform as service PaaS, and Infrastructure as service IaaS.
Cloud Capabilities Types: These types were developed as an improvement
of the NIST cloud service models. This new re-structuring is provided by
ISO/IEC and is characterized by three capability types, Application,
Platform, and Infrastructure [5].
Cloud Characteristics: These general characteristics of the cloud are not
limited to any specific type or model [5] and include: On-demand self-
service, broad network access, resource pooling, rapid elasticity, and
measured service[32].
Cloud Service Customer:
Cloud customers are a party within the cloud system who use the cloud services and
deal with the cloud provider; they may or may not deal with a cloud partner. For
example: they may select a cloud partner to manage cloud services, perform cloud
business activities, and manage their cloud services users [5].
12
Cloud service provider:
Cloud Providers are parties within the cloud system which provide the cloud services
available to cloud customers. For example, they provide service, set up and monitor
cloud service, perform service maintenance, maintain system security, etc.[5].
Providers may or may not deal with a cloud partner depending on the service level
agreement between the provider and the customer.
Cloud service partner: The cloud partner is a party within the cloud system
in between the customer and the provider; based on their agreement, the cloud
partner can provide support or assistance with cloud activities for both the
cloud service provider and the cloud service customer or either one of them
[5].
2.1.2. Cloud Service Models vs. Capabilities Types
NIST Cloud Service Models
Software as Service (SaaS): The cloud provider allows customers to run software
on provider resources with limited access, control, and management ability; the
underlying cloud infrastructure is fully controlled by the cloud provider [33].
Platform as Service (PaaS): The cloud provider allows customers to run software
and use applications, languages, libraries, services, and tools supported by the
provider. The underlying cloud infrastructure is mainly controlled by the cloud
provider [33].
Infrastructure as Service (IaaS): The cloud provider allows customers to run
software and control provider resources, including owned applications, storage,
networks, and processing and operating systems. The underlying cloud infrastructure
is partially controlled by the cloud provider [33].
13
ISO/IEC Cloud Capabilities Types
Another prospective is provided by ISO/IEC, which renamed the model as Cloud
Capabilities Types.
Application: Similar to SaaS, this is a cloud capability type where the customer of a
specific cloud service has the ability to use the applications of a cloud service
provider. Through the application capabilities, the customers of a cloud service are
able to make use of the applications on the cloud [5].
Platform: Similar to PaaS, this is a type of cloud service capability where the
customer of the cloud service can set up, run, and manage customer-developed or
customer-attained applications through the use of one or more languages of
programming. The customer can also operate using one or more than one service
which is partly under the control of the provider of cloud services [5].
Infrastructure: Similar to IaaS, This is a capabilities type of cloud service where
the customer of the cloud service can use provision resources, storage, and
processing as well as networking resources [5].
The new approach provided by ISO/IEC allows the industry to encompass new,
unique, innovative cloud services currently and in the future.
14
Table 1 Cloud service categories and Cloud capabilities types [21]
C Service Categories
C Capabilities Types
I
P
A
Compute as a service
X
Communications as a Service
X
X
Data Storage as a Service
X
X
X
Infrastructure as a Service
X
Network as a Service
X
X
X
Platform as a Service
X
Software as a Service
X
As shown in the table above, ISO/IEC has re-categorized cloud service using a
different model based on cloud capabilities types. With the growth of the cloud and
its variety of services, it became impractical to limit cloud services to a specific
category.
The same SaaS, PaaS, IaaS services that already exist have to be mapped with either
one or all of the cloud capabilities types (infrastructure, platform and application).
Any new cloud services will be able to fit into one these capabilities types. Services
like Compute as a Service, Data Storage as a Service, and Network as a Service are
referred to as representative categories of cloud service; as a result of the growing
cloud industry, there has been some difficulty in categorizing based on the NIST
Models. Some of these are defined as follows:
CaaS (Communication as a Service): This is a category of cloud service where the
capabilities allow the customer of the cloud service to be involved in real time
collaboration and interface.
15
CompaaS (Compute as a Service): This is a category of cloud service in which the
abilities granted to the cloud service customer are the use of and terms of processing
of resources that are required in deploying and running software.
DSaaS (Data Storage as a Service): Data storage and other closely related
capabilities are provided to the cloud service customer according to the category of
cloud service.
NaaS (Network as a Service): Connectivity of transport and other closely related
network capacities are offered by the cloud service category to the customer of the
cloud service [5].
2.1.3. Customers’ and providers’ activities and responsibilities
Table 2 Cloud customer and Cloud provider [34][35]
C Services Typ
C Activities
P Activit
E
ISO/IEC Application
NIST (SaaS)
Uses application/service for
business process operations.
Installs, manages, maintains, and
supports the software application on a
Cloud infrastructure.
Google Apps
Microsoft
Office 365
ISO/IEC Platform
NIST (PaaS)
Develops, tests, deploys, and
manages applications hosted in
a cloud environment
Provisions and manages cloud
infrastructure and middleware for the
platform consumers; provides
development, deployment, and
administration tools to platform
consumers.
Google App
Engine
Apache
Stratos
ISO/IEC Infrastructures
NIST ( IaaS)
Creates/installs, manages, and
monitors services for IT
infrastructure operations
Provides and manages the physical
processing, storage, networking, and
the hosting environment and cloud
infrastructure for IaaS consumers.
Amazon
Web
Services
(AWS)
Windows
Azure
Table 2 shows different activities between customers and providers based on the type
of cloud service.
16
An example of an application or SasS type: The Google doc allows the user to save
and edit the user’s document on the cloud while the user is working on it from
different locations.
An example of a platform or PaaS type: the Google App Engine allows the user to
compute and process resources while deploying and running software with limited
control.
An example of infrastructure or IaaS type: Amazon Web Services (AWS) provide an
infrastructure to the user to control resources.
C Services Model or Capabilities Types
ISO/IEC A
NIST (S S)
ISO/IEC P
NIST (P S)
ISO/IEC I
NIST ( I S)
Services responsibility
Cloud Customer
Cloud Customer
Cloud Customer
Cloud Provider
Cloud Provider
Cloud Provider
Figure 3 Services responsibility among Cloud capabilities types and model
The Services responsibility is among the cloud capabilities types and models. Figure
3 shows that a cloud provider has much more control of resources as opposed to the
cloud customers in the application capability type, while in the platform capability
type, there is an increase in control on the part of the customer while using provider
resources. In the Infrastructure capability type, customers have a high level of control
of resources, but the provider still has the authority to manage their resources.
17
2.1.4. Cloud Deployment Models
Public: The provider has deployed a shared cloud for multiple customer
organizations, and it is available for public use.
Private: The provider has deployed the cloud to a single customer organization.
Hybrid: This is a combination of two or more cloud deployment types.
Community: The provider has deployed a shared cloud to a group of customers or
organizations that share similar activities [33].
The most recognized cloud deployment models in Asia and Europe are Hybrid,
Public, and Private Clouds. The Community Cloud is less popular and is only used
minimally in Asia and Europe [34]. The Hybrid Cloud is the most popular, where
customers utilize both Private and Public Clouds collaboratively. In Europe, the
national governments utilize a unique model called the G-Cloud model [35]. It is a
hybrid model that combines the features of private and community cloud models
[33].
2.1.5. Cloud Characteristics
On-demand self-service: This is a characteristic of the cloud service where the
customers have the ability of provisioning the capabilities of computing as required
without any significant involvement and as needed automatically without needing
human interaction [33] of the cloud service provider. The significance of this cloud
computing aspect is that it reduces the costs for the users. It eliminates the need for
additional user interference, hence reducing overhead [5].
Broad network access: This is an aspect in which the physical and the virtual
components are present in a network and can be accessed via basic mechanisms that
18
promote usage of assorted platforms to the clients. This aspect increases the
convenience of cloud computing where users readily access all resources as soon as
they are on the network (e.g., mobile phones, laptops) [5][33].
Resource pooling: This is a characteristic of cloud computing where the physical
and virtual resources can be combined in order to serve more than one customer of
cloud services. This advancement shows that cloud computing supports multi-
tenancy while utilizing abstraction to maximize customer experience with location
independence but with some ability and right to specificity (e.g., country, state, or
datacenter) [5][33].
Rapid elasticity: This ability of virtual and physical resources can be easily
increased or decreased through rapid and elastic adjustments. This feature allows the
customers of cloud computing to have accessibility to an unlimited number of
resources in any capacity at any time [5][33].
Measured Service: This is an aspect of cloud service that facilitates the tracking of
activities. Through this facility, cloud service usage can be tracked, measured,
reported, managed, monitored, controlled, and billed. This feature enables users of
cloud computing services to only pay for the extent of services rendered [5][33].
2.1.6. Cloud Security Issues
Overview
Risks are associated with data wherever they are located. Moving data to the cloud
moves the risk to that data to the cloud, too. There is thus a requirement for a
modification to the set of security requirements to maintain the cloud’s needs.
Moreover, these transfers of data to the cloud introduce new risks and increase some
already known risks to data which become more vulnerable in the area of the Cloud.
19
A report published by the Cloud Security Alliance (CSA) shows survey results
reporting on likely cloud risks [14]. These risks have been evaluated using the
STRIDE threat model [36]. A number of 12 threats known as “The Treacherous 12
- Cloud Computing Top Threats in 2016 “and demonstrate that while the cloud does
not hold a monopoly on risks, it does increase those risks [14].
In this section we will briefly explain these 12 threats with the STRIDE threat
evaluation. Furthermore, there will be a brief mention of the business impacts
associated with these threats.
CSA Top 12 Threats in 2016
The following list of 12 threats has been provided by CSA [37] :
Data Breaches
A Data Breach refers to an incident where an individual with no authority gets access
to sensitive, confidential, and protected data. In the process, these individuals may
use, steal, or release this data. In some incidents, data loss could occur due to human
errors or due to poor security practices in an organization system. Data loss can be
seen in different categories ranging from trade secrets to competitors seeking
strategies and tactics.
Weak Identity, Credential and Access Management
Different forms of attacks, whether they are data breaches or system flaws, occur
since most systems are characterized by the application of weak passwords and the
rotation of cryptographic key certificates and passwords. Professionals in the field
recommend that data credentials should not be embedded in the source code or
20
distributed among repositories. Identity systems must be scaled to handle high
numbers of users and CSPs.
Insecure APIs
In cloud, user interfaces or the application programming interfaces facilitate the
customers’ interactions with the cloud platforms. However, even as they do so, the
security and availability of the cloud is based solely on the interfaces. Challenges
then arise since the API’s have to be designed to control encryption, monitor activity,
and protect malicious attempts. By being the exposed part of the systems, they can
become a link to the outside world.
System and Application Vulnerabilities
System vulnerabilities refer to the bugs or open ports, etc., which malicious attackers
can rely on in a program to infiltrate the whole computing system. The main threat
of system vulnerabilities can be traced to the fact that they give the attackers absolute
control of the systems, which could allow them to disrupt service operations, steal
data, or manipulate the systems to perform as they please. Bugs have been a common
phenomenon ever since networks were created. System upgrades, regular
vulnerability scanning, and control of the systems are some of the suggestions for
addressing this threat.
Account Hijacking
Account hijacking refers to the practice of relying on different methods such as fraud,
exploitation of software vulnerabilities, and phishing to attain credentials and
passwords. Once an attacker has these details, they can eavesdrop on one’s account
data, manipulate the data, redirect clients to illegitimate sites, or falsify information.
21
Since most individuals tend to store their information in these accounts, they can be
platforms where the attacker gains leverage power over the vulnerable party.
Malicious Insiders
A Malicious Insider is often an individual who is a current or former employee, a
business partner, or a contractor who may have had authorized access to an
organizations system, networks and data and who maliciously uses this data to
infringe negatively on the confidentiality, integrity, and availability of an
organization’s systems.
Advanced Persistent Threats (APTs)
APTs refers to those threats which exist in the nature of parasites, infiltrating systems
and establishing a foothold in the computing infrastructure of a company, hence
facilitating the smuggling of data or intellectual property. Most of the APTs focus on
vulnerabilities over an extended period and often adapt to the systems’ defense
mechanisms. The main points of entry are spear heading, USB deliver, penetration
of networks, direct system hacks, and unsecured third party networks.
Data Loss
Data Loss has become a primary concern to every organization in the modern world,
since they range from service provider data deletion on cloud to physical
catastrophes, or loss of encrypted data without encryption keys. Data backups and
offsite storage have been recommended as some of the practices to avoid data loss.
Insufficient Due Diligence
Whenever executives develop any business strategies, cloud computing and its
technologies should be considered. The development of checklists and good road
22
maps should be considered to facilitate due diligence. cloud should be adopted
sparingly to minimize incidents of exposure and to facilitate legal, technical, and
compliance solutions.
Abuse and Nefarious Use of Cloud Services
Poorly deployed cloud models, free trial services, and fraudulent account sign ups
through payment instructions expose the different cloud computing models, whether
they are the IaaS, SaaS or PaaS, to attackers. The actors can leverage on computing
resources in order to target the organizations and other cloud providers. Misuse of
cloud-based attacks may succeed whether they are attempted through a launch of
Email Spams, DDoS attacks, Phishing, Brute Force, Automated Clicks, or stolen
credentials.
Denial of Service
Targeting the availability of systems, DOSs focus on preventing users from accessing
data or applications. They do so by forcing the targeted cloud services to consume
excessive amounts of finite system resources such as processor power, network
bandwidth, disk space, and memory. In the process, legitimate users cannot access
the systems which leaves them frustrated and confused as to why the system is not
responding.
Shared Technology Issues
The development of cloud is usually done in such a way that it facilitates the sharing
of the platforms, application, and infrastructure. Sometimes this is done without
changing the Off the Shelf Hardware, which is undertaken at the expense of security.
Challenges usually arise since the Isolation features may not be as strong, which
means that hackers can rely on such a vulnerability. In resolving this issue, consistent
23
monitoring of the systems and user based security, network, application, and storage
should often be considered.
Threat Analysis STRIDE
The threat analysis model STRIDE has been used by Microsoft [36]. STRIDE is an
acronym that classifies threat into the following classification components: Spoofing
identity, Tampering with data, Repudiation, Information Disclosure, Denial of
service and Elevation of privilege.
Thompson, and Sunkara, defined [38] each component as follows:
Spoofing identity: an unauthorized use of identity when the attacker is able to act as
an authorized user of a system.
Tampering with data: when the integrity of that data is successfully modified,
added to, deleted, or reordered by the attacker.
Repudiation: when the user denies the performance of a certain action on the system,
and there is no ability to prove that the action was or was not performed.
Information Disclosure: when information is available and exposed to an
unauthorized user.
Denial of service: when an attack occurs and is able to deny the services to valid
users.
Elevation of privilege: the ability to get higher access and privilege occurs when an
unprivileged user or attacker gains higher privileges in the system than their
authorized level [38].
24
The CSA report used this model to evaluate the 12 threats, by mapping each threat
to the most likely category of the STRIDE Model
CSA 12 Critical Issues to Cloud Security
(Ranked in Order)
Threat
Categories
Spoofing identity
(S)
Tampering with data
(T)
Repudiation
(R)
Information Disclosure
(I)
Denial of service
(D)
Elevation of privilege
(E)
S
T
R
I
D
E
1: Data Breaches
2:Insufficient Identity, Credential and Access Management
3: Insecure Interfaces and APIs
4: System Vulnerabilities
5: Account Hijacking
6: Malicious Insiders
7: Advanced Persistent Threats (APTs)
8: Data Loss
9: Insufficient Due Diligence
10: Abuse and Nefarious Use of Cloud Services
11: Denial of Service
12: Shared Technology Issues
Figure 4 STRIDE Threat analysis [37]
The figure above shows where cloud threats occur most commonly, according to the
STRIDE model. It is possible to fit issues 2, 4, 5, and 9 in figure 4 into all of these
threat categories, as claimed in the CSA report [37].
25
Treats Business Impacts
Many of these threats have significant impact upon business and users, with some
possible impacts including:
The possibility of large lawsuits, fines, and criminal charges, since a company
has a duty to protect the data.
The likelihood of an effect on a company’s ability to respond to the market
needs appropriately.
The possibility of exposing an organization’s lack of integrity, availability,
and accountability, which could be catastrophic to an organization’s
reputation.
The costs of preventing attacks in addition to the cost of undoing the damage,
which ranges from recovery systems to repairing the vulnerabilities.
The loss of sensitive materials or information, which are part of its
craftsmanship, in process copyright, or patent.
Economic damages as a result of the attack facilitates stealing of data.
The disabling of the continuous functioning of the company in planning for
future needs; attacks may force the company out of business due to a lack of
the information required to respond to its market.
The vulnerability of customers as a result of the attack will affect the whole
cloud network [37].
2.1.7. Cloud Security Standards
Overview
Cloud security standards and their support through potential cloud service providers
and in the enterprise is a critical focus area for both cloud consumers and providers.
This is an evolving sub-domain of information security and continues to expand as
26
the technology grows. According to the Cloud Standards Customer Council – CSCC
[20], these standards offer a level of assurance that best practices are applied both
internally and by the service providers; for several security standards, certifications
are offered. Further, standards support hybrid cloud computing, easing integration of
on-premises security technologies and the cloud. Standards also encourage
interoperability, doing away with vendor locking while simplifying the migration
from one cloud provider to another [20]. The advantages of supporting key security
standards are numerous.
This literature review of standards offers an overview of computer standards that
touch on cloud security management. In addition, this report further examines how
these standards provide solutions to strategic decisions for an organization from a
cloud challenges perspective.
Table 3 Standards list overview
L
A
ISO
ISO/IEC
Standards
27001
27002
27004
27017
ISACA’s
CoBIT
Governance
HIPPA
HI-TECH
Health
NIST
NIST
800-145
NIST SP
800-146
NIST SP
800-144
NIST SP
500-291
NIST SP
500 -292
Measurement
Standards
FISMA
FISMA
Regulation
FedRAMP
FedRAMP
Regulation
As shown in table 3, this section contains a preview of a variety of standards that are
related to our future research work. It starts with the area of information security
management and control. Further, it considers other related areas such as governance,
27
privacy, and security regulations to review them from a cloud perspective. The report
relies on organizations such as:
ISO
NIST
ISACA
Also, the report provides an overview of acts and framework including:
Control Objectives for Information and Related Technologies (CoBIT).
Health Information Technology for Economic and Clinical Health (HIPAA).
Health Insurance Portability and Accountability Act (HITECH).
The Federal Information Security Management Act (FISMA).
Federal Risk and Authorization Management Program (FedRAMP).
28
The first section summarizes some of the ISO 27K Family, Figure 5 showing 27K
Family standards. Some of them will be briefly explained in this dissertation such as
ISO 27001, 27002, 27004, and 27017.
ISO 27001
This section is the summarization of ISO 27001 main points [39].
ISO 27001: 2013 – Information technology / Security techniques / Information
Security Management Systems / Requirements: specifies the necessity for
establishing, maintaining, implementing, and continually advancing an information
security system inside an organization. Further, it entails requirements for the
Figure 5 ISMS Family of Standards Relationships [70]
29
treatment and assessment of information security risks customized to the necessities
of the organization, such as security requirements, structure and size, as well as the
organizational processes used [39].
This standard consists of six domains: Leadership, planning, support, operation,
performance evaluation, and improvement.
Leadership:
As part of the top management of an organization, the leader is responsible for
ensuring that information security objectives and policies are established in addition
to the proper integration of the information security management system needs into
the enterprise’s processes. It further ensures that required resources in regard to
information security management systems are available. Also, it confirms the
significance of effective information security management, as well as supporting
other related management roles.
Planning:
By exploiting this domain, organizations are able to consider information security
risks and opportunities and to ensure that information security management systems
(ISMS) attain their intended results. Additionally, it ensures that organizations
reduce or avoid undesired effects. Organizations are, therefore, able to plan
operations that address these risks and exploit opportunities.
Support:
This domain makes it possible for organizations to identify and offer resources
required for the establishment, execution, maintenance, and progressive support of
the ISMS. The domain helps the management to establish a necessary individual’s
30
competence on the basis of education, training, or experience. Further, it ensures
these individuals are aware of information security policies and determines the need
for external and internal communications associated with the ISMS including when,
what, who, and with whom to communicate.
Operations:
By adopting this domain, the organization is able to plan, implement, and manage
processes needed to satisfy information security requisites, in addition to
implementing processes established under planning. The organization is further able
to control planned changes and to evaluate the outcomes of unintended changes.
Performance evaluation:
This domain offers guidelines for evaluating the information security performance
and effectiveness of the ISMS by determining what needs to be measured and
monitored, which evaluation and monitoring techniques are to be applied, and who,
what, and when this shall be done. In addition, it offers audit criteria and management
review guidelines.
Improvement:
In the event of nonconformity, the domain allows the organization applicable actions
to control and deal with the consequences. In addition, the organization is able to
evaluate the need for action in eliminating the causes of nonconformity, to avoid
recurrence and to effectively implement required action and changes to the ISMS, if
required.
31
ISO 27002
This section is a summarization of the ISO 27002 main point [40].
ISO 27002: 2013 – Information technology / Security techniques / Code of practice
for information security controls: offers guidelines for organizational information
security standards, in addition to information security management practices,
including the selection, implementation and administration of controls accounting
for the organization’s information threat environment(s). it provides a process of
deploying an Information Security Management System based on ISO / IEC 27001
[40].
One of the objectives of ISO 27002 is to offer management support and direction for
information security in conformity with business requirements in addition to
appropriate laws and regulations at the highest level of management.
This standard contains 14 security controls:
1) Information security policies, 2) Organization of information security, 3) Human
resource security, 4) Asset management, 5) Access control, 6) Cryptography, 7)
Physical and environmental security, 8) Operations security, 9) Communications
security, 10) System acquisition, development and maintenance, 11) Supplier
relationships, 12) Information security incident management, 13) Information
security aspects of business continuity management and 14) Compliance.
Information Security Policies:
Approved information security policies set out the organization’s strategy to achieve
its information security objectives. As such, information security policies should
encompass requirements established by regulations, legislation, and contracts,
32
business strategy, and the present and forecasted information security threat
environment. Additionally, the information security policies should clarify
statements concerning the assignment of specific and general responsibilities for
information security management to identified roles; this includes the definition of
information security, the principles and objectives to guide all activities associated
with information security, and processes for handling exceptions.
Organization of information security:
The purpose of this standard is to establish an internal management framework to
accomplish and control the implementation, as well as the operation of information
security. It requires guidelines on the allocation of information security
responsibilities to be established in line with the organization’s security policies. It
also offers guidelines on the segregation of conflicting duties, in addition to
maintaining suitable contacts with relevant authorities and special interest groups,
such as specialist security forums.
Human Resource Security:
This control helps organizations ensure that contractors and employees understand
their responsibilities and are appropriate for the roles for which they are considered.
This standard offers screening guidelines and sets direction on the terms and
conditions of employment / engagement. It further helps ensure that contractors and
employees are conscious of and satisfy their information security obligations. Other
areas covered are termination and change of contract.
Asset Management:
This control of the standard helps in identification of organizational assets, in
addition to the definition of fitting protection obligations. As such, an enterprise
33
should identify assets related to information lifecycle and document their
importance. It further clarifies ownership of assets where individuals, as well as other
entities – following approved asset management responsibility – qualify to be
allocated as asset owners.
Access Control:
ISO 27002 limits access to information along with information processing facilities.
It requires that an access control policy be developed, documented, and reviewed on
the basis of enterprise and information security requirements. These policies should
take account of the security requisites for business application, with consistency
among the access rights in addition to information classification policies of networks
and systems.
Cryptography:
This control of ISO 27002 ensures effective and proper exploitation of cryptography
to secure the authenticity, confidentiality, and integrity of information. It requires
that a policy on the exploitation of cryptographic controls for the security of
information be established and implemented throughout their entire lifecycle. The
management of cryptographic keys is crucial to the efficient use of cryptographic
techniques.
Physical and environmental security:
This control is aimed at preventing unauthorized physical access, harm and
interference to the enterprise’s information / data and data processing facilities. It
requires that perimeter security be identified to secure areas that hold either critical
or sensitive information / data and data processing facilities. It further recommends
that secure facilities be protected by suitable entry controls to guarantee that only
34
authorized personnel are granted access. As such, offices’, facilities’ and rooms’
physical security should be considered, designed, and implemented.
Operations security:
This control ensures secure and accurate operation of information processing
facilities. It requires that operating procedures be documented and available to all
users who need them. As such, changes to the enterprise, information processing
facilities, business processes and systems that impact information security should be
controlled. Use of resources should be monitored and fine-tuned, with future capacity
requirements considered to maintain required system performance. This includes
protection from malware, logging, and regular information backup.
Communication Security:
This control ensures the protection of information within networks and the support
of information processing facilities. It requires that networks be controlled and
managed to protect information in network, applications, and systems. All network
levels and services need to be identified in the network service agreements at the
different levels.
System acquisition, development and maintenance:
This control ensures that information security is an essential part of the information
systems throughout the entire lifecycle. This includes the requisites for information
systems that offer services across public networks. It requires that information
engaged in service application transactions be safeguarded to avert incomplete
transmission, unauthorized message alteration, unauthorized message duplication, or
misrouting.
35
Supplier relationships:
This control ensures secure access of the organization’s assets to suppliers. It requires
that information security requirements for diminishing risks linked to suppliers’
access to enterprise assets be discussed with the suppliers and documented.
Additionally, it requires that all appropriate information security requirements be
developed and discussed with every supplier that may access, store, communicate,
process or offer IT infrastructure components for, the enterprise’s information as to
agreement and implementation in any later process.
Information security incident management:
Ensures an effective and consistent approach to any management information
security incidents, as well as communication on security weaknesses and events. This
domain requires that management procedures and responsibilities be established to
ensure a rapid, efficient and orderly response to incidents related to information
security in a speedy manner.
Information security aspects of business continuity management:
This covers information security continuity, where the standard requires that
information security continuity be embedded within the enterprise’s business
continuity management systems. In planning for IS continuity, the enterprise is
required to establish its information security requirements along with the continuity
of information security management in undesirable situations.
Compliance:
This control is to assure agreement with any required law or regulations and other
policy or followed standards by the organization along with an increase to the highest
36
possible level of effectiveness of the information systems audit process.
ISO 27004
This section is a summarization of ISO 27004 main points [41].
ISO/IEC 27004 Information technology — Security techniques — Information
security management — Measurement is concerned with measurements regarding
information security management – security metrics. This standard is intended to
assist organizations measure, report, and, as a result, analytically advance the
effectiveness of their Information Security Management Systems (ISMS). The
standard offers guidance on the use and development of measurement and measures
so as to evaluate the effectiveness of implemented ISMS(s) alongside controls or
clusters of controls [41], as specified in ISO/IEC 27001.
This standard provider an overview of the information security measurement model
with managing responsibilities of measurements and measurement constructs and the
processes that need to be implemented (i.e. developing and planning, operating,).
These provide a guideline to help organizations to implement the Information
Security Measurement and record measurement activities and their outcomes [41].
ISO 27017
ISO 27017:2015 – Information technology — Security techniques — Code of
practice for information security controls based on ISO/IEC 27002 for Cloud
services. ISO 27017 provides a practice to apply 27002 with consideration to cloud
issues; it provides change to control sections – major changes are suggested in the
Access Control area:
37
9.2.1 User registration and deregistration.
9.2.2 User access provisioning.
9.2.3 Management of privileged access rights.
9.4.1 Information access restriction.
9.4.4 Use of privileged utility programs [42].
ISO 27017 also provides new controls that meet and fit with the cloud environment.
6.3.1 Shared roles and responsibilities within a cloud computing
environment.
8.1.5 Removal of Cloud service customer assets.
9.5.1 Segregation in virtual computing environments.
9.5.2 Virtual machine hardening.
12.1.5 Administrator’s operational security.
12.4.5 Monitoring of Cloud services.
13.1.4 Alignment of security management for virtual and physical networks
[43].
This section is a summarization of the ISO 27017 main points [21].
Generally used along with the ISO/IEC 27001 series of standards, ISO/IEC 27017
offers advanced controls for cloud computing service providers as well as cloud
service customers. In contrast to many other technology-related standards, ISO/IEC
27017 offers clarification to both party’s responsibilities and roles and helps make
cloud services as secure and safe as the rest of the information contained in a certified
information management system (ISMS).
As provided in ISO 27002, 270017 provides a sector with specific guidelines for the
cloud, the same Code of practice for information security controls that has been
38
explained in 27002 with modification to comply with cloud responsibilities between
cloud customers and cloud providers.
Similar to 27002, 27017 has 14 controls.
Information Security Policies: The objectives outlined in clause 5.1 of the ISO / IEC
27002 pertain to this standard as well. The cloud service provider is expected to
augment its information security policy to cover the provision and exploitation of its
cloud service, while considering the following: threats from authorized insiders;
cloud service customer isolation and multi-tenancy – including virtualization;
virtualization security; communication of information sharing and breach guidelines
to assist forensics and investigations; lifecycle management of cloud service
customer accounts; protection of and access to cloud service customer data; and
access control procedures, such as strong authentication meant for administrative
access to cloud services. In addition to the baseline information security requisites,
there are specific requirements applicable to the implementation and design of the
cloud service provider.
On the other hand, an information security policy intended for cloud computing
should be described as a topic-specific policy of the customer. In regard to cloud
computing, the customer’s information security policy should be in harmony with
the organization’s acceptable position for its information, among other assets.
When describing cloud computing information security policy, the cloud service
customer should consider: the service in the addition to the context in which they
exploit the cloud service; which assets can be held in the cloud environment; which
processes can run on virtualized and multi-tenant cloud service; whether stored
information within the cloud computing environment can be exposed to access and
administration by the cloud service provider; and the cloud provider’s geographical
39
location, as well as the countries in which the cloud provider is likely to store the
customer’s data – even momentarily [21].
In regard to organizing information security, the cloud service provider should
document and consent to an appropriate allocation of information security
responsibilities and functions with its cloud customers, suppliers, and service
providers. The cloud service customer on the other hand should be in agreement with
the cloud provider on a suitable allocation of information security responsibilities
and functions, in addition to confirming that they can satisfy their responsibilities
and functions. As such, the information security responsibilities and functions of
both parties should be affirmed inside an agreement. Additionally, the cloud
customer should manage and identify its relationship with the customer care and
support functions of the cloud provider [21].
Access Control: To manage cloud access, the cloud provider should afford the cloud
customer user registration and deregistration functions, as well as guidelines for the
use of these functions. In addition, the provider should access rights management
functions, as well as user guidelines for these functions. The cloud customer’s control
policy for exploiting network services should describe user access requisites for each
cloud service used. The provider should offer sufficient authentication practices for
authenticating administrators to administrative capabilities consistent with identified
threats. For instance, the cloud provider can offer to allow the use of third-party
multifactor authentication techniques or offer multifactor authentication functions.
The ISO 27017:2015 standard requires that the customer ensure that access to data
in the cloud is restricted and in compliance with its access control policy. This covers
access to the cloud service, service functions, and user data held within the service.
This cloud service provider should offer access controls that allow the customer to
limit its services, service functions, and customer data held within the service.
40
Cryptography: Further, the standard requires the cloud service provider to offer
information to users regarding the conditions in which it exploits cryptography to
secure the data it processes [21].
In addition, the provider should also offer information to customers regarding any
functions it offers that can support users in applying their own cryptographic
protection. The user should thus employ cryptographic controls if justified by a risk
analysis. These controls should be sufficiently strong to mitigate identified risks.
As described in ISO 27000, the organization of information security’s objective in
ISO 27017 is to establish an internal management framework to kick off and control
the implementation, as well as the operation of information security. It requires that
guidelines on the allocation of information security responsibilities be established in
line with the organization’s security policies. It requires the cloud customer to
identify and manage relationships with care functions of the provider, including
customer support. This is an addition to the ISO 27002.
Similar to ISO 27002, Human Resource Security helps organizations ensure that
contractors and employees understand their responsibilities and are fit for the roles
for which they are considered. This standard offers screening guidelines and sets
direction on the terms and conditions of employment / engagement. It further helps
ensure that contractors and employees are conscious of and satisfy their information
security obligations. Other areas covered are termination and change of contract [44].
Asset Management is aimed at the identifying of organizational assets, in addition to
the definition of fitting protection obligations. As such, an enterprise should identify
assets related to information lifecycle and document their importance. Its inventory
of assets is similar to ISO 27002 and requires that asset inventory of the cloud
41
provider be explicitly identified, including both customer data and cloud service
derived data.
Physical and environmental security is similar to clause 11.1 of ISO 27002; this
segment is aimed at preventing unauthorized physical access, harm, and interference
to the enterprise’s information / data and data processing facilities. It requires
perimeter security to be identified to secure areas that hold either critical or sensitive
information / data and data processing facilities. Additionally, this standard offers
implementation guidelines for cloud services, where it requires the customer to
request confirmation that the provider has procedures and policies for secure reuse
and disposal of resource. On the other hand, it requires that the provider ensures that
arrangements are in place for secure reuse or disposal of resources such as files,
memory, and data storage.
Operations security entails components described in ISO 27002 clause 12.1, as well
as implementation guidelines such as requiring that the customer’s change
management process should consider the effect of any changes made by the provider.
In addition, the provider should present the customer with information concerning
changes to the cloud that could negatively affect the service. For example, time and
date of backup and maintenance schedules must be provided [27].
Communication security offers implementation guidelines for cloud services in
addition to the components covered in ISO 27002 clause 13.0, such as requiring that
the cloud customer should define its requisites for segregating networks to attain
tenant isolation within a shared environment and verify that it meets those
requirements. As such, the provider should enforce network segregation between
tenants within a multitenant environment and between the provider’s administration
environment and the customer’s computing environment.
42
The System acquisition, development and maintenance segment offers additional
implementation guidance for cloud services that are not included in ISO 27002. It
requires the customers to determine IS requirements and evaluate whether services
provided by the provider satisfy those requirements. The Provider on the other hand
should present information to the customer about IS capabilities they exploit. This
information ought to be instructive without disclosing information that could be
taken advantage of by someone with malicious intent.
Supplier relationship ensures secure access to the organization’s assets by suppliers
in both direction, assets that are owned by cloud customers and, in the opposite
direction, assets that are owned by the cloud provider. It requires that information
security requirements for retreating risks linked to suppliers’ access to enterprise
assets be discussed with the suppliers and documented. 27017 suggest an extra
implementation process to cloud customers only. To mitigate risks associated with
the cloud service provider, the policy cloud customer needs to include the provider
as a supplier while accessing and managing data security.
Information security incident management is the same as 27002, in that it ensures an
effective and consistent approach to management information security incidents, as
well as communication on security weaknesses and events. Customer and provider
should verify the allocation of responsibilities and verify meeting requirements. The
provider needs to clearly document incidents that the cloud service provider must
report to the customer, the level of disclosure of the detection, and contact
information for the handling of any issues that could occur.
Information security aspects of business continuity management covers information
security continuity, where the standard requires that information security continuity
be embedded within the enterprise’s business continuity management systems and
there are not specific requirements for the cloud.
43
Compliance is provided to assure agreement with any required law or regulations
and other policies or followed standards by the organization along with an increase
to the highest possible level of effectiveness of information systems audit process. In
the cloud, the customer should consider the issue of related laws and regulations that
need to be met by the provider and request evidence of having met those laws and
regulations, while the provider should inform the customer of the legal jurisdictions
governing the applicable area.
ISO/IEC DIS 19086-2
This section is a summarization of the ISO/IEC Information technology — cloud
computing — Service level agreement (SLA) framework — Part 2: Metric Model
[45] which is still under development. The Draft offers a preview of some challenges
including the definition of expectations and determining how to reach an agreement
on the specific requirements of the cloud service providers and the cloud service
customers. The need for cloud metrics thus arises to address such a gap between these
two parties. For instance, the metrics explain the layers of clouds that require such
measures such as hardware, logic, governance, or service layers or parts of the
lifecycle. Metrics also lead to better management of clouds enabling the
measurement of the sharing of control by stakeholders such as auditors, attorneys,
cloud service providers, and cloud service customers.
The metrics model further elaborates on the procedure of adopting computational
figures, which give measurement scales. These scales can be relied in setting service
level objectives and qualitative objectives. The underlying principle is commitment
and availability. There are different approaches in this process, which are built and
developed in the metrics model. These models are mainly automated approaches. At
the same time, there are the goals tagged to the development of the metrics model.
44
The consistent representation addresses efficient organization, sharing, and the use
of information for the metrics. Explicit relationships indicate the organization or how
concepts relate to one another. The repository of definitions determines reusability,
searchability, or what can be delivered. In addition to these are comparability, which
involves the accumulation of information to facilitate identification of differences
and similarities. Adaptability addresses the ease of migration between models.
Composability involves reusability to the extent that one can identify particular
characteristics such as accuracy, precision, and uncertainty.
CoBIT
This section is a summarization of the CoBIT main points [14].
ISACA’s COBIT has advanced to be a well-recognized IT control and risk
framework. Broadening its use to cloud governance, COBIT’s flexibility supports
innovation as the framework, is platform-agonistic in both type and complexity,
offers clearly described risk assessment measures – not only binary, but also a
maturity model scale of 1 to 5, and has ample depth to tackle nearly all technical
cloud computing aspects. This standard addresses IT controls and risks throughout a
full program lifecycle. COBIT controls are categorized into:
(i) Plan and Organize (PO), which offers direction to service delivery (Deliver
and Support), and solution delivery (Acquire and Implement).
(ii) Deliver and Implement (DS), which receives the solutions offering usability
to end users.
(iii) Acquire and Implement, which offer the solutions and transfers to be
converted into services.
(iv) Monitor and Evaluate (ME), which monitors all processes to make sure that
the direction offered is followed. The COBIT information criteria reflect
45
information needs that include: reliability, compliance, availability,
integrity, confidentiality, efficiency and effectiveness [14].
In governance considerations, IT is mainly a strategic asset for an enterprise. When
accurately implemented, the cloud can be an extension of this asset. COBIT’s
business aspects consist of linking business objectives to IT objectives, offering
metrics and maturity models that determine their achievement, and recognizing the
associated duties of IT and business and IT process owners.
Nonetheless, migrating to a cloud environment could introduce a paradigm shift in
business processes; thus, the IT business risk analysis must be shared, understood,
and mutually governed.
Notably, legal professionals are beginning to tackle newly documented issues with
the cloud computing environment exploiting legal precedents from other realms of
business law. Anchored on these precedents, legal professionals are offering advice
to various legislatures with possible new worldwide cloud law enactments.
FISMA
This section is a summarization of the FISMA [46].
FISMA refers to the Federal Information Security Management Act of 2002, which
recognizes the significance of information security to the national and economic
interests of the United States. It specifies a comprehensive framework to secure
government information, assets and operations against man-made and natural threats.
FISMA allocates responsibilities to different agencies to guarantee the information
security of the Federal Government [46]. It is thus used by all government agencies
and by all organizations that exchange information directly with government systems
– such as military sub-contractors, state government departments, clearinghouses,
46
and government contractors. The Act necessitates program officials, along with the
head of every agency, to carry out annual information security program reviews, with
the goal of diminishing risks or maintaining them below certain acceptable levels
within a timely, efficient, and cost effective manner [47].
The FISMA holds federal agencies accountable in secure government information.
Failure to comply with FISMA can result in (for affected agencies): significant
administrative sanctions, budget cuts, and unfavorable publicity [48].
FedRAMP
This section is a summarization of FedRAMP[49].
FedRAMP, federal risk and authorization management program.
A recommendation issued by the executive office of the president of the management
and budget, the official letter that introduces Fed RAMP outlines the benefit of
adopting cloud technology in government institutions, and it is also a memo giving
specific reference to the processes that are involved in determining the feasibility of
adopting cloud computing. In the memo, FedRAMP requests the adoption of cloud
use while attempting to determine the most effective approach to implementing cloud
services. FedRAMP investigates the security requirements and expectations of such
a system, determining the standards that should reflect the needs and responsibilities
of the third party who implements security controls, verifying the required
authorization packages and bodies to be involved in the process, and recommending
approaches that are associated with the security authorization processes for use and
adoption of such systems. The memo gives specific reference to its applicability and
purpose. Under purposes, it reiterates the policies required to adopt cloud services,
gives reference to the responsibilities of management, and clarifies the requirements
that guide departments seeking to use FedRAMP to acquire cloud services.
47
FedRAMP is a parallel standard to FISMA; their differences are largely attributed to
the dates when they were enacted and the control requirements in the standard. In
their article, the three-note FISMA was created for the security of the information
system while FedRAMP addresses security needs that are related to the cloud. A
comparison to the NIST control family portrays the differences and similarities
which exist between these two standards. FedRAMP has high requirements for the
access control set, monitoring of employee and personal use, and ensuring the
integrity of the information system in the cloud [50].
HIPPA – HITECH
This section is a summarization of HIPPA – HITECH [51][52].
HIPAA, the Healthcare Insurance Portability and Accounting Act, publishes Privacy
and Security regulations, where the privacy rule imposes controls aimed at averting
unauthorized disclosure of Protected Health Information (PHI) [51]. It requires
suitable security to shield the privacy of PHI, setting conditions and restrictions
around the exploitation and disclosure that may be made of such information devoid
of patent authorization; the security rule, on the other hand, covers the integrity,
confidentiality, and availability of protected electronic health information (ePHI).
Health Information Technology for Economic and Clinical Health (HITECH)
specifies how health information technology may be adopted and used with
consideration to privacy and security.
This regulation prescribes several requisite procedures, policies, and reporting
mechanisms that should be in place for each information system that stores,
processes, and transmits ePHI within and among covered entities and applies to
deployments pertaining to cloud computing [53]. The rules for HIPAA/HITECH
compliance do not clearly address storing ePHI into cloud services, but it does
48
mention business associates, which can include cloud providers, as being required to
meet the requirements of HITECH [54][51].
NIST
NIST 800-145 The NIST Definition of Cloud Computing:
The NIST definitions set a taxonomy of cloud computing which describes an
important aspect of cloud computing. It provides a baseline for the direction of the
cloud industry. “The service and deployment models defined form a simple
taxonomy that is not intended to prescribe or constrain any particular method of
deployment, service delivery, or business operation”, according to NIST [2].
NIST defines cloud computing as a model for allowing convenient, ubiquitous, on-
demand network access to a shared pool of configurable computing resources, which
can be quickly provisioned and released with minimum management effort or cloud
service provider interaction. It provide a cloud model of five cloud essential
characteristics, three service models, and four deployment models [2].
NIST SP 800-146 Cloud Computing Synopsis and Recommendations
explains the cloud computing technology subject and offers recommendations for
information technology decision makers. Since cloud computing is an evolving
research area, its ultimate strengths and weaknesses have not yet been fully examined
The 800-146 offers recommendations on when and how cloud computing is a fitting
tool and indicates the restrictions of present knowledge and subjects for future
analysis. It is meant for diverse enterprise information system professionals such as
information system developers, chief information officers, system designers, system
programmers, network and system administrators, application programmers, and
49
information system security officers as well as system owners who are involved with
cloud computing [27].
NIST SP 800-144 Guidelines on Security and Privacy in Public Cloud Computing:
These guidelines offer an overview of public cloud computing along with privacy
and security challenges involved. It discusses technology risks and threats to public
cloud environments and offers the insight needed to establish informed resolutions
on their treatment. That includes the need for consideration of the risks taken by
organizations while outsourcing data, applications, and infrastructure to a public
cloud environment.
It lists several key security and privacy issue risks including governance, compliance,
trust, data protection, and availability [31].
NIST SP 500-291: NIST Cloud Computing Standards Roadmap:
This standard seeks to offer guidance and leadership around the cloud computing
paradigm to speed up its use within the industry and government. The purpose of this
special publication is to establish a Roadmap, which can be integrated into the NIST
Cloud Computing Technology for public and private agencies. Thus, it describes the
parties involved within the various cloud deployments and offers general direction
and responsibilities [28].
NIST SP 500 -292: NIST Cloud Computing Reference Architecture
This seeks to offer guidance in the area of cloud computing reference architecture by
identifying the major collaborators in the cloud environment and their functions and
activities within cloud computing. This cloud computing reference architecture
50
defines five key actors: the cloud carrier, the cloud auditor, the cloud consumer, the
cloud provider, and the cloud broker, in addition to describes their activities [18].
2.2. Related Work
2.3. Security Metrics
Good information security metrics should support enterprise managers and system
owners in their decision making. They should further quantify failures and successes
of past and present security investments, objectively compare the effectiveness of an
enterprise’s security processes with those of other enterprises, and justify the
investment in such processes and procedures.
As a definition of security metrics terminology:
Security metrics is an objective and quantitative basis for security assurance. It
quantitatively or qualitatively describes the security level of a system which is
exploited by decision makers to analyze data. A security metric can therefore be
described as a measurable attribute of the result of a security engineering process,
which could serve as proof of its correctness and effectiveness.[41][55][56] It is used
in regard to the control and effectiveness of information security management
systems, using a measurement function, approach, analytical model, and decision
criteria[47].
Security metrics can be divided into two classes: Verification is confirmation by
examination and provision of objective evidence that specified requirements have
been fulfilled, while Validation is ascertaining that the requirements for a specific
intended use can be systematically perform [57].
These two categories would correspond to the questions, “Did we build the system
right?” and “Did we build the right system?” [57].
51
The key requirement of measurement is having to identify what is being measured.
Why? How? Who is responsible? and following up by documenting and evaluating
the results [58] which help measure the level of security. For instance, managers
require tools to help them answer the questions
- Is my organization’s information secure?
- Is my organization acting in accordance with the regulations on securing and
managing critical data?
- How do we measure security risks of service or emerging technology offered to our
clients?’ [59]
52
Table 4 Cyber security metrics categories [61]
Metrics Categories
Management
Metrics
Offer information on business
performance functions, and
how they impact the
performance
Incident costs
Mean incident costs
Patch policy compliance
IT security budget
Operational
Metrics
Exploited to understand and
optimize business functions
Mean incident discovery time
Mean incident recovery time
Mean-time to patch
Mean-time to incident recovery
Technical
Metrics
Offer technical details and
establish a foundation for
other metrics
Number of incidents
Risk assessment coverage
% of critical applications
Existing anti-malware
compliance
In our Guideline and Framework, we use key examples of information security
metrics: the Center For Internet Security (CIS) Security Metrics [60] which covers
Management Metrics, Operational Metrics, and Technical Metrics, as shown in
Table 4 above.
2.4. Cloud Security Metrics
A metric enables us to characterize the properties of a cloud by defining it in terms
of rules, units, and expressions. It also makes possible the ability to reproduce and
to verify both measurement results and observations [26].
53
Figure 6 Differences between enterprise computing and cloud computing [61]
Generally, “The need for security metrics in the cloud is not much different from the
need for security metrics” [62]. Therefore, the majority of existing metrics could be
used on cloud environments, but it is very important to distinguish the shared
responsibilities between cloud customers and provider. Consequently, cloud security
metrics provide visibility to both cloud provider and customers, in addition to
providing a common language for understanding between cloud customers and
provider and helping to improve the performances of the cloud services [62].
From a security metrics prospective, and in cloud case scenario:
- Both need to measure security to have the ability to define who is doing what.
- Both need to manage functional components to secure the cloud.
- Both need to continue the assessment of risk and threats that are associated with
the cloud [61].
54
2.5. Shared Security Responsibilities
Taking shared security responsibilities into consideration has been emphasized and
recommended by NIST, PCI, CSCC, and ISO/IEC 27017 [18][19][20][21].
The NIST 500 292 NIST Cloud Computing Reference Architecture document
explains the need to understand the responsibilities of parties involved in managing
the cloud application due to the fact that there are different degrees of control over
the computing resources in a cloud system between Cloud Provider and the Cloud
Customer [18]. Control is divided in the cloud system, which means that the
responsibilities to secure the system need to be clear, with no conflict or overlap
between both parties. Therefore, security, including security controls, measurements,
protections, etc., [18] is definitely a shared responsibility.
The PCI Guideline states that “Clear policies and procedures should be agreed upon
between client and cloud provider for all security requirements, and clear
responsibilities for operation, management, and reporting need to be defined for each
requirement” [19].
For this reason, lines of accountability and responsibility will become more aligned
and will close the gap that is associated with the differences between each service
and deployment model.
The CSCC Guideline provides similar reasons along with another example, related
to personally identifiable information PII. An example is the EU General Data
Protection Regulation (GDPR) which applies to handling of PII of European Union
citizens. “PII protection related issues should be dealt with in the cloud service
agreement. It should be clear how responsibilities are allocated between the provider
and the customer and also which jurisdictions are involved” [20]. The level of
55
responsibility varies depending on the cloud service involved and needs to be clearly
set, making it possible to determine the shared security responsibilities of PII.
ISO/IEC 27017 is Security techniques — Code of practice for information security
controls based on ISO/IEC 27002 for Cloud services provides guidance as to how to
apply the standards specified in ISO/IEC 27002 with consideration to Cloud issues.
It provides modification to these controls by clarifying the relationship regarding
shared roles and responsibilities between the Cloud service customer and the Cloud
service provider. “Responsibilities for shared information security roles in the use of
the Cloud service should be allocated to identified parties, documented,
communicated, and implemented by both” [21] based on every party’s assigned
tasks.
Several organizations include the Shared Security Responsibility Model in their
Cloud process, including Microsoft [63] ,Amazon [23], and Oracle [24].
In the case of Microsoft, their model names seven responsibilities that organizations
should consider. These seven suggested responsibilities contribute to the security and
privacy of a computing environment with different Cloud service models.
These seven responsibilities are:
Data classification and accountability.
Client and end point protection.
Identify and access management.
Application level control.
Network control.
Host infrastructure.
Physical security.
56
In the case of Amazon, their model did mention the responsibilities between
provider and customers but did not provide details regarding how different Cloud
service models impact their users. Amazon states clearly their responsibility for
securing the underlying infrastructure that supports the Cloud while the Cloud
customers are responsible for anything they put on the Cloud or connect to the Cloud
[23].
The model divides the responsibilities based on:
Amazon Web Services (AWS) Security Responsibilities spells out the manner in
which Amazon takes protection responsibility for the global infrastructure that runs
all the services they provide. “This infrastructure is comprised of the hardware,
software, networking, and facilities that run AWS services” [23]. Regarding
Customer Security Responsibilities, a determination of needed configuration
security responsibilities will be decided on at a later stage between customers and
AWS. However, customers are able to provision virtual servers, storage, databases,
and desktops. Customer are also allowed to “use Cloud-based analytics and
workflow tools” [23] that allow them to process data and to store it in the Cloud.
In the case of Oracle, Oracle addresses this concern by emphasizing the role of the
customer in determining the security measures that are executed in their oracle Cloud
infrastructure. The customer who has an Oracle Cloud is responsible for securing
their workloads alongside the platform services that they have adopted. Oracle’s
shared responsibility model shows that Oracle is responsible for the security in the
infrastructure and service such as database, compute, storage, and other services
while the customers are responsible for securing the operating systems, applications,
and other security management tasks at the level of platform and applications such
as:
57
Virtual machine security.
Identify and access management.
Network and dynamic firewall configuration.
Data security.
Under the heading of security, Oracle has adopted the practice of setting up different
user accounts allowing a manager to review and access all the employee activities
that take place in such a system. Other features that customers gain under
infrastructure as service are configurations of the systems, maintenance, protection
archiving, data backup, and encryption [24].
58
Chapter 3: Methodology
The dissertation goals are to develop and validate a taxonomy, framework, and
guidelines for Cloud security metrics. A literature review has identified the need for
Cloud security shared responsibility. To confirm the need to proceed with further
research efforts to determine the Cloud security shared responsibility, we made a
short visit to Cloud related organizations and brief discussion with Cloud experts and
researcher.
To help organize existing security metrics adopted by the Cloud computing area, we
created a taxonomy that classifies the components of Cloud computing and security
metrics and sets a foundation that helps group metrics responsibility between the
Cloud provider and customers. Based on that, we developed a framework to allocate
Cloud Security Metrics Responsibility. We provided guidelines to help implement
metrics in relation to the framework. In order to test our work in the early stages, we
created and conducted a pilot study which was sent to six participants. Two of them
were Cloud providers, two were Cloud customers and two were neutral participants
who worked as Cloud researchers. The pilot study was analyzed and used to finalize
the taxonomy and framework.
For the validation process, an online broad survey has been created along with
institutional review board (IRB) approval (Appendix A). We used a Survey Monkey
[64], which was spread online via email and social media to a list of 350 Members
of a Saudi Group for Information Assurance – Hemaya [65] who work in
government, academia, companies, and organizations. Members with Cloud and
security metrics experiences helped by participating on the survey and the rest helped
by spreading the survey to their co-worker or friends who had experience in Cloud
and security metrics.
59
We conducted one survey for both taxonomy and framework. The taxonomy and
framework were shown as discrete units for evaluation separately by the participants.
A valid total number of 81 Cloud expert participants completed the evaluation of the
taxonomy, and a total number of 65 Cloud expert participants completed the
evaluation of the framework.
The guidelines were developed to assist the customer and provider in determining
the set of Cloud security metrics that are their individual and shared responsibility
and has been validated by implementing real-life examples. As a case study, we
adopted existing FISMA metrics security for use with the Cloud.
The results have been analyzed using a statistics software, the IBM Statistical
Package for the Social Sciences SPSS [66], to get better descriptive analyzed results
and to ascertain that our research questions have been determined and explained.
Thus, final conclusions and future work have been identified and will be discussed
in chapter seven of the dissertation.
60
Chapter 4: Cloud Security Metrics Taxonomy
4.1. Early Work
For a better understanding of our taxonomy, guidelines and framework, the following
is a list that explains each of the terms and concepts that has been used in each
component:
Security Metrics Categories
The following metrics categories were suggested by the Center of Internet Security
[60]:
Management Metrics: used to “provide information on the performance of
business functions, and the impact on the organization. Audience: Business
Management” [60]; for example, they may specify patch policy compliance
and the percentage of configuration compliance.
Operational Metrics: “Used to understand and optimize the activities of
business functions. Audience: Security Management” [60]; for example, they
may specify mean-time to incident recovery and mean-time to patch.
Technical Metrics: used to “provide technical details as well as a foundation
for other metrics. Audience: Security Operations” [60]; for example, they
may specify the percent of critical applications and number of known
vulnerability instances.
61
Cloud Metric Responsibility:
Cloud Customer (C): Cloud customers are responsible for collecting and
evaluating specific metrics within a Cloud system.
Cloud Provider (P): Cloud providers are responsible for collecting and
evaluating specific metrics within a Cloud system.
Both – Customer & Provider (B): Both Cloud customer and provider share
responsibility for collecting and evaluating specific metrics within a Cloud
system.
Cloud Capabilities Types:
The following Cloud capabilities types were suggested by ITU and ISO [5]:
Application: the Cloud provider allows customers to run software on provider
resources with limited access, control, and management ability, but the
underlying Cloud infrastructure is fully controlled by the Cloud provider.
Platform: the Cloud provider allows customers to run software and use
applications, languages, libraries, services, and tools supported by the
provider. The underlying Cloud infrastructure is mainly controlled by the
Cloud provider.
Infrastructure: the Cloud provider allows customers to run software and
control provider resources, including owned applications, storage, networks,
processing and operating systems. The underlying Cloud infrastructure is
partially controlled by the Cloud provider.
Cloud Model Deployment
The following Cloud model deployment was suggested by NIST, ITU and ISO
[2][5]:
62
Public (Pu): the provider has deployed a shared Cloud for multiple customer
organizations, and it is available for public use.
Private (Pr): the provider has deployed the Cloud to a single customer
organization.
Hybrid (Hy): this includes a combination of two or more Cloud deployment
types.
Community (Co): the provider has deployed a shared Cloud to a group of
customers or organizations that share similar activities.
Cloud Security Metrics Taxonomy
A taxonomy can be used to organize these concepts in a way that helps balance the
responsibilities for metrics evaluation between the Cloud provider and the customer.
We classify Cloud security metrics based on two components:
1.
Cloud computing [Capabilities Type + Cloud Deployment].
2.
Security metrics [Metrics Categories + Metrics Responsibility].
In Cloud computing components, we rely on the International Telecommunication
Union suggested model. ISO/IEC re-categorized Cloud service using a different
model based on Cloud capabilities types [5]. With the growth of the Cloud and its
type of services, it became impractical to limit Cloud services to the NIST service
model. The same Software as service (SaaS), Platform as service (PaaS) and
Infrastructure as service (IaaS) already existing in NIST [2] has to map all of the
Cloud capability types (infrastructure, platform and application). Any new Cloud
services will be able to fit one these capabilities types. Services such as Compute as
a Service, Data Storage as a Service, and Network as a Service, which are referred
to as representative categories of Cloud service, are the results of the growing Cloud
63
industry and present some difficulty in categorization based on the NIST Model.
Cloud Deployment remains the same on both ISO/IEC and NIST.
In regard to the Security metrics components, Metrics Categories has been suggested
as a grouping by the Center for Internet Security (CIS) [60]. These metrics are
organized into a hierarchy based on their users and purpose. In Metrics
Responsibility, we suggest that the responsibility be divided between the customer
and the provider, including an area where the responsibility will be shared and the
metrics will be used by both.
The taxonomy will be of benefit to:
Classify the area of Cloud security metrics.
Show the relationship between Cloud computing and security metrics.
Identify and label the areas where security is the shared responsibility of both
the Cloud customers and the provider.
Be fundamental for future development in the area of Cloud security metrics
classification.
Provide a basis upon which to develop our Cloud Security Metrics
Responsibility Framework and guidelines for implementing the framework.
64
4.1.1. Taxonomy
Figure 7 Cloud Security Metrics Taxonomy
65
As shown in Figure 7, we classify Cloud security metrics based on two
components. The first is Cloud Computing which includes Capabilities Type and
Cloud Deployment. The second is Security Metrics which includes Metrics
Categories and Metrics Responsibility.
In the first component, the Cloud customer and provider must decide which
Capabilities Type and Cloud Deployment options will be used.
In the second component, based on the selections in the 1st component, a set of
metrics should be divided into three categories: Management, Operational, and
Technical. Then, for each category the balance of Metric Responsibilities between
the provider and customer is determined.
At this time, we are only looking at Public and Private Clouds without considering
Community and Hybrid Cloud deployment.
4.1.2. Pilot study
In early work, we conducted a Pilot Study to help maintain maximum objectivity of
our early work with taxonomy before moving forward. The taxonomy pilot study
was shared with Cloud experts to get their primary evaluation and thoughts that
would help with our efficacy studies in the main survey at a later stage of our research
[67]. The pilot study sought multiple points of view and perspectives from
participants related to our research focus. We chose experts who are working in
Cloud provider organizations, others who are working as Cloud customers, and other
participants who are Cloud researchers, and, therefore, neutral rather than either
Cloud customers or Cloud providers.
The pilot study questionnaires sent to the six participants included nine qualitative
questions, the first two dealing with their Cloud perspective (Cloud customer, Cloud
66
provider, or neutral) and their years of Cloud experience. There were 2 questions
about the taxonomy. The pilot study participants were then asked, using a Five-level
Likert approach (strongly agree, agree, neither agree nor disagree, disagree, and
strongly disagree) the following questions to gain their opinions regarding our early
work taxonomy:
Table 5 Taxonomy pilot study Q 1
Q 1: Do agree or disagree that the three categorie of security metrics (Ma ,
O , T ) cover the range of metrics t can be used for eval a Cloud
?
Level of Agreement
Strongly
Agree
Agree
Neither
Agree nor
Disagree
Disagree
Strongly
Disagree
Number of Responses
0
6
0
0
0
Table 6 Taxonomy pilot study Q 2
Q 2: D agree disagree b the pro should
evaluating the security for a specific C system?
Level of Agreement
Strongly
Agree
Agree
Neither
Agree nor
Disagree
Disagree
Strongly
Disagree
Number of Responses
3
1
1
0
1
In summary, when we look at the level of agreement percentage without counting
the neutral responses from those who decided not to indicate whether they agreed or
disagreed, we determined that in both questions, the results showed that experts
mainly agreed with the comprehensiveness of our suggested security metrics
taxonomy for the covered range of Cloud security metrics which led us to proceed in
conducting a broad survey for our taxonomy research.
67
4.2. Taxonomy Validation
The approach and process of the taxonomy and framework are fully described in the
Methodology chapter. To validate both the framework and the taxonomy, we
conducted a single survey that had the responders evaluate both. However, both
framework and taxonomy have been shown separately on the survey to be evaluated
by measuring User Satisfaction [68]. We targeted users who are Cloud experts
exclusively.
These experts are but may not be limited to one of these types: Cloud provider, Cloud
customer /decision maker, Cloud user/technical staff and Cloud
researcher/regulator/authority.
In the case of Taxonomy, the survey was formulated to elicit an evaluation by Cloud
experts of different components and processes of taxonomy, such as determining
their opinion of the comprehensiveness of our suggested security metrics in terms of
the range of metrics and the ability to use them for evaluating a Cloud solution.
The online survey started with a letter to prospective participants identifying the
reason for the survey and providing contact information for participant concerns. An
Informed Consent form was provided to the participants explaining their rights and
offering information about the IRB approval number, the study title, the purpose of
the study, procedures, potential risks of participating, potential benefits of
participating, compensation, confidentiality, the voluntary nature of participation,
and finally, an explanation of the right to withdraw from the study at any time during
the online survey.
More details are included in the dissertation (Appendix A,B).
68
The next part of the survey asked about the work experiences. Participants were
asked to provide information about the following:
Indicate the experience they have had in using or providing Cloud services.
Indicate whether they have experience with Cloud systems and Cloud
metrics.
Indicate whether or not they have had any experience as a Cloud regulator,
Cloud Researcher Educator or trainer for Cloud users or providers.
Indicate their area of expertise, whether in using security metrics or in Cloud
services.
Indicate the years of experience in using security metrics
Indicate the years of experience in using Cloud services
Indicate their specific area of expertise in Cloud computing
Indicate their specific area of expertise in security metrics
A total of 198 participants started the survey. A total of 89 completed the survey
section on the taxonomy. Those who withdrew either did not agree to the consent
form or did not feel familiar with the research area. Of the participants, eight
indicated that they had zero experience in the area of Cloud and security metrics;
therefore, we excluded them. The total valid sample for the taxonomy were 81
participants who completed the taxonomy section.
69
Table 7 Distribution of taxonomy participant countries
By indicating their Internet Protocol address (IP) locations, the participants showed
their distribution across several countries in the world. However, the majority of
participants were from Saudi Arabia, where the online survey was originally
distributed.
In the taxonomy section we defined each component of the taxonomy in detail to
provide a better understating. To allow participants to evaluate the taxonomy, we
used a Likert scaling approach (1 Strongly disagree, 2 disagree, 3 Neither Agree nor
Disagree, 4 Agree, 5 Strongly Agree) to determine the participants’ level of
agreement to the following questionnaire:
Q: Do you agree or disagree that the three categories of security metrics
(Management, Operational, Technical) cover the range of security metrics that can
be used for evaluating a Cloud solution?
BY COUNTRY
#
%
Austria
1
1.2
Malaysia
1
1.2
Oman
1
1.2
Saudi Arabia
59
72.8
Turkey
1
1.2
United Arab Emirates
1
1.2
United Kingdom
10
12.3
United States
7
8.6
V Sample Total
81
100
70
Q: Do you agree or disagree that the proposed taxonomy could be used to adopt
existing security metrics to the Cloud?
Q: Do you agree or disagree that both the customer and the provider should be
involved in evaluating the security metrics for a specific Cloud system?
Q: Do you agree or disagree that the proposed taxonomy could be used to determine
if the Cloud security metrics cover all of the combinations of capability and metric
type?
4.3. Taxonomy Results Analysis
The taxonomy results include two sections: work background and taxonomy
evaluation. Each result has been shown in a table or figure along with the related
question as follows:
Please indicate whether you have experience with Cloud systems and Cloud
metrics?
Table 8 Taxonomy participants experience perspective
EXPERIENCE PERSPECTIVE
#
%
Cloud and metrics background
22
27.2
Cloud background
36
44.4
Metrics background
12
14.8
No experience with Cloud or metrics
11
13.6
T
81
100
The focus of this dissertation research is on a combination of two fields, the cloud
security and metrics. Therefore, the participants were asked to identify their
background experience in order to determine their perspective on the taxonomy. The
results showed that there were a total of 36 participants who had a Cloud background
71
and 12 who had a metrics background along with 22 participants who had experience
in both metrics and the Cloud.
Please indicate the experience you have in using or providing Cloud services?
Table 9 Taxonomy participants Cloud use perspective
CLOUD USE PROSPECTIVE
#
%
Provide Cloud ( Cloud Provider )
23
28.4
Use Cloud ( Cloud Customers )
52
64.2
Neither Cloud Customer nor Cloud Provider (Neutral)
6
7.4
T
81
100
The participants were asked to respond to the following: “Please indicate the
experience you have in using or providing Cloud services.” to analyze their possible
different perspectives in future work, if needed. Some of the participants were
neutral, in that they claimed to be neither Cloud customers nor Cloud providers.
Note: Participants identifying themselves as both “user and provider of the Cloud”
are considered as Cloud providers in our analysis.
Table 10 Taxonomy participants others experience
H experience as
C regulator
C researcher
E or trainer
N
Total : 81
24
37
18
18
A number of participants who were Neutral were neither Cloud customers nor
providers. Their evaluation was considered as neutral, in between customer and
provider, as, for example, participants who work in academia or as Cloud authorities.
The participants were allowed to indicate multiples choices if they were responsible
for more than one task that was provided in the survey questions.
72
How many years of Cloud Computing related work experience do you have?
Table 11 Taxonomy participants Cloud years of experience
C experience
#
%
No experience
2
2.5
0-1
17
21.0
2-4
36
44.4
4-6
17
21.0
7+
9
11.1
Total
81
100
Results showed the years of experience for Cloud experts only. Regarding work
experience, participants were asked about their years of Cloud experience, so that
participants who selected “No experience” were assumed to have only metrics
expertise.
Do you believe that your Cloud Computing expertise is more toward?
Table 12 Taxonomy participants Cloud expertise area
C expertise area
#
%
Cloud Management
30
37.0
Cloud Operations
17
21.0
Cloud Technical
18
22.2
I am not Cloud expert
16
19.8
In order to be more specific about the perspective of cloud expertise participation,
participants were asked to identify their cloud area.
73
Participants identified their expertise area based on the following criteria:
Management Team: Responsible for tasks such as: strategy, rules, policy, guideline,
evolution, recommending, governance, compliance, key performance indicator
(KPI) etc...
Operations Team: Responsible for tasks such as: monitoring, implementation,
optimization, applying new configurations, continuity, daily operations,
maintenance, update, reporting etc...
Technical Team: Responsible for tasks such as: hardware and system installation,
set-up basic configuration, support to end-users, Diagnose and resolve issues, collect
preliminary information etc…
How many years of Metrics related work experience do you have?
Table 13 Taxonomy participants metrics years of experience
M experience
#
%
No experience
23
28.4
0-1
20
24.7
2-4
29
35.8
4-6
5
6.2
7+
4
4.9
Total
81
100
Result showed the years of experience for metrics experts only. As work experience,
participants has been asked about their years of Cloud experience, participants whom
selected “No experience” are Cloud expertise only.
74
Do you believe that your Metrics expertise is more toward?
Table 14 Taxonomy participants metrics expertise area
M expertise area
#
%
Management Metrics
29
35.8
Operational Metrics
15
18.5
Technical Metrics
18
22.2
I am not metrics expert
19
23.5
In order to be more specific about the prospective of cloud expertise participation,
they have been identify regarding to their metrics area.
Participants identified their expertise area based on the following criteria:
Management Metrics: responsible for metrics such as:
· Mean Cost of Incidents.
· IT Security Spending as % of IT Budget.
· Patch Policy Compliance.
· Percentage of Configuration Compliance.
Operational Metrics: responsible for metrics such as:
· Mean-Time to Incident Discovery.
· Mean-Time to Mitigate Vulnerabilities.
· Mean-Time to Incident Recovery.
· Percent of Changes with Security Exceptions.
Technical Metrics: responsible for metrics such as:
· Current Anti-Malware Compliance.
· Number of Known Vulnerability Instances.
· Patch Management Coverage.
· Security Testing Coverage.
75
Figure 8 Taxonomy Q1 histogram
Taxonomy evolution items were measured using a 5 level Likert scale (strongly
agree, agree, neither agree nor disagree, disagree, and strongly disagree) and the
results were as follows:
Q1: Do you agree or disagree that the three categories of security metrics
(Management, Operational, Technical) cover the range of security metrics that
can be used for evaluating a Cloud solution?
Table 15 Level of agreements to taxonomy Q1
Table 15 and figure 8 present the participants’ responses regarding their level of
agreement to the three categories of security metrics coverage of the range of security
metrics and ability to be used for evaluating a Cloud solution. The mean values of
the level of agreement is 3.901 showing that the responses were more positive (agree)
and the standard deviation = .6443, where its mode is 4.0.
Q2: Do you agree or disagree that the proposed taxonomy could be used to
adopt existing security metrics to the Cloud?
Q1: L of Agreements
#
%
1.0 Strongly disagree
0
0.0
2.0 disagree
3
3.7
3.0 Neither Agree nor Disagree
12
14.8
4.0 Agree
56
69.1
5.0 Strongly Agree
10
12.3
Mean Mode Std. Deviation
3.901 4.0 .6443
76
Figure 10 Taxonomy Q3 histogram
Figure 9 Taxonomy Q2 histogram
Table 16 Level of agreement to taxonomy Q2
Table 16 and figure 9 present the participants’ responses regarding their level of
agreement to the usability to our taxonomy to adopt existing security metrics to the
Cloud. The mean values of the level of agreement is 3.716 showing that the responses
were more positive (agree) and the standard deviation = .6752, where its mode is 4.0.
Q3: Do you agree or disagree that both the customer and the provider should
be involved in evaluating the security metrics for a specific cloud system?
Table 17 Level of agreements to taxonomy Q3
Q2: L of Agreements
#
%
1. Strongly disagree
0
0.0
2. Disagree
4
4.9
3. Neither Agree nor Disagree
21
25.9
4. Agree
50
61.7
5. Strongly Agree
6
7.4
Mean Mode Std. Deviation
3.716 4.0 .6752
Q3: L of Agreements
#
%
1. Strongly disagree
1
1.2
2. Disagree
7
8.6
3. Neither Agree nor Disagree
7
8.6
4. Agree
30
37.0
5. Strongly Agree
36
44.4
Mean Mode Std. Deviation
4.148 5.0 .9888
77
Figure 11 Taxonomy Q4 histogram
Table 17 and figure 10 present the participants’ responses regarding their level of
agreement to the involvement of customer and the provider to evaluating the security
metrics for a specific Cloud system. The mean values of the level of agreement is
4.148 showing that the responses were more positive (agree) and the standard
deviation = .9888, where its mode is 5.0, indicates they (Strongly Agree).
Q4: Do you agree or disagree that the proposed taxonomy could be used to
determine if the Cloud security metrics cover all of the combinations of
capability and metric type?
Table 18 Level of agreements to taxonomy Q4
Table 18 and figure 11 present the participants’ responses regarding their level of
agreement to the ability of the taxonomy could be used in determine the coverage of
Cloud security metrics to all combinations of capability and metric type. The mean
values of the level of agreement is 3.790 showing that the responses were more
positive (agree) and the standard deviation = .7700, where its mode is 4.0.
Q4: L of Agreements
#
%
1. Strongly disagree
0
0.0
2. Disagree
5
6.2
3. Neither Agree nor Disagree
19
23.5
4. Agree
45
55.6
5. Strongly Agree
12
14.8
Mean Mode Std. Deviation
3.790 4.0 .7700
78
4.4. Discussion
In conclusion to the taxonomy validation results analysis process, it can be seen that
the 1th research question is directly related to taxonomy:
Will Cloud adoption of existing information security metrics be more useful by
sharing responsibility between Cloud customer and provider?
From the result we realized that the participants who decided not to take sides
whether agreeing or disagreeing cannot be included. Therefore, we removed them,
and we looked at the level of agreement percentage without counting the neutral
responses.
Figure 12 Taxonomy agreement vs disagreement
The results of the survey on figure 12 showed that, for Question 1, 95.7% of 69
participants agree to the coverage of three categories of security metrics
(Management, Operational, Technical), while, for Question 2, 93.3% of 60
participants agree that the taxonomy would be helpful to adopt existing security
metrics to the cloud. Also, the result of the survey showed that, for Question 3, 89.2%
of 74 participants are in agreement with the involvement of both customer and
provider in evaluating the security metrics (shared responsibility). Furthermore, in
Question 4, 91.9 % of 62 participants agree that the taxonomy can be used to
L A # % Q 1 # % Q2 # % Q3 # % Q4
Strongly disagree 0 0.0% 0 0.0% 1 1.4% 0 0.0%
disagree 3 4.3% 4 6.7% 7 9.5% 5 8.1%
Agree 56 81.2% 50 83.3% 30 40.5% 45 72.6%
Strongly Agree 10 14.5% 6 10.0% 36 48.6% 12 19.4%
Tot al 69 100.0% 100.0% 60 100.0% 100.0% 74 100.0% 100.0% 62 100.0% 100.0%
6.7%
93.3%
10.8%
89.2%
8.1%
91.9%
4.3%
95.7%
79
determine the coverage of Cloud security metrics for all combinations of capabilities
and metrics types.
In the final analysis, the experts’ responses indicate that they clearly agree that a
taxonomy such as we proposed can be useful in determining the shared responsibility
and we find that there is no need to revise it. Based on the survey results, we used
the taxonomy for the Cloud security metrics allocation of a shared responsibility
framework.
80
Chapter 5: Sharing Responsibility for Cloud
Security Metrics
The Framework is a shared security responsibility approach to determine and manage
possible different degrees of control between Cloud Providers and Cloud Customers.
Our framework contribution is to focus deeply on determining the shared
responsibility within Cloud security metrics. Through the process of decision
making, the Cloud provider and customer must decide on the Capabilities Type,
Cloud Deployment, and Metrics Categories; these impact factors need to be
determined to help set up shared responsibility on Cloud security metrics.
The framework is used to provide a mechanism for organizations to:
Document the security responsibility on the Service Level Agreement.
Support the decision process of selecting the best Cloud type and
deployment based on the level of cost and the work load, etc.
Close any possible gap or overlap risk between customers and providers.
Measure the shared security responsibility.
Identify and prioritize responsibility within the Cloud security metrics
process.
Provide clarity in communication between customers and provider about
cybersecurity risk.
5.1. Early work
As we explained, we assume that the proportion of responsibilities could vary, based
on the Capabilities Type that was chosen and that the proportions could be different
for each of the Metric Categories (Management, Operational and Technical) Note
81
that the proportions shown in the diagram are for illustration only and have not been
determined.
Figure 13 Framework for the allocation of shared responsibility in cloud security
metrics
As shown in Figure 13, we assumed that the proportion will vary, based on three
cloud capability types. Moreover, this variation will differ based on the Metric
Categories (Management, Operational and Technical).
82
For Cloud capability types our assumptions are generally as follows:
In Application: Normally, the control of the cloud system will be more in the hands
of the provider in general, as shown in the framework.
In Platform: Normally, the control of the cloud system will be fairly evenly divided
between the provider and the customer, as shown in the framework.
In Infrastructure: Normally, the control of the cloud system will be more in the
hands of the customer in general, as shown in the framework.
For Metric Categories type our assumptions are generally as follows:
Management: We assume that control will be more in the hands of the customer in
all capabilities types.
Operational: We assume that control will be in the middle, evenly divided between
the provider and the customer.
Technical: We assume that control will be more in the hands of the provider in all
capabilities types.
In conclusion, as shown in our taxonomy and framework, there are the following
multiple factors and impacts:
Capabilities Type + Cloud Deployment
+
Metrics Categories + Metrics Responsibility
83
These are organized in such a way as to provide a clear view of the determination of
Cloud security metrics responsibility.
5.1.1. Pilot study
In early work, we conducted a pilot study, the framework was shared with Cloud
experts to get their primary evaluation and thoughts that would help with our efficacy
studies in the main survey at a later stage of our research [67]. We conducted a Pilot
Study to help maintain maximum objectivity of our early work with taxonomy before
moving forward. The pilot study sought multiple points of view and perspectives
from participants related to our research focus. We chose experts who are working
in Cloud provider organizations, others who are working as Cloud customers, and
other participants who are Cloud researchers, and, therefore, neutral rather than either
Cloud customers or Cloud providers.
The pilot study questionnaires sent to the six participants included nine qualitative
questions, the first two dealing with their Cloud perspective (Cloud customer, Cloud
provider, or neutral) and their years of Cloud experience. Among the remaining
questions, as were five questions are about the framework.
They were then asked, using a Five-Level Likert approach (strongly agree, agree,
neither agree nor disagree, disagree, and strongly disagree) the following questions
to gain their opinions regarding our early work framework:
84
The pilot study questionnaire and preliminary result are following
Table 19 Framework pilot study Q 3
Q 3: D you agree that it importa to consider perce of individual
? (A ; refe
of determining the di of responsibilities.)
Level of Agreement
Strongly Agree
Agree
Neither Agree
nor Disagree
Disagree
Strongly
Disagree
Number of Responses
0
6
0
0
0
Table 20 Framework pilot study Q 4
Q 4: D
C
(A , Platfor , and Infrastructure)?
Level of Agreement
Strongly Agree
Agree
Neither Agree
nor Disagree
Disagree
Strongly
Disagree
Number of Responses
2
3
1
0
0
Table 21 Framework pilot study Q 5
Q 5: D
between the cus and the pro could vary bet public and private Cloud ?
Level of Agreement
Strongly Agree
Agree
Neither Agree
nor Disagree
Disagree
Strongly
Disagree
Number of Responses
5
1
0
0
0
Table 22 Framework pilot study Q 6
Q 6 : D y
C (public,
private)?
Level of Agreement
Strongly Agree
Agree
Neither Agree
nor Disagree
Disagree
Strongly
Disagree
Number of Responses
2
4
0
0
0
85
Table 23 Framework pilot study Q 7
Q 7: D
organizational capabilitie would help in a clear id about the security ?
Level of Agreement
Strongly Agree
Agree
Neither Agree
nor Disagree
Disagree
Strongly
Disagree
Number of Responses
1
3
1
1
0
In summary, when we look at the level of agreement percentage, without counting
the neutral responses from those who decided not to indicate whether they agreed or
disagreed, we determined that the results show that experts mainly agree with the
following, which led us to proceed in conducting a broad survey for our framework
research:
- The importance of considering the percentage of individual or shared
responsibility that the customer and the provider have for evaluating the
security metrics.
- The possibility of variation in the proportion of responsibility for evaluating
security metrics between the customer and the provider and between public
and private Clouds.
- The provision of such security metrics to assess and evaluate the
organizational capabilities as an aid in deciding about specific Cloud
deployment.
5.2. Framework Validation
The approach and process of the taxonomy and framework are fully described in the
Methodology chapter. To validate both the framework and the taxonomy we
conducted a single survey that had the responders evaluate both. However, both
86
framework and taxonomy have been shown separately on the survey to be evaluated
by measuring cloud experts satisfaction . We target users who are cloud and metrics
experts only.
These experts are but are not limited to being one of these types: Cloud provider,
Cloud customer /decision maker, Cloud user/technical staff and Cloud
researcher/regulator/authority.
In the case of framework, the survey was formulated to elicit an evaluation by Cloud
experts of different components and processes framework such as:
Determining their opinion in the importance of considering the percentage of
individual or shared responsibility that the customer and the provider have
for evaluating the security metrics.
Determining their opinion on the use of security metrics to assess and
evaluate the organizational capabilities as an aid in deciding about specific
Cloud capabilities types, metrics categories and shared responsibility.
The online survey started with a letter to prospective participants identifying the
reason for the survey and providing contact information for participant concerns. An
Informed Consent form was provided to the participants explaining their rights and
offering information about the IRB approval number, the study title, the purpose of
the study, procedures, potential risks of participating, potential benefits of
participating, compensation, confidentiality, the voluntary nature of participation,
and finally, an explanation of the right to withdraw from the study at any time during
the online survey.
More details are included in the dissertation Appendix (A,B).
87
The next part of the survey asked about the work experiences. Participants were
asked to provide information about the following:
Indicate their experience, if they have any, in using or providing Cloud
services.
Indicate whether they have experience with Cloud systems and Cloud
metrics.
Indicate if they have any experience as a Cloud regulator, Cloud Researcher
Educator or Trainer for Cloud users or providers or none.
Indicate the experience they may have in using security metrics or Cloud
services.
Indicate the years of experience in using security metrics.
Indicate the years of experience in using Cloud services.
Indicate their area of expertise in Cloud computing.
Indicate their area of expertise in security metrics.
A total of 198 participants started the survey. 71 completed the survey until the end.
Those who withdrew either did not agree to the consent form or did not feel familiar
with the research area. Of the participants, six indicated that they had zero experience
in the area of Cloud and security metrics; therefore, we excluded them. We only take
the responses from participants who completed the survey framework sections. In
particular, not every participant who completed the survey taxonomy section
completed the survey framework section. Therefore, the total valid sample for the
framework was 65 participants who completed the taxonomy section.
88
Table 24 Distribution of framework participants countries
By indicating their Internet Protocol address (IP) locations, the participants showed
their distribution across several countries in the world. However, the majority of
participants were from Saudi Arabia, where the online survey was originally
distributed.
As in the taxonomy portion of the survey, the framework has been defined in each
component in detail to provide a better understating. Moreover, we explained how
we classified the Cloud capabilities and the deployment + security metrics categories
and responsibilities by explaining each stage.
We showed the participants the framework without our assumptions being
provided, and we asked the participants to choose one of six choices:
a. Equal sharing
b. Towards the provider
c. Towards the customer
d. Less sharing between both
e. More shared between both
f. No sharing is necessary
BY COUNTRY
#
%
Malaysia
1
1.5
Oman
1
1.5
Saudi Arabia
47
72.3
United Arab Emirates
1
1.5
United Kingdom
8
12.3
United States
7
10.8
T
65
100
89
Figure 14 Framework as displayed in the survey
We asked the participants to indicate the sharing of responsibility of Cloud metrics
that they find most appropriate for the combination of components that is provided.
There were nine combinations of the three capabilities and three metric categories:
Application-Technical Metrics
Application-Operational Metrics
Application- Management Metrics
Platform – Technical Metrics
Platform – Operational Metrics
Platform – Management Metrics
Infrastructure – Technical Metrics
Infrastructure – Operational Metrics
Infrastructure – Management Metrics
90
5.3. Framework Results Analysis
The results contain two sections: the work background and the framework
evaluation. Each result has been shown in a table or figure along with the related
question as follows:
Please indicate whether you have experience with Cloud systems and Cloud
metrics.
Table 25 Framework participants experience perspective
EXPERIENCE PERSPECTIVE
#
%
Cloud and metrics Background
16
24.6
Cloud Background
30
46.2
Metrics Background
11
16.9
No experience with Cloud and Metrics
8
12.3
T
65
100
The focus of this dissertation research is a combination of two fields, the cloud
security and metrics. Therefore, the participants were asked to identify their
background experience in order to determine their perspective on the framework
separately. The results showed that there were a 30 of participants who had a Cloud
background and 11 who had a metrics background along with 16 participants who
had experience in both metrics and the Cloud.
91
Please indicate the experience you have in using or providing Cloud services.
Table 26 Framework participants Cloud use prospective
CLOUD USE PROSPECTIVE
#
%
Neither Cloud Customer nor Cloud Provider (Neutral)
4
6.2
Provide Cloud ( Cloud Provider )
15
23.1
Use Cloud ( Cloud Customers )
46
70.8
T
65
100
The participants were asked to respond to the following: “Please indicate the
experience you have in using or providing Cloud services.” to analyze their possible
different perspectives in future work, if needed. Some of the participants were
neutral, in that they claimed to be neither Cloud customers nor Cloud providers.
Note: Participants identifying themselves as both “user and provider of the Cloud”
are considered as Cloud providers in our analysis.
Table 27 Framework participants others experience
H experience as
C regulator
C researcher
E or trainer
N
Total : 65
20
32
13
14
A number of participants who were Neutral were neither Cloud customers nor
providers. Their evaluation was considered as neutral, in between customer and
provider, as, for example, participants who work in academia or as cloud authorities.
The participants were allowed to indicate multiples choices if they were responsible
for more than one task that was provided in the survey questions.
92
How many years of Cloud Computing related work experience do you have?
Table 28 Framework participants Cloud years of experience
C experience
#
%
No experience
11
16.9
0-1
30
46.2
2-4
13
20.0
4-6
9
13.8
7+
2
3.1
Total
65
100
Results showed the years of experience for Cloud experts only. Regarding work
experience, participants were asked about their years of Cloud experience, so that
participants who selected “No experience” were assumed to have only metrics
expertise.
Do you believe that your Cloud Computing expertise is more toward?
Table 29 Framework participants Cloud expertise area
C expertise area
#
%
Cloud Management
27
41.5
Cloud Operations
15
23.1
Cloud Technical
11
16.9
I am not Cloud expert
12
18.5
In order to be more specific about the perspective of Cloud expertise participation,
participants were asked to identify their Cloud area.
Participants identified their expertise area based on the following criteria:
93
Management Team: Responsible for tasks such as: strategy, rules, policy, guideline,
evolution, recommending, governance, compliance, key performance indicator
(KPI). etc...
Operations Team: Responsible for tasks such as: monitoring, implementation,
optimization, applying new configurations, continuity, daily operations,
maintenance, update, reporting. etc...
Technical Team: Responsible for tasks such as: hardware and system installation,
set-up basic configuration, support to end-users, Diagnose and resolve issues, collect
preliminary information. etc…
How many years of Metrics related work experience do you have?
Table 30 Framework participants metrics years of experience
M experience
#
%
No experience
15
23.1
0-1
23
35.4
2-4
5
7.7
4-6
4
6.2
7+
18
27.7
Total
65
100
Result showed the years of experience for metrics experts only. As work experience,
participants has been asked about their years of Cloud experience, participants whom
selected “No experience” are Cloud expertise only.
94
Do you believe that your Metrics expertise is more toward?
Table 31 Framework participants metrics expertise area
M expertise area
#
%
Management Metrics
28
43.1
Operational Metrics
13
20.0
Technical Metrics
9
13.8
I am not metrics expert
15
23.1
In order to be more specific about the prospective of Cloud expertise participation,
they have been identify regarding to their metrics area.
Participants identified their expertise area based on the following criteria:
Management Metrics: responsible for metrics such as:
· Mean Cost of Incidents
· IT Security Spending as % of IT Budget
· Patch Policy Compliance
· Percentage of Configuration Compliance
Operational Metrics: responsible for metrics such as:
· Mean-Time to Incident Discovery
· Mean-Time to Mitigate Vulnerabilities
· Mean-Time to Incident Recovery
· Percent of Changes with Security Exceptions
Technical Metrics: responsible for metrics such as:
· Current Anti-Malware Compliance
· Number of Known Vulnerability Instances
· Patch Management Coverage
· Security Testing Coverage
95
Cloud Application: Technical, Operational and Management Metrics
Table 32 Results of responsibility allocation - cloud application
Application
Technical
Operational
Management
#
%
#
%
#
%
Valid
65
a. Equal sharing
8
12.3
16
24.6
19
29.2
b. Towards the provider
24
36.9
19
29.2
12
18.5
c. Towards the customer
15
23.1
13
20.0
15
23.1
d. Less sharing between both
5
7.7
9
13.8
9
13.8
e. More shared between both
10
15.4
6
9.2
7
10.8
f. No sharing is necessary
3
4.6
2
3.1
3
4.6
Table 32 shows the allocation of shared responsibility between Cloud customers and
provider by number (#) and percentage (%) of the 65 experts with valid responses.
In the case of Cloud Application capabilities types, in consideration of the different
metrics categories:
Technical, Operational and Management Metrics
The results showed variation in shared responsibility in different cases, effected by
the different capabilities types and metrics categories.
96
Table 33 Cloud application both shared responsibility
Table 33 only included the responses from Table 32 that related to agreement to
shared responsibility with different proportions of sharing (Less Sharing in between,
Equal sharing, More Sharing in between, No Sharing is necessary).
Table 34 Cloud application customers vs provider shared responsibility
In Table 34, we only included those choices in Table 32 that lean toward the cloud
customer or lean toward the cloud provider.
Cloud Application Amount of Sharing
L:Less Sharing in between E:Equal sharing M:More Sharing in between N:No Sharing
Management Metrics
Operational Metrics
Technical Metrics
sharing
L
E
M
N
L
E
M
N
L
E
M
N
#
9
19
7
3
9
16
6
2
5
8
10
3
%
13.8
29.2
10.8
4.6
13.8
24.6
9.2
3.1
7.7
12.3
15.4
4.6
Cloud Application Balance of Sharing
C P
Cloud Customers Cloud Provider
Management Metrics
Operational Metrics
Technical Metrics
sharing
C
P
Total
C
P
Total
C
P
Total
#
15
12
27
13
19
32
15
24
39
%
23.1
18.5
41.6
20.0
29.2
49.2
23.1
36.9
60
97
Since the questions regarding less sharing, more sharing or equal showed no
preference for customer or provider, we merged all of those responses from Table 33
into the category of “shared responsibility” in Figure 15.
The percentages assigned to customer and provider in Figure 15 came from Table
34.
Figure 15 Application:
participants allocation percentage
98
Cloud Platform: Technical, Operational and Management Metrics
Table 35 Results of responsibility allocation - Cloud Platform
Platform
Technical
Operational
Management
#
%
#
%
#
%
Valid
65
a. Equal sharing
17
26.2
14
21.5
19
29.2
b. Towards the provider
23
35.4
17
26.2
17
26.2
c. Towards the customer
6
9.2
11
16.9
11
16.9
d. Less sharing between both
6
9.2
8
12.3
4
6.2
e. More shared between both
7
10.8
10
15.4
7
10.8
f. No sharing is necessary
6
9.2
5
7.7
7
10.8
Table 35 shows the allocation of shared responsibility between Cloud customers and
provider by number (#) and percentage (%) of the 65 experts with valid responses.
In the case of Cloud Platform capabilities types, in consideration of the different
metrics categories:
Technical, Operational and Management Metrics
The results showed variation in shared responsibility in different cases, effected by
the different capabilities types and metrics categories.
99
Table 36 Cloud platform both shared responsibility
In Table 36, only included the responses from Table 35 that related to agreement to
shared responsibility with different proportion of sharing (Less Sharing in between,
Equal sharing, More Sharing in between, No Sharing is necessary).
Table 37 Cloud platform customers vs provider shared responsibility
In Table 37, we only included those choices in Table 35 that lean toward the cloud
customer or lean toward the cloud provider.
Cloud Platform Amount of Sharing
L:Less Sharing in between E:Equal sharing M:More Sharing in between N:No Sharing
Management Metrics
Operational Metrics
Technical Metrics
sharing
L
E
M
N
L
E
M
N
L
E
M
N
#
4
19
7
7
8
14
10
5
6
17
7
6
%
6.2
29.2
10.8
10.8
12.3
21.5
15.4
7.7
10.8
26.2
1.8
9.2
Cloud Platform Balance of Sharing
C P
Cloud Customers Cloud Provider
Management Metrics
Operational Metrics
Technical Metrics
sharing
C
P
Total
C
P
Total
C
P
Total
#
11
17
28
11
17
28
6
23
29
%
16.9
26.2
43.1
16.9
26.2
43.1
9.2
35.4
44.6
100
Since the questions regarding less sharing, more sharing or equal showed no
preference for customer or provider, we merged all of those responses from Table 36
into the category of “shared responsibility” in Figure 16. The percentages assigned
to customer and provider in Figure 16 came from Table 37.
Figure 16 Platform: participants
allocation percentage
101
Cloud Infrastructure: Technical, Operational and Management Metrics
Table 38 Results of responsibility allocation - Cloud Infrastructure
Infrastructure
Technical
Operational
Management
#
%
#
%
#
%
Valid
65
a. Equal sharing
13
20.0
13
20.0
13
20.0
b. Towards the provider
25
38.5
23
35.4
17
26.2
c. Towards the customer
10
15.4
13
20.0
17
26.2
d. Less sharing between both
3
4.6
6
9.2
8
12.3
e. More shared between both
6
9.2
6
9.2
6
9.2
f. No sharing is necessary
8
12.3
4
6.2
4
6.2
Table 38 shows the allocation of shared responsibility between Cloud customers and
provider by number (#) and percentage (%) of the 65 experts with valid responses.
In the case of Cloud Infrastructure capabilities types, in consideration of the different
metrics categories:
Technical, Operational and Management Metrics
The results showed variation in shared responsibility in different cases, effected by
the different capabilities types and metrics categories.
102
Table 39 Cloud infrastructure both Shared Responsibility
In Table 39, only included the responses from Table 38 that related to agreement to
shared responsibility with different proportion of sharing (Less Sharing in between,
Equal sharing, More Sharing in between, No Sharing is necessary).
Table 40 Cloud infrastructure customers vs provider shared responsibility
In Table 40, we only included those choices in Table 38 that lean toward the cloud
customer or lean toward the cloud provider.
Cloud Infrastructure Amount of Sharing
L:Less Sharing in between E:Equal sharing M:More Sharing in between N:No Sharing
Management Metrics
Operational Metrics
Technical Metrics
sharing
L
E
M
N
L
E
M
N
L
E
M
N
#
8
13
6
4
6
13
6
4
3
13
6
8
%
12.3
20.0
9.2
6.2
9.2
20.0
9.2
6.2
4.6
20.0
9.2
12.3
Cloud Infrastructure Balance of Sharing
C P
Cloud Customers Cloud Provider
Management Metrics
Operational Metrics
Technical
Metrics
sharing
C
P
Total
C
P
Total
C
P
Total
#
17
17
34
13
23
36
10
25
35
%
26.2
26.2
52.4
20.0
35.4
55.4
15.4
38.5
53.9
103
Since the questions regarding less sharing, more sharing or equal showed no
preference for customer or provider, we merged all of those responses from Table 39
into the category of “shared responsibility” in Figure 17. The percentages assigned
to customer and provider in Figure 17 came from Table 40.
Figure 17 Infrastructure:
participants allocation percentage
104
5.4. Discussion
There are mainly two indirect questions, the first question about the shared
responsibility and the second question about the proportion of responsibility shared
between cloud customer and provider:
Q A = Should we have shared responsibility? If Yes, should it be (a) equal sharing,
(d) less sharing or (e) more sharing?
No
Yes
Q B = If we have sharing, should it be (a) equal sharing, (b) more toward the provider
or (c) should more toward the customer?
105
In Question A, we assume that when the participant chose a (equal sharing), they
meant that there is a preference for equally sharing. However, we assume that when
the participant’s choice was b (less sharing), or e (more sharing), there is a preference
for sharing but they did not specify whether it should lean toward the customer or
toward the provider.
In Question B, we assume that when the participant chose a (equal sharing), they
meant that the control is equally shared between Cloud customer and provider. At
the same time, we assume that when the participant chose b (toward a provider), they
meant that there should be sharing but that the control is more with the provider. We
assume as well that when the participant chose c (toward a customer), they meant
that there is a sharing but the control is more with the customer.
Because we didn’t ask these two questions directly, it is difficult to know if the
survey participant understood some questions, and it will be considered in future
work for more investigation.
Framework keys
Cloud Customer
Both Sharing
Cloud Provider
106
Figure 18 Framework with all participants’ allocation percentage
For the final analysis in regard to answering the research questions related to the
framework:
1. Should the proportion of responsibility for evaluating security metrics
between the customer and the provider vary for the different Cloud
capabilities (Application, Platform and Infrastructure)?
2. Should the proportion of Metrics responsibility for evaluating security
metrics between the customer and the provider vary for the different metrics
categories (Management, Operational and Technical)?
3. Does the relationship of Cloud computing and security need to be organized
to distinguish the shared responsibility of Cloud security metrics?
107
In regard to research question one, as we can see in Tables 32, 35, and 38 and Figure
18, results show that the proportion of metrics categories shared responsibility
between Cloud customer and provider are different based on each capabilities types,
whether Application, Platform or Infrastructure. Figure 18 shows that there is a
variation among the proportions of the Cloud capabilities. For instance, Technical
metrics in platform is 36.7% larger than in Infrastructure, while operational metrics
in platform is 27.8% larger than in Infrastructure. In contrast, with management
metrics, Application is 29.6% larger than in Infrastructure.
The results answered question one by showing clearly that the proportion of
responsibility for evaluating security metrics between the customer and the provider
vary for the different Cloud capabilities (Application, Platform and Infrastructure).
In regard to research question two, as we can see in Tables 32, 35, and 38 and Figure
18, the survey results show the proportion of shared responsibility between the Cloud
customer and the provider are different based on each security metrics category.
Figure 18 shown that there are a vary of proportion the security metrics categories.
For instance, Management is 22.8.0% larger than technical metrics in Infrastructure
capability type; however, in Platform, the Operational metrics is only 6.5% larger
than management, not a large difference between capabilities. In application,
Management is 52.0% larger than technical.
The results answered question two by showing clearly that the proportion of Metrics
responsibility for evaluating security metrics between the customer and the provider
vary for the different metrics categories (Management, Operational and Technical).
In regard to research question three, as we can see in Figure 18, results show that in
application capability type more than 95% agreed with the need for sharing of
responsibility. In the platform capability type, almost 90% agreed with sharing, while
108
in the infrastructure capability type, 87% agreed with sharing for technical metrics,
and over 93% agreed with sharing in the management and operational metrics
categories.
The results answered question three by showing clearly that the majority of
participants agreed with the need for shared responsibility. For this reason, cloud
customers and providers need to allocate their shared responsibilities to avoid
possible gaps or overlaps between them.
109
Chapter 6: Guidelines for Allocating Cloud Shared
Responsibility
The guidelines go through a process of four stages to end up with clear output that is
used in organizing responsibility in the framework.
6.1. Framework Component
When developing the taxonomy, we assumed that the proportion of responsibility for
evaluating security metrics will be shared between the customer and the provider and
will be impacted by multiple factors defined by the taxonomy, for example:
Capabilities Type + Cloud Deployment + Metrics Categories + Metrics
Responsibility
After presenting the complete framework in Figure 13, we explain all of its
components in detail in Figure 7. The guidelines explain the process of implementing
the stages step by step.
110
Figure 19 Guidelines for framework implementation stages
As shown in Figure 19, the guidelines have four stages:
Stage One – customer will determine the Capabilities Type that fits their needs.
The survey shows that the experts believe that different Capabilities Types will
impact the proportional level of responsibility in later stages.
Stage Two – customer will determine the Cloud Deployment that fits their needs.
The survey shows that the experts believe that the choice between Public and Private
Clouds will impact the level of security in later stages.
111
The output of stage one and two will lead toward stage three.
Stage Three – based on the selected Cloud Deployment and Capabilities Type, the
metrics will be divided into the categories of management, operational,
and technical.
Stage Four – the proportion of metrics responsibility is dynamic between the
customer and the provider, the choices made in stages one and two can change the
proportion of metrics responsibility.
The shared responsibilities shown in Figure 19 are for illustration, and the survey
shows that the experts believe that the actual number could be affected by choices
made in stages one and two and can change the proportion of metrics responsibility.
6.2. Case Study: Guidelines for Implement FY15 CIO Annual
FISMA Metrics
As we explain in Chapter 2, FISMA refers to the Federal Information Security
Management Act of 2002, which recognizes the significance of information security
to the national and economic interests of the United States. It specifies a
comprehensive framework to secure government information, assets, and operations
against man-made and natural threats. FISMA allocates responsibilities to different
agencies to guarantee the information security of the Federal Government [46].
The FISMA provides a list of metrics to measure the compliance with their
framework. Although their existing metrics were not designed for nor are they related
to the Cloud, in this case study, we use them only to demonstrate how our guidelines
can be used by implementing the FISMA metrics.
112
FY15 CIO Annual FISMA Metrics has nine categories:
1. System Inventory
2. Information Security Continuous Monitoring
3. Identity Credential and Access Management
4. Anti-Phishing and Malware Defense
5. Data Protection
6. Network Defense
7. Boundary Protection
8. Training and Education
9. Incident Response
Each category group contains a list of metrics with a total of 53 Metrics across all of
the categories. We chose one metric randomly from each of the nine categories.
Moreover, we decided the capability type randomly to test more possible
combinations.
We assumed that the deployment type was public in all cases. Each of the metrics
was assigned to one of the metrics categories based of its type following the criteria:
Management Metrics: Offer information on business performance functions, and
how they impact the performance.
Operational Metrics: Exploited to understand and optimize business functions.
Technical Metrics: Offer technical details and establish a foundation for other
metrics [60].
In the final analysis, we determined the responsibility by assigning an individual
metric to a Cloud customer only or to a Cloud provider only or to both.
113
1. SYSTEM INVENTORY
1.2. How many endpoints belong to systems without a valid Authorization to
Operate ATO?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: ATO is based on a risk based framework to implement the security
controls within the IT environment. The Cloud customer management team
assesses the risk and knows how many endpoints exist without (ATO)[69].
2. INFORMATION SECURITY CONTINUOUS MONITORING
2.8. How many major application databases does the organization maintain?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: since it is a platform, maintenance is shared between both. Thus, we
count the number of application databases the organization maintains as part of
the management metrics
114
3. IDENTITY CREDENTIAL AND ACCESS MANAGEMENT
3.8 How many users are enabled to remotely log onto the organization's
LAN/WAN resources or services from mobile devices?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: since it is an infrastructure where the majority resources are under
the customer control, the number of remotely accessed mobile devices has been
identified as a management metric
4. ANTI-PHISHING AND MALWARE DEFENSE
4.3. Percent (%) of hardware assets covered by a host-based intrusion prevention
system.
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: the provider owns the resources, even when the customer controls
the infrastructure, so the operational team has to confirm that hardware assets are
covered by a host-based intrusion prevention system.
115
5. DATA PROTECTION
5.1. What is the estimated number of hardware assets in each of the following
mobile and portable asset types, and how many are encrypted?
Mobile and Portable Device
Types
Estimated number of
mobile hardware assets of
the types indicated in each
row.
Estimated
number of assets
from 5.1.1 with
FIPS 140-2
compliant
encryption of data
on the device.
Laptop computers and netbooks
Tablet-type computers
Smartphones
Other mobile devices
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: knowing the number of hardware assets and how many are encrypted
is part of the management metrics. However, the customer is responsible for
managing their assets.
116
6. NETWORK DEFENSE
6.1. What is the estimated percent (%) of remote access connections that have each
of the following properties?
6.1.1. Percent (%) that utilize Federal Information Processing Standard FIPS 140-
2-validated cryptographic modules.
6.1.2. Percent (%) that prohibit split tunneling and/or dual-connected remote hosts
where the mobile device has two active connections.
6.1.3. Percent (%) configured in accordance with Office of Management and
Budget OMB M-07-16 to time-out after 30 minutes of inactivity (or less) and
requires re-authentication to reestablish session.
6.1.4. Percent (%) scanned for malware upon connection
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: remote access needs to be a consideration of both customer and
provider, especially when it is required to be in compliance with government
standards. Being aware of the estimated percent is an operational metric.
117
7. BOUNDARY PROTECTION
7.1. Percent (%) of the required Trusted Internet Connections TIC 2.0 Capabilities
implemented.
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: being aware of the estimated percent is an operational metrics, but
the Internet connections in between the customer and the provider need to be the
responsibility of both.
8. TRAINING AND EDUCATION
8.1. Percent (%) of users that successfully completed annual Cybersecurity
Awareness and Training (CSAT).
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: since it involves applications, Cloud customers need to consider the
efficient use of resources so that the Cloud user employee training is part of
customer responsibility. Training is also a part of management metrics.
118
9. INCIDENT RESPONSE
9.3. When will the agency transition to the new United States Computer
Emergency Readiness US-CERT reporting format?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Justification: Meeting the reporting requirement is part of the management metrics
but is also the responsibility of the customer
The remaining FISMA metrics are listed in the (Appendix C), with the classifications
of metrics categories and shared responsibility providing an example of the use of
the guidelines and framework. For a real-life case, the Cloud customer and provider
would have to set up an agreement by following the same process and allocating the
responsibility.
119
Chapter 7: Conclusion
7.1. Summary
With the increasing importance of measuring the security of Cloud systems, and the
increasing need to focus attention on the security metrics that are specific to Cloud
computing, the use of existing metrics can be adopted to the Cloud computing
industry, thus saving effort in improving service selection, service agreement, and
service verification. The goal of our research was to develop a taxonomy that cloud
be uses as basis for a framework that would guide the cloud customers and providers
by supporting their decisions in selecting and using security metrics to better protect
Cloud systems by allocating shared responsibility between Cloud customer and
provider.
In this dissertation, we present and discuss a taxonomy of Cloud security metrics and
a framework for allocating shared responsibility for selecting and managing Cloud
security metrics, as well as providing guidelines for implementing the framework.
These components will all support cloud users in making decisions regarding
selecting and using security metrics to better protect Cloud systems.
In chapter 4, we presented the taxonomy which considers several novel viewpoints
in regard to how the Metrics are organized by Cloud capability type (Application,
Platform, Infrastructure) along with the type of Cloud deployment (public, private,
hybrid, community), and the different needs of managerial, operational and technical
staff within a cloud system. The survey results show that exports believe there is a
need for allocating shared responsibility where the cloud customer and provider need
to avoid possible gaps or overlaps between them which will cause vulnerability to
their organization.
120
In chapter 5, we presented the framework for allocating cloud security metrics which
help to determine the proportion of shared responsibility for Cloud security metrics
between potential Cloud customers and service providers based on factors such as
the type of Cloud capability, the deployment model, and the metrics categories.
In chapter 6, we provided guidelines as to how to implement our framework. The
case study shows how to implement any existing security metrics to the Cloud.
To validate the taxonomy and framework, we conducted a survey of cloud security
and metrics experts with a total of 86 responses. The survey results show that the
majority of participants supported the need for a taxonomy and agreed with the
necessity for shared responsibility in selection and management of cloud security
metrics. Moreover, they also agreed that the proportion of responsibility for
evaluating security metrics between the customer and the provider may vary for the
different Cloud capabilities (Application, Platform and Infrastructure). Similarly, the
results show that the balance of responsibilities between the customer and the
provider for evaluating security metrics may vary for the different metrics categories
(Management, Operational and Technical).
7.2. Future Directions
The collected results from the survey can be analyzed further by consider several
points of view. For instance, it would be useful to compare different perspectives of
Cloud customer vs provider and to compare different perspectives on management,
operational, and technical metrics and study the impact of eliciting responses from
each category of participant separately to gain different points of view.
In the future, we will continue working towards evaluation and implementation of
our taxonomy, guidelines, and framework. Further research can be conducted in this
area as a continuation of our work, for instance, using our guidelines and implanting
121
the framework in a real-life example of SLA, allocating the shared responsibility
between the cloud customer and provider along with different validation methods .
Moreover, studying the impact of different deployment types, especially
distinguishing the shared responsibility with public and private cloud deployment
would be effective. Furthermore, details guideline will be develop for classification
of secure data within the framework to assist government agency in regards to
classify data.
There are lessons to be learned from our survey, and we will be considering them in
future research. For instance, using an even number, such as a four-Likert scale to
get clear responses in regards to level of agreement, would be preferable. The two
indirect questions that have been mentioned in section 5.4, the first question about
the shared responsibility (How much sharing?) and the second question about the
proportion of responsibility shared between cloud customer and provider (The
direction of the sharing), will be dealt with using direct questions in future research
in order to get clearer and better results for further evaluation.
122
References
[1] T. J. Kwasniewski and E. Puig, “Cloud computing in the government: a DACS
critical review and technology assessment,” 2011.
[2] P. Mell and T. Grance, “NIST SP 800-145, The NIST Definition of Cloud
Computing,” NIST Spec. Publ., vol. 145, p. 7, 2011.
[3] Enisa, Security Framework for Governmental Clouds, no. February. 2015.
[4] “INTERNATIONAL STANDARD ISO/IEC 17789 Information technology —
Cloud computing — Reference architecture,” 2014.
[5] ISO/IEC, “ISO/IEC 17788 -Y. 3500 SERIES Y- Information technology - Cloud
computing - Overview and vocabulary,” ITU SERIES Y, no. ITU Y.3500. pp. 1–16,
2014.
[6] R. Moreno-Vozmediano, R. S. Montero, and I. M. Llorente, “Key challenges in
cloud computing: Enabling the future internet of services,” IEEE Internet Comput.,
vol. 17, no. 4, pp. 18–25, 2013.
[7] W. Jansen and T. Grance, “NIST 800-144 Guidelines on Security and Privacy in
Public Cloud Computing,” NIST Spec. Publ. 800-144, p. 80, 2011.
[8] R. Pardo, Jorge. Andrew, Flavin. Michael, “2016 Top Markets Report Cloud
Computing,” 2016.
[9] R. Pardo, Jorge. Andrew, Flavin. Michael, “2016 Top Markets Report Cloud
Computing,” 2016.
[10] H. Schulze, “Cloud security SPOTLIGHT REPORT 2016,” 2016.
[11] M. Weinberger, “Google And Cisco Have Teamed Up In The Cloud Wars Against
Amazon And Microsoft.” [Online]. Available:
http://www.businessinsider.com/google-cloud-and-cisco-sign-partnership-2017-
10?IR=T. [Accessed: 02-Nov-2017].
[12] Julie Spitzer, “Amazon and Microsoft expand AI partnership,” beckers hospital
review. [Online]. Available: https://www.beckershospitalreview.com/artificial-
intelligence/amazon-and-microsoft-expand-ai-partnership.html.
[13] Caroline Wong, “The New Normal : Security Metrics and Cloud Computing,”
2013. [Online]. Available: https://www.symantec.com/connect/blogs/new-normal-
security-metrics-and-cloud-computing.
[14] I. T. Control, IT Control Objectives for Cloud Computing: Controls and Assurance
in the Cloud. ISA, 2011.
[15] M. Al Morsy, J. Grundy, and I. Müller, “An Analysis of The Cloud Computing
Security Problem,” 2010.
[16] Hassan, T. And, J. B.D., Joshi, U. Of, Pittsburgh, Gail-Joon, Ahn, A. State, and
University, “Security and Privacy Challenges in Cloud Computing Environments.” .
[17] “IBM Point of View : Security and Cloud Computing Cloud computing,” no.
November, 2009.
123
[18] R. B. Bohn, J. Messina, F. Liu, J. Tong, and J. Mao, “NIST SP 500 -292 NIST
cloud computing reference architecture,” Proc. - 2011 IEEE World Congr. Serv.
Serv. 2011, pp. 594–596, 2011.
[19] C. Special and I. Group, “PCI Data Security Standard,” no. February, 2013.
[20] Cloud Standards Customer Council, “Cloud Security Standards : What to Expect &
What to Negotiate,” 2013.
[21] “ISO/IEC 27017 INTERNATIONAL STANDARD - Information technology —
Security techniques — Code of practice for information security controls based on
ISO/IEC 27002 for cloud services technologies,” vol. 2015, no. 1st, 2015.
[22] F. Simorjay, “Microsoft® AzureTM Shared Responsibilities For Cloud Computing,”
2016.
[23] A. W. Services and S. P. August, “Amazon Web Services : Overview of Security
Processes,” vol. 2016, no. August, 2016.
[24] Oracle, “Oracle Infrastructure and Platform Cloud Services Security,” 2016.
[25] R. B. Bohn, “NIST Sp - 500-293 US Government Cloud Computing Technology
Roadmap 2014,” Nist Spec. Publ., vol. I; II, p. 85, 2014.
[26] F. de Vaulx, E. Simmon, and R. Bohn, “Cloud Computing Service Metrics
Description,” Nist Spec. Publ., vol. 500, no. 307, p. 24, 2015.
[27] L. Badger, T. Grance, R. Patt Corner, and J. Voas, “NIST 800-146 Special
Publication - Cloud Cloud Computing Synopsis and Recommendations,” Recomm.
Natl. Inst. Stand. Technol., vol. C O M P, p. 84, 2011.
[28] J. T. Michael Hogan, Fang Liu, Annie Sokol, “NIST Special Publication 500‐291
Cloud Computing Standards Roadmap,” 2011.
[29] J. L. Bayuk, “Measuring systems security: an initial security theoretical construct
framework,” 2011.
[30] J. Luna, H. Ghani, D. Germanus, and N. Suri, “A Security Metrics Framework for
the Cloud,” Int. Conf. Secur. Cryptogr., pp. 245–250, 2011.
[31] W. Jansen and T. Grance, “NIST Guidelines on Security and Privacy in Public
Cloud Computing,” Director, vol. 144, no. 7, pp. 800–144, 2011.
[32] NIST, “The NIST Definition of Cloud Computing Recommendations of the
National Institute of Standards and Technology,” Nist Spec. Publ., vol. 145, p. 7,
2011.
[33] W. Bumpus, “NIST Cloud Computing Standards Roadmap,” NIST Cloud Comput.
Stand., pp. 1–3, 2013.
[34] Australian Government, “Cloud Computing Regulatory Stock Take Report,” 2014.
[35] G. Ict and S. March, “UK Government Cloud Strategy,” 2011.
[36] “The STRIDE Threat Model,” Microsoft, 2016. [Online]. Available:
https://msdn.microsoft.com/en-us/library/ee823878.
[37] Cloud Security Alliance, “Cloud Computing Top Threats in 2016: Treacherous 12,”
2016.
[38] D. R. Thompson, J. Di, H. Sunkara, C. Thompson, and E. Hall, “Categorizing RFID
Privacy Threats with STRIDE.”
[39] “ISO / IEC 27001 INTERNATIONAL STANDARD Information technology —
Security techniques — Information security management systems —
Requirements,” 2013.
124
[40] ISO/IEC, “ISO 27002 INTERNATIONAL STANDARD ISO / IEC Information
technology — Security techniques — Code of practice for information security
controls,” vol. 2013, no. attachment 1, p. 90, 2013.
[41] I. Standard, “ISO 27004 Information technology — Security techniques —
Information security management — Measurement Technologies,” vol. 2009, 2009.
[42] T. Trunk, “ISO: In Search of cloud infrastructure compliance,” 2016. [Online].
Available: https://waxie.us/blog/iso-standards.
[43] T. Trunk, “ISO: In Search of cloud infrastructure compliance,” 2016. .
[44] P. Hofmann and D. Woods, “Cloud computing: The limits of public clouds for
business applications,” IEEE Internet Comput., vol. 14, no. 6, pp. 90–93, 2010.
[45] ISO/IEC, “DRAFT INTERNATIONAL STANDARD ISO / IEC DIS 19086-2
Information technology — Cloud computing — Service level agreement ( SLA )
framework — Part 2 : Metric Model,” vol. 2017, 2017.
[46] Public Low 107-347, “Federal Information Security Management Act (FISMA) -
Information security,” Fed. Inf. Secur. Manag. Act 2002, vol. 48, no. 1, pp. 48–63,
2007.
[47] Cloud Lock, “FISMA Compliance for Cloud Applications.” [Online]. Available:
https://www.cloudlock.com/wp-content/uploads/2015/04/FISMA-Compliance-
Guide-CloudLock.pdf. [Accessed: 27-May-2015].
[48] NIST, “NIST 800-53 Security and Privacy Controls for Federal Information
Systems and Organizations Security and Privacy Controls for Federal Information
Systems and Organizations,” Sp-800-53Ar4, p. 400+, 2014.
[49] V. Steven, Federal Risk and Authorization Management Program (FedRAMP). .
[50] A. S. AN and A. W. L. A. S, “FISMA VS. FEDRAMP: CONTROLS AND
AUTHORI ZATION DIFFERENCES.”
[51] D. of H. and H. Services, “Modifications to the HIPAA Privacy, Security,
Enforcement, and Breach Notification Rules Under the Health Information
Technology for Economic and Clinical Health Act and the Genetic Information
Nondiscrimination Act; Other Modifications to the HIPAA Rules,” Fed. Regist. /,
vol. 78, no. 17, pp. 1–138, 2013.
[52] Office of Civil Rights Department of Health and Human Services, “Summary of the
HIPAA Privacy Rule,” OCR Priv. Br., p. 25, 2003.
[53] T. A. Ragone, “The Future of HIPAA in the Cloud,” 2013.
[54] United States Congress, Health Information Technology (HITECH Act), vol. 2009.
2009, pp. 112–164.
[55] R. Barabanov, S. Kowalski, and L. Yngstrom, “Information Security Metrics State
of the Art,” Stock. Univ., no. 11, p. 62, 2011.
[56] A. Jaquith, Security Metrics: Replacing Fear." Uncertainty, and Doubt. Boston:
Addison-Wesley, 2007.
[57] J. Bayuk, “Cloud security metrics,” in 2011 6th International Conference on System
of Systems Engineering, 2011, pp. 341–345.
[58] B. G. Johnson, “Measuring ISO 27001 ISMS processes,” 2014.
[59] C. Kormos, N. S. Agency, A. Systems, N. Givans, and N. Bartol, “USING
SECURITY METRICS TO ASSESS RISK MANAGEMENT CAPABILITIES,” in
National Information Systems Security Conference, 1999, no. 410.
125
[60] Center for Internet Security, “The CIS Security Metrics,” Cent. Internet Secur., vol.
1.1.0, no. 28, p. 175, 2010.
[61] C. Wong, “Security Metrics , A Beginner ’ s Guide,” 2011.
[62] Caroline Wong, “The New Normal : Security Metrics and Cloud Computing,”
2013. .
[63] F. S. Reviewers:, “Microsoft® AzureTM Shared Responsibilities For Cloud
Computing,” pp. 1–11, 2016.
[64] About us, “Survey Monkey,” 2014. [Online]. Available:
https://www.surveymonkey.com/mp/aboutus/.
[65] “Saudi Group for Information Assurance - Hemaya.” [Online]. Available:
hemayagroup.org. [Accessed: 12-4-2017].
[66] IBM Corporation, “IBM Statistical Package for the Social Sciences SPSS,” 2016.
[Online]. Available: https://www-01.ibm.com/software/ch/de/analytics/spss/.
[67] M. A. Hertzog, “Considerations in determining sample size for pilot studies,” Res.
Nurs. Health, 2008.
[68] J. A. Busch, “How to Validate a Taxonomy,” 2009.
[69] fedramp, “What is an Authority to Operate (ATO)?,” fedramp. [Online]. Available:
https://www.fedramp.gov/resources/faqs/what-is-an-authority-to-operate-ato/.
[70] ISO/IEC, “ISO/IEC 27000 Information technology — Security techniques —
Information security management systems — Overview and vocabulary,” October,
vol. 3, p. 38, 2014.
126
Appendix A: IRB approval letter
127
Appendix B: Survey
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
Appendix C: FISMA Metrics
Federal Information Security Management Act:
Public Low 107-347, “Federal Information Security Management Act (FISMA) -
Information security,” Fed. Inf. Secur. Manag. Act 2002, vol. 48, no. 1, pp. 48–63,
2007
1. SYSTEM INVENTORY METRICS
1.1 For each FIPS 199 impact level, what is the total number of operational
unclassified information systems by organization (i.e. Bureau or Sub-
Department Operating Element) categorized at that level?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
1.2. How many endpoints belong to systems without a valid ATO? (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
1.3. How many public facing systems are without a valid ATO? (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
148
2. INFORMATION SECURITY CONTINUOUS MONITORING
METRICS
2.1. What is the total number of the organization’s hardware assets connected to
the organization’s unclassified5 network(s)?6 (Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.2. Percent (%) of the organization’s network fabric covered by a capability to
detect and alert on the addition of unauthorized hardware assets onto the
organization’s network. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.3. Percent (%) of the organization's network fabric covered by an automatic
capability (scans/device discovery processes) that provides enterprise-level
visibility into the current state of all hardware assets. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.4. What is the mean time7 to detect a new device (time between scans in 2.2)?
(AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.5. Percent (%) of the organization’s registered network fabric covered by a
Network Access Control switching technology that blocks unauthorized devices.
(Base)
149
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.6. Percent (%) of endpoints from 2.1.2 covered by an automated software asset
inventory capability to scan the current state of installed software (e.g., .bat,
.exe, .dll). (AP)Percent (%) of the organization's network fabric covered by an
automatic capability (scans/device discovery processes) that provides enterprise-
level visibility into the current state of all hardware assets. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.7. Percent (%) of endpoints from 2.1.2 covered by a desired-state software
asset management capability to detect and block unauthorized software from
executing (e.g., AppLocker, certificate, path, hash value, services, and behavior
based whitelisting solutions).8 (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.8. How many major application databases9 does the organization maintain?
(Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
150
2.9. Percent (%) of the organization’s network fabric that undergoes periodic
discovery scanning specifically for the purpose of identifying and enumerating
databases. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.10. Please complete Table 2. Future configurations will be added as needed.
Data calls for layer 2, layer 3, mobile, printers, or other devices or operating
systems will be used as needed.
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.11. Percent (%) of hardware assets listed in 2.1 assessed using credentialed
scans with Security Content Automation Protocol (SCAP) validated
vulnerability tools.10 (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.12. What is the mean time11 between vulnerability scans? (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
151
2.13. Percent (%) of the databases in 2.8 that undergo periodic vulnerability
scanning with a special purpose database vulnerability scanner. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
2.14. What is the mean time12 to mitigate for high13 findings? (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3. IDENTITY CREDENTIAL AND ACCESS MANAGEMENT
3.1. How many users have unprivileged network accounts?15 (Exclude
privileged network accounts and non-user accounts .) (Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3.3. Percent (%) of privileged network users19 that had their privileges reviewed
this year. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
152
3.4. Percent (%) of privileged network users that had their privileges adjusted or
terminated after being reviewed this year. (Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3.5. Percent (%) of the organization’s internal systems20 configured to require
PIV authentication. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3.6. Percent (%) of the organization’s government service portals (e.g., Max.gov
Portal, MyEPP) that enforce PIV authentication for cross-agency federal
customers. (If none are provided, answer N/A.) (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3.7. How many users log onto the organization’s remote access solution(s)21 to
obtain access to the organization’s desktop LAN/WAN resources or services?
(Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
153
3.8 How many users are enabled to remotely log onto the organization's
LAN/WAN resources or services from mobile devices? (Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3.9. Percent (%) of agency’s operational Physical Access Control Systems
(PACS) that comply with procurement requirements for purchasing products and
services from the FIPS 201 Approval Products List maintained by General
Services Administration (GSA) (per OMB M06-18). (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
3.10. Percent (%) of agency’s operational PACS that electronically accept and
authenticate internal users’ PIV credentials for routine access in accordance with
NIST standards and guidelines (e.g., FIPS 201-2 and NIST SP 800-116). (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4. ANTI-PHISHING AND MALWARE DEFENSE
4.1. Percent (%) of privileged user accounts that have a technical control
preventing internet access. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
154
4.2. Percent (%) of incoming email traffic analyzed for clickable URLs,
embedded content, and attachments. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.3. Percent (%) of hardware assets covered by a host-based intrusion prevention
system. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.4. Percent (%) of hardware assets covered by an antivirus (AV) solution using
file reputation services, checking files against Cloud-hosted, continuously
updated malware information. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.5. Percent (%) of email attachments opened in sandboxed environment or
detonation chamber. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
155
4.6. Percent (%) of incoming emails using email sender authentication protocols
such as DomainKeys Identified Mail (DKIM), Author Domain Signing Practices
(ADSP), Domain-based Message Authentication, Reporting & Conformance
(DMARC), Vouch by Reference (VBR), or IP Reverse (iprev). (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
Percent (%) of incoming emails scanned using a reputation filter24 tool to
perform threat assessment of email sender. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.8. Percent (%) of hardware assets covered by an anti-exploitation tool (e.g.,
Microsoft’s Enhanced Mitigation Experience Toolkit (EMET) or similar). (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.9. Percent (%) of inbound email traffic passing through anti-phishing/anti-
spam filtration technology at the outermost border Mail Transport Agent or
email server. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
156
4.10. Percent (%) of inbound network traffic that passes through a web content
filter that provides anti-phishing, anti-malware, and blocking of malicious
websites (e.g., fake software updates, fake antivirus offers, and phishing offers).
(AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.11. Percent (%) of hardware assets that have implemented a browser-based
(e.g., Microsoft Phishing filter) or enterprise-based tool to block known phishing
websites and IP addresses. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.12. Percent (%) of outbound communications traffic checked at the external
boundaries to detect covert exfiltration of information. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.13. Percent (%) of sent email that is digitally signed. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
4.14. Percent (%) of email traffic quarantined or otherwise blocked. (AP)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
157
5. DATA PROTECTION
5.1. What is the estimated number of hardware assets in each of the following
mobile and portable asset types, and how many are encrypted?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
6. NETWORK DEFENSE
What is the estimated percent (%) of remote access connections that have each
of the following properties?
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
7. BOUNDARY PROTECTION
7.1. Percent (%) of the required TIC 2.0 Capabilities implemented. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
158
7.2. Percent (%) of external network traffic to/from the organization’s networks
that passes through a TIC/MTIPS. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
7.3. Percent (%) of external network/application interconnections to/from the
organization’s networks that passes through a TIC/MTIPS. (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
7.4. Percent (%) of public-facing servers28 use IPv6 (e.g., web servers, email
servers, DNS servers, etc.). (Exclude low-impact networks, Cloud servers, and
Internet Service Provider (ISP) resources unless they require IPv6 to perform
their business function.) (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
8. TRAINING AND EDUCATION
8.1. Percent (%) of users that successfully completed31 annual Cybersecurity
Awareness and Training (CSAT). (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
159
8.2. Percent (%) of all users that participated in cybersecurity-focused exercises.
(KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
8.3. Percent (%) of the organization’s network users and other staff32 that have
significant security responsibilities.33 (KFM)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
9. INCIDENT RESPONSE
9.1. Of the information security incidents reported to US-CERT in FY2015,
what was the total number of incidents reported to Congress? (Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
9.2. Of all of the cyber related (electronic) incidents with confirmed loss of
confidentiality, integrity or availability reported to US-CERT in FY15 (per
OMB M-15-01), what was the average meantime (in hours) between detection
and notification to the Agency’s top-level Computer Security Incident Response
Team (CSIRT), Security Operations Center (SOC), or Information Technology
(IT) department? (Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
160
9.3. When will the agency transition to the new US-CERT reporting format?
(Base)
Applications
Public
Management
Customer
Platform
Operational
Both
Infrastructure
Private
Technical
Provider
