WK 6 ASSIGNMENT ************* DRAFT Thesis **********

The ISA-CMM contains thirty-five (35) Information Security Assurance base practices, organized into nine process areas that address the specific performance of an Information Security Assurance Activity. Each process area has a set of goals that represent the expected state of an organization that is successfully performing the process area. An organization that performs the base practices of the process area have to also achieve its goals.

A process area:

· Assembles related base practices in one area for ease of use

· Relates to Information Security Assurance activities (Domain)

· Can be implemented in multiple organization and product contexts

· Can be improved as a distinct process area

· Can be improved by a group with similar interests in the process area

· Includes all base practices that are required to meet the goals of the process area

The nine process areas of the ISA-CMM are listed below. These process areas and the base practices that define them are provided in the next section.

· ISA-PA01 Provide Training

· ISA-PA02 Coordinate with Customer Organization

· ISA-PA03 Specify Initial Information Security Needs

· ISA-PA04 Assess Threat

· ISA-PA05 Assess Vulnerability

· ISA-PA06 Assess Impact

· ISA-PA07 Assess Information Security Risk

· ISA-PA08 Provide Analysis and Results

· ISA-PA09 Manage Information Security Assurance Processes

As shown below, six of the process areas relate to the activities performed to complete an Information Security Assurance Activity, while three of the process areas contain activities required for organizational support for an Information Security Assurance services capability.

Organizational support activities that ensure the assessment organization is prepared and capable to perform Information Security Assurance Activities:

· ISA-PA01 Provide Training

· ISA-PA02 Coordinate with Customer Organization

· ISA-PA09 Manage Information Security Assurance Processes

On-site customer coordination where information criticality and customer concerns are identified:

· ISA-PA03 Specify Initial Information Security Needs

On-site information gathering processes to gather information and perform an analysis of the site’s security:

· ISA-PA04 Assess Threat

· ISA-PA05 Assess Vulnerability

· ISA-PA06 Assess Impact

· ISA-PA07 Assess Information Security Risk

Document Information Security Assurance plan and document final report of findings and recommendations:

· ISA-PA08 Provide Analysis and Results

As Information Security Assurance Activities are dependent on all the Process Areas, none of the ISA-PA in the ISA-CMM can be “tailored out”, and therefore, they will all receive ratings. In other CMMs, the annotation of Not Applicable (NA) is used to identify PA that are not part of the organizational mission objectives.

For example, even if an organization does not do Assess Threat, it must have access to credible threat data from some source in order to perform Assess Risk. If there is no evidence that some form of agreement/oversight is in place to collect appropriate threat data, the organization would receive a zero in Assess Threat.

Generic practices are activities that apply to all processes. They address the management, measurement, and institutionalization aspects of a process. In general, they are used during an appraisal to determine the capability of an organization to perform a process.

Generic practices are grouped into logical areas called “Common Features” that are organized into six “Capability Levels” which represent increasing organizational capability. Unlike the base practices of the domain dimension, the generic practices of the capability dimension are ordered according to maturity. Therefore, generic practices that indicate higher levels of process capability are located at the top of the capability dimension.

The common features are designed to describe major shifts in an organization's characteristic manner of performing work processes (in this case, the security engineering domain). Each common feature has one or more generic practices. The lowest common feature is 1.1 Base Practices are Performed. This common feature simply checks whether an organization performs all the base practices in a process area.

Subsequent common features have generic practices that help to determine how well a project manages and improves each process area as a whole. The generic practices are grouped to emphasize any major shift in an organization's characteristic manner of doing security engineering. Table 21 lists some principles captured in the generic practices.

There is more than one way to group practices into common features and common features into capability levels. The following discussion addresses these common features.

The ordering of the common features stems from the observation that implementation and institutionalization of some practices benefit from the presence of others. This is especially true if practices are well established. Before an organization can define, tailor, and use a process effectively, individual projects have to have some experience managing the performance of that process. Before institutionalizing a specific estimation process for an entire organization, for example, an organization has to first attempt to use the estimation process on a project. However, some aspects of process implementation and institutionalization have to be considered together (not one ordered before the other) since they work together toward enhancing capability.

Common features and capability levels are important both in performing an assessment and improving an organization's process capability. In the case of an assessment where an organization has some, but not all common features implemented at a particular capability level in a particular process area, the organization usually is operating at the lowest completed capability level for that process. For example, an organization that performs less than 90% of the Level 2 generic practices for a process area has to receive a Level 1 rating even if they seem to perform all the generic practices at level 3. Simply put, you can not receive a rating at the current level if all of the previous levels and 90% of the current level common features are not implemented.

In the case of improvement, organizing the practices into capability levels provides a organization with an “improvement road map“. An appraisal has to be performed to determine the capability levels for each of the process areas. This indicates that different process areas can and probably will exist at different levels of capability. The organization will then be able to use this process-specific information as a means to focus improvements to its processes. The priority and sequence of the organization's activities to improve its processes have to take into account its business goals.

Business goals are the primary drivers in interpreting a model such as the ISA-CMM. But, there is a fundamental order of activities and basic principles that drive the logical sequence of typical improvement efforts. This order of activities is expressed in the common features and generic practices of the capability level side of the ISA-CMM architecture.

The ISA-CMM contains six levels, which are depicted in the figure below:

These six levels are:

Level 0, “Not Performed,” indicates that an organization does not meet all of the base practices of a process.

Level 1, “Performed Informally,” focuses on whether an organization or project performs a process that incorporates the base practices. This level can be characterized by the statement, “you have to do it before you can manage it.”

Level 2, “Planned and Tracked,” focuses on project-level definition, planning, and performance issues. This level can be characterized by the statement, “understand what's happening on the project before defining organization-wide processes.” Foundation for measurement is established.

Level 3, “Well Defined,” focuses on disciplined tailoring from defined processes at the organization level. This level can be characterized by the statement, “use the best of what you've learned from your projects to create organization-wide processes.” Measurement parameters are refined.

Level 4, “Quantitatively Controlled,” focuses on measurements being tied to the business goals of the organization. Although it is essential to begin collecting and using basic project measures early, measurement and use of data is not expected organization wide until the higher levels have been achieved. This level can be characterized by the statements, “you can't measure it until you know what ‘it’ is” and “managing with measurement is only meaningful when you're measuring the right things.”

Level 5, “Continuously Improving,” gains leverage from all the management practice improvements seen in the earlier levels, then emphasizes the cultural shifts that will sustain the gains made. This level can be characterized by the statement, “a culture of continuous improvement requires a foundation of sound management practice, defined processes, and measurable goals.”

The common features below represent the attributes of mature security engineering necessary to achieve each level.

Level 0

No Capability

Level 1

1.1 Base Practices are Performed

Level 2

2.1 Planning Performance

2.2 Disciplined Performance

2.3 Verifying Performance

2.4 Tracking Performance

Level 3

3.1 Defining a Standard Process

3.2 Perform the Defined Process

3.3 Coordinate the Process

Level 4

4.1 Establishing Measurable Quality Goals

4.2 Objectively Managing Performance

Level 5

5.1 Improving Organizational Capability

5.2 Improving Process Effectiveness

The ISA-CMM also does not imply specific requirements for performing the generic practices. An organization is generally free to define, manage, measure, control and make use of effective processes in any way or sequence they choose. However, because some higher level generic practices are dependent on lower level generic practices, organizations are encouraged to work on the lower level generic practices before attempting to achieve higher levels.

The general format of the process areas is shown in Figure 31: Process Area Format. The summary description contains a brief overview of the purpose of the ISA-PA. Each ISA-PA is decomposed into a set of base practices (BPs). The ISA-BPs are considered mandatory items (i.e., they must be successfully implemented to accomplish the purpose of the process area they support). Each base practice is described in detail following the PA summary. Goals identify the desired end result of implementing the ISA-PA.

ISA-PA 01 – Process Area Title (in verb-noun form)

Summary Description – An overview of the process area.

Goals – A list indicating the desired results of implementing this Process Area.

Process Area Notes – Any other notes about this process area.

Base Practices List – A list showing the number and name of each Base Practice.

ISA-BP 01.01 – Base Practice Title (in verb-noun form)

Descriptive Name – A sentence describing the base practice.

Description – An overview of this base practice.

Example Work Products – A list of examples illustrating some possible output.

Notes – Any other notes about this base practice.

ISA-BP 01.02...

Summary Description

The purpose of Provide Training is to ensure that Information Security assessors and evaluators have the necessary knowledge and skills to achieve project and organizational objectives. Knowledge encompasses the ability to grasp the technical concepts involved in Information Security Assurance Activities. Skills encompass the ability to execute or effectively identify resources to achieve Information Security Assurance objectives. An effective training program addresses identifying training needs, selecting training methods, ensuring the availability of training, the actual training of personnel, and assessing training effectiveness.

Needed knowledge and skills can be acquired by training within the organization and by timely acquisition of critical resources. Critical resources may include site personnel, training programs, and lab facilities, along with relationships with government and academia to provide training services or the acquisition of qualified temporary hires, new hires, consultants, and subcontractors.

Key staff personnel include a number of Information Security analysts to support the identification, collection, and dissemination of threat, vulnerability, and impact data in addition to management personnel and assessors and evaluators to support the activity. If the organization does not have sufficient staff to adequately support an Information Security Assurance capability, it may need to rely on the skills provided by subject matter experts (SMEs).

If there is a need for SMEs, these subject matter experts would be responsible for ensuring their own training for maintaining proficiency within the area where they are expected to perform. That being said, the organization is still responsible for providing evidence that the SMEs meet the minimum criteria for training (i.e. the organization can defend its understanding of the requirements identified by the ISA-BPs and that the SME meets them.)

Goal

· Ensure Information Security assessors and evaluators and supporting staff have the necessary knowledge and skills to appropriately support and execute the Information Security Assurance Activities.

Process Area Notes

The knowledge and skills required to properly perform the necessary functions include Information Security analysis and technical and non-technical Information Security information. If the organization does not have in-house assessors and evaluators and wishes to receive a rating higher than ‘1’, they will need to provide the training evidence for the assessor and/or evaluator resources (i.e., obtain the evidence from the sub-contracted organization that provides the subject matter experts).

Successful training programs result from an organization’s commitment to continuing education agendas. In addition, they are administered in a manner that optimizes the learning process, are repeatable, assessable, and easily modifiable to meet the ever-changing education needs of the organization. Training is not limited to “classroom” events: it includes many vehicles that support the enhancement of skills and the building of knowledge. When training is not a viable approach due to compressed schedules or lack of availability of training resources, SMEs may be required to address the immediate needs.

Base Practices List

The following list contains the base practices that are essential elements to provide training in Information Security processes:

ISA-BP01.01 Identify Training Needs.

ISA-BP01.02 Select Method of Information Security Training.

ISA-BP01.03 Ensure Availability of Information Security Training.

ISA-BP01.04 Train Personnel.

ISA-BP01.05 Assess Training Effectiveness.

Identify needed knowledge and skills throughout the organization using project requirements, organizational strategic plans, and employee skills and deficiencies as guidance.

Description

This Base Practice requires the organization to identify the needed Information Security knowledge and skills. Needs are determined by reviewing project and organizational plans and mission objectives. Comparison of the needs with current employee skills will identify deficiencies that may be remedied through training or acquisition of knowledge and skills from SMEs.

Example Work Products

· Training needs assessment

· List of Information Security training needs

· Training gap analysis

· Training plans

Notes

None.

Determine the appropriate mechanisms (mode and provider) for delivering Information Security training in order to address the identified training needs.

Description

This Base Practice requires the organization to identify the appropriate mechanisms for providing Information Security knowledge and skills training (e.g. Computer Based Training, hands-on lab, formal instruction, apprenticeship programs). Project and organizational needs are analyzed, and training solutions are selected from alternatives which would include on-the-job training (OJT), apprenticeship, mentoring, and formal education.

Example Work Products

· Training profile

· Individual training/development plans

· Established relationships with academia

· Formal and informal in-house training

Notes

Evidence must show an established organizational structure responsible for selecting the method of training. Although OJT is an acceptable form of training, OJT itself will not meet the intent of this ISA-BP. The organization must demonstrate that the appropriate method of training is provided (see examples above) based on each individual’s needs.

Ensure that appropriate training is available to support an organization’s Information Security capability.

Description

This Base Practice requires that the organization provide an avenue for assessors and evaluators and supporting staff (e.g., management support) to take advantage of the identified training mechanisms to address their specific training needs. The organization must provide the support (e.g., facilities, materials) and resources (e.g., time, funding) needed to: ensure employee awareness of available training; allow employees to fulfill their individual training/development plans; and maintain the training program.

Example Work Products

· Description of processes for ensuring availability of training

· Resources for training (e.g., labs, SMEs, computer based training, training program [e.g., manuals, website])

· Employee indoctrination (e.g., new-hire training, periodic course announcements, individual training/development plans)

Notes

An important factor for ensuring reasonable availability of training is a prioritization process that matches needs determined in ISA-BP01.01 (Identify Training Needs) with resources (e.g. funding, time) and management support to ensure availability.

Ensure that assessors and evaluators have received the appropriate training to support their assigned Information Security roles.

Description

This Base Practice requires the organization to verify that training is being properly conducted using the mechanisms identified in ISA-BP01.02 (Select Method of Information Security Training) to support the Information Security training needs outlined in ISA-BP01.01 (Identify Training Needs) and delivered in ISA-BP01.03 (Ensure Availability of Information Security Training).

Example Work Products

· Verification that training is conducted (e.g., training records, certificates, college transcripts)

· Individual training plans/individual development plans

Notes

While an organization can meet immediate technical needs by hiring / contracting subject matter experts, continuing education is an integral part of maintaining technical proficiency. A repository of baseline training materials and revisions to training materials are examples of supporting evidence that can demonstrate that a process for maintaining course materials is in place.

Records have to be kept for all students who complete each training course or other approved training activity. Also, records of successfully completed training are made available for consideration in the assignment of the staff and managers.

If the organization relies solely on the expertise of contractors it is still incumbent on them to verify that the contractor personnel are maintaining their proficiencies. While it is not likely or practical for the organization to maintain the training records of every contractor, there has to be provisions for reviewing said records at any time. If training is provided to contractors by the organization, a local copy of the training has to be maintained and copies provided to the contractor for inclusion in personnel files.

Ensure that the training uses the identified training methods to effectively address the training needs.

Description

This Base Practice requires the organization to effectively implement the needs identified in ISA-BP01.02 (Select Method of Information Security Training) to address the training needs provided in ISA-BP01.01 (Identify Training Needs) to support the organization’s objectives. A process has to exist to determine the skill level of the employee before and after receiving the training to assess the adequacy of the training. This may be accomplished via formal testing, on-the-job skill assessment, or by assessment mechanisms embedded in the courseware.

Example Work Products

· Individual training plans/individual development plans

· Training feedback reporting

· Training program modifications

· Proficiency evaluations

Notes

All personnel have to be held accountable to this ISA-BP (both internal, SMEs and contractors alike.) This means that an organization that relies on contractors to fulfill a set of capabilities needs to ensure that the contractors meet the minimum training and qualifications applied to their own employees.

The purpose of the Coordinate with Customer Organization Process Area is to ensure that Information Security Assurance providers perform coordination activities with the customer organizations. Various mechanisms may be used to communicate the Information Security Assurance decisions and recommendations with customer organizations (e.g. memoranda, documents, e-mail, meetings, and working groups).

Goals

· All participating members of the assessment organization are aware of Information Security coordination activities to the extent necessary to perform their functions.

· Coordination methods to communicate Information Security decisions to the customer organization are agreed upon.

· Assessment decisions are effectively communicated to the customer organization.

Process Area Notes

The Assessment Plan is the primary mechanism for documenting coordination activities. Coordination activities occur continuously throughout the assessment process.

Base Practices List

ISA-BP02.01 Identify Coordination Mechanisms.

ISA-BP02.02 Facilitate Coordination.

ISA-BP02.03 Coordinate Decisions and Recommendations.

Ensure assessors and evaluators are aware of the Information Security coordination activities with the customer and the mechanisms used to perform them.

Description

This ISA-BP requires that Information Security analysts are aware of and involved with coordination activities. There are many ways that the Information Security decisions and recommendations can be shared between the team and customer. It is not uncommon to have multiple assessors and evaluators working on the same project. In these situations, all assessors and evaluators have to be working toward the commonly understood agreements as determined during assessment planning.

Example Work Products

· Communication plans, which include what information is to be shared, meeting times, processes and procedures to be used between assessment team and customer

· Templates for meeting reports, messages, memoranda, status reports

Notes

The assessment team has to establish how they will be communicating and coordinating with the customer site. Agreements on how they will coordinate have to be documented in one of the following: Assessment Plan, Project Book, or Red Team Book.

Facilitate effective communication between Information Security Assurance team and customer site.

Description

This ISA-BP requires continual use of communication mechanisms for communicating with and providing decisions to the customer.

Example Work Products

· Agreements on coordination/communication mechanisms

· Documented decisions in meeting reports, messages, memoranda, status reports

· Conflict resolution procedures

Notes

Agreements regarding coordination and communication mechanisms between the assessment team and the customer site must be documented, for example in an assessment plan or memoranda. These agreements have to also define the relationship between the team lead and site point of contact. In particular, the team lead is responsible for ensuring that the site is informed about what the team needs for access to provide an accurate assessment of the site. The site point of contact is responsible for ensuring that the team has adequate access to the site and resources for performing the assessment.

Use the identified mechanisms to communicate decisions and recommendations related to the Information Security Assurance Activity.

Description

This ISA-BP requires that the assessment team coordinate with the customer in a form that has been agreed to in ISA-BP02.01 (Identify Coordination Mechanisms). Decisions and recommendations have to be communicated to the customer in a documented form.

Example Work Products

· Assessment Plan

· Project Book

· Red Team Book

Notes

During an assessment, the assessment team could provide informal recommendations to the site that may result in mitigation of vulnerabilities or other issues as the assessment is progressing. In this case, it is important that the team document any of these recommendations in their Final Report so that all results of the assessment are captured.

The purpose of the Specify Initial Information Security Needs Process Area is to determine an assessor and/or evaluator’s ability to identify the Information Security requirements of the system being assessed. This includes identifying information criticality, regulatory documents, and customer concerns and constraints. This information is analyzed to identify the high-level Information Security goals. These goals are then translated into initial security requirements for each system being assessed and become the basis for the Information Security Assurance analysis.

Goal

· A common understanding of initial Information Security needs is established between the Information Security Assurance team and the customer site.

Process Area Notes

This process area addresses how the customers' information is obtained and refined into a coherent baseline of initial Information Security requirements to be used in the analysis of the customers' systems.

Base Practices List

ISA-BP03.01 Understand Criticality of the Customer’s Assets.

ISA-BP03.02 Identify Applicable Constraints.

ISA-BP03.03 Identify Customer's Concerns.

ISA-BP03.04 Capture High-Level Objectives.

ISA-BP03.05 Identify Initial Information Security Needs.

Understand the mission critical assets for the site.

Description

This base practice requires that the assessment team collect all of the information necessary for a comprehensive understanding of the site’s critical mission assets, which include information, systems, personnel and functions. An understanding of the potential impact (i.e., criticality) resulting from a compromise of any or all of these assets is essential.

Example Work Products

· Site Mission Description

· System Information Criticality Analysis

Notes

The Assessment Plan and/or Project Book have to contain the site mission description as well as the organizational and system information criticality analysis.

Identify policies, standards, procedures, laws, and other regulatory influences that are applicable to the organization.

Description

This base practice requires that the assessment team understand the constraints under which the site is held accountable when conducting their activities. Constraints can be imposed from the laws, regulations, policies and commercial standards that govern the organization. A determination of precedence between global and local policies has to be performed. In addition, customer constraints could impose restrictions to the site’s access to resources, personnel, and facilities due to operational needs. The resulting set of constraints along with the customer’s concerns determined in ISA-BP03.03 (Identify Customer’s Concerns) would then govern the activities of the team and determine the scope of the assessment.

Example Work Products

· An analysis of the Information Security policies, regulations, standards, procedures, and laws to identify constraints that applies to the specific systems being assessed.

· An analysis of customer-specific organizational and operational constraints.

Notes

Central to this activity is the understanding of the operational environment. Particular consideration is required when multiple domains are involved. Conflict may occur between laws and regulations that are applicable in different countries and different types of business.

Customer satisfaction will depend on an assessor and/or evaluator’s ability to adequately address the customer’s constraints. If the customer imposes constraints that have a major impact on the outcome of the assessment, this has to be noted in the final report.

Understand the site-specific concerns raised by the customer.

Description

This base practice requires the assessment team to gain an understanding of the customer’s concerns. Customer concerns (e.g. known vulnerabilities, confirmed incidents, threat agents, and upcoming certifications) need to be addressed throughout the remaining Information Security Assurance Activities (e.g., Outbrief, Final Report). The assessor and/or evaluator will reply on their understanding criticality of the customer’s assets defined in ISA-BP03.01 (Understand Criticality of the Customer’s Assets) along with the applicable constraints determined in ISA-BP03.02 (Identify Applicable Constraints) to focus the on-site activities and to determine the scope of the Information Security Assurance Activity.

Example Work Products

· List of Customer Concerns

Notes

In addition to understanding a customer’s constraints, customer satisfaction will depend on an assessor and/or evaluator’s ability to adequately address the customer’s site-specific Information Security concerns. Having an adequate understanding of the concerns will ensure a more accurate assessment of the security of the organization’s systems.

Additional factors may influence the Information Security concerns and constraints of the organization. These factors include the political orientation and changes in political focus, technology developments, economic influences, global events, and Information Warfare activities. As none of these factors are static, they require monitoring and periodic assessment of the potential impact of change.

Capture a high-level understanding of the organization’s Information Security objectives.

Description

This base practice requires the assessment team to analyze the information from ISA-BP03.01 (Understand Criticality of the Customer’s Assets), ISA-BP03.02 (Identify Applicable Constraints), ISA-BP03.03 (Identify Customer's Concerns), and determine (at a high level) the Information Security objectives of the organization. These objectives represent the overall information protection philosophy and are the basis for analyzing vulnerabilities and developing recommendations in ISA-PA08 (Provide Information Security Analysis and Results).

Example Work Products

· List of High-Level Goals

· Security Policy

· Mission Objectives

Notes

While there is no specific section in the Assessment Plan and/or the Project Book that lists these high-level objectives, the objectives have to be addressed throughout the Plan. The most likely source of the organization’s overall protection philosophy is an Information Security Policy document. At a minimum, confidentiality, integrity, and availability of critical information need to be considered when determining organizational objectives.

Determine the initial Information Security needs for the customer’s systems.

Description

This base practice requires that the assessment team analyze the high-level goals identified in ISA-BP03.04 (Capture high-level goals) to establish a set of needs to be addressed by the Information Security Assurance Activity. Failing to meet any of these needs will most likely cause vulnerabilities. The needs will be the basis for establishing requirements for conducting the technical aspects of the Information Security Assurance Activity identified in ISA-PA04 (Assess Threat), ISA-PA05 (Assess Vulnerabilities), ISA-PA06 (Assess Impact), and ISA-PA07 (Assess Information Security Risk).

Example Work Products

· List of Information Security needs

· Analysis of high-level goals

· Scoping Questionnaire

· Rules of Engagement

Notes

While there is no specific section in the Assessment Plan, Project Book, or Red Team Book, that specifically lists the Information Security needs, these needs have to be addressed throughout the plan and final report.

The understanding of the system(s) environment is critical in determining Information Security needs. The Information Security needs of the system are not restricted to the internal system(s). For example, Information Security needs may be addressed by the facility in which the system resides and the personnel operating the system. This enables physical measures to be considered as part of the Information Security needs analysis.

The purpose of the Assess Threat Process Area is to identify applicable security threats. A threat is any agent, circumstance, or event with the potential to cause harm to information or systems.

Goals

· Threats to the organization’s mission, assets (e.g., personnel, information, physical structures), and systems (IT) are identified and characterized.

Process Area Notes

This Process Area addresses the assessor and/or evaluator’s ability to determine applicable potential and known threats to the organization. Many approaches can be used to perform threat analysis. An important consideration for determining which approach to use is how it will interface with other activities.

While the activities involved with gathering threat, vulnerability, and impact are unique and have been grouped into separate Process Areas, they are interdependent when performing risk analysis. Therefore, threat analysis has to be focused, to a certain extent, by the existence of potential corresponding vulnerabilities and impacts.

This process area focuses on the analysis required to identify the applicable threats. The analysis required for proper use of the identified threats will be addressed in ISA-PA07 (Assess Risk).

Since threats are constantly changing, this Process Area must incorporate periodic updates of threat sources to ensure the validity of the threat analysis.

Base Practices List

ISA-BP04.01 Identify Applicable Threats.

ISA-BP04.02 Identify Threat Impact Potential.

ISA-BP04.03 Assess Threat Agent Capability.

ISA-BP04.04 Assess Threat Likelihood.

ISA-BP04.05 Monitor Threats.

Identify threats arising from a natural source and man-made sources (either accidental or deliberate) that have the capability to impact the site operations.

Description

This Base Practice requires that the assessment team develop a list of potential natural threats that are applicable to a particular location and analyze man-made threats that are applicable to a particular organization. Man-made threats may be accidental (e.g., user error) or deliberate (e.g., intrusions). Of critical concern are the threat potential for impacting operations and any subsequent loss of capability.

Example Work Products

· Threat database

· Threat analysis

· List of applicable natural threats

· List of applicable man-made threats

Notes

Threats arising from natural causes include (but are not limited to): earthquakes, tsunami, tornadoes, electrical failure, and floods. Not all threats can occur in all locations. For example it is not reasonably possible for a tsunami to occur in the center of a large continent. Thus it is important to identify which natural threats can occur in a specific location.

Much of the information required for identifying natural threats can be obtained from actuarial lists and natural phenomena occurrence databases. While this information is valuable, it has to be used with caution as it may be highly generalized and therefore may need to be interpreted to address the specific environment in terms of actual potential impact.

There are two types of man-made threats: those that are inadvertent, and those that result from a deliberate act. To understand the applicability of man-made threats, it is important to develop scenarios describing how the threat might potentially impact site operations. Use of generic man-made threat sources have to be reviewed for completeness and relevancy. It is important to remain current with known, as well as postulated tools and techniques that could potentially be used by an adversary.

Identify units of measure and applicable ranges for a threat in a specified environment.

Description

This Base Practice requires that the assessment team identify the potential for a threat to inflict damage to an asset, taking into consideration the environmental protection mechanisms that are in place. Natural threats typically have units of measure associated with them. An example is the Richter scale for earthquakes. In most cases the total range of the unit of measure will not be applicable in a particular location. It is therefore appropriate to establish the maximum, and in some cases the minimum, probability of an event that can occur in the particular location under consideration.

Example Work Products

· Threat table with associated units of measure and ranges based on site location.

Notes

In cases where a unit of measure for a particular threat does not exist, an acceptable unit of measure and acceptable ranges has to be created that are specific to the location.

Assess capability and motivation of threat agent for threats arising from man-made sources.

Description

This process area focuses on the determination of a potential human adversary’s ability and capability of executing a successful attack against the system. Ability addresses the adversary’s knowledge of attacks (e.g., do they have the training / knowledge). Capability is a measure of the likelihood that an able adversary can actually execute the attack (e.g., do they have the resources).

Example Work Products

· Threat agent descriptions – capability assessments and descriptions

Notes

Deliberate man-made threats are to a large extent dependent upon the capability of the threat agent and the resources that the threat agent has at its disposal. Thus a relatively inexperienced hacker who has access to the hacking tools of much more experienced and capable hackers, is a much more dangerous threat, but not as dangerous as the experienced hackers themselves. However, the inexperienced hacker may well cause unintended damage, which the experienced hacker is less likely to do. In addition to the agent capability, an assessment of the resources that the agent has available have to be considered along with their motivation for performing the act which may be affected by the agent’s likely assessment of the attractiveness of the target (asset).

A threat agent may use multiple attacks in sequence or concurrently to achieve the desired goal. The effect of multiple attacks occurring in sequence or concurrently needs to be considered. The development of scenarios can assist in performing this task.

Assess the likelihood that a threat can exploit a given target.

Description

This Base Practice requires that the assessment team determine how likely a threat event is to occur. Many factors need to be considered in making this assessment ranging from the chance occurrence of a natural event to the deliberate or accidental act of an individual.

Example Work Products

· Threat table with associated likelihood of exploitation

Notes

Determining the likelihood of man-made threats depends upon characteristics of the threat agent including the motivation, ability, and resources at the agent’s disposal. The likelihood of a natural threat will depend primarily on site location.

The probability of a successful attack depends upon the ability of the threat agent, motivation (e.g., political, religious), and the resources that the threat agent has at its disposal. Motivation for performing the act may be affected by the agent’s assessment of the value (e.g., military, strategic, or financial importance, psychological impact) of the target. A threat agent may use multiple attacks in sequence or concurrently to achieve the desired goal. The effect of multiple attacks occurring in sequence or concurrently needs to be considered. The development of scenarios can assist in performing this task.

Monitor ongoing changes in the threat capabilities and changes to threat characteristics.

Description

This Base Practice requires that the assessment team monitor ongoing changes to threat capabilities and characteristics. Since new threats can become relevant and the characteristics of existing threats can change, it is important to monitor both existing threats and their characteristics, and to check for new threats on a regular basis.

Example Work Products

· Threat monitoring reports – documents describing the results of the threat monitoring effort

· Threat change reports – documents describing changes in the threat spectrum

· Evidence of changes to the threat list from ISA-BP04.01 (Identify Applicable Threats)

Notes

The monitoring of threats needs to take into account any changes to the system or physical environment. Such changes could potentially have an effect on how new or already known threats could compromise an organization’s assets.

Summary Description

The purpose of the Assess Vulnerability Process Area is to determine the assessment team’s ability to identify and characterize vulnerabilities. Vulnerability refers to a weakness within the system, organization, or procedures that can potentially be exploited in some way that is detrimental to the organization’s systems, assets and/or capability to perform its mission.

Goals

· Vulnerabilities to the organization’s mission, assets, and systems are identified and characterized.

Process Area Notes

While the activities involved with gathering threat, vulnerability, and impact information have been grouped into separate Process Areas, they are interdependent when performing risk analysis. Therefore, the search for vulnerabilities has to be guided to a certain extent by the existence of corresponding threats. However, the absence of a known threat does not imply the absence of vulnerabilities. Potential vulnerabilities can exist that are not commonly known in the public arena.

Since new vulnerabilities are constantly arising, this Process Area must incorporate periodic up-dates of vulnerability sources and analysis techniques.

Base Practices List

ISA-BP05.01 Identify Applicable Vulnerabilities.

ISA-BP05.02 Define Exploitation Potential.

ISA-BP05.03 Determine Overall Vulnerability.

ISA-BP05.04 Monitor Exploitation Potential.

Identify applicable vulnerabilities for a specified operational system.

Description

This base practice requires that the assessment team determine the assessor and/or evaluator's ability to identify applicable vulnerabilities for a specified operational system. This would include vulnerabilities that could potentially put the assets and systems at risk. In this base practice, vulnerabilities are seen as inherent to the system without consideration to the likelihood of any threats.

In order to increase the confidence that all applicable vulnerabilities are identified, the assessment team has to define the method for conducting vulnerability assessments. Defining the method for identifying security vulnerabilities for the system in a way that permits them to be identified and characterized may include a scheme for categorizing and prioritizing the vulnerabilities based on threats and their likelihood, operational functions, security requirements, or other areas of concern when provided.

Example Work Products

· Vulnerability databases

· Vulnerability assessment method

· Vulnerability analysis

· List of applicable vulnerabilities

Notes

At least two fundamentally different approaches exist for the identification of vulnerabilities. These two approaches are characterized as analysis-based approaches and testing-based approaches. Analysis-based approaches are generally more effective for identifying new vulnerabilities. Testing-based approaches are generally more effective for identifying known vulnerabilities that are present and not implementing the required patch or defense mechanism.

The applicability of vulnerabilities must encompass a systems approach. Information Security functions are sometimes supported by non-Information Security mechanisms that may have exploitable vulnerabilities that could adversely impact the system. As such, the search for potential vulnerabilities has to not be strictly limited to Information Security functions and mechanisms.

Analyze the vulnerabilities identified in ISA-BP05.01 (Identify Applicable Vulnerabilities) to determine the susceptibility to attack.

Description

This base practice requires that the assessment team understand the probability of a known threat exploiting a known vulnerability. As such, assessors and evaluators need to understand the underlying vulnerabilities. The susceptibility for exploitation would be based in part on the defense mechanisms, site policies, procedures, and environment.

Example Work Products

· Vulnerability Table with associated threats and exploitation potential

Notes

It is important to note that vulnerabilities can exist without a threat capable of exploitation.

All vulnerabilities have to be monitored to provide for future changes in threat capabilities or the inclusion of new threats.

Analyze combinations of vulnerabilities identified in ISA-BP05.01 (Identify Applicable Vulnerabilities) and analyzed in ISA-BP05.02 (Define Exploitation Potential) in order to determine the combined effect of multiple attacks.

Description

This base practice requires that the assessment team be able to determine the potential for multiple exploitations. Multiple exploitations can be accomplished through a single attack against a set of vulnerabilities (one to many), multiple attacks against a single vulnerability (many to one), or multiple independent attacks (one to one).

Example Work Products

· Vulnerability analysis.

· Overall vulnerability table – a compilation of known threats and susceptible vulnerabilities

· System Vulnerability Criticality Matrix (SVCM)

· Organizational Vulnerability Criticality Matrix (OVCM)

· Information Security Posture Profile

Notes

The assessor and/or evaluator must analyze the interdependencies of the vulnerabilities identified in ISA-BP05.01 (Identify Applicable Vulnerabilities). These interdependencies can create additional vulnerabilities or increase the likelihood of exploitation of known vulnerabilities.

The relationships between threats and vulnerabilities can be categorized in three ways: one-to-many, many–to-one, and one-to-one. The one-to-many relationship implies a single threat can attack multiple vulnerabilities at the same time. The many-to-one relationship implies multiple threats attacking a single vulnerability at the same time. The one-to-one relationship implies one threat attacking a single vulnerability at the same time but independent of any other attack. The overall vulnerability analysis needs to address these three categories of aggregate vulnerabilities.

Monitor ongoing changes in vulnerabilities and changes to their characteristics.

Description

This Base Practice requires that the assessment team monitor ongoing changes to vulnerabilities and their characteristics and understand the potential for exploitation. Since new vulnerabilities can become relevant and the characteristics of existing vulnerabilities can change, it is important to monitor both existing vulnerabilities and their characteristics, and to check for new vulnerabilities on a regular basis.

Example Work Products

· Vulnerability monitoring reports – documents describing the results of the vulnerability monitoring effort

· Vulnerability change reports – documents describing new or changed vulnerabilities

· Evidence of changes to the vulnerability list from ISA-BP05.03 (Determine Overall Vulnerability)

Notes

The monitoring of vulnerabilities needs to take into account any changes to the system or physical environment. Such changes could potentially have an effect on how new or already known vulnerabilities could be exploited in order to compromise an organization’s assets.

The purpose of the Assess Impact process area is to determine an assessor and/or evaluator’s ability to identify the severity and type of impact to organizational mission, assets, and/or systems caused by the exploitation of vulnerabilities. Impacts may be tangible, such as the loss of physical assets or revenue, or intangible, such as loss of reputation or goodwill.

Goals

· Potential impacts to the organization’s mission, assets, and systems are identified and characterized.

Process Area Notes

The organizational capabilities are impacted as a consequence of the exploitation of vulnerabilities, either deliberate or accidental. The result of a successful exploitation would be a breach of confidentiality, loss of integrity, or reduction of availability. The results of the exploitations need to be translated into a quantifiable impact, e.g., increased cost, extended schedule, loss of resources and/or capability.

While the activities involved with gathering threat, vulnerability, and impact information have been grouped into separate Process Areas, they are interdependent when performing risk analysis. The probability of sustaining an impact is directly related to the relationship between known threats and vulnerabilities. The activities that result in the tables in ISA-BP04.05 (Assess Threat Likelihood) and ISA-BP05.03 (Determine Overall Vulnerability) will provide the foundation for determining impact probabilities.

The severity of impacts will play a central role in determining overall risk. Once the overall risk is established, a return on investment decision can be made between accepting the risk and the cost to mitigate.

Base Practices List

ISA-BP06.01 Analyze Capabilities.

ISA-BP06.02 Identify Potential Impacts.

ISA-BP06.03 Monitor Impacts.

Analyze operational, business, and/or mission capabilities.

Description

This base practice requires that the assessment team analyze the organization’s capabilities based on the mission objectives outlined in ISA-BP03.01 (Understand Criticality of the Customer’s Assets). This will provide an understanding of the relative value of the mission capabilities to the organization. Value is determined by the importance of the asset and its intended use in support of organizational capabilities.

Example Work Products

· List of organizational capabilities based on mission objectives.

· System capability profile – describes the capabilities of a system and their importance to the mission of the organization.

Notes

The analyzed capabilities imply a level of criticality of the organization. Criticality can be interpreted as the impact on the system operation, on human lives, on operational cost and other critical factors, when a leveraged function is compromised, modified, or unavailable in the operational environment. Assets may also be defined in relation to their applicable security requirements. For example, assets may be defined as the confidentiality of a client list, the availability of interoffice communication, or the integrity of payroll information. The risk assessment method selected has to address how capabilities and assets are to be valued and prioritized.

All potential critical systems have to be examined to properly scope the assessment effort. Assessors and evaluators and customers have to use a list of systems, along with time constraints, size of the systems, and other considerations to assist in the planning of the assessment effort.

Identify impacts to the organizational mission.

Description

This base practice requires that the assessment team determine the assessor and/or evaluator's ability to describe the type and severity of impacts in terms of their effect on organizational capabilities. Starting with the assets identified in ISA-BP03.01 (Understand Criticality of the Customer’s Assets) and the prioritized capabilities identified in ISA-BP06.01 (Analyze Capabilities), the consequences of a successful exploitation can be determined. For each asset and capability, the consequences may be corruption, disclosure, obstruction, or destruction. The severity of an impact is a measure of its affect on the organizational mission objectives.

Example Work Products

· List of potential impacts and their associated severity

Notes

The assessment plan has to contain an analysis of impacts, to include criticality matrices along with impact definitions and assignments.

Severity will be reported in measures appropriate to the impact (e.g., cost, schedule, performance).

Monitor ongoing changes in the impacts and changes to their severity.

Description

This base practice requires that the assessment team monitor ongoing changes to the impacts. Since new impacts can become relevant and the severity of existing impacts can change, it is important to monitor both existing impacts and their severity, and to check for new impacts on a regular basis.

Example Work Products

· Impact monitoring reports – documents describing the results of the impact monitoring effort

· Impact change reports – documents describing changes in the impacts

· Evidence of changes to the impact list

Notes

The monitoring of impacts needs to take into account any changes to the system or physical environment. Such changes could potentially have an effect on how new or already known impacts could compromise an organization’s assets.

The purpose of the Assess Information Security Risk Process Area is to identify and analyze Information Security risks to an organization based on threats, vulnerabilities, and impacts to its mission, assets, and systems.

This process area focuses on ascertaining risks based on an established understanding of how capabilities and assets are vulnerable to threats. Specifically, this activity involves identifying and assessing the likelihood of realizing an impact.

Goals

· The interdependencies of threats, vulnerabilities, and impacts are identified.

· The security risk associated with operating the system within a defined environment is determined.

· Countermeasures to the identified risks are determined.

Process Area Notes

Information Security risk is the likelihood and the impact that a successful exploitation has on an organization’s mission, assets, or systems. While Information Security risk deals specifically with protection of organizational Information Security capability, the resulting impact could be characterized in terms of cost and schedule impacts to the organization.

The first part of Assess Information Security Risk is the identification of the interdependence of input from threat, vulnerability, and impact process areas in order to form exploitation scenarios. The second part of Assess Information Security Risk is to analyze exploitations with respect to proposed countermeasures.

All potential risks have to be identified whether they will be realized or not. Risks have to be analyzed considering interdependencies between the threat, vulnerability, and impacts, and associated potential for exploitability, using the results of analysis in ISA-BP05.03 (Determine Overall Vulnerability) and ISA-BP06.02 (Identify Potential Impacts).

While activities involved with gathering threat, vulnerability, and impact information, are grouped into separate PAs, they are interdependent. This information forms the basis for the results provided in ISA-PA08 (Provide Analysis and Results).

Since risk environments are subject to change, they must be periodically monitored to ensure that the understanding of risk generated by this PA is maintained at all times.

Base Practices List

ISA-BP07.01 Determine Threat / Vulnerability / Impact Triples.

ISA-BP07.02 Assess Risk Associated with Exploitations.

ISA-BP07.03 Identify Potential Countermeasures.

ISA-BP07.04 Monitor Risks.

Identify threat/vulnerability/impact triples that have potential to adversely affect an organization’s mission, assets, and/or systems.

Description

This Base Practice requires that the assessment team identify exploitations by recognizing the interdependencies between threats, vulnerabilities, and impacts of exploitations.

These exploitations will be inputs to the analysis and selection of recommended countermeasures for the specified system.

Example Work Products

· Table with impacts and associated threats identified in ISA-PA04 (Assess Threat) and vulnerabilities (utilizing vulnerability table from ISA-BP05.03 [Determine Overall Vulnerability] and identified impacts from ISA-BP06.01 [Prioritize Capabilities])

Notes

The components of risk have been identified and analyzed independently in ISA-PA04 (Assess Threat), ISA-PA05 (Assess Vulnerability) and ISA-PA06 (Assess Impact). This PA brings threat, vulnerability, and impact together to determine interdependencies, resulting in identification of exploitations.

Identify and analyze risks.

Description

This ISA-BP requires that the assessment team analyze the exploitations identified in ISA-BP07.01 (Determine Threat /Vulnerability /Impact Triples) to determine interdependencies that would result in a risk to the mission, assets, or system.

Example Work Products

· List of risks (e.g., groups of similar exploitations)

Notes

The likelihood of exploitation is the combination of the likelihood of the threat and the likelihood of a vulnerability being successfully targeted by a specified threat. A risk statement is based on the likelihood of exploitation combined with the potential damage incurred from a realized impact. The output from the analysis performed in this PA is the foundation for an organizational risk management program.

Determine potential countermeasures to identified exploitations.

Description

This ISA-BP requires that the assessment team analyze exploitations identified in ISA-BP 07.01 (Determine Threat /Vulnerability /Impact Triples) and assessed in ISA-BP 07.02 (Assess Risk Associated with Exploitations ) to determine if there are viable countermeasures that could eliminate or reduce the effect of the exploitation. The assessors and evaluators then must identify countermeasures that the organization can implement to mitigate or eliminate the possibility of exploitation. Exploitations can be mitigated by countermeasures that reduce threat capability, close vulnerabilities, and/or minimize an impact.

Example Work Products

· Table of potential countermeasures associated with potential exploitations

Notes

The Final Assessment Report has to contain recommended countermeasures for each identified vulnerability. The assessor and/or evaluator's analysis have to provide alternate countermeasures (when applicable) so the customer can analyze them along with other factors (e.g., cost) when determining proper actions.

Although risk can be reduced by solutions for threat, vulnerability, or impact, the majority of solutions will pertain to the elimination or mitigation of vulnerabilities.

Countermeasures can be categorized in three ways: one-to-many, many–to-one, and one-to-one. The one-to-many relationship implies a single countermeasure can mitigate multiple vulnerabilities at the same time. The many-to-one relationship implies multiple countermeasure can be used to mitigate a single vulnerability (e.g., secure Guards, Firewalls, Routers on same system each help reduce overall vulnerability). The one-to-one relationship implies one countermeasure is used to mitigate a single vulnerability (this is generally the most costly solution). The overall countermeasure analysis needs to look at the cost-benefit tradeoff for each countermeasure based on its ability to effectively reduce the likelihood of a vulnerability being exploited vs. the impact (i.e., cost) to the organization if the exploitation were to occur.

Monitor ongoing changes in the risks and changes to their characteristics.

Description

This Base Practice requires that the assessment team monitor ongoing changes to the risks. Since new risks can become relevant and the characteristics of existing risks can change, it is important to monitor existing risks and their characteristics, and to check for new risks on a regular basis.

Example Work Products

· Risk monitoring reports – documents describing the results of the risk monitoring effort

· Risk change reports – documents describing changes in risks

· Evidence of changes to the risk list from ISA-BP07.02 (Assess Risk Associated With Exploitations).

Notes

The monitoring of risks needs to take into account any changes to the system or physical environment. Such changes could potentially have an effect on how new or already known risk could affect the Information Security capability of an organization.

The purpose of the Provide Analysis and Results PA is to accurately report the findings and recommendations of an Information Security Assurance Activity to the site and interested third parties. This information is identified in ISA-PA03 (Specify Initial Information Security Needs) and includes the analysis performed in ISA-PA04 (Assess Threat), ISA-PA05 (Assess Vulnerability), ISA-PA06 (Assess Impact) and ISA-PA07 (Assess Risk).

Goals

· Any constraints or special emphasis requested by the customer are incorporated into the assessment results.

· A summary of the assessment findings and recommendations is provided to the customer.

Process Area Notes

The assessment team needs to keep in mind that the reports provided to the customer have to provide a foundation for a risk management program. It is good practice for the assessment team’s organization to establish a standard template for the final reports in order to compare assessment results and to easily incorporate modifications that may be needed due to changes in any guiding standards.

Base Practices List

ISA-BP08.01 Address Customer’s Concerns and Constraints.

ISA-BP08.02 Provide Findings and Recommendations.

Respond to concerns and constraints raised by customer in ISA-BP03.02 (Identify Applicable Constraints) and ISA-BP03.03 (Identify Customer’s Concerns).

Description

This base practice requires that the assessment team ensure that any customer concerns, constraints, or other considerations have been identified in the Information Security Assurance plan.

Example Work Products

· Information Security Assessment Plan

· Project Book

· Red Team Book

· Final Outbrief

Notes

The plans must identify any customer concerns and constraints. These may include considerations on the requirements, design, implementation, configuration, and documentation. The final report will demonstrate that the concerns and constraints were incorporated in the assessment analysis.

Convey the assessment analysis and results to the customer.

Description

This ISA-BP requires that the assessment team provide results from the assessment that will support the customer in making informed risk management decisions.

Example Work Products

· Assessment out-brief materials

· Final Report

Notes

The Final Assessment Report is the definitive document for reporting the assessment team’s findings and recommendations, regardless of any other discussions between the team members and the customer site.

Summary description

The purpose of Manage Information Security Assurance Processes is to determine the level of oversight and control applied to the processes associated with the organization’s Information Security Assurance Activities. This includes the overall management structure as well as the management of individual Information Security Assurance Activities. Effective management includes analysis and control of changes to the process. In addition, this process area is applicable to all work products that are generated throughout an Information Security Assurance activity. The specific requirements for each Information Security Assurance address by the ISA-CMM are provided as appendices to this document.

Process Area Notes

Proper Information Security Assurance management processes will allow for the effective and efficient use of Information Assurance resources with respect to customer priorities.

A mature organization will have a set of work products placed under process management. These work products would include, but are not limited to, the Information Security Assessment Methodology (or any other methodology used by the organization), Assessment reports, Assessments Plans, Assessment schedules, and any Analyst tools used by the team.

Base Practices List

The following list contains the base practices that are essential elements of good systems engineering:

ISA-BP09.01 Identify Information Security Assurance Process Management Structure.

ISA-BP09.02 Define the Information Security Assurance Process.

ISA-BP09.03 Maintain Work Product Baselines.

ISA-BP09.04 Manage the Information Security Assurance Program.

Verify the proper management structure is in place to manage Information Security Assurance processes.

Description

This base practice requires that the Information Security Assurance team verify the presence of a dedicated management infrastructure to support Information Security Assurance Activities. Dedicated support implies evidence that personnel, time, money, and resources are made available in support of Information Security Assurance Activities.

example Work Products

· Identified organizational management structure

Notes

Analyzing the organizational chain of command and lines of authority is one way to help make this determination. Some examples of the presence of a mature management structure would be an organization chart defining roles and responsibilities for management and team members.

Verify the presence of a set of activities used to conduct Information Security Assurance Activities.

Description

This base practice requires that the Information Security Assurance team understand the set of steps that the organization uses to conduct Information Security Assurance Activities. At a minimum, one would expect three general phases to be addressed. These three phases are the pre-activity, onsite activity, and report generation phases.

example Work Products

· Information Security Assessment process description

· Information Security Evaluation process description

Notes

The pre-activity phase would include logistics support, general site awareness, and preliminary system and organization descriptions. The onsite phase would include the implementation of the specified Information Security Assurance methodology, and addresses the technical aspects of conducting Information Security Assurance Activities. The report generation phase would include production and delivery of the final report.

Verify the collection of information used to support the Information Security Assurance program.

Description

This Base Practice requires that the assessment organization provide evidence to support the presence of an Information Security Assurance program. This evidence has to address the overall Information Security Assurance program and the technical activities conducted by the Information Security Assurance team.

example Work Products

· A set of templates, process descriptions, and program management documents

Notes

It is important to understand that the baseline may change over time. Therefore, some evidence of configuration control has to be present.

Verify that the organization is taking an active role in managing the Information Security Assurance Activities.

Description

This Base Practice requires that the assessment organization provide evidence that the proper decisions are being made to support the Information Security Assurance program.

example Work Products

· Program management documents

Notes

This chapter contains the Generic Practices (GPs) that are applied to the Process Areas (PAs) in order to determine the capability of the organization. The GPs are grouped according to Common Features and Capability Levels. In order to achieve a rating, the organization must perform at least 90% of the Common Features within a Capability Level and 100% of the Common Features in the previous levels.

The following is the list of Capability Levels, each of which may have Common Features and GPs associated with them:

· Capability Level 0 - Not Performed

· Capability Level 2 - Planned and Tracked

· Capability Level 3 - Well Defined

· Capability Level 4 - Quantitatively Controlled

· Capability Level 5 - Continuously Improving

Notes

It is important to remember that GPs measure how well the ISA-BPs are conducted throughout the organization. The higher the GP, the more standardized a process is implemented. Meeting higher GPs also implies a greater awareness and enforcement of the process activities across the entire organization.

While the various Process Areas are loosely related, there are no mandated minimum ratings required in one ISA-PA in order to achieve a rating at or above in another ISA-PA. That being said, there are tight binding relationships within the GPs. For example if you do not perform GP 2.1.1 – Allocate Resources you can not do GP 2.1.2 – Assign Responsibilities because you have no pool to draw from. A complete relational description is provided in Appendix B.

The ISA-CMM can not be tailored. As such all the GPs will be applied to each PA and a rating will be provided for each Process Area.

Summary Description

The purpose of the Not Performed Capability Level is to identify organizations that fail to support all of the Base Practices within a Process Area. Support implies that someone (i.e. ANYONE) in the organization has knowledge of and performs the activity.

Notes

None.

The purpose of the Performed Informally Capability Level is to identify that activities are being implemented at a minimum acceptable level in support of a Process Area. All the base practices of the process must be performed. The implementation of these base practices may not be rigorously planned and tracked (i.e., they are being done ad hoc). An ad hoc organization does not imply lack of performance; rather that performance relies heavily on the experience of individuals because there is not a documented process.

Common Features List

This capability level comprises the following common features:

· Common Feature 1.1 – Base Practices Are Performed

Notes

Ad Hoc organizations can be 100% effective in doing their mission. The risk lies in the potential for a total loss of capability based on the loss of one subject matter expert who carries all the knowledge of a specific activity in his/her head. In the event that the organizations were to lose such an individual, there would be no way to transfer the knowledge to his/her replacement, thereby putting the entire activity at risk.

This Common Feature requires that the organization provide evidence that all the Base Practices of a Process Area are being performed in some manner. However, the consistency of performance and the quality of the work products produced are likely to be highly variable due to the ad hoc nature of the controls that are in place.

This common feature comprises the following generic practices:

· GP 1.1.1 – Perform the Process

This GP requires that the assessment organization provide evidence that supports the claim that all ISA-BPs of a PA are being implemented.

Example Work Products

· Individual knowledgeable in the ISA-BPs

· Services and products that meet a customer demand

Notes

This process is an informal process. The customer(s) of the process area may be internal or external to the organization. The presence of deliverables (i.e., products and services) can be used as evidence that a process exists. If there is one person in the organization that can do this, it will count as sufficient. This is why analysis of individuals performing the tasks in ad hoc organizations is so important.

Relationships

This Generic Practice provides the base for GP 2.1.3 (Document The Process).

The primary distinction from the Performed Informally Capability Level is that the performance of the process is planned and managed via some form of documented process. The purpose of the Planned and Tracked Capability Level is to establish the presence of documented (formal or informal) and maintained processes at the project, team or activity level. Performance according to specified procedures is verified. Work products conform to specified standards and requirements. Measurement is used to track process area performance, thus enabling the project, team or activity to manage its actions based on actual performance.

Common Features List

This capability level comprises the following common features:

· Common Feature 2.1 – Planning Performance

· Common Feature 2.2 – Disciplined Performance

· Common Feature 2.3 – Verifying Performance

· Common Feature 2.4 – Tracking Performance

Notes

At level 2, we rely on the knowledge provided by a team of individuals instead of just one person. This implies a stronger likelihood that the organization will not lose its entire capability if one person were to leave. At level 2, the various teams have documented processes for their internal use, but have not provided this knowledge or process for implementation throughout the organization.

This Common Feature requires that the project or activity provide evidence that a documented (formal or informal) process exists. The purpose is to establish a baseline capability that is used within a provider organization. This does not imply a standardized process across the organization, but the presence of organized processes within specific groups of individuals (i.e. Assessment team, Network team, Threat analysis team).

This common feature comprises the following generic practices:

· GP 2.1.1 – Allocate Resources

· GP 2.1.2 – Assign Responsibilities

· GP 2.1.3 – Document the Process

· GP 2.1.4 – Provide Tools

· GP 2.1.5 – Ensure Training

· GP 2.1.6 – Plan the Process

This generic practice requires that the project or activity allocate resources (including time, tools, and people) for performing the process area.

Example Work Products

· Organization chart

· Roles and responsibilities

· Property accountability

Notes

The effective allocation of resources requires an organization to provide a responsible entity for allocation functions (e.g. requirements analysis, needs identification, levels of effort, qualifications). At this level, the responsible entity may be defined on a project-by-project basis, but as the organization matures, one would expect to see a centralized allocation method/authority.

Relationships

This Generic Practice provides the base for GP 2.1.2 (Assign Responsibilities), GP 2.1.4 (Provide Tools), and GP 2.1.5 (Ensure Training).

This generic practice requires that the project or activity assign responsibilities for developing the work products and/or providing the services of the process area.

Example Work Products

· Spreadsheet that maps organization structures to roles and responsibilities identified in GP 2.1.1 (Allocate Resources)

Notes

This activity generally falls to the senior member of the team for the execution of a specific task. It is not until higher levels of maturity that clear lines of responsibility (e.g. PM, Task Leaders) and authority are established at the organization level and tasking and process execution are enforced from the top down.

Relationships

GP 2.1.3 – Document the Process

This generic practice requires that the project or activity document the approach to performing the process area in standards and/or procedures.

Example Work Products

· Standards and procedures

· Vulnerability assessment guides

· Risk mitigation plans

Notes

The participation of the process owners (i.e. the people that use the process from day to day) is essential for creating a usable process. It is important to note that the documentation at this level need only apply at the task or project level and may not apply across the entire organization.

Relationships

GP 2.1.4 – Provide Tools

This generic practice requires that the project or activity provide appropriate tools to support performance of the process area.

Example Work Products

· Process management tool-set

· Information Security Assurance technical tool-set

· Appropriate hardware

· Communications systems

Notes

Process-related tools can range from formal process tracking tools developed in house to hardware, COTS scheduling software, e-mail, databases, best-practice matrices or status meetings. In addition to this set of tools, the organization need to provide the set of technical tools (e.g. Network scanners, risk tools, vulnerability identification and assessment tools).

The selection of tools has to support the day to day activities at the task or project level. As the organizational process maturity gets better, the tools will also become more formal in nature (e.g. Project Management tools, data dictionaries that drive the DB activities).

Relationships

The tools identified in this generic practice have to support the processes identified in GP 1.1.1 (Perform the Process), captured in GP 2.1.3 (Document the Process) and will help implement the plans and procedures identified in GP 2.2.1 (Use Plans, Standards and Procedures).

This generic practice requires that the project or activity ensure that the individuals performing the process areas are appropriately trained in how to perform the process.

Example Work Products

· PA-specific training objectives

· Evidence of training (e.g., training records)

Notes

ISA-PA01 (Provide Training) measures whether a training program exists. This GP determines whether the specific training for a given PA is adequate to support the activities defined within the PA. For example, it is possible to have a level 5 training program (e.g. ISA-PA01 = Level 5) and still not train in the proper areas (e.g., GP 2.1.5 is not implemented for ISA-PA04 (Assess Threat).

Relationships

Training outlined in this Generic Practice have to support the processes identified in GP 1.1.1 (Perform the Process), captured in GP 2.1.3 (Document the Process).

This generic practice requires that the project or activity provide an execution plan in support of the documented process.

Example Work Products

· Project plans

· Schedules

· Milestones

Notes

This generic practice does not imply execution of the plan. It is the strategy (e.g., schedule, milestones) for implementing the process using the documented plan, resources, and tools identified in the previous generic practices.

Relationships

This Generic Practice uses the output from GP 2.1.3 (Document the Process) to develop accurate plans, schedules and milestones. This Generic Practice provides the base for GP 2.2.1 (Use Plans, Standards, and Procedures).

This Common Feature requires that the project or activity provide evidence that a baseline set of processes are being implemented appropriately within the individual assessment activity to achieve Level 2.

Generic Practices List

This common feature comprises the following generic practices:

· GP 2.2.1 – Use Plans, Standards, and Procedures

· GP 2.2.2 – Work Product Management and Control

This generic practice requires that the project or activity provide evidence that supports the use of documented plans, standards, guidelines, and/or procedures in implementing the process area.

Example Work Products

· Presence of products and services delivered as specified by the process

· Informal process controls (e.g., email, meeting minutes, spreadsheets)

Notes

This set of plans, standards and procedures provides consistency in the products produced at the project or activity level, not the organizational level. While there are documented plans, standards, and procedures, they are not standardized across the organization. The assessment team has to expect to see documentation at the project and activity level.

Relationships

The standards and procedures used here were documented in GP 2.1.3 (Document the Process), and the plans used were documented in GP 2.1.6 (Plan the Process) evolve to GP 3.1.1 (Standardize the Process).

This generic practice requires that the project or activity provide evidence that the work products of the process area are placed under version control and appropriately managed.

Example Work Products

· Informal version control methodology

· Evidence of a review and release process

Notes

The appropriate management of the work products implies that various documents and procedures are maintained and disseminated throughout the project or activity to ensure a common approach to the task. In addition, this activity is designed to keep the set of plans, standards and procedures used by the project or activity current.

Relationships

Where process area ISA-PA09 (Manage Information Security Assurance Processes) focuses on the general practices of the Information Security process, this generic practice is focused on the management (e.g., version control) of the various work products within a specific process area.

This Generic Practice uses the output from GP 2.2.1 (Use Plans, Standards and Procedures) to identify configuration items and develop a change management system. This generic practice lays the foundation for GP 2.3.2 (Audit Work Products).

This Common Feature requires that the project or activity provide evidence that they are implementing their plans and procedures as directed.

This common feature comprises the following generic practices:

· GP 2.3.1 – Verify Process Compliance

· GP 2.3.2 – Audit Work Products

This generic practice requires that the project or activity lead provide evidence of oversight to ensure that processes are performed as directed.

Example Work Products

· Evidence of milestones or tasks defined and tracked (e.g., status, action tracking)

· Evidence of directed change to original schedule and milestones

Notes

This generic process focuses on the management's ability to ensure that the procedures are followed as specified for the project or activity. This can be done informally (e.g., through e-mail, kickoff meeting minutes) or formally (e.g., project plans). Informal controls imply that the management reviews actions and makes decisions based on what they feel is good enough. Formal controls imply that a plan is laid out with deliverables and milestones. The managers then track against a known set of critical events to determine if the project or activity is compliant with the processes. All of this is still done on a project or activity basis vice an organizational perspective.

Relationships

The applicable standards and procedures were documented in GP 2.1.3 (Document the Process) and GP 2.1.6 (Plan the Process), and used in GP 2.2.1 (Use Plans Standards and Procedures). This generic practice lays the foundation GP 3.2.2 (Perform Defect Reviews).

This generic practice requires that work products are verified for compliance with applicable standards and/or customer requirements.

Example Work Products

· Change logs

· Internal review/release process

Notes

Work products are those documents that support the activities of and are the resultant output of an assessment (i.e. The Assessment report). At Level 2, this practice is met by project lead and/or customer review and feedback.

Relationships

The applicable standards and procedures were documented in GP 2.1.3 (Document the Process) and used in GP 2.2.1 (Use Plans Standards and Procedures). This GP leverages from Generic Practice GP 2.2.2 (Work Product Management and Control) which ensures that the product or service undergoing review is the correct version and has complied with the appropriate configuration management process. This generic practice lays the foundation GP 2.4.1 (Track with Measurement).

This Common Feature requires that the project or activity provide evidence that they are gathering process related measurements.

Generic Practices List

This common feature comprises the following generic practices:

· GP 2.4.1 – Track with Measurement

· GP 2.4.2 – Take Corrective Action

This generic practice requires that the project or activity track actions against schedules and milestones and measure deviations.

Example Work Products

· Process for tracking

· Status reports

· Meeting minutes

· Action item tracking methodology

Notes

Tracking activities involve maintaining a record of activity (e.g., status reports, meeting minutes). Measurement involves identifying deviations from the original plan and procedures. Effective tracking and measurement cannot be done without some foresight or planning that specifies target objectives. Both a description of the measures and the evidence that the measures are used are needed for compliance with this generic practice.

Relationships

The use of measurement implies that the measures have been defined and selected in GP 2.1.3 (Document the Process) and GP 2.1.6 (Plan the Process), data has been collected in GP 2.2.1 (Use Plans Standards and Procedures) and the work products are valid (GP 2.3.2 – Audit Work Products) and can offer effective points of measure. This generic practice lays the foundation for activities conducted in GP 2.4.2 (Take Corrective Action) and evolves to GP 3.2.1 (Use Well-Defined Data).

This generic practice requires that the project or activity identify thresholds for taking corrective action when progress varies significantly from that planned.

Progress may vary from documented or past performance because estimates were inaccurate, performance was affected by external factors, or the requirements on which the plan was based have changed. Corrective action may involve changing the process, changing the plan, or both. The corrective action is the result of an unexpected occurrence during an assessment activity that the team was not prepared to address. These actions are documented and reported for use in refining future assessment activities.

Example Work Products

· Documented process for corrective action

· Examples of acceptable limits to deviation (thresholds)

· Results showing corrective action taken

Notes

An example of a threshold would be: You expect to produce 5 reports and are concerned if you produce less than 4 in the given time frame. If you produce 3, there have to be an expected reaction (i.e. have to work double time to catch up or hire an expert for the next period to cover the backlog).

Relationships

This GP assimilates the data collected in GP 2.4.1 (Track with Measurement) to determine what (if any) action needs to be taken to refine the current process.

The primary distinction from the Planned and Tracked Capability Level is that the process is planned and managed using an organization-wide standard process as opposed to the individual project, team or assessment activity level. The purpose of the Well Defined Capability Level is to establish the presence of Standardized (i.e. formally documented and maintained) processes. Base practices are performed using well-defined data, approved and tailored versions of a standard, documented process and process work products and capability are communicated effectively.

This capability level comprises the following common features:

· Common Feature 3.1 – Defining a Standard Process

· Common Feature 3.2 – Perform the Defined Process

· Common Feature 3.3 – Coordinate Practices

Notes

At level 3, we should see standardized processes being applied across the organization. This implies a level or rigor to ensure consistency of execution across all tasks. A “defined process” will typically be tailored from the organization’s standard process definition. A well-defined process is one in which needs are identified and addressed through policies, standards, inputs, entry criteria, activities, procedures, specified roles, measurements, validation, templates, outputs, and exit criteria that are documented, consistent, and complete. A well-defined process uses standardized reporting and collection techniques to gather data on performance and quality.

This Common Feature requires that the organization provide evidence of the institutionalization of a standard process. The origin of institutionalized processes may be one or more similar processes used successfully on a specific project or activity. A key element of a standardized process at this level is formal documentation. An organization’s standard process is likely to need tailoring to specific situations. Therefore, documentation of a standard process for the organization and a method for tailoring the standard process to specific uses is addressed.

This common feature comprises the following generic practices:

· GP 3.1.1 – Standardize the Process

· GP 3.1.2 – Tailor the Standard Process

This generic practice requires that the organization document a standard process or family of processes for the organization that provides a formal description (i.e., documented) of how to implement the base practices of the process area.

Example Work Products

· A set of formal process documents that are applied across the entire organization and tailored to suit the specific needs of a project or activity.

Notes

The critical distinctions between the Level 2 Generic Practices and Level 3 Generic Practices are the scope of application of the policies, standards, and procedures and the level of rigor applied to the documentation of the process. At Level 2, the project or activity lead has the discretion for establishing standards and procedures for the activity. These standards and procedures may only be used exclusively in their specific area. At Level 3, policies, standards, and procedures are being established at an organizational level for common use.

Relationships

The Level 2 process description was provided in GP 2.1.3 (Document the Process) and implemented in GP 2.2.1 (Use Plans, Standards, and Procedures). This generic practice establishes the Level 3 process description that is tailored in GP 3.1.2 (Tailor the Standard Process).

This generic practice requires that the organization provide evidence (in documented form) that they are modifying the set of formal process documents to address the specific needs of each project or activity.

Example Work Products

· Documented process for tailoring

· Verification of the use of standardized processes (a documented process)

· Verification of tailoring where applicable (proof of specific use)

Notes

There is no requirement to tailor if the standardized process can effectively meet the needs of the project or activity. The tailoring activity could include addendums to the standardized process documents and/or modifications directly to the standardized document baseline.

Relationships

The organization's standard process is documented in GP 3.1.1 (Standardize the Process). The tailored process definition is used in GP 5.1.2 (Continuously Improve the Standard Process).

This Common Feature requires that the organization provide evidence of a repeatable well-defined process. Thus the use of the institutionalized process, review of the results of the process, and work products for defects, and development of well-defined data for assessing performance and results are essential to meeting the intent of this feature.

This common feature comprises the following generic practices:

· GP 3.2.1 – Use a Well-Defined Process

· GP 3.2.2 – Perform Defect Reviews

· GP 3.2.3 – Use Well-Defined Data

This generic practice requires that the organization provide evidence that the standardized process is being implemented in a manner that addresses the specific needs of the project or activity.

Example Work Products

· Project process descriptions that includes policies, standards, inputs, entry criteria, activities, procedures, specified roles, measurements, validation, templates, outputs, and exit criteria

Notes

A well defined process uses standardized reporting and collection techniques to gather data on performance and quality.

Relationships

GP 3.2.2 – Perform Defect Reviews

This generic practice requires that the organization provide evidence that standardized quality assurance is performed against the work products produced.

Example Work Products

· Documented work product review process

· Organizational change management system

· Organizational actions tracking database

· Revised work product standards, templates, and outlines

Notes

In order to have effective defect reviews, there have to be a central point of control for the review and release of information.

Relationships

This Generic Practice uses the output from GP 2.3.1 (Verify Process Compliance) to identify potential concerns of the process that need to be addressed. This generic practice lays the foundation for GP 5.2.1 (Perform Causal Analysis).

This generic process requires that the organization provide evidence that the data associated with the process and work products is verified and validated throughout the activity.

Example Work Products

· Documented process for using well-defined data

· Database Management System

· Data item descriptions/data dictionary

· Update and modification process

Notes

Process and data verification implies that the appropriate data is being used to support an activity. Validation implies that the data is relevant to the mission. Data at Level 3 can be qualitative or quantitative in nature as long as it is defined and applied across the entire organization. In order to transition to Level 4, all data must be quantitative.

Relationships

This Generic Practice uses GP 3.2.1 (Use a Well-Defined Process) to identify critical data elements that need to be refined. This generic practice lays the foundation for GP 4.1.1 (Establish Quality Goals).

This Common Feature requires that the organization provide evidence that the coordination of activities is done throughout the projects and the organization. Various groups throughout the projects and organization perform significant activities. A lack of coordination can cause delays or incompatible results. Thus the coordination of intra-group, inter-group, and external activities must be appropriately addressed. These generic practices form an essential foundation to having the ability to quantitatively control processes.

This common feature comprises the following generic practices:

· GP 3.3.1 – Perform Intra-Group Coordination

· GP 3.3.2 – Perform Inter-Group Coordination

· GP 3.3.3 – Perform External Coordination

A group can be an assessment team, red team, appraisal team, quality assurance team, risk configuration management team, IT team or any other subset within the organization.

GP 3.3.1 – Perform Intra-Group Coordination

This GP requires that the organization provide evidence of effective communication at the team level.

Example Work Products

· Project/activity communications (e.g., e-mail, meeting minutes, schedules with tasks/milestones)

Notes

This type of coordination addresses the need for the individuals within a specific project or activity to share their experiences with one another to mitigate potential failures.

Relationships

This Generic Practice relies on GP 2.1.2 (Assign Responsibilities) as the activity to establish a clear organizational structure in which to operate.

This GP requires that the organization provide evidence of effective communication among the individual assessment teams or other teams as appropriate (e.g. QA, CM) within the organization.

Example Work Products

· Organizational agreements (e.g., MOUs, SLAs, or informal such as e-mail)

· Lessons learned

· Quality Assurance activities

· Configuration Management Activities

Notes

Relationships between the individual assessment teams are established by management to ensure a common understanding of the commitments, expectations, and responsibilities of each team within the organization. These understandings are documented and agreed upon throughout the organization and address the interaction among various teams. Issues are tracked and resolved among all the affected teams.

Relationships

This Generic Practice relies on GP 2.1.2 (Assign Responsibilities) as the activity to establish a clear organizational structure in which to operate.

This GP requires that the organization provide evidence of effective communication with people/organizations outside of the assessment organization.

Example Work Products

· Organizational agreements (e.g., formal MOUs, SLAs)

· Statements of Work

· Proposals

Notes

This type of coordination addresses the needs of external entities that request or require assessment results (e.g., customers, research organizations, sponsor organizations).

A relationship between external groups is established via a common understanding of the commitments, expectations, and responsibilities of each assessment team within an organization.

Relationships

This Generic Practice relies on GP 2.1.2 (Assign Responsibilities) as the activity to establish a clear organizational structure in which to operate.

Summary Description

The primary distinction from the Well-Defined level is that the defined process is quantitatively understood and controlled. The purpose of the Quantitatively Controlled Capability Level is to define detailed measures of performance and establish procedures to ensure they are collected and analyzed. This leads to a quantitative understanding of process capability and an improved ability to predict performance. Performance is objectively managed, and the quality of work products is quantitatively known.

This capability level comprises the following common features:

· Common Feature 4.1 – Establishing Measurable Quality Goals

· Common Feature 4.2 – Objectively Managing Performance

Notes

While the time, money and effort expended to transition for levels 0-3 is relatively linear in nature, the level of effort to achieve level 4 is an order of magnitude more costly. All aspects of process and performance metrics used to achieve level 4 ratings are quantitative in nature!

This Common Feature requires the organization to provide evidence of established, measurable targets (i.e., quality goals) for the work products resulting from the organization’s processes. These generic practices form an essential foundation to objectively managing the performance of a process.

This common feature comprises the following generic practices:

· GP 4.1.1 – Establish Quality Goals

This GP requires that the organization provide evidence that measurable (i.e., quantitative) quality goals for the work products of the organization's standard processes are identified.

Example Work Products

· Quantitative quality goals

Notes

These quality goals can be tied to the strategic quality goals of the organization, the particular needs and priorities of the customer, or to the tactical needs of the project. The measures referred to here go beyond the traditional end-product measures. They are intended to imply sufficient understanding of the processes being used to enable intermediate goals for work product quality to be set and used and determine whether quality has been achieved.

An example of a goal is: “To be #1”

An example of a quality goal is: “To be #1 by achieving a 95% pass rate in…..”

Relationships

This Generic Practice relies on GP 3.2.3 (Use Well-Defined Processes) as input for developing and measuring against a set of quality goals. All data collected in GP 3.2.3 (Use Well-Defined Data) must be converted to quantitative data to be accepted at this level. This generic practice lays the foundation for GP 4.2.1 (Determine Process Capability).

Generic Practices List

This common feature comprises the following generic practices:

· GP 4.2.1 – Determine Process Capability

· GP 4.2.2 – Use Process Capability

This GP requires that the organization provide evidence that measurement against quality goals is done to determine process effectiveness (i.e., process capability).

Example Work Products

· Quality goals gap analysis

· Feedback studies

· Progress against goals

· Metrics

Notes

Measurements inherent in the process definition are collected as the process is being performed. Quantitative metrics are required at this level to demonstrate process capability (e.g., achieving customer satisfaction of 80% when the goal is 90%).

Relationships

This Generic Practice uses the output from GP 4.1.1 (Establish Quality Goals) to measure how closely the organization outputs match the defined goals. This generic practice lays the foundation for GP 4.2.2 (Use Process Capability).

This GP requires that the organization provide evidence of corrective action as appropriate when the process is not performing within its process capability.

Example Work Products

· Lessons learned

· Evidence of corrective action taken to improve processes

Notes

Special causes of variation are used to understand when to use corrective action and what kind of corrective action is appropriate so that future projects or activities are not adversely impacted. The corrective action being taken here is intended to aid future programs. The key here is that the organization has the ability to accurately capture the cause of a process failure within the current program and feed the information back to prevent future failures.

Relationships

Once GP 4.2.1 (Determine Process Capability) has been established, the organization can react to problems in the process set. This practice is an evolution of GP 3.2.3 (Use Well-Defined Data), with the addition of quantitative process capability to the defined process. This generic practice lays the foundation for GP 5.1.1 (Establish Process Effectiveness Goals).

The primary distinction from the Quantitatively Controlled level is that the defined process and the standard process undergo continuous refinement and improvement, based on a quantitative understanding of the impact of changes to these processes. The purpose of the Continuously Improving Capability Level is to use quantitative performance goals (targets) for process effectiveness and efficiency based on the business goals of the organization. Continuous process improvement against these goals is enabled by quantitative feedback from performing the defined processes and from piloting innovative ideas and technologies.

This capability level comprises the following common features:

· Common Feature 5.1 – Improving Organizational Capability

· Common Feature 5.2 – Improving Process Effectiveness

Notes

Accomplishing level 5 implies that you have fully grasped the quantitative controls and metrics at level 4 and are able to apply them in REAL TIME! At level 4 you can use all your quantitative inputs to make corrections for the next project. At level 5 you use the same inputs to correct the current project in such a manner as to not miss the initial requirements within the acceptable thresholds of cost you forecast!

This Common Feature requires the organization to provide evidence of standard processes throughout the organization and making quantitative comparisons between their different uses. As a process is used, opportunities are sought for enhancing the standard process and defects are analyzed to identify potential enhancements to the standard process. Thus goals for process effectiveness are established and improvements to the standard process are identified and analyzed for potential changes to the standard process. These generic practices form an essential foundation to improving process effectiveness.

This common feature comprises the following generic practices:

· GP 5.1.1 – Establish Process Effectiveness Goals

· GP 5.1.2 – Continuously Improve the Standard Process

This generic practice requires that the organization establish quantitative goals for improving the effectiveness of the standard process based on the business goals of the organization and the current process capability.

Example Work Products

· Quantitative goals indicating process effectiveness (e.g., ROI due to process improvement)

Notes

Where goals established at Level 4 address product quality, this GP addresses goals with respect to business objectives, for example increased profits due to process improvement activities. These goals lay the foundation for establishing thresholds for determining whether to continue specific process improvement activities, for example use of the quality goals set in Level 4. It is possible that a quality goal, once implemented could be counter to business objectives.

Relationships

The outputs from GP 4.2.2 (Use Process Capability) are leveraged to determine business relevance in order to set process effectiveness goals.

GP 5.1.2 – Continuously Improve the Standard Process

Description

This generic practice requires that the organization provide evidence that the process is continuously improved by changing the organization's standard process to increase its effectiveness.

Example Work Products

· Versions of the standard process

· Change control process

Notes

The information learned from managing individual projects is communicated back to the organization for analysis and deployment to other applicable areas. Changes to the organization's standard process may come from innovations in technology or incremental improvements. Innovative improvements will usually be externally driven by new technologies. Incremental improvements will usually be internally driven by improvements made in tailoring of the defined process. Improving the standard process attacks common causes of variation.

Relationships

This Generic Practice uses GP 3.1.2 (Tailor the Standard Process) to identify critical data elements that need to be refined.

This Common Feature requires the organization to provide evidence of making the standard process one that is in a continual state of controlled improvement. The significant difference between this level and a Level Four activity is that the correction is done in real time.

This common feature comprises the following generic practices:

· GP 5.2.1 – Perform Causal Analysis

· GP 5.2.2 – Eliminate Defect Causes

· GP 5.2.3 – Continuously Improve the Defined Process

This generic practice requires that the organization perform real-time analysis to determine common failures in the standard process.

Example Work Products

· Method for causal analysis

· Causal analysis results

Notes

Causal analysis of the process identifies rudimentary problems that prevent a process from realizing its goals. Such problems, if not corrected, will be propagated through all instantiations of the process.

Relationships

This Generic Practice uses the output GP 3.2.2 (Perform Defect Reviews) to identify the potential source (i.e. cause) of the defect. Results of this analysis are used in GP 5.2.2 (Eliminate Defect Causes)

This generic practice requires that the organization take steps to eliminate the causes of defects in the defined process.

Example Work Products

· Mitigation strategies

· List of mitigation steps

· Revised process

Notes

The organization must take action to mitigate or eliminate the causes of defects in the process. Both common causes and special causes of variation are implied in this generic practice, and each type of defect may result in different action.

Relationships

Causes were identified in GP 5.2.1 (Perform Causal Analysis).

This generic practice requires that the organization continuously improve process performance by changing the defined process to increase its effectiveness.

Example Work Products

· Revised process

Notes

The improvements may be based on incremental improvement or innovative improvements such as new technologies (perhaps as part of pilot testing). Improvements will typically be driven by the goals established in GP 5.1.1 (Establish Process Effectiveness Goals). The key to this GP is that the corrections are made in "real time", not passed to the next program. Real-time means that corrections to the process happen within the scope of the current activity. They are not just captured to be used as part of a lessons learned for future activities.

Relationships

This Generic Practice leverages off GP 3.2.1 (Use a Well-Defined Process) to establish procedures for improving performance in “real-time”.

Accreditation Formal declaration by a designated approving authority that a system is approved to operate in a particular security mode using a prescribed set of safeguards.

Appraisal Analysis by a trained team of professionals to determine the state of an organization’s current process, to determine the high-priority process-related issues facing an organization, and to obtain the organizational support for process improvement.

Assessment A review of the security posture of a specified operational system for the purpose of identifying potential vulnerabilities. When using the Information Security Assessment Methodology (ISAM), analysts gather and analyze information covering 18 baseline Information Security categories to identify potential vulnerabilities and ensure that the team has not overlooked any important areas.

Assessment Plan A living document which details the steps that are being performed during the pre-assessment, on-site phase, and post-assessment activities. This document designates the who, when and where of a specific vulnerability assessment.

Asset Anything that has value to the organization. [ISO 13335-1: 1996]

Assurance Degree of confidence that security needs are satisfied. [NIST94a]

Assurance Argument A set of structured assurance claims, supported by evidence and reasoning, that demonstrate clearly how assurance needs have been satisfied.

Assurance Claim An assertion or supporting assertion that a system meets a security need. Claims address both direct threats (e.g., system data are protected from attacks by outsiders) and indirect threats (e.g., system code has minimal flaws).

Assurance Evidence Data on which a judgment or conclusion about an assurance claim may be based. The evidence may consist of observation, test results, analysis results, and appraisals providing support for the associated claims.

Authenticity The property that ensures that the identity of a subject or resource is the one claimed. Authenticity applies to entities such as users, processes, systems and information. [ISO 13335-1:1996]

Availability The property of being accessible and useable upon demand by an authorized entity. [ISO 7498-2: 1988]

Base Practice An engineering or management practice that contributes most to the effective implementation of a process area.

Baseline A specification or product that has been formally reviewed and agreed upon, that thereafter serves as the basis for further development, and that can be changed only through formal change control procedures. [IEEE-STD-610]

Baseline Information Categories Eighteen specific areas that are examined and investigated during an Information Security assessment for potential security vulnerabilities.

Capability Maturity Model A description of the stages through which an organization evolves as it defines, implements, measures, controls, and improves its processes. The model provides a guide for selecting process improvement strategies by facilitating the determination of current process capabilities and the identification of the issues most critical to software quality and process improvement. [CMU/SEI-93-TR-025]

Causal Analysis The analysis of defects to determine their underlying root cause. [CMU/SEI-93-TR-025]

Certification Comprehensive evaluation of security features and other safeguards of an AIS to establish the extent to which the design and implementation meet a set of specified security requirements.

Common Cause (of a Defect) A cause of a defect that is inherently part of a process or system. Common causes affect every outcome of the process and everyone working in the process. [CMU/SEI-93-TR-025]

Common Feature A logical grouping of generic practices that are designed to describe a major shift in an organization’s process capability. Common Features indicate whether the implementation and institutionalization of a process area is effective, repeatable, and lasting.

Confidentiality The property that information is not made available or disclosed to unauthorized individuals, entities, or processes. [ISO 7498-2:1988]

Consistency The degree of uniformity, standardization, and freedom from contradiction among the documents or parts of a system or component. [IEEE-STD-610]

Correctness A property of a representation of a system or product such that it accurately reflects the specified security requirements for that system or product.

Customer The individual or organization that is responsible for accepting the product and authorizing payment to the service / development organization.

Criticality Matrix The organizational criticality matrix is the output of a systematic method used to create a matrix that will represent and define organizational information, assign the highest value of all information types, and assign attributes and loss impact factors in the areas of confidentiality, integrity, and availability. This is followed by development of a system criticality matrix that maps information types and associated values from the organizational criticality matrix to a matrix for each system being assessed.

Data Integrity The property that data has not been altered or destroyed in an unauthorized manner. [ISO 7498-2:1988]

Defined Process The operational definition of the standard process used by a project. It is developed by tailoring the organization’s standard process to fit the specific characteristics of the project. [CMU/SEI-93-TR-025]

Effectiveness A property of a system or product representing how well it provides security in the context of its proposed or actual operational use

Engineering Group A collection of individuals (both managers and technical staff) which is responsible for project or organizational activities related to a particular engineering discipline (e.g. hardware, software, software configuration management, software quality assurance, systems, system test, system security).

Evidence Directly measurable characteristics of a process and/or product that represent objective, demonstrable proof that a specific activity satisfies a specified requirement.

Exploitation Result of a successful attack by a threat against a vulnerability

Exposure The measure of risk associated with a specific Threat / Vulnerability / Impact triple being realized.

Formal Documentation Written documentation that is under configuration management and control. It is generally developed under strict style and content rules.

Generic Practice A management practice that contributes most to the effective institutionalization of a process area.

Group The collection of departments, managers, and individuals who have responsibility for a set of tasks or activities. The size can vary from a single individual assigned part-time, to several part-time individuals assigned from different departments, to several dedicated full-time individuals.

Impact The resulting effect to an organization from a successful exploitation of a specific vulnerability by a threat.

Informal Documentation Written documentation that can be gathered together to provide evidence that a specific activity is being done. It need not be in any specific format.

Information Criticality The customer’s perceived value and importance (e.g. sensitivity, criticality, integrity) of the information being processed or stored on a customer’s system.

Institutionalization The building of infrastructure and corporate culture that support methods, practices, and procedures so that they are the ongoing way of doing business, even after those who originally defined them are gone. [CMU/SEI-93-TR-025]

Integrity See data integrity and system integrity

Logical Boundaries Where the responsibility for or authority over information changes hands.

Maintenance The process of modifying a system or component after delivery to correct flaws, improve performance or other attributes, or adapt to a changed environment. [IEEE-STD-610]

Methodology A collection of methods, procedures, and standards that define an integrated synthesis of engineering approaches to the development of a product or system.

Objective (adj.) From a non-biased perspective

On-site Activities A series of meetings to explore and confirm the information and conclusions made during the Pre-assessment Phase. This includes data gathering (interviews, documentation, system demonstrations) and validation in order to provide initial analysis and feedback to the customer.

Organization A unit within a company or other entity (e.g., government agency) within which many projects are managed as a whole. All projects within an organization share a common top-level manager and common policies. [CMU/SEI-93-TR-025]

Organization’s Standard Process The operational definition of the basic process that guides the establishment of a common process across projects. It describes the fundamental process elements that each project is expected to incorporate into its defined process. It also describes the relationships between the process elements. [CMU/SEI-93-TR-025]

Out-Briefing The final step in the on-site activities phase in which the assessment team briefly discusses their preliminary findings. These findings, along with associated analyses, will later be detailed in the final report.

Penetration Profile A delineation of the activities required to effect a penetration.

Performed Process The process that is actually executed on a project.

Physical Boundaries The locations (e.g. room, building, complex) of the system equipment and local procedures regarding the handling and processing particular types of information.

Post-Assessment The final analysis of all information gathered during the assessment, and completion of any additional research that is required, concluding with the preparation and delivery of a final report that outlines all identified potential vulnerabilities, and corresponding countermeasures.

Pre-Assessment The initial meeting and subsequent coordination activities to provide the customer an in-brief of services, refine their needs, gain an understanding of the criticality of the customer’s information: identify systems, including boundaries: coordinate logistics with the customer; and write an assessment plan. During this phase it is determined what information have to be protected and why. The goal is to define and agree upon the scope of the assessment.

Procedure A written description of a course of action to be taken to perform a given task. [IEEE-STD-610]

Process A sequence of steps performed for a given purpose. [IEEE-STD-620]

Process Area A defined set of related process characteristics which, when performed collectively, can achieve a defined purpose. A process area is composed of base practices.

Process Capability Process capability refers to an organization's potential. It is a range within which an organization is expected to perform.

Process Maturity The extent to which a specific process is explicitly defined, managed, measured, controlled, and deemed effective.

Process Performance The measure of actual results on a particular project that may or may not fall within the Process Capability range.

Project An activity that is bounded by time, materials and personnel.

Quantitative Process A process in what all performance measures and reporting can be quickly and logically converted to cost in terms of time, material and resources.

Recommended Countermeasures Suggestions to eliminate or mitigate security risks used to improve an organizations security posture.

Reliability The property of consistent behavior and results. [IEEE 13335-1:1996]

Residual Risk The risk that remains after safeguards have been implemented. [IEEE 13335-1:1996]

Risk The potential that a given threat will exploit vulnerabilities of an asset or group of assets to cause loss or damage to the assets. [IEEE 13335-1:1996]

Risk Analysis The process of identifying security risks, determining their magnitude, and identifying areas needing safeguards. [IEEE 13335-1:1996]

Risk Management Process of assessing and quantifying risk and establishing acceptable level of risk for the organization. [IEEE 13335-1:1996]

Security Policy Rules, directives and practices that govern how assets, including sensitive information, are managed, protected and distributed within an organization and its systems

Security Posture An organization’s current susceptibility to exploitation.

Security Related Requirements Requirements which have a direct effect on the secure operation of a system or enforce conformance to a specified security policy.

Special Cause (of a Defect) A cause of a defect that is specific to some transient circumstance and not an inherent part of a process. Special causes provide random variation in process performance. [CMU/SEI-93-TR-025]

Standardized Standardized implies control at the highest levels and implementation through to the lowest level. Tailoring can occur at the various levels as long as the original intent is not lost.

System A collection of components organized to accomplish a specific function or set of functions. [IEEE-STD-610] A system may include many products. A product can be the system.

System Identification Determining what information needs protecting and why, then having the customer identify where the information is stored, transferred and by what systems and networks.

Tailored Process A process, standard, or procedure that is modified to better match process or product requirements. [CMU/SEI-93-TR-025]

Team A collection of individuals (both managers and technical staff) which is responsible for project or organizational activities related to a particular discipline (e.g. hardware, software, software configuration management, software quality assurance, systems, system test, system security).

Threat Capabilities, intentions, and attack methods of adversaries to exploit, or any circumstance or event with the potential to cause harm to information or a system.

Validation The process of assessing a system to determine whether it satisfies the specified requirements.

Verification The process of assessing a system to determine whether the work products of a given development phase satisfy the conditions imposed at the start of that phase.

Vulnerability A weakness of an asset or group of assets which can be exploited by a threat. [IEEE 13335-1:1996]

Well-Defined Process A process that includes readiness criteria, inputs, standards and procedures for performing the work, verification mechanisms (e.g., reviews), outputs, and completion criteria. [CMU/SEI-93-TR-025]

Work Product Output of a process.

This appendix is designed to provide specific guidance to organizations seeking Information Security Assessment Methodology (ISAM) compliance in accordance with the Information Security Assessment Training and Rating Program (ISATRP). A brief overview of the work products and actions required for each Process Area is provided below. In some cases there are no additional actions needed to address ISAM and ISATRP requirements. If there is no additional action, it is so stated:

ISA-PA01: Provide Training

The knowledge and skills required to properly perform the necessary functions must include successful completion of the ISAM course.

ISA-BP01.01 – Identify Training Needs

To be ISAM compliant, the technical ISAM proficiency of employees (e.g., ISAM certificates) along with the organizational ISAM capability to support assessor activity (e.g., ISA-CMM organizational rating) are required.

ISA-BP01.02 – Select Method of Information Security Training

Organizational capability is measured based on the output from an official ISA-CMM appraisal activity which includes training, appraisal and rating.

Below is a matrix that outlines the evidence needed to comply with the ISAM for a specific ISA-PA at a specified level:

Information Security Assurance Capability Maturity Model

161 of 161 September 2010

This appendix is designed to provide specific guidance to organizations seeking Information Security Red Team Methodology (ISRM) compliance in accordance with the Information Security Assessment Training and Rating Program (ISATRP). It is important to note that the success of the ISRM process does not rely on the output from an ISAM assessment. Although some of the artifacts of the ISAM can be used to support the technical aspects of an ISRM assessment.

A brief overview of the work products and actions required for each Process Area is provided below. In some cases there are no additional actions needed to address ISAM and ISATRP requirements. If there is no additional action, it is so stated.

ISA-PA01: Provide Training

The knowledge and skills required to properly perform the necessary functions must include successful completion of the ISRM and the ISA-CMM courses as appropriate.

ISA-BP01.01 – Identify Training Needs

To be ISRM compliant, the technical ISRM proficiency of employees (e.g., ISRM certificates) along with the organizational ISRM capability to support assessor activity (e.g., ISA-CMM organizational rating) are required.

ISA-BP01.02 – Select Method of Information Security Training