Software testing Excercises, due within 12 hrs maximum

These exercises are designed to help you get the most from the McCabe training course and will demonstrate how you can use the McCabe Toolset effectively. The exercises are designed to be used alongside the information contained in the course notes; however, there is enough material supplied to allow the student to work through the exercises alone.

Using this book

The exercises follow the class presentations. Each section is concluded with an exercise. The exercises use several small public domain systems and some written in-house for the purposes of this training course. This book includes the options for both the UNIX and Windows implementations.

Anything inside a box like this details a

task for you to perform

These are infoboxes and contain extra

information about the Toolset

Obtaining Extra Help

If, after the course has concluded, you require any further help or assistance then please do not hesitate to contact the McCabe Technical Support Helpdesk on the numbers below.

Phone: 800-634-0150

Email : support@mccabe.com

Exercise 1 - Flowgraphs

Objectives

To enable students to gain an understanding of how McCabe IQ represents the logic in program source code by manually constructing flowgraphs from source/pseudo code.

To examine the tools’ construction of flowgraphs

Activity 1 - Manual Flowgraphs

Draw a flowgraph for the

following code using the graphical

constructs provided on the next page

1 IF condition

2 statement

3 ENDIF

4 statement

5 WHILE condition

6 DO SWITCH

7 Label 1

8 statement

9 Label 2

10 statement

11 Default

12 statement

13 END SWITCH

14 END WHILE

Activity 2 - Automated Flowgraphs

The tools can be used to generate

flowgraphs representing source code

Using the source Graph/Listing ,

evaluate the software

Invoke the flowgraphs Battlemap by:

Windows - clicking on the icon named flowgraphs

UNIX - typing ‘./start_flowgraphs’

Select No

Right Click on each function, select the Graph/Listing option, and answer the following questions:

Which is the easiest to understand (source 1,2 or 3)? : ______

Which would you prefer to maintain (source 1,2 or 3)? : ______

Which would you prefer to test (source 1,2 or 3)? : ______

Selecting nodes on the graph or listing

allows you to cross-reference between

the two

Exercise 2 - Manual Application of v(g)

Objectives

To obtain an understanding of the McCabe metrics by manually calculating cyclomatic complexity values

Activity 1

Use the flowgraph regions method and

the code predicate(s) method to

calculate v(g)

?

v(g) = Regions = ________

v(g) = Predicates + 1 = _

Activity 2

Study the sample flowgraph and write

the equivalent pseudo code

?

Flowgraph Code

What is the Cyclomatic Complexity?

?

v(g) = ( e - n + 2 ) = - + 2 =

v(g) = ( Predicates + 1 ) = + 1 =

v(g) = Regions =

Activity 3

Use the formal method to discover the

complexity of the following graph.

?

..\..\bitmaps\exercise5.bmp

Cyclomatic Complexity v(g) __________

Exercise 3 - Unit Level Metrics

Objectives

To evaluate and compare complexities (v(g) and ev(g)) of modules.

To gain insight and speed in interpreting unit level outputs.

To practice on unit level tool tasks.

Activity 1

Use the tool to generate metrics

?

Invoke the less Battlemap.

Select File Preferences General Sort Reports Alphabetically Close

Select Metrics Module Metrics . Fill in the table below from the report generated.

Name of module

v(g)

ev(g)

Expected Maintenance Difficulty

jump_back

back_line

commands

edit

forw_line

forward

main

pr_string

Activity 2

Explore the difference between v(g)

and ev(g)

?

From the chart above, compare the metrics of the two functions back_line and edit.

· Use the View Find option to find the functions (note that the Find dialog moves the mouse pointer to point at the ‘found’ module).

· Bring up the Graph/Listing and answer the following questions.

Which of the two modules is more difficult to maintain?

Which of the two modules has a higher v(g)?

How would you interpret the discrepancy of the two answers above? Explain.

Activity 3

Use graphical techniques to measure

a system.

?

· Select Metrics Graphical Scatter Plot.

· Enlarge the graph.

The Scatter Plot is context sensitive. Right clicking on a point will allow you to access information about the module it represents. Answer the following questions.

Which is the most unstructured module?

Which is the most complex but completely structured module?

Exercise 4 - System Level Metrics (Procedural)

Objectives

To gain an understanding of how McCabe IQ determines the Design complexity of a System by Calculating the System Design Complexity and by looking at the theoretical Execution of Paths in a program

Activity 1

Answer some basic questions about

the design metric

?

Invoke the Corp4 Battlemap by :

Windows - clicking on the icon named Corp4

UNIX - typing ‘./start_corp4’

Select No

Use the Module Metrics report learned in Exercise 3 Activity 1 to fill in the following table.

Module Name

iv(g)

Maintenance

Revenue

Development

Query

Support

Marketing

Main

Using Metrics Program Metrics

The S0 and S1 metrics can only be calculated from a single entry point

!

What is the S0 of “main”?

What is the S1 of “main”?

S1 describes the number of paths (subtrees) which traverse the entire system that are needed to execute the integration paths within each module of corp4

Exercise 5 – Project Configuration

Objectives

To gain a basic understanding of the McCabe IQ tool set-up and to use the McCabe IQ tools to analyse a small set of source files.

Activity 1

Create a new project.

?

The description of the code for which you will build a program configuration file is as follows:

i. The project name is “ exercise5”.

ii. The code to be parsed is C code.

ii. The code is already preprocessed.

iii. The code is in c:\McCabe\exercises\c\less\code ( PC )

OR<student directory>/exercises/less/code ( UNIX ).

iv. These are the source files: ch.i, command.i, line.i, main.i, prompt.i and prim.i.

v. There should be no instrumentation information produced.

vi. The Program name is “less”

Invoke the config Battlemap by :

Windows - clicking on the icon named config

UNIX - typing ‘./start_config’

From the Battlemap main menu select File and then New Project.

The Create a New Project window is displayed. Click the Browse button to change to the Unix <student_directory>/exercises or PC c:\mccabe\exercises\c\ directory. Type exercise5 in the Enter a name for your project text box. This name will be used to create the name of your project configuration file (work.pcf). Notice how the project name entered is used as the directory name. If the folder does not exist, the user is prompted to create the folder.

Click Yes. The location for project files should be <student_directory>/exercises/pcf/exercise5 for Unix or c:\mccabe\exercises\c\exercise5 for PCs. Click Next.

Select a language and dialect.

?

From the Language: drop-down menu, select C. From the Dialect drop-down menu, select ANSI 1989 Standard C.

Click Next in the Select a Programming Language and Dialect window.

Click Next in the Source of Configuration Information window to ensure that you are prompted for the source files.

Select source files for the project.

?

Click Add/Remove Files. In the Add/Remove Files window, browse to the directory <student_directory>/exercises/less/code for Unix or to c:\mccabe\exercises\c\less\code for the PC. In the Look In text box containing preprocessed source files, you will see ch.i, command.i, line.i, prim.i, prompt.i and main.i.

Click Add Folder to add the files to the text box at the bottom of the Add/Remove Files window. Click OK. The files are now listed in the Source Files in Program box in the Select Source Files window. Click Next.

Select the files to be instrumented.

?

In the Instrument Files window, check the Don’t Instrument Files radio button and click Next.

Review your project settings.

?

In the Project Summary window, confirm your selections click Next.

Create the project.

?

In the Finish window, check the Yes, Create the project and then show the Battlemap radio button. Click Finish

When the software displays

‘Do you wish to Draw Structure Map’ Click YES

Activity 2

Configure a project to allow you to

perform your own analysis

?

Invoke the config Battlemap by :

Windows - clicking on the icon named config

UNIX - typing ‘./start_config’

From the Battlemap main menu select File and then New Project.

Select your own source files or try any of the existing Demo code folders in the C subdirectories, and repeat the process above.

Ensure that the folder/project you create is called ACTIVITY2

Monitor the McCabe console window for error messages

during the analysis

!

Exercise 6 - Using Unit Level Metrics

Objectives

To gain an understanding if how the McCabe IQ tool can be used to generate custom reports, and then modifying the tools to use other metrics

Activity 1

Use the tools to create a report that lists

cyclomatic & essential complexity in addition

to number of comment lines of code

?

Invoke the less Battlemap.

· Select Metrics Create Custom Reports .

· You can select the Include Predefined and Global Reports check box to see the default reports available.

· Click on the Add button.

· In the next window, make sure the Module radio button is selected, then press OK.

Custom report lists may use the

abbreviation of the metric name

e.g. v(g) for Cyclomatic Complexity

!

The displayed window is the custom report generation window.

· Fill in the appropriate Module Report Name, Title, and Author boxes.

· In the Available Metrics: box select the metrics required (v(g), ev(g) and comments) and press Add .

· Click comments in the Column Metrics: window.

· Click Column Info button.

· Under the Sorting: heading select Ascending radio button and press OK to save your changes.

· Preview the report to verify that the report will truncate mod_name at 30 characters and will sort ascending on comments.

· Close the preview window using Close.

· Select Save, ensure that the new report is highlighted in the Custom Report list, and click Run.

Which module in your opinion does not have enough comment lines of code

(CLOC)?

Note that Custom Reports can

include Scatterplots and Histograms

for the measures you include

!

Activity 2

Modify the Scatterplot to measure the

v(g) and CLOC

?

· Select File Preferences Metric Settings.

· Under the Selections For: section ensure that the Scatterplot radio button is highlighted.

· Select ev(g) in the Selected Metrics column and click on Remove.

· Select comments from the Available Metrics column, and click on Add.

· Click Save then Close.

Bring up a Scatterplot graph as described in Exercise 3 Activity 3.

Describe how you would move the midpoint from v(g)=10, CLOC=4.

(Hint: Bring up the Metrics Thresholds Window again)

Exercise 7 - Battlemap Usage

Objectives

To become familiar with the McCabe IQ Battlemap interface.

To visualise a system layout, module relationship and design quality; to formulate initial opinions about quality and improvements

Activity 1

Use the Battlemap Analysis Tool to

make initial decisions about a system

?

Invoke the less Battlemap.

Using the options in the File Preferences Battlemap Box dialog, reduce the structure chart so that all of it is visible on the screen.

To view a system first adjust the

vertical and horizontal gaps to

proportion the diagram. Always keep

the gaps as small as possible.

!

By visual inspection on the structure establish the module that you feel has the greatest controlling influence in the system. Click on that module to establish its name.

Module Name: __________

[Note: Systems consisting of a few highly complex modules that have a lot of functionality, with the rest of the modules being relatively simple are often the result of ‘chaotic’ programming techniques or ‘legacy’ code which has been modified over many years. The balance between unit level complexity and function hierarchy complexity is ‘good’ when all modules have medium complexity - between 1 and 20 - and the calling hierarchy is not highly complex. The complexity of the calling hierarchy can be empirically calculated (and may be covered later in the course) For the purposes of this exercise you should use visual inspection to determine it. ]

Give your assessment of the balance between the unit level complexity and function hierarchy complexity.

Very Good / Good / Average / Bad / Very Bad

Based on the applications structure, what module or modules require further design work?

Activity 2

Navigate through the Battlemap

finding modules quickly

?

The Find… option can accept wildcards.

e.g. b* will find all modules beginning

with the letter b.

!

Using the View Find dialog, locate the only module on the Battlemap with “fc” in its name. (Remember that the Find dialog moves the mouse pointer to the found module.)

What is the module called?

· Select the Graph/Listing option from the module menu.

In your opinion is this function too complex? Yes / No

In your opinion is it well structured? Yes / No

· Select the Module Summary option from the module menu.

How many lines of code contain executable statements?

How many lines of code contain comments? _

NCSLOC : Non Comment Source Lines of Code

CSLOC : Comment Lines of Code

BLOC : Blank lines of Code

C&SLOC : Comment and Source Lines of Code

!

What metric of this module could be regarded as above average?

Activity 3

Use the tools to extract details

about a module

?

Find the only module that ends with the letters “_back”. Provide the following information about that module:

Module Reference Number:

Number of Parameters Used:

Degree of fan-out vs. fan-in:

The module number can be used in the

Find window to search for the module

!

Exercise 8 - The Exclude Tool

Objectives

To use the McCabe IQ Exclude Tool for impact analysis and system management

Activity 1

Use the Exclude tool to isolate a

specific area of the system

?

When the exclude window is active any

mouse click will update this window and

NOT activate the menu

!

Invoke the less Battlemap.

Using the options in the File Preferences Battlemap Box dialog, reduce the structure chart, set the box type Terse .

View a restricted system in relation to

the entire system to gauge impact

?

· Find the function edit.

· Select View Advanced Exclude from the main menu bar.

· Check the Art iculate box and ensure Su perimpose Exclude is on (ticked).

· Click on the module edit in the Battlemap followed by E xclude in the Exclude window.

Gauge the impact of change of edit? Localized / Medium / Extensive

Using the same technique evaluate the impact of change on the following moduless. Use the Cl ear and Restore button on the exclude window to reset the view.

Gauge the impact of change of flush? Localized / Medium / Extensive

Gauge the impact of change of psignals ? Localized / Medium / Extensive

Keeping the last exclusion.

· Uncheck Capture Box Selection (ensuring that this selection is OFF).

· Uncheck Su perimpose Exclude (ensuring that this selection is OFF).

· Click E xclude button.

Enlarge the system using the File Preferences Battlemap Box menu so that it fits on the screen (Keep the gaps as small as possible)

What modules directly call module psignals?

How many modules are directly called by psignals? ___________

!

Modules on the Battlemap with vertical lines on either side are library modules. These are modules for which the parser could find no source code. An example would be a system call.

· Find prim:forw. Using the Advanced Exclude function, Root on prim:forw.

· Select Code Onl y and E xclude.

What has happened to the Battlemap?

The Exclude window can accept the

‘’ wildcard operator.

!

Activity 2

Use your knowledge to create a sub-

system described below

?

Using the following exclusion, reduce the system. (Type this text directly into the Exclude dialog).

ROOT search

DEPTH 3

OMIT back_line

OMIT back_raw_line

OMITTREE forw_raw_line

How many code functions remain? _

What is the total cyclomatic complexity v(g) of this reduced system? ___

(Hint : Exercise 3 Activity 1)

All reports, including graphical and

Test reports display only the exclusion

!

Remove the exclusion parameters using Cl ear and E xclude

Exercise 9 - Partitioning

Objectives

To learn how the McCabe IQ tool can be used to group parts of the system together, and to create new groupings to manage large systems.

Activity 1

Invoke the multipad Battlemap by :

Windows - clicking on the icon named multipad

UNIX - typing ‘./start_multipad’

Select No

Identifying separate areas of the system

?

C does not have the formal concept of a class, which would exist in C++, however it is possible to group functions within the system. In this example we will use the file as a natural system grouping. Files are not necessarily a good technique of group but in general they do contain important information on the architecture of the system.

Although called classes by the tool, in

this exercise they will be referred to as

groups

!

Collapse all the modules into their owner files groups.

Tools ® Class Editor… Tools®Bind All

Remove the external library calls from the system and root on the Multipad file group

(Hint : Tools Exclude … Code Onl y)

How many file groups make up this system ? ______

Click on MULTIPAD file group on the Battlemap and select Unbind

Which modules of MULTIPAD make CALLs to other file groups?

Rebind the system, Tools ® Class Editor… Tools®Bind All

Using the same technique how many modules call MPFILE from within the MPFIND file group ?

Unbind the entire system and close the class editor

Activity 2

Use the exclusion and classing together

to identify an impact of change

?

Make an articulation on the module AddFile

Rebind the system, Tools® Class Editor… Tools®Bind All

How many file groups remain ? _

Why ? _______________________________________

Clear all the exclusions except the Code Only and rebind the system.

Using the popup menu on each file group, use the Module Metrics to view the combined complexity of each system.

Which system requires the greatest effort to test?

Activity 3

Combine three systems to simplify

the system

?

· Invoke the Tools® Class Editor .

· Use the File® Add User Class to create a new group,

· Name the group “NewSystem”

· Select first MPINIT, MPPRINT and finally MPFIND from the Battlemap

· Save the changes to the class, select File® Save User Class .

· Rebind the system

The tool stores the grouping (class)

Information within a text file specified by

the CLASSFILE keyword in the PCF

!

How many groups are now displayed?

How many Lines Of Code (LOC) exist in the new group?

Exercise 10 - The Data Dictionary

Objectives

To gain an understanding of how to use the McCabe IQ Data Dictionary.

Activity 1

Use the Data Dictionary to visualise

and isolate various data elements in

the system

?

Invoke the less Battlemap.

· Select Reengineering Data Data Dictionary. The window will display all the data elements used within the current system.

· Use the Data Set button within the Data Dictionary* to limit the view to only global, local and parameter data variables that contain the letter ‘a’. This is done by entering information into the Specify Data Set window as shown below and Apply.

How many references meet this criteria?

Note that the Declaration Type also

Includes Modules and Variables

!

Use the Reports Specified Metrics option within the Data Dictionary to establish which single function makes the greatest use of this particular set of data.

Function Name

[ Hint : sdv(g) quantifies the complexity of a module as it relates to that modules use of a defined data set]

Activity 2

The Data Dictionary window can accept

a number of wildcards and logical

operators

!

Operators:

* Any series of characters

| OR

& AND

! NOT

Change the set to select all variables that contain the string er but not any that contain the string error

How many references meet this criteria?

Finally reduce the previous set of variables to only those of type int

How many actual variables remain?

QA Exercise 1 - Derived Metrics

Objectives

To gain an understanding of how to create a derived metric in the McCabe IQ tool, and to then use derived metrics in reports

Activity 1

Invoke the less Battlemap.

Create a metric to identify which function has the lowest CLOC to complexity ratio.

?

· Select Quality Derived Metrics.

· Select Add.. and ensure that the Module radio button is checked, and then press OK.

· Enter CLOC_VG_RATIO for Module Metric Name and Column Heading,

· Enter Author name.

· Select the vg metric, and press the divide ( / ) button on the calculator.

· Select comments from the Metrics window.

· Check the Make Globally Available box.

· Press Save.

Activity 2

Create a report that lists cyclomatic & essential complexity, number of comment lines of code and CLOC to complexity ratio

?

· Select MetricsCreate Custom Reports .

· You can select the Include Predefined and Global Reports check box to see the default reports available.

· Click on the Add button.

· In the next window, make sure the Module radio button is selected and then press OK.

Custom report lists may use the

Abbreviation of the metric name

e.g. v(g) for Cyclomatic Complexity

!

The displayed window is the custom report generation window.

· Fill in the appropriate Module Report Name, Title, and Author boxes.

· In the Available Metrics: box select the metrics required (v(g), ev(g), comments, and CLOC_VG_RATIO) and press Add-> .

· Click on CLOC_VG_RATIO in the Column Metrics: window.

· Click on the Column Info button.

· Under the Sorting: heading select the Ascending radio button and press OK to save your changes.

· Preview the report to verify that the report will truncate mod_name at 30 characters and will sort ascending on CLOC_VG_RATIO.

· Close the preview window using Close.

· Select Save, ensure that the new report is highlighted in the Custom Report list, and click Run.

Note that Custom Reports can include Scatterplots and Histograms for the measures you include.

!

What do you think could be causing the n/a to be generated in the report line?

(Hint : Look at the figures and try to work out the ratio manually)

How would you modify the metric to remedy this problem?

Activity 3

Create a report that lists cyclomatic & essential complexity, number of comment lines, and is categorized in terms of risk

?

Create a metric to quantify risk by evaluating the difference between v(g) and ev(g)

?

· Select Quality Derived Metrics.

· Select Add.. and ensure that the Module radio button is checked, and then press OK.

· Enter RISK_RATIO for Module Metric Name and Column Heading,

· Enter Author name.

· Select the vg metric, and press the subtract (-) button on the calculator.

· Select evg from the Metrics window.

· Check the Make Globally Available box.

· Press Verify

· Press Save

We now have our RISK_RATIO metric, now to create a custom report with it.

Create a new Risk report with hidden column and selection criteria

?

· Select MetricsCreate Custom Reports .

· You can select the Include Predefined and Global Reports check box to see the default reports available.

· Click on the Add button.

· In the next window, make sure the Module radio button is selected and then press OK

The displayed window is the custom report generation window.

· Fill in the appropriate Module Report Name, Title, and Author boxes.

· In the Available Metrics: box select the metrics required (v(g), ev(g), RISK_RATIO,comments & nl) and press Add-> .

· Click on v(g) in the Column Metrics: window.

· Click on the Column Info button.

· Under the Sorting: heading select the Descending radio button

· Set the Range minimum value to 10 and press OK to save your changes.

· Click on RISK_RATIO in the Column Metrics: window

· Click on the Column Info button.

· Click on the Hide Column check box.

· Set the Range maximum value to 6.

· Preview the report to verify that the report will truncate mod_name at 30 characters and will sort descending on v(g) and will not show the RISK_RATIO values.

· Close the preview window using Close.

· Select Save, ensure that the new report is highlighted in the Custom Report list, and click Run.

How many modules are identified as HIGH RISK? _

Which module(s) represents the highest risk based on difference alone?

QA Exercise 3 - Reports that Assess Quality

Objectives

To use the McCabe IQ tool to generate quality reports, the maintainability report and a logic density report.

Activity 1

Generate a Maintainability report.

?

Invoke the less Battlemap.

· Select Quality Reports Maintainability .

The Maintainability Information for the Program is displayed. The maintainability report displays metrics for a program.

Module details are obtained by moving the mouse onto a totals value (in blue), right hand mouse click Expand Column

What module would be the most difficult to maintain?

Activity 2

Generate a Density report.

?

· Select Quality Reports Density .

· Select Help. The Density help page is displayed. At the bottom of the page under Related Topics, select Density Report: Assessing Quality. Read how density is defined and calculated.

· High quality modules or systems will have density metric values close to 0.

How many modules in this system have ed(G)=1?

What does this number tell you about the structuredness of those modules? ______________

TEST Exercise 1 – A First Look at Instrumentation

Objectives

To use the McCabe IQ Test tool to perform the following activities

· To configure a project for instrumentation

· Identify where parser options are to be changed

· Examine an instrumented source file

· Learn how to rebuild an instrumented file

Activity 1

Configure a project.

?

Invoke the Corptest Battlemap by :

Windows - clicking on the icon named Corptest

UNIX - typing ‘./start_corptest’

Select No

a. From the Battlemap menu, select File and then New Project. The Create a New Project window in the configuration wizard appears. Click Browse to change to the c:\mccabe\exercises\c\corptest\ directory on a [PC] Windows platform or to the <student_directory>/exercises/corptest/ directory on a UNIX platform. Type work in the Enter a name for your project text box. The name “work” will be used to label your project configuration file work.pcf. Notice how the project name entered is also used as the directory name. The location for the project file should be c:\ mccabe\exercises\c\corptest\work\ on a [PC] Windows platform or <student_directory>/exercises/corptest/work/ on a Unix platform. Click Next.

b. In the Select a Programming Language and Dialect window, select C from the Language drop-down menu. Next, select Microsoft C from the Dialect drop-down menu on a [PC] Windows platform or ANSI 1989 Standard C from the Dialect drop-down menu on a UNIX platform. Click Next.

c. In the Source of Configuration Information window with I’ll supply the information selected, click Next.

d. In the Select Source Files window, type CorpTest in the Enter a program name text box and then click Add/Remove Files.

In the Add/Remove Files window, browse to the

c:\mccabe\exercises\c\corptest\code\ directory on a [PC] Windows platform or to the <student_directory/exercises/corptest/code/ directory on a Unix platform. In the Look in box, select the file corp4.c, click Add, and then click OK. The corp4.c file is now listed in the Source Files in Program box in the Select Source Files window. Click Next.

e. In the Instrument Files window with Instrument all files selected, click Next.

f. From the Project Summary window, click the Advanced Options... button. Parser options are listed here. Notice the BOOLINST, EVENTMAP, INST, INSTMOD, NOINSTMOD, and PATHVEC options. To obtain a description of an option, move the mouse over the option.

g. What is the difference between INSTMOD and NONINSTMOD? __________________________________

h. For this example do not change any of the options. From the Advanced Options window click the OK button.

i. After verifying that your project configuration information in the Project Summary window is correct, click Next.

j. In the Finish window with Yes, create the project and then show the Battlemap and Export the instrumented source code, which I'll rebuild myself selected, click Finish.

NOTE: Corp4 is a program that

simulates the original telephone

answering process at McCabe.

!

Activity 2

Examine instrumented source code.

?

a. Browse to the directory

C:\mccabe\exercises\c\corptest\work\CorpTest_inst\

directory on a [PC] Windows platform or to the

<student_directory>exercises/corptest/work/CorpTest_inst/

directory on a Unix platform.

b. What files were created in the instrumentation directory? _____________________

c. With the editor of your choice examine the file corp4.c.

d. List two functions that were inserted during the instrumentation process. _

TEST Exercise 2 - Unit and System Testing

Objectives

To use the McCabe IQ tool to calculate System Design Complexity and to study the theoretical Execution of Test Paths

Activity 1

Answer some basic questions about

the design metric

?

Invoke the Corp4V2.

Using Metrics Program Metrics

The S0 and S1 metrics can only be

calculated from a single entry point

!

What is the S0 of “main”?

What is the S1 of “main”?

S1 describes the number of paths (subtrees) which traverse the entire system that are needed to execute the integration paths within each module of corp4

The Integration Level Test Plan window allows the user to generate reports and visualize either a complete set of test paths (subtrees) or just the integration subtrees (also called design subtrees)

· Select T esting Test Plan Integration Level

· Select OK. Using the Subtree Number navigation bar, scroll through the integration test and observe the Battlemap changes.

Name all modules which are executed by the baseline (1st) design subtree.

Activity 2

List the conditions and specify the test

scenario that exercise the baseline

?

Integration test conditions are required for each integration test.

· Select T esting Test Plan Integration Level

· In the Integration Level Test Plan window, select the radio button Text and then OK.

· The Design Subtrees window contains the test conditions for each integration test.

List the conditions and specify the test scenario that will exercise the baseline.

Condition

Value

How many design subtrees are needed to achieve integration testing?

(iv(g) Subtrees)

(Hint : Testing->Test Plan->Integration Level->select Detail Level Design)

How many design subtrees are needed to achieve complete unit testing?

(Cyclomatic v(g) Subtrees)

(Hint : Testing->Test Plan->Integration Level->select Detail Level Cyclomatic )

Why are there more subtrees based on v(g) rather than iv(g)?

TEST Exercise 3 – Procedural Software Testing

Objectives

To gain experience in using the McCabe IQ tools for:

-- classifying modules by their degree of testedness.

-- completing the integration tests of a system.

Activity 1

Use the instrumented version of the

source code to measure test case

effectiveness

?

Before commencing this exercise, delete the file, inst.out.

· Run the Corp4 program by clicking on the executable, named Corp4.exe.

· Answer the questions asked by the program exactly as directed below:

Is this a purchase? y <Enter>

Do you need new features? y <Enter>

Do you have the research results? n <Enter>

Was initial research successful? y <Enter>

The program should terminate and return you to the command prompt. In the Battlemap,

· Select View Switch to Coverage Mode .

· Select File Preferences Testing .

· Ensure that the Testedness Indicator is set to Both .

· Close the Testing Preferences window by selecting Close.

· Select Project Testing Data Import .

· Click on the file inst.out and select Open.

List the modules that are:

Fully Path tested: _

Partially tested: _

Never tested:

The colors represented test coverage

Red : Untested

Yellow: Partially Tested

Green : Fully Path tested

!

Now examine the reports under the T esting menu item.

What was the overall Average % Code Coverage achieved?

What was the overall Average % Branch Coverage achieved?

· Select T esting Testing Data Load/Save.

· Select Save from the Load/Save Testing Data dialog.

· Type ‘Initial Test’ in the Title section of the dialog.

· Type an Author and Description as appropriate.

· Save the testing data by selecting Save.

Activity 2

Examine the module marketing to determine what testing has been performed as follows:

· Right click on the module marketing.

· Select the Sl ice… option from the popup menu.

Examine the resulting flowgraph & listing.

How many paths have been tested?

How many remain untested? ____

This decision is controlled by the first question of the execution. Determine an sequence of inputs that will fully test marketing. This will also cause the function support to be executed, ensure the test developed ONLY tests this function and NOT maintenance.

Response 1

Response 2

Response 3

Response 4

Response 5

Example

Y

Y

N

Y

Activity 2

What was new overall Average % Code Coverage achieved?

What was new overall Average % Branch Coverage achieved?

Activity 3

Description of End-to-End Testing Scenarios

McCabe Test is able to determine all possible execution paths for an application from “end-to-end”, i.e. all the possible ways that parent modules can call child modules.

This information can be displayed graphically by highlighting all the calling lines in the Battlemap in turn, or can be produced in a textual format.

McCabe Test can also take into account the testing already performed on a software system, and ‘subtract’ paths already performed from the theoretical complete set of end-to-end tests. The textual listing will contain paths (called subtrees) of the following format:

SUBTREE #1:

main > marketing > query > [printf] < query > [gets] < query < marketing

> development > query > [printf] < query > [gets] < query < development

> query > [printf] < query > [gets] < query < development < marketing

< main

END-TO-END TEST CONDITION LIST FOR SUBTREE #1:

marketing 22(2): purchase==1 ==> TRUE

development 37(2): new_features==1 ==> TRUE

development 41(4): research_results==1 ==> TRUE

Decision values required to be set to List describing modules called during

exercise this path. End-to-End test.

Line number and node number (in ‘>’ signifies a call to a module.

brackets) where this decision can be ‘<’ signifies a return from a module.

found in the code is written here. ‘[module]’ signifies library module.

Obtain the End-to-End subtree listing describing the remaining (Untested) subtrees for the program under test as follows:

· Select Testing Test Plan Integration Level

· Ensure Cyclomatic Detail Level is selected.

· Ensure Text Output Type is selected.

· Ensure Show Only Untested Subtrees is selected ON (a check in the box).

· Create the report by pressing OK.

You may also request a visual diagram of the effect of each of these subtrees as follows:

· Return to the Integration Level Test Plan dialog.

· Ensure that the Graph Output Type is selected.

· Press OK.

[ Note: The lines in the Battlemap are now highlighted to show ALL the calls performed in Subtree #1. You may advance through the subtrees by altering the Subtree Number slider in the Integration Level Test Plan dialog.

There are 7 subtrees in total. ]

Examine the End-to-End subtree textual report created above to determine the number of remaining (Untested) subtrees for the program under test.

How many more tests need to be performed to achieve full path testing? _

Using the pop up Sl ice option and the textual subtrees report generated above, design the remaining 5 tests and insert them into the table below.

The Example test case has been entered already, you should add the test case from Activity 2.

Response 1

Response 2

Response 3

Response 4

Response 5

Example

Y

Y

N

Y

Activity 2

Test 3

Test 4

Test 5

Test 6

Test 7

ONLY after completing the design of the tests, invoke the corp4 system again and execute the remaining tests (Test 1 to Test 7) on corp4 in turn.

Did all the test execute as expected? Yes / No

If not, import the test results and use the reports available under the Testing Test Plan Integration Level dialog to establish the remaining test.

What is the remaining test (if any)?

Activity 4

Using the regression test suite

developed in Activity 2, test a modified

version of the same system

?

Before commencing this exercise delete the inst.out file.

· Run the Corp4v2 program.

· Apply all of the tests from the table developed in Activity 3.

· Invoke the Corp4v2 Battlemap.

· Select Project Testing Data Import

Now examine the reports under the T esting menu item.

What was the overall % Code Coverage achieved? ___

Select View Switch to Coverage Mode .

Examine the Battlemap. Is the old regression suite sufficient to test the new version? Yes / No

If not, determine additional path required to test the system using the Testing Test Plan Integration Level menu

Is it possible to path test this version? Yes / No

TEST Exercise 7 - Test Slices

Objectives

To examine how to use the McCabe IQ tool to save individual test executions as slices and subsequently compare the difference between tests.

Activity 1

Save two test slices in the repository

?

· Delete the trace file for Corp4 by :

Windows - clicking on the icon named Corp4 Deltrace

UNIX - typing ‘./deltrace_corp4’

· Invoke the Corp4 Battlemap by :

Windows - clicking on the icon named Corp4

UNIX - typing ‘./start_corp4’

Select No

· Delete all previous testing data from corp4 by Project Testing Data Clear

· Invoke the Corp4 program by :

Windows - clicking on the icon named Corp4.exe

UNIX - typing ‘./run_corp4’

· Answer the questions asked by the program exactly as directed below:

Is this a purchase? y <Enter>

Do you need new features? y <Enter>

Do you have the research results? n <Enter>

Was initial research successful? y <Enter>

The program should terminate.

· Invoke the Corp4 Battlemap by :

Windows - clicking on the icon named Corp4

UNIX - typing ‘./start_corp4’

Select No

· Select View Switch to Coverage Mode .

· Select Preferences Testing…

· Ensure that the Testedness Indicator is set to Both .

· Close the Testing Preferences dialog by selecting Close.

· Select Project Testing Data Import .

Now save the testing coverage results as follows:

· Select T esting Load/Save Testing Data .

· Select Save… from the Load/Save Testing Data dialog.

· Type ‘Test 1’ in the Title section of the dialog.

· Type an Author and Description as appropriate.

· Save the testing data by selecting Save.

Repeat the above operation, including the removal of the trace file and internal trace data, using the following test set.

Is this a purchase? y <Enter>

Do you need new features? y <Enter>

Do you have the research results? n <Enter>

Was initial research successful? n <Enter>

Was further research successful? y <Enter>

Name this slice Test 2.

Now subtract the previous slice from this one by following these steps:

· Select Preferences Testing…

· Ensure that the Coverage Type is set to Slice .

· Close the Testing Preferences dialog by selecting Close.

[ Note: Slice manipulation can ONLY be viewed when in Slice coverage mode. The steps above ensure the VTT is showing slice coverage before continuing. ]

· Select Reengineering Slice Operations .

· Select Construct Slice from the Slice Operations dialog.

[ Note: This dialog allows existing test slices to be manipulated and combined using set theory. The symbol ‘~’ equates to Complement (NOT), and the symbol ‘^’ equates to Intersect. The following diagram describes the process of creating the subtracted slice below: ]

Test B

Test A

Test B

Test B

Test A

Test A

~(Test A) (Test B) (Test B) ^ ~(Test A)

Compliment of Test A Test B Intersect Complement of Test A

· Select the Test 1 Slice you saved by clicking on it.

· Click Complement.

· Click Save Expression to save this expression to the list.

· Select the Test 2 Slice by clicking on it.

· Click Intersect .

· Select the previously saved Expression with title ‘ ~(Test 1)’.

· Click Save Expression .

· Select this newly saved expression with title ‘ (Test 2) ^ (~(Test 1))’.

· Select Save Slice…

· Insert an appropriate Author and Description, leaving the title as it is.

· Save this slice by selecting Save.

Name of module	v(g)	ev(g)	Expected Maintenance Difficulty
jump_back
back_line
commands
edit
forw_line
forward
main
pr_string

Module Name	iv(g)
Maintenance
Revenue
Development
Query
Support
Marketing
Main

	Response 1	Response 2	Response 3	Response 4	Response 5
Example	Y	Y	N	Y
Activity 2

Condition	Value

Software testing Excercises, due within 12 hrs maximum

TABLE OF CONTENTS

Cyclomatic Complexity v(g) ________________

Identifying separate areas of the system

Activity 3 Create a report that lists cyclomatic & essential complexity, number of comment lines, and is categorized in terms of risk ?

Create a report that lists cyclomatic & essential complexity, number of comment lines, and is categorized in terms of risk ?

Activity 3

Activity 4

Activity 3

Create a report that lists cyclomatic & essential complexity, number of comment lines, and is categorized in terms of risk

?

Create a report that lists cyclomatic & essential complexity, number of comment lines, and is categorized in terms of risk

?