White Paper for System testing
helpneeded2016IT355 Unit 3A Manual (Static) Testing Writing the White Paper
Spring 2016
Dr Diane Murphy
1/27/2016
1
1
Manual Testing
Finding Errors Early
Inspections
Walkthroughs
Code Inspections
Checklists
Desk Checking
Peer Ratings
Common Coding Errors
1/27/2016
2
Evolution of Software Testing
Original assumption that programs written solely for machine execution
Not intended for people other than programmer to read
Only way to test was to execute on a computer
Changed in the 1970s when programmers saw the value of someone else reading their code
Earlier detection of errors
More comprehensive testing
Made important in the 2000s with the advent of agile methodologies
Static testing
1/27/2016
3
Factors Affecting Code Reading as Part of Software Testing
Size or complexity of the application
Size of the development team
The timeline for applications development
Relaxed
Intense
Background and culture of the programming team
Acceptance of “informal” methods of testing
Pair programming in Agile methodologies
1/27/2016
4
Value of Static Testing
Quite effective in finding errors
Every development project should employ at least one of the human testing techniques
Pair programming (Agile)
Program inspections
Walkthroughs
Applied at the time the program is being coded and before computer-based testing begins
Finds errors early in the process
1/27/2016
5
Effectiveness of Static Testing
Programmers more likely to fix bugs found in the static testing process
Particularly when they have been involved in the review process
Peer pressure also important
Programmers tend to make more mistakes when correcting an error found in computer-based testing
1/27/2016
6
Pair Programming
Two developers work together at one workstation
The driver, writes code while the observer, pointer or navigator, reviews each line of code as it is typed in
The two developers switch roles frequently
While reviewing, the observer also considers the "strategic" direction of the work, coming up with ideas for improvements and likely future problems
This frees the driver to focus all of his or her attention on the "tactical" aspects of completing the current task, using the observer as a safety net and guide
Inspections and Walkthroughs
Two primary static testing methods in other SDLCs
Code inspections
Walkthroughs
Code-oriented methods can be used at virtually any stage in software development
As each unit or module is completed
After application is considered complete
Errors are located specifically in code
Easier to debug
Frequently exposes a batch of errors
1/27/2016
8
Common Characteristics of Inspections and Walkthroughs
Team of people reading or visually inspecting code
Participants examine the code independently and then meet to discuss at a participant’s conference
“Meeting of the minds”
Find errors but not solutions to the errors
Testing not debugging
1/27/2016
9
Value of Walkthroughs and Inspections
Find from 30% to 70% of the logic design and coding errors found in a typical program by the end of testing
Not effective in detecting high-level design errors
For example, errors in requirements documents
Typically find specific types of programming errors
Uninitialized variables
Divide by zero
Equally valuable in new code and in modifications to existing code
1/27/2016
10
Code Inspections
More formal approach with a set of procedures for error detection techniques for group code reading
Inspection team usually 4 people
Moderator
Programmer
Program Designer
Test Specialist
Must familiarize themselves with code before session
1/27/2016
11
Activities that Occur in the Session
Programmer narrates, statement by statement, the logic of the program
Other participants raise questions
Programmer may also find his own errors as he reads through the code
Program is analyzed with respect to checklists of historically common programming errors
Moderator responsible for ensuring discussions are productive and focus on finding errors, not correcting them
1/27/2016
12
At the End of the Session
Programmer is given a list of the errors uncovered
If many errors or substantial corrections required, moderator may call for a re-inspection
List of errors also used to refine the error checklist to improve the effectiveness of future inspections
Analyze errors
Categorize errors
Update error checklist
Results may be considered confidential to preserve integrity of process
1/27/2016
13
Potential Design Issues
Some teams may find that when a minor problem is discovered discussions may lead to a design change to handle the situation
In the discussion, additional problems may also be uncovered
Can result in a design change which affects other parts of the application
1/27/2016
14
The Error Checklist
Use of the error checklist is important part of inspection process
Examine code for common errors
Language independent
Usually categorized into specific areas:
Data Reference Errors
Data Declaration Errors
Comparison Errors
Control Flow Errors
Interface Errors
Input/output Errors
Other Checks
1/27/2016
15
Other Checks
Compiler tools can be used to check for variables that are never referenced or referenced only once
Need to verify any warning or information messages from the compiler as indicators of potential issues
Must also check if any functionality is missing from the program
1/27/2016
16
Walkthroughs
Set of procedures and error-detection techniques for group code reading
Slightly different from the inspection process
Uninterrupted meeting of 1-2 hours
Consists of 3 to 5 members
Moderator
Recorder
Tester
Others?
1/27/2016
17
Unique Characteristics of a Walkthrough
Performed by a group of developers
3 or 4 is optimal
One participant is the coder
Majority of testing performed by other developers
Good testing principle
Better than traditional process of desk checkers by coders themselves
1/27/2016
18
Other Members in Walkthroughs
Programmer with similar role on project
Highly experienced programmer
A programming-language expert
A new programmer
Person who will eventually maintain the code
Someone from a different project
Someone from the same programming team but different role
1/27/2016
19
Walkthrough Procedures
Participants are given materials several days in advance to review
Need to be familiar with code before meeting
Tester comes to meeting with a set of paper test cases
Each test case is mentally executed
“Walked through” the logic the program
Status of program monitored on a whiteboard
Focus on values in variables
1/27/2016
20
Desk Checking
Older techniques which is a one-person inspection or walk-through
Read program
Check with respect to an error list
Walk test data through it
For most people desk checking is relatively unproductive
Undisciplined coder often does testing
No synergism between team
More valuable than doing nothing but not as effective as code inspections or walkthroughs
1/27/2016
21
Peer Ratings
Technique of evaluation of anonymous programs in terms of:
Overall quality
Maintainability
Extensibility
Usability
Clarity
Purpose is programmer self-evaluation
Objective is not to find errors so is not technically a program testing technique
1/27/2016
22
After the Peer Ratings Review
Participants are given the anonymous evaluation forms for their two contributed programs
Also given a statistical summary showing the overall and detailed ranking of their original programs
Purpose is to allow programmer to assess their programming skills and those of their peers
Useful in both commercial and classroom environments
1/27/2016
23
Code Reviews at Google
"All code that gets submitted needs to be reviewed by at least one other person, and either the code writer or the reviewer needs to have readability in that language. Most people use Mondrian to do code reviews, and obviously, we spend a good chunk of our time reviewing code."
-- Amanda Camp, Software Engineer, Google
1/27/2016
24
Code Reviews at Yelp
“At Yelp we use review-board. An engineer works on a branch and commits the code to their own branch. The reviewer then goes through the diff, adds inline comments on review board and sends them back. The reviews are meant to be a dialogue, so typically comment threads result from the feedback. Once the reviewer's questions and concerns are all addressed they'll click "Ship It!" and the author will merge it with the main branch for deployment the same day.”
-- Alan Fineberg, Software Engineer, Yelp
1/27/2016
25
Code Reviews at Facebook
"At Facebook, we have an internally-developed web-based tool to aid the code review process. Once an engineer has prepared a change, she submits it to this tool, which will notify the person or people she has asked to review the change, along with others that may be interested in the change -- such as people who have worked on a function that got changed. At this point, the reviewers can make comments, ask questions, request changes, or accept the changes. If changes are requested, the submitter must submit a new version of the change to be reviewed. All versions submitted are retained, so reviewers can compare the change to the original, or just changes from the last version they reviewed. Once a change has been submitted, the engineer can merge her change into the main source tree for deployment to the site during the next weekly push, or earlier if the change warrants quicker release."
- Ryan McElroy, Software Engineer, Facebook
1/27/2016
26
Rationale for Reviews
Can catch most bugs, design flaws early
More than 1 person has seen every piece of code
– Insurance against author’s disappearance
– Accountability (both author and reviewers are accountable)
Forcing documentation and code improvements
Authors to articulate their decisions
Authors participate in the discovery of flaws
Prospect of your code review raises quality threshold
Inexperienced personnel get hands-on experience
Pairing them up with experienced developers
Can learn by being a reviewer as well
Explicit non-purpose:
Assessment of individuals for promotion, pay, ranking, etc.
1/27/2016
27
Common Error Types
Data Reference Errors
Data Declaration Errors
Comparison Errors
Control Flow Errors
Interface Errors
Input/output Errors
Syntax Errors
1/27/2016
28
Data Reference Errors
Uninitialized variables are the most frequent programming errors
Arrays also present common problems:
Subscript value must remain within range and must usually be an integer
If a data structure is referenced in multiple procedures, it must always be defined in the same way
1/27/2016
29
1. Example of Data Reference Error
C language example:
void count( void )
{
int k, i;
for (i = 0; i < 10; i++)
{
k = k + 1;
}
printf("%d", k);
}
1/27/2016
30
Data Declaration Errors
Variables should be explicitly declared
Not necessarily an error in some programming languages, but good programming practice
Variables should be checked to make sure that they are the correct size and data type
Variables with similar names should be checked since they often result in errors or misuse
Capitalization common problems
1/27/2016
31
2. Example Data Declaration Error
Javascript Example
<SCRIPT>
function SumDigits(num) { var i, sum = 0;
for (i = 1; i <= num; i++) { Sum += i; }
alert("The sum of the digits from 1 to "+ num + " is:\n\n\t " + sum); }
</SCRIPT>
32
3. Computational Errors
Most common error is doing computations with non-numeric data types
Size of variable can also produce errors if target variable is not large enough for the result
Divide by zero is another common computational error
Variables within limited ranges should be checked and values outside that range noted as errors
1/27/2016
33
Example of Java with Code to Catch
public class MainClass {
public static void main(String args[]) {
int d, a;
try { d = 0;
a = 42 / d;
System.out.println("This will not be printed.");
} catch (ArithmeticException e) { //
System.out.println("Division by zero.");
}
System.out.println("After catch statement.");
}
}
1/27/2016
34
Comparison Errors
Comparisons must be made with variables of the same data type
Comparison operators must be used correctly for requirements, often confusion between greater than and less than
For complex operators, such as in Boolean logic, the order of operations is significant
Positioning of parentheses is a common error
1/27/2016
35
4. Example of Comparison Error
if($finalg > 91)
{
$finalletter = "A";
}
else if($finalg > 81)
{
$finalletter = "B";
}
else if($finalg > 71)
{
$finalletter = "C";
}
else
{
$finalletter = "F";
}
Requirements:
A is 91 to 100
B is 81 to 90 et
1/27/2016
36
Control Flow Errors
Use of loops is often an error
They must terminate
They must execute
Too many or too few loops may also result from inappropriate coding of different types of loops
Logic must cover all possible conditions and not make erroneous decisions
1/27/2016
37
5. Example of Control Flow Error
while (theMark == true)
{
alert("Hello, world");
}
1/27/2016
38
Interface Errors
Today’s programming environment involves a modular approach where different pieces of code must interface with code that may or may not be written by the same coder
Parameters must match between modules
Number of parameters
Data type and size of each
Programmer must understand the module interface requirements and send and receive parameters appropriately
1/27/2016
39
6. Example of Interface Error
function addTaxes (amount, rate) ( tax = amount*rate; return tax; }
The function is called in the code:
<button onclick = “addTaxes (amount):”>Calculate</button>
1/27/2016
40
Input/Output Errors
Attributes must be correct for any files being used
Files must be opened and closed appropriately
Input/output errors must be handled in the program
Spelling, grammar, and clarity must be considered in all text messages displayed by the program
1/27/2016
41
7. Example of Input/Output Error
1/27/2016
42
8. Examples of Syntax Error
result = (firstVal - secondVal / factor;
double x = 2.0, y = 3.1415, product;
7 x * y = product;
int RoundFloat (float floatToRound) {
int roundedValue;
roundedValue = int (floatToRound + 0.5); }
1/27/2016
43
Exercise: Unit 3
Detecting and Classifying
Look at examples on worksheet, classify errors
Think how you might test
Submit results to Unit 3
1/27/2016
44
Assignment 2 Requirements
Each of you has been assigned a testing technique to prepare a white paper
What am I looking for:
Follow my instructions (check before you post)
Marketing document: looking for style (images, format, etc.)
Content about your test technique and when it will be used
Meet the deadline
Questions?
1/27/2016
45
What is a White paper?
Authoritative report or guide helping readers understand an issue, solve a problem, or make a decision
Since the early 1990s, the term applied to documents used as marketing or sales tools in business
Use selected facts and logical arguments to build a case favorable to the organization sponsoring the document
“Persuasive writing”
1/27/2016
46
What is Persuasive Writing?
Definition: persuasive writing…
seeks to convince its readers to embrace the point-of-view presented by appealing to the audience’s reason and understanding through argument and/or entreaty
47
47
First, let’s establish what we mean by persuasive writing.
Persuasive Genres
You encounter persuasion every day.
TV Commercials
Letters to the Editor
Junk mail
Magazine ads
Blogs
Can you think of other persuasive contexts?
48
48
Steps for Effective Persuasion
Understand your audience
Support your opinion
Know the various sides of your issue
Respectfully address other points of view
Find common ground with your audience
Establish your credibility
49
49
When to Persuade an Audience
Examples from the workplace
Your organization needs funding for a project
Your boss wants you to make recommendations
You need to shift someone’s current point of view to build common ground so action can be taken
You are asked to write a white paper which “markets” items offered by the company
Product
Services
Techniques
50
50
Researching an Issue
Become familiar with all sides of an issue.
-find common ground
-understand the history of the topic
-predict the counterarguments your
audience might make
-find strong support for your own
perspective
51
51
Researching an Issue
Find common ground with your audience
For example:
Point of Opposition: You might support open-source, whereas your audience might not
Common ground: Both sides want to minimize the cost of the product
52
52
.
Researching an Issue
Predict counterarguments
Example:
Your Argument: Open-source is available for free
The Opposition: There is no support for open-source and so we have to absorb all the costs for training and support
53
53
Support Your Perspective
Appeal to the audience’s reason
Use statistics and reputable studies
Cite experts on the topic
Do they back up what you say?
Do they refute the other side?
54
54
Cite Sources with Some Authority
Which source would a reader find more credible?
The New York Times
http://www.myopinion.com
Which person would a reader be more likely to believe?
Joe Smith from Fort Wayne, IN
Dr. Susan Worth, Prof. of Computer Science at Purdue University
55
55
Establish Credibility
Cite credible sources
Cite sources correctly and thoroughly
Use professional language and ensure there are no grammatical errors
Use good design and layout
Edit out all errors
56
56
Assignment 2
Write a white paper for the “testing method” listed next to your name
Need to research the testing technique and other competing techniques
Establish the credibility of your statements
Need to follow the format described in the assignment
Assignment 2 DRAFT due Feb 3, 2016
1/27/2016
57
Quiz 2
SDLC and testing (Waterfall, Incremental, RUP, Spiral, Agile)
Purpose of testing
Uniqueness of mobile app testing
Developers doing their own testing
Creativity behind testing
Characteristics of a successful test
1/27/2016
58