Algorithm design problem set
LE/EECS3101
Design and Analysis of Algorithms
Iterative Algorithms & Loop Invariants
Karim Jahed
Iterative algorithms - Basic Structure
1 loop u n t i l <c o n d i t i o n >
2 <do s o m e t h i n g >
3 end
• Many iterative algorithms follow the same basic structure
• Store key data about the computation in some data
structure
• At each iteration, take one step by modifying the data
• Each step takes you closer to the destination and exit
condition
1
Iterative Algorithms - FindMax()
input: an array A[1..n] of integers
output: m such that m ∈ A and m ≥ A[i] for all 1 ≤ i ≤ n
1 max = A [ 1 ] , i = 2
2 loop u n t i l i > n
3 i f max < A [ i ] then
4 max = A [ i ]
5 i = i + 1
6 end
7 end
8 return max
How can we tell the algorithm is correct?
• It is not!
2
Iterative Algorithms - FindMax()
input: an array A[1..n] of integers
output: m such that m ∈ A and m ≥ A[i] for all 1 ≤ i ≤ n
1 max = A [ 1 ] , i = 2
2 loop u n t i l i > n
3 i f max < A [ i ] then
4 max = A [ i ]
5 i = i + 1
6 end
7 end
8 return max
How can we tell the algorithm is correct?
• It is not!
2
Iterative Algorithms - FindMax()
input: an array A[1..n] of integers
output: m such that m ∈ A and m ≥ A[i] for all 1 ≤ i ≤ n
1 max = A [ 1 ] , i = 2
2 loop u n t i l i > n
3 i f max < A [ i ] then
4 max = A [ i ]
5 ((( (
i = i + 1
6 end
7 i = i + 1
8 end
9 return max
How can we tell the algorithm is correct?
• It is not!
2
Correctness - Abstraction levels
• Prove that a concrete algorithm, e.g., FindMax(), works on a specific input, e.g., A = [53, 10, 69, 1]
• Prove that a concrete algorithm, e.g., FindMax(), works on an arbitrary input, e.g., A = [1..n]
• Prove that a ”meta algorithm”, e.g., iterative algorithms, work on an arbitrary input
• This course teaches you the latter. We will develop notations and techniques to prove the correctness of meta-algorithms
3
Correctness - Abstraction levels
• Prove that a concrete algorithm, e.g., FindMax(), works on a specific input, e.g., A = [53, 10, 69, 1]
• Prove that a concrete algorithm, e.g., FindMax(), works on an arbitrary input, e.g., A = [1..n]
• Prove that a ”meta algorithm”, e.g., iterative algorithms, work on an arbitrary input
• This course teaches you the latter. We will develop notations and techniques to prove the correctness of meta-algorithms
3
Correctness - Abstraction levels
• Prove that a concrete algorithm, e.g., FindMax(), works on a specific input, e.g., A = [53, 10, 69, 1]
• Prove that a concrete algorithm, e.g., FindMax(), works on an arbitrary input, e.g., A = [1..n]
• Prove that a ”meta algorithm”, e.g., iterative algorithms, work on an arbitrary input
• This course teaches you the latter. We will develop notations and techniques to prove the correctness of meta-algorithms
3
Loop invariants
• A loop invariant is a statement about the state of the computation • It keeps track of the progress of the computation and makes sure the
computation never gets lost
4
Loop invariants - FindMax()
1 max = A [ 1 ] , i = 2
2 loop u n t i l i > n
3 i n v a r i a n t ‘max is the maximum in A[1..i-1]’
4 i f max < A [ i ] then
5 max = A [ i ]
6 end
7 i = i + 1
8 end
9 return max
• We can now argue if invarianti ∧code =⇒ invarianti+1
• Notice the similarities with induction!
• But what about the correctness of the invariant before the first iteration?
5
Loop invariants - FindMax()
1 max = A [ 1 ] , i = 2
2 loop u n t i l i > n
3 i n v a r i a n t ‘max is the maximum in A[1..i-1]’
4 i f max < A [ i ] then
5 max = A [ i ]
6 end
7 i = i + 1
8 end
9 return max
• We can now argue if invarianti ∧code =⇒ invarianti+1
• Notice the similarities with induction!
• But what about the correctness of the invariant before the first iteration?
5
Loop invariants - FindMax()
1 max = A [ 1 ] , i = 2
2 loop u n t i l i > n
3 i n v a r i a n t ‘max is the maximum in A[1..i-1]’
4 i f max < A [ i ] then
5 max = A [ i ]
6 end
7 i = i + 1
8 end
9 return max
• We can now argue if invarianti ∧code =⇒ invarianti+1
• Notice the similarities with induction!
• But what about the correctness of the invariant before the first iteration?
5
Precondition
1 max = A [ 1 ] , i = 2
2 precond ‘max is the first element of A’
3 loop u n t i l i > n
4 i n v a r i a n t ‘max is the maximum in A[1..i-1]’
5 i f max < A [ i ] then
6 max = A [ i ]
7 end
8 i = i + 1
9 end
10 return max
• A precondition is an assertion about the state of the computation before the loop
• The validity of the loop invariant depends on the truth value of the precondition, i.e., precond =⇒ invariant1. In other words, the precondition establishes the loop invariant.
• How about the correctness of the entire program? How do we define correctness? 6
Postcondition
1 max = A [ 1 ] , i = 2
2 precond ‘max is the first element of A’
3 loop u n t i l i > n
4 i n v a r i a n t ‘max is the maximum in A[1..i-1]’
5 i f max < A [ i ] then
6 max = A [ i ]
7 end
8 i = i + 1
9 end
10 postcond ‘max is the maximum in A[1..n]’
11 return max
• A postcondition is an assertion about the state of the computation after the loop
• We can now define correctness of the program as: precond∧code =⇒ postcond
• i.e., if the precondition holds then the postcondition must also hold.
7
Proof of the Loop Invariant - Partial Correctness
LI: ‘At the beginning of the ith iteration, max contains the maximum of
A[1..i-1]’
• Initialization: the precondition asserts that max contains A[1] so LI(1) is true.
• Maintenance: For i > 2, assume LI(i-1) is true, i.e., max = maximum element in A[1..i-2]. We have two cases:
1. A[i-1] is the maximum in A[1..i-1]. In that case, max is updated to
A[i-1] (line 6).
2. A[i-1] is not the maximum in A[1..i-1]. The maximum is therefore in
A[1..i-2]. By our assumption, max already has this value and max
remains unchanged in this iteration.
8
Proof of the Loop Invariant - Termination
• The correctness of the program requires that the program terminates, i.e., partial correctness ∧ termination =⇒ correctness
• Termination: when the loop terminates, i = n + 1. Therefore, by the proof of the loop invariant maintenance, max contains the
maximum of A[1..n] (i.e., the postcondition!).
9
Proof of Loop Invariants - Summary
We must prove three things about loop invariants to prove the
correctness of iterative algorithms:
• Initialization - given the precondition the loop invariant is true before the first iteration
• Maintenance - if the loop invariant is true before this iteration it remains true before the next iteration
• Termination - when the loop terminates, the invariant implies the postcondition
10