Algorithm design problem set

LE/EECS3101

Design and Analysis of Algorithms

Iterative Algorithms & Loop Invariants

Karim Jahed

Iterative algorithms - Basic Structure

1 loop u n t i l <c o n d i t i o n >

2 <do s o m e t h i n g >

3 end

• Many iterative algorithms follow the same basic structure

• Store key data about the computation in some data

structure

• At each iteration, take one step by modifying the data

• Each step takes you closer to the destination and exit

condition

1

Iterative Algorithms - FindMax()

input: an array A[1..n] of integers

output: m such that m ∈ A and m ≥ A[i] for all 1 ≤ i ≤ n

1 max = A [ 1 ] , i = 2

2 loop u n t i l i > n

3 i f max < A [ i ] then

4 max = A [ i ]

5 i = i + 1

6 end

7 end

8 return max

How can we tell the algorithm is correct?

• It is not!

2

Iterative Algorithms - FindMax()

input: an array A[1..n] of integers

output: m such that m ∈ A and m ≥ A[i] for all 1 ≤ i ≤ n

1 max = A [ 1 ] , i = 2

2 loop u n t i l i > n

3 i f max < A [ i ] then

4 max = A [ i ]

5 i = i + 1

6 end

7 end

8 return max

How can we tell the algorithm is correct?

• It is not!

2

Iterative Algorithms - FindMax()

input: an array A[1..n] of integers

output: m such that m ∈ A and m ≥ A[i] for all 1 ≤ i ≤ n

1 max = A [ 1 ] , i = 2

2 loop u n t i l i > n

3 i f max < A [ i ] then

4 max = A [ i ]

5 ((( (

i = i + 1

6 end

7 i = i + 1

8 end

9 return max

How can we tell the algorithm is correct?

• It is not!

2

Correctness - Abstraction levels

• Prove that a concrete algorithm, e.g., FindMax(), works on a specific input, e.g., A = [53, 10, 69, 1]

• Prove that a concrete algorithm, e.g., FindMax(), works on an arbitrary input, e.g., A = [1..n]

• Prove that a ”meta algorithm”, e.g., iterative algorithms, work on an arbitrary input

• This course teaches you the latter. We will develop notations and techniques to prove the correctness of meta-algorithms

3

Correctness - Abstraction levels

• Prove that a concrete algorithm, e.g., FindMax(), works on a specific input, e.g., A = [53, 10, 69, 1]

• Prove that a concrete algorithm, e.g., FindMax(), works on an arbitrary input, e.g., A = [1..n]

• Prove that a ”meta algorithm”, e.g., iterative algorithms, work on an arbitrary input

• This course teaches you the latter. We will develop notations and techniques to prove the correctness of meta-algorithms

3

Correctness - Abstraction levels

• Prove that a concrete algorithm, e.g., FindMax(), works on a specific input, e.g., A = [53, 10, 69, 1]

• Prove that a concrete algorithm, e.g., FindMax(), works on an arbitrary input, e.g., A = [1..n]

• Prove that a ”meta algorithm”, e.g., iterative algorithms, work on an arbitrary input

• This course teaches you the latter. We will develop notations and techniques to prove the correctness of meta-algorithms

3

Loop invariants

• A loop invariant is a statement about the state of the computation • It keeps track of the progress of the computation and makes sure the

computation never gets lost

4

Loop invariants - FindMax()

1 max = A [ 1 ] , i = 2

2 loop u n t i l i > n

3 i n v a r i a n t ‘max is the maximum in A[1..i-1]’

4 i f max < A [ i ] then

5 max = A [ i ]

6 end

7 i = i + 1

8 end

9 return max

• We can now argue if invarianti ∧code =⇒ invarianti+1

• Notice the similarities with induction!

• But what about the correctness of the invariant before the first iteration?

5

Loop invariants - FindMax()

1 max = A [ 1 ] , i = 2

2 loop u n t i l i > n

3 i n v a r i a n t ‘max is the maximum in A[1..i-1]’

4 i f max < A [ i ] then

5 max = A [ i ]

6 end

7 i = i + 1

8 end

9 return max

• We can now argue if invarianti ∧code =⇒ invarianti+1

• Notice the similarities with induction!

• But what about the correctness of the invariant before the first iteration?

5

Loop invariants - FindMax()

1 max = A [ 1 ] , i = 2

2 loop u n t i l i > n

3 i n v a r i a n t ‘max is the maximum in A[1..i-1]’

4 i f max < A [ i ] then

5 max = A [ i ]

6 end

7 i = i + 1

8 end

9 return max

• We can now argue if invarianti ∧code =⇒ invarianti+1

• Notice the similarities with induction!

• But what about the correctness of the invariant before the first iteration?

5

Precondition

1 max = A [ 1 ] , i = 2

2 precond ‘max is the first element of A’

3 loop u n t i l i > n

4 i n v a r i a n t ‘max is the maximum in A[1..i-1]’

5 i f max < A [ i ] then

6 max = A [ i ]

7 end

8 i = i + 1

9 end

10 return max

• A precondition is an assertion about the state of the computation before the loop

• The validity of the loop invariant depends on the truth value of the precondition, i.e., precond =⇒ invariant1. In other words, the precondition establishes the loop invariant.

• How about the correctness of the entire program? How do we define correctness? 6

Postcondition

1 max = A [ 1 ] , i = 2

2 precond ‘max is the first element of A’

3 loop u n t i l i > n

4 i n v a r i a n t ‘max is the maximum in A[1..i-1]’

5 i f max < A [ i ] then

6 max = A [ i ]

7 end

8 i = i + 1

9 end

10 postcond ‘max is the maximum in A[1..n]’

11 return max

• A postcondition is an assertion about the state of the computation after the loop

• We can now define correctness of the program as: precond∧code =⇒ postcond

• i.e., if the precondition holds then the postcondition must also hold.

7

Proof of the Loop Invariant - Partial Correctness

LI: ‘At the beginning of the ith iteration, max contains the maximum of

A[1..i-1]’

• Initialization: the precondition asserts that max contains A[1] so LI(1) is true.

• Maintenance: For i > 2, assume LI(i-1) is true, i.e., max = maximum element in A[1..i-2]. We have two cases:

1. A[i-1] is the maximum in A[1..i-1]. In that case, max is updated to

A[i-1] (line 6).

2. A[i-1] is not the maximum in A[1..i-1]. The maximum is therefore in

A[1..i-2]. By our assumption, max already has this value and max

remains unchanged in this iteration.

8

Proof of the Loop Invariant - Termination

• The correctness of the program requires that the program terminates, i.e., partial correctness ∧ termination =⇒ correctness

• Termination: when the loop terminates, i = n + 1. Therefore, by the proof of the loop invariant maintenance, max contains the

maximum of A[1..n] (i.e., the postcondition!).

9

Proof of Loop Invariants - Summary

We must prove three things about loop invariants to prove the

correctness of iterative algorithms:

• Initialization - given the precondition the loop invariant is true before the first iteration

• Maintenance - if the loop invariant is true before this iteration it remains true before the next iteration

• Termination - when the loop terminates, the invariant implies the postcondition

10