Algorithm design problem set

LE/EECS3101

Design and Analysis of Algorithms

Divide and Conquer I

Karim Jahed

Divide and Conquer

• Divide: the problem into smaller instances to the same problem.

• Conquer: solve the smaller instances recursively.

• Combine: glue the answers together to obtain the answer to the larger instance.

1

Divide and Conquer - FindMax

• Divide: the problem into smaller instances to the same problem. • What is an instance of the problem that we know how to trivially

solve?

• How should we split the input array A[1..n] into smaller instances?

• Conquer: solve the smaller instances recursively. • What is the solution to the smallest instances?

• Combine: glue the answers together to obtain the answer to the larger instance.

• How ca we combine the smaller instances into a larger one?

2

Divide and Conquer - FindMax

• Divide: the problem into smaller instances to the same problem. • What is an instance of the problem that we know how to trivially

solve?

• How should we split the input array A[1..n] into smaller instances?

• Conquer: solve the smaller instances recursively. • What is the solution to the smallest instances?

• Combine: glue the answers together to obtain the answer to the larger instance.

• How ca we combine the smaller instances into a larger one?

2

Divide and Conquer - FindMax

• Divide: the problem into smaller instances to the same problem. • What is an instance of the problem that we know how to trivially

solve?

• How should we split the input array A[1..n] into smaller instances?

• Conquer: solve the smaller instances recursively. • What is the solution to the smallest instances?

• Combine: glue the answers together to obtain the answer to the larger instance.

• How ca we combine the smaller instances into a larger one?

2

Divide and Conquer - FindMax (2)

1 fun f i n d M a x ( A [ 1 . . n ] , l , h ) :

2 i f l >= h :

3 return A [ l ]

4 end

5

6 mid = ( l + h ) / 2

7 max1 = f i n d M a x ( A , l , mid )

8 max2 = f i n d M a x ( A , mid + 1 , h )

9 return max( max1 , max2 )

10 postcond returns the maximum element in A[l..h]

• Smallest problem: l >= h, i.e., a sub-array of one element. • Divide by splitting the array into two sub-arrays at the middle. • Conquer by recursively calling findMax on each sub-array. • Combine by taking the maximum of the result of the two recursive

calls.

• What is the running time of findMax?

3

Divide and Conquer - FindMax (2)

1 fun f i n d M a x ( A [ 1 . . n ] , l , h ) :

2 i f l >= h :

3 return A [ l ]

4 end

5

6 mid = ( l + h ) / 2

7 max1 = f i n d M a x ( A , l , mid )

8 max2 = f i n d M a x ( A , mid + 1 , h )

9 return max( max1 , max2 )

10 postcond returns the maximum element in A[l..h]

• Smallest problem: l >= h, i.e., a sub-array of one element. • Divide by splitting the array into two sub-arrays at the middle. • Conquer by recursively calling findMax on each sub-array. • Combine by taking the maximum of the result of the two recursive

calls.

• What is the running time of findMax?

3

Divide and Conquer - FindMax (2)

1 fun f i n d M a x ( A [ 1 . . n ] , l , h ) :

2 i f l >= h :

3 return A [ l ]

4 end

5

6 mid = ( l + h ) / 2

7 max1 = f i n d M a x ( A , l , mid )

8 max2 = f i n d M a x ( A , mid + 1 , h )

9 return max( max1 , max2 )

10 postcond returns the maximum element in A[l..h]

• Smallest problem: l >= h, i.e., a sub-array of one element. • Divide by splitting the array into two sub-arrays at the middle. • Conquer by recursively calling findMax on each sub-array. • Combine by taking the maximum of the result of the two recursive

calls.

• What is the running time of findMax? 3

Runtime Analysis - FindMax

How many comparisons do we need? We can try and example and draw

the recursion tree:

4

Runtime Analysis - FindMax (2)

Let T (n) be the number of comparison needed by FindMax to find the

maximum of n elements. We get the following recurrence relation:

T (n) =

{ c1 n = 1

2T (n/2) + c2 n > 1

Or,

T (n) =

{ Θ(1) n = 1

2T (n/2) + Θ(1) n > 1

5

Solving recurrences - The substitution method

We can solve T (n) using the substitution method. First, we keep on

substituting n until we find a pattern.

T (n) = 2T (n/2) + 1

= 2(2T (n/22) + 1) + 1

= 2(2(2T (n/23) + 1) + 1) + 1

= ...

= 2T (n/2) + 1

= 22T (n/22) + 21 + 1

= 23T (n/23) + 22 + 21 + 20

= ...

= 2iT (n/2i ) + i−1∑ j=0

2j

= 2iT (n/2i ) + 2i − 1

6

Solving recurrences - The substitution method (2)

Next, we figure out how many substitutions are needed to get to the base

case T (1)

T (n) = 2iT (n/2i ) + 2i − 1

We basically want to replace T (n/2i ) with T(1), i.e., we need n 2i

= 1

n

2i = 1 =⇒ n = 2i =⇒ log n = i

Let’s substitute i:

2log nT (1) + 2log n − 1

7

Solving recurrences - The substitution method (3)

What is 2log n?

2log n = nlog2 2 = n

Finally, we get:

T (n) = nT (1) + n − 1 = 2n − 1 ∈ Θ(n)

8

Divide and Conquer - MergeSort

1 fun m e r g e S o r t (A [ 1 . . n ] , l , h ) :

2 i f l >= h :

3 return

4 end

5

6 mid = ( l + h ) / 2

7 m e r g e S o r t ( A , l , mid )

8 m e r g e S o r t ( A , mid + 1 , h )

9 m e r g e ( A , l , mid , h )

10 postcond A[l..h] is sorted

9

Divide and Conquer - MergeSort

1 fun m e r g e S o r t (A [ 1 . . n ] , l , h ) :

2 i f l >= h :

3 return

4 end

5

6 mid = ( l + h ) / 2

7 m e r g e S o r t ( A , l , mid )

8 m e r g e S o r t ( A , mid + 1 , h )

9 m e r g e ( A , l , mid , h )

10 postcond A[l..h] is sorted

• Smallest problem: l >= h, i.e., a sub-array of one element is trivially sorted.

• Divide by splitting the array into two sub-arrays at the middle. • Conquer by recursively calling MergeSort to sort each sub-array. • Combine by merging the two (now) sorted sub-arrays into a larger

sorted sub-array. 9

Divide and Conquer - MergeSort

1 fun m e r g e S o r t (A [ 1 . . n ] , l , h ) :

2 i f l >= h :

3 return

4 end

5

6 mid = ( l + h ) / 2

7 m e r g e S o r t ( A , l , mid )

8 m e r g e S o r t ( A , mid + 1 , h )

9 m e r g e ( A , l , mid , h )

10 postcond A[l..h] is sorted

• Smallest problem: l >= h, i.e., a sub-array of one element is trivially sorted.

• Divide by splitting the array into two sub-arrays at the middle. • Conquer by recursively calling MergeSort to sort each sub-array. • Combine by merging the two (now) sorted sub-arrays into a larger

sorted sub-array.

• What is the running time of mergeSort? 9

Runtime Analysis - Merge

1 fun m e r g e ( A [ 1 . . n ] , l , m, h )

2 s = (m− l +1) + ( h−m) 3 C = empty a r r a y o f s i z e s

4 i = l , j = m+1

5 for k = 1 to s

6 # e d g e c a s e s o m i t t e d

7 i f A [ i ] <= B [ j ]

8 C [ k ] = A [ i ++]

9 e l s e

10 C [ k ] = B [ j ++]

11 end

12 i = l

13 for k = 1 to s

14 A [ i ++] = C [ k ]

15 end

• What is the running time of merge?

10

Runtime Analysis - Merge

1 fun m e r g e ( A [ 1 . . n ] , l , m, h )

2 s = (m− l +1) + ( h−m) 3 C = empty a r r a y o f s i z e s

4 i = l , j = m+1

5 for k = 1 to s

6 # e d g e c a s e s o m i t t e d

7 i f A [ i ] <= B [ j ]

8 C [ k ] = A [ i ++]

9 e l s e

10 C [ k ] = B [ j ++]

11 end

12 i = l

13 for k = 1 to s

14 A [ i ++] = C [ k ]

15 end

• What is the running time of merge?

• In each iteration, an element in A[l..h] is merged

into C.

• Merge takes linear time, i.e., T (n) ∈ Θ(n) (where n = h − l + 1)

10

Runtime Analysis - MergeSort

Sometimes, we can find the bound by simply inspecting the recursion tree

11

Runtime Analysis - MergeSort

Sometimes, we can find the bound by simply inspecting the recursion tree

11

Runtime Analysis - MergeSort

We can define the recurrence relation for MergeSort as:

T (n) =

{ Θ(1) n = 1

2T (n/2) + Θ(n) n > 1

12

Runtime Analysis - MergeSort (2)

Solving the recurrence by substitution gives:

T (n) = 2T (n/2) + n

= 2(2T (n/22) + n) + n

= 22T (n/22) + 2n

= 23T (n/23) + 3n

= ...

= 2iT (n/2i ) + i.n

= 2log nT (1) + log n.n

= n log n + n

T (n) ∈ Θ(n log n)

13