Algorithm design problem set

LE/EECS3101

Design and Analysis of Algorithms

Lower Bounds

Karim Jahed

Lower Bounds

For any algorithm for a given problem, and for each n > 0, there exists

an input that makes the algorithm takes Ω(f (n)). f (n) is then a lower

bound on the worst case running time.

• Reason about problems rather than specific algorithms.

• Prove a lower bound on the running time of a problem • i.e., a lower bound on any algorithm that solves the problem.

• We must not assume anything about how the algorithm works.

1

Lower Bounds - FindMax

Claim: Any comparison-based algorithm for finding the maximum of n

(n > 0) distinct elements must uses at least n − 1comparisons. Note that the number of comparisons is a lower bound on the running time of

the algorithm.

Proof: Only the winner does not lose.

• If x and y are compared, and x > y, call x the winner and y the loser.

• Any comparison produces exactly one loser.

• For n distinct elements, there is exactly one winner. Hence, there are n − 1 losers.

• Therefore, n − 1 comparisons have to be made.

Conclusion: A lower bound on the running time of any comparison

based algorithm for finding the maximum is Ω(n).

2

Lower Bounds - Observations

We proved a claim about any algorithm that only uses comparisons to

find the maximum. We made no assumptions about:

• The nature of the algorithm

• How the algorithm works (other than its comparison-based)

• The optimality of the algorithm

• Whether the algorithm is reasonable or not

3

Lower Bounds - Sorting

• Can we beat the Ω(n log n) lower bound for sorting? • In general, no. But, in some special cases were assumptions are

made about the input, YES!

• We will see linear time sorting in the next lecture.

• We will prove the Ω(n log n) lower bound for sorting.

4

Lower Bounds - Sorting (2)

• We will consider algorithms that uses the results of comparisons, not the value of elements.

• Does not make assumptions much about the type or value of the data being sorted.

• In this model, it is reasonable to count the number of comparisons. The number of comparisons is a lower bound on the running time of

the algorithm.

5

Lower Bounds - Sorting (3)

• Finding lower bounds for problems is rarely simple. There are no known general techniques we can use.

• We must try ad-hoc methods for each problem.

• Sorting lower bounds: • Trivial: Ω(n) - every element must be in a comparison. • Better: Ω(n log n) - we already know several O(n log n) algorithms. • How do we reason about all possible comparison-based sorting

algorithms?

6

The Decision Tree Model

Consider the problem of sorting the array A = {a, b, c} where a, b, and c are distinct elements. We can construct a decision tree that shows all the

result of all comparisons.

• Each internal node is one comparison • Each leaf is one possible ordering

7

The Decision Tree Model (2)

• The result of a sorting algorithm is a path from the root to one of the leaves. Therefore, the worst case running time is Θ(h) where h

is the height of the tree.

• The height of the tree is h = log k where k is the number of leaves.

• There are k = n! leaves (all permutations of A).

• Therefore, the running time is Θ(log n!) ∈ Θ(n log n). • Conclusion: A lower bound on any comparison-based sorting

algorithm is Ω(n log n).

8