Divide and Conquer - Thu, January 21

Review Example

image-20210121091846451

Master Theorem for Recurrences

In general, , where . The Master Theorem states the following:

We need to bound our generic total-work formula, in the three cases above. Recall that for a power series, .

Case 1: Root-Heavy

If , then , because all the other terms are constants.

Case 2: Balanced

If , then .

Case 3: Leaf-Heavy

If , then .

Matrix Multiplication

We'll use input matrices and of size x each. In general, matrix multiplication is not commutative.

The Naive Algorithm

For i = 1, ..., n:      n times
    For j = 1, ..., n:	n times
        compute Z_ij	O(n)

This algorithm runs in , which is pretty bad.

The Recursive Algorithm

We can leverage blockwise multiplication.

Here, , because there are eight multiplications and the recombination cost (summing two x matrices takes time) is . Applying the third case of Master Theorem will tell us that our new algorithm is... also .

The Strassen Algorithm

image-20210121100820910

Beyond Strassen

Suppose you figure out how to multiply x matrices in multiplications. Then, via divide and conquer, you get an algorithm in time. This boils down to . We can hence "lift" any finite-size improvement into an asymptotic improvement.

Currently, the fastest matrix multiplication algorithm runs in .

Recap of Examples

We have now seen