Fast Fourier Transform - Thu, Jan 28

We have applied divide and conquer to arithmetic problems:

Today we'll study faster polynomial multiplication (). The key tool underlying this huge speedup is the Fast Fourier Transform (FFT), which has numerous applications across all sciences and engineering.

Polynomial Multiplication

We are given two polynomials:

We want to output coefficients of polynomial

The naive algorithm follows the definition of polynomial multiplication.

The algorithm itself, then, is:

For i = 0, 1, ..., 2d:
    compute C_i := \sum_{k=0}^i a_k * b_{i-k}

This runs in time ( iterations of an sum ).

We can say that any polynomial multiplication algorithm must run in time. But, can we do better than ?

Observation: the multiplication of polynomial evaluations is easy -- for every , is obtained in one multiplication of and .

Idea: evaluate , at many points, then pointwise multiply, then interpolate to get .

Interlude: Representations of Polynomials

image-20210128092453475

Where the last equation is a linear system of equations in variables .

Back to Polynomial Multiplication by Evaluation & Interpolation

We want to multiply (, ).

  1. Choose domain with
  2. Compute
  3. Compute
  4. For , compute , which uniquely determines . This step takes time.
  5. Compute

The running time here is .

The Fast Fourier Transform enables us to do Eval/Interp in operations, which leads to Polynomial Multiplication in operations. Today, we will only focus on Polynomial Evaluation.

The naive algorithm is:

For u \in S:
    compute A(u) := \sum_{i=0}^d a_i * u^i

Which uses operations, which is when . The most important issue now is evaluating polynomials faster. How?

Idea: make a clever choice os to support a divide and conquer approach.

View polynomial as even and odd powers: .

To compute on , it suffices to compute , on and then reverse. Let's choose so that has size .

We can use roots of unity for the set .

Interlude: Roots of Unity

The -th roots of unity are all the solutions to the equation . These are .

Recall that , so . This means that and have the same square. The polar coordinates of on the Imaginary-Real plane are .

We can use roots of unity to create a set such that . Let's fix to be a power of 2, and let . Then, and so on, because and both square to .

Note: .

Back to FFT: Evaluation

Our input is a list of coefficients ( power of 2), and our output is where .

  1. ,
  2. For , and
  3. Output

The running time is .

This is the evaluation part of Fast Fourier Transform. The lecture slides will contain a slide on interpolation.