1. Continuous and discrete equations

(a) Given a PDE with Dirichlet boundary conditions, an linear elliptic differential operator, we wish to find the solution .

(b) The solution can be written formally as . is a linear elliptic integral operator, given by , the convolution of the right hand side with some Green’s function of .

(c) and can be discretized (somehow) as (a matrix) and (a vector). The goal is to find the discretized version of , given by .

(d) The formal solution of the discretized problem is . But inverting directly would be to expensive, that’s where the iterative solver springs into action.

2. The iterative solver

(a) The goal is to build an iteration that converges against the solution .

(b) While increases, we wish that the error in ,

,

decreases and finally reaches 0. We’ll never know the value of because is unknown. But that doesn’t matter as long as the error in ,

is known. The name of is simply “the error” while is called “the residual”.

(c) The relation between the error and the residual is , i.e. scales with . The “anisotropic” scaling-factor is .

(d) The relation between the exact solution and the residual is . If we replace with an approximation , we’ll get an approximation of :

.

I would frame this equation now if I could, because it defines , the iterator that’ll hopefully converge against .

(e) To be sure that it does, has to approach 0 for . The related proof is always done by induction, i.e. has to be related to . Note that our iteration can be rewritten as (can be simply done with (c)), so that . So eventually

.

To conclude whether approaches , we have to make sure that approaches 0. The key is the following relation between and the spectral radius of :

.

You’ll see it if you expand in the Eigenbase of . The conclusion is:

If then convergence.

Introducing a rectangular grid (step-width and points in each direction) on a finite domain (which need not to be rectangular) and substituting the continuous functions and operators with discretized versions

gives (provided the special case and zero boundary conditions)

- Convergence ( for ) can be shown (even though it’s a bit tricky, the proof makes use of the discrete version of the maximum-principle).

- The matrix belongs to the family of block-matrices, i.e. it’s built by (N-1)^2 blocks of quadratic sub-matrices of size N-1 (the “bandwith”), so the total number of elements is . Each block is either diagonal or tridiagonal. This tridiagonal-block-matrix shape is characteristic for all elliptic problems.

- Solving the system with the wrong algorithm can be highly expensive: Gauss-elimination costs and bandsolvers (with ), while the price of the most sophisticated algorithms is (Buneman’s Total Reduction) or even (adaptive multilevel Schwartz + Krylov).The problem with Total Reduction is that it can’t be applied to all elliptic problems. Good general-purpose solvers are iterative solvers, i.e. Jacobi, Gauss-Seidel, gradient method (all ), SOR, conjugated gradient () method plus some more sophisticated methods (adaptive multilevel Schwartz + Krylov, ).

- In d dimensions: The bandwith of the matrix-blocks is given by , or approximately . The multilevel-method wins in all dimensions, but note that in 1D, bandsolvers aren’t worse.

full    Gauss elim.   bandsolver  Jacobi, GS, gradient    SOR, CG     fast solvers

d = 1        3            1                3                 2             1

d = 2        6            4                4                 3             2

d = 3        9            7                5                 4             3

d is the number of dimensions, the numbers # describe the asymptotic (i.e. large N) costs to reach a given accuracy as . In general, bandsolver’s cost is . With w = n/N it’s , which is cheap for d=1, but expensive for d > 1.