We begin by picking up a theme from the end of last semester, when you were solving systems of linear equations. In problems of fluid dynamics, elasticity, E&M, quantum mechanics, one must solve PDEs of the form
in a region say 0 < x, y < 1, with boundary conditions on u or du/dn. In one space dimension, this equation is an ODE, a 2-point boundary value problem. Discretizing on a regular grid results in a matrix system Av =b, where the matrix A is tri-diagonal. A solution is easily computed by the Thomas algorithm.
In two dimensions, the resulting discrete system is "tri-diagonal with fringe"; with a band size of N, the number of grid points in any one direction. Direct methods exist for solving such a sparse system, methods which may be classified as sparse LU-solvers (details depend on the boundary conditions!) Mike Heath at Illinois has been in the forefront in developing such methods. Here we concentrate on iterative methods.
For simplicity in exposition, restrict to one dimension, that is, the ODE problem. The most well known iterative methods are Jacobi and Gauss-Siedel. Another very popular method is Conjugate Gradient, which can be applied only if A is positive-definite (usually the case if the system arises from the kind of DE described above but derivative boundary conditions can make the system only positive semi-definite). We will discuss stationary methods, and multigrid, as well as CG and the more general Krylov methods.