We will start off with elliptic equations. Consider first the examples
given above. We discritize the unit square by a mesh of
N+1 points by N+1 points, so 
.
Let 
.
We have
and on the boundary, 
.
We must solve a linear system A u = b. The matrix A is sparse, but it
is not tridiagonal (like in 1D). It is `block tridiagona' - that is,
the 
matrix has three (N-1) blocks along the
diagonal, sub- and super-diagonals. There are reasonably fast
block solvers for systems of this type. But the general level of complexity
suggests why iterative methods are attractive.
An additional level of complexity is suggested by the equation
where 
is a variable coefficient. For example, there
is a standard test problem of this form, with f=1 and 
piecewise
constant, say 10 or 100 for 
and 1 in the rest
of the square, with zero Dirichlet conditions. Standard iterative methods
have troubles with this problem. Multigrid, or variants of CG, are
the usual solvers.
Boundary conditions pose an additional difficulty. If we consider a Neumann
problem, it should be clear that we could (probably) add a constant
to the solution and still satisfy the equation and BCs. A trivial
example is 
with zero Neumann data on the square.
u=const. is a solution.
HOMEWORK EXERCISE: Write down the discritization for the
1D problem 
with zero Neumann data, on a grid with N=4.
Is the finite difference matrix symmetric, positive definite? What can you
say about applying standard solution algorithms?
Will this behavior persist in 2D?
What is the matrix for zero Dirichlet data? What is the difference?