Basic operators and adjoints

Inverse operator

A common practical task is to fit a vector of observed data $\bold d_{\rm obs}$ to some modeled data $\bold d_{\rm model}$ by the adjustment of components in a vector of model parameters $\bold m$.

$$\bold d_{\rm obs} \quad\approx\quad \bold d_{\rm model} \eq \bold F \bold m$$ (42)

A huge volume of literature establishes theory for two estimates of the model, $\hat {\bold m}_1$ and $\hat {\bold m}_2$, where

$$\hat {\bold m}_1 = (\bold F\T\bold F)^{-1}\bold F\T\bold d$$ (43)

$$\hat {\bold m}_2 = \bold F\T(\bold F\bold F\T)^{-1}\bold d$$ (44)

Some reasons for the literature being huge are the many questions about the existence, quality, and cost of the inverse operators. Let us quickly see why these two solutions are reasonable. Inserting equation (42) into equation (43), and inserting equation (44) into equation (42), we get the reasonable statements:

$$\hat {\bold m}_1 = (\bold F\T\bold F)^{-1}(\bold F\T\bold F)\bold m \eq\bold m$$ (45)

$$\hat {\bold d}_{\rm model} = (\bold F\bold F\T)(\bold F\bold F\T)^{-1}\bold d \eq\bold d$$ (46)

Equation (45) says the estimate $\hat {\bold m}_1$ gives the correct model $\bold m$ if you start from the modeled data. Equation (46) says the model estimate $\hat {\bold m}_2$ gives the modeled data if we derive $\hat {\bold m}_2$ from the modeled data. Both these statements are delightful. Now, let us return to the problem of the inverse matrices.

Normally, a rectangular matrix does not have an inverse. Surprising things often happen, but commonly, when $\bold F$ is a tall matrix (more data values than model values), then the matrix for finding $\hat {\bold m}_1$ is invertible while that for finding $\hat {\bold m}_2$ is not; and when the matrix is wide instead of tall (the number of data values is less than the number of model values), it is the other way around. In many applications neither $\bold F\T\bold F$ nor $\bold F\bold F\T$ is invertible. This difficulty is solved by ``damping'' as we see in later chapters. If it happens that $\bold F\bold F\T$ or $\bold F\T\bold F$ equals $\bold I$ (unitary operator), then the adjoint operator $\bold F\T$ is the inverse $\bold F^{-1}$ by either equation (43) or (44).

Current computational power limits matrix inversion jobs to about $10^4$ variables. This book specializes in big problems, those with more than about $10^4$ variables. The iterative methods we learn here for giant problems are also excellent for smaller problems; therefore we rarely here speak of inverse matrices or worry much if neither $\bold F\bold F\T$ nor $\bold F\T\bold F$ is an identity.

EXERCISES:

1. Consider the matrix
   

   $$\left[ \begin{array}{cccccccc} 1 & 0 & 0 & 0 & 0 & 0 & 0 ... ...0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{array} \right]$$ (47)

   
   
   and others like it with $\rho$ in other locations. Show what combination of these matrices will represent the leaky integration matrix in equation (19). What is the adjoint?
2. Modify the calculation in Figure 5 so that there is a triangle waveform on the bottom row.
3. Notice that the triangle waveform is not time aligned with the input in2. Force time alignment with the operator ${\bold C\T \bold C}$ or ${\bold C \bold C\T}$.
4. Modify leakint by changing the diagonal to contain 1/2 instead of 1. Notice how time alignment changes in Figure 5.
5. Suppose a linear operator $\bold F$ has its input in the discrete domain and its output in the continuum. How does the operator resemble a matrix? Describe the operator $\bold F\T$ that has its input in the discrete domain and its output in the continuum. To which do you apply the words ``scales and adds some functions,'' and to which do you apply the words ``does a bunch of integrals''? What are the integrands?

Basic operators and adjoints