Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial

Second step of the improvement

Now let us assume $n > 2$ and add some amount of the step from the $(n-2)$-th iteration to the search direction, determining the new direction ${\bf s}_n^{(n-2)}$, as follows:

$${\bf s}_n^{(n-2)} = {\bf s}_n^{(n-1)} + \beta_n^{(n-2)} {\bf s}_{n-2}\;.$$ (17)

We can deduce that after the second change, the value of numerator in equation (9) is still the same:

$$\left({\bf r}_{n-1}, {\bf A s}_n^{(n-2)}\right)^2 = \left[... ...ght)\right]^2 = \left({\bf r}_{n-1}, {\bf A c}_n\right)^2\;.$$ (18)

This remarkable fact occurs as the result of transforming the dot product $\left({\bf r}_{n-1}, {\bf A s}_{n-2}\right)$ with the help of equation (4):

$$\left({\bf r}_{n-1}, {\bf A s}_{n-2}\right) = \left({\bf r... ...{n-1} \left({\bf A s}_{n-1}, {\bf A s}_{n-2}\right) = 0\;.$$ (19)

The first term in (19) is equal to zero according to formula (7); the second term is equal to zero according to formula (15). Thus we have proved the new orthogonality equation

$$\left({\bf r}_{n-1}, {\bf A s}_{n-2}\right) = 0\;,$$ (20)

which in turn leads to the numerator invariance (18). The value of the coefficient $\beta_n^{(n-2)}$ in (17) is defined analogously to (14) as

$$\beta_n^{(n-2)} = - {{\left({\bf A s}_n^{(n-1)}, {\bf A ... ...f A s}_{n-2}\right)} \over {\Vert{\bf A s}_{n-2}\Vert^2}}\;,$$ (21)

where we have again used equation (15). If ${\bf A s}_{n-2}$ is not orthogonal to ${\bf A c}_n$, the second step of the improvement leads to a further decrease of the denominator in (8) and, consequently, to a further decrease of the residual.

Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial