Model fitting by least squares

Differentiation by a complex vector

Complex numbers frequently arise in physical applications, particularly those with Fourier series. Let us extend the multivariable least-squares theory to the use of complex-valued unknowns $\bold m$. First, recall how complex numbers were handled with single-variable least squares; i.e., as in the discussion leading up to equation (5). Use an asterisk, such as $\bold m\T$, to denote the complex conjugate of the transposed vector $\bold m$. Now, write the positive quadratic form as:

$$Q(\bold m\T, \bold m) \eq (\bold F\bold m - \bold d)\T\, (\b... ... (\bold m\T\,\bold F\T - \bold d\T) (\bold F\bold m - \bold d)$$ (45)

Recall from equation (4), where we minimized a quadratic form $Q(\bar X,X)$ by setting to zero, both $\partial Q/\partial \bar X$ and $\partial Q/\partial X$. We noted that only one of $\partial Q/\partial \bar X$ and $\partial Q/\partial X$ is necessarily zero, because these terms are conjugates of each other. Now, take the derivative of $Q$ with respect to the (possibly complex, row) vector $\bold m\T$. Notice that $\partial Q/\partial \bold m\T$ is the complex conjugate transpose of $\partial Q/\partial \bold m$. Thus, setting one to zero also sets the other to zero. Setting $\partial Q/\partial \bold m\T =\bold 0$ gives the normal equations:

$$\bold 0 \eq \frac{\partial Q}{\partial \bold m\T} \eq \bold F\T\,(\bold F \bold m - \bold d)$$ (46)

The result is merely the complex form of our earlier result (43). Therefore, differentiating by a complex vector is an abstract concept, but it gives the same set of equations as differentiating by each scalar component, and it saves much clutter.

Model fitting by least squares