An Example of Changing Coordinates in 2D

As a simple example, let's pick the following pair of new coordinate vectors in 2D:

$$\begin{eqnarray*} \underline{s}_0 &\isdef & [1,1] \\ \underline{s}_1 &\isdef & [1,-1] \end{eqnarray*}$$

These happen to be the DFT sinusoids for $N=2$ having frequencies $f_0=0$ (``dc'') and $f_1=f_s/2$ (half the sampling rate). (The sampled complex sinusoids of the DFT reduce to real numbers only for $N=1$ and $N=2$.) We already showed in an earlier example that these vectors are orthogonal. However, they are not orthonormal since the norm is $\sqrt{2}$ in each case. Let's try projecting $\underline{x}$ onto these vectors and seeing if we can reconstruct by summing the projections.

The projection of $\underline{x}$ onto $\underline{s}_0$ is, by definition,

$$\begin{eqnarray*} {\bf P}_{\underline{s}_0}(\underline{x}) &\isdef & \frac{\left... ...e{1})}{2} \underline{s}_0 = \frac{x_0 + x_1}{2}\underline{s}_0. \end{eqnarray*}$$

Similarly, the projection of $\underline{x}$ onto $\underline{s}_1$ is

$$\begin{eqnarray*} {\bf P}_{\underline{s}_1}(\underline{x}) &\isdef & \frac{\left... ...e{1})}{2} \underline{s}_1 = \frac{x_0 - x_1}{2}\underline{s}_1. \end{eqnarray*}$$

The sum of these projections is then

$$\begin{eqnarray*} {\bf P}_{\underline{s}_0}(\underline{x}) + {\bf P}_{\underline... ...0 - x_1}{2}\right) \\ [5pt] &=& (x_0,x_1) \isdef \underline{x}. \end{eqnarray*}$$

It worked!

Now consider another example:

$$\begin{eqnarray*} \underline{s}_0 &\isdef & [1,1], \\ \underline{s}_1 &\isdef & [-1,-1]. \end{eqnarray*}$$

The projections of $x=[x_0,x_1]$ onto these vectors are

$$\begin{eqnarray*} {\bf P}_{\underline{s}_0}(\underline{x}) &=& \frac{x_0 + x_1}{... ...ne{s}_1}(\underline{x}) &=& -\frac{x_0 + x_1}{2}\underline{s}_1. \end{eqnarray*}$$

The sum of the projections is

$$\begin{eqnarray*} {\bf P}_{\underline{s}_0}(\underline{x}) + {\bf P}_{\underline... ...-1,-1) \\ &=& \left(x_0+x_1,x_0+x_1\right) \neq \underline{x}. \end{eqnarray*}$$

Something went wrong, but what? It turns out that a set of $N$ vectors can be used to reconstruct an arbitrary vector in ${\bf C}^N$ from its projections only if they are linearly independent. In general, a set of vectors is linearly independent if none of them can be expressed as a linear combination of the others in the set. What this means intuitively is that they must ``point in different directions'' in $N$-space. In this example $s_1 = - s_0$ so that they lie along the same line in $N$-space. As a result, they are linearly dependent: one is a linear combination of the other ( $s_1 = (-1)s_0$).

Consider this example:

$$\begin{eqnarray*} \underline{s}_0 &\isdef & [1,1] \\ \underline{s}_1 &\isdef & [0,1] \end{eqnarray*}$$

These point in different directions, but they are not orthogonal. What happens now? The projections are

$$\begin{eqnarray*} {\bf P}_{\underline{s}_0}(\underline{x}) &=& \frac{x_0 + x_1}{... ...{\bf P}_{\underline{s}_1}(\underline{x}) &=& x_1\underline{s}_1. \end{eqnarray*}$$

The sum of the projections is

$$\begin{eqnarray*} {\bf P}_{\underline{s}_0}(\underline{x}) + {\bf P}_{\underline... ...x_1}{2}, \frac{x_0 + 3x_1}{2}\right) \\ &\neq& \underline{x}. \end{eqnarray*}$$

So, even though the vectors are linearly independent, the sum of projections onto them does not reconstruct the original vector. Since the sum of projections worked in the orthogonal case, and since orthogonality implies linear independence, we might conjecture at this point that the sum of projections onto a set of $N$ vectors will reconstruct the original vector only when the vector set is orthogonal, and this is true, as we will show.

It turns out that one can apply an orthogonalizing process, called Gram-Schmidt orthogonalization to any $N$ linearly independent vectors in ${\bf C}^N$ so as to form an orthogonal set which will always work. This will be derived in Section 5.7.3.

Obviously, there must be at least $N$ vectors in the set. Otherwise, there would be too few degrees of freedom to represent an arbitrary $\underline{x}\in{\bf C}^N$. That is, given the $N$ coordinates $\{x(n)\}_{n=0}^{N-1}$ of $\underline{x}$ (which are scale factors relative to the coordinate vectors $\underline{e}_n$ in ${\bf C}^N$), we have to find at least $N$ coefficients of projection (which we may think of as coordinates relative to new coordinate vectors $\underline{s}_k$). If we compute only $M<N$ coefficients, then we would be mapping a set of $N$ complex numbers to $M<N$ numbers. Such a mapping cannot be invertible in general. It also turns out $N$ linearly independent vectors is always sufficient. The next section will summarize the general results along these lines.