Hello My MIMO Control Friends,

PURPOSE OF NOTE
Every mxn matrix A has four (4) fundamental vector spaces associated with
it. In this note the four (4) fundamental vector spaces associated with an
mxn matrix A are precisely defined. These vector spaces are referred to in
the linear algebra literature as follows:

      (1) the column space of A,
      (2) the right null space of A,
      (3) the row space of A, and 
      (4) the left null space of A.

Understanding the structure of these four vector spaces is essential in
order to "deeply understand" systems of linear equations Ax = b.
____________________________________________________________________________

COLUMN SPACE (OR RANGE SPACE)
The column space (or range space) of a matrix A is the set of all vectors
Ax where x is an arbitrary n-dimensional vector.

>From this definition, one notes that the column space of a matrix A is the
set of all possible linear combinations of the columns of A, or
equivalently, the set of all possible vectors b for which Ax = b has a
solution.

____________________________________________________________________________

RIGHT NULL SPACE
The right null space of a matrix A is the set of all n-dimensional vectors
x such that Ax = 0.

In some linear algebra texts, the right null space of A is referred to as
the right kernel of A.

____________________________________________________________________________

ROW SPACE
The row space of a matrix A is the set of all vectors y^H A where y is an
arbitrary m-dimensional vector.

>From this definition, one notes that the row space of a matrix A is the set
of all possible linear combinations of the rows of A, or equivalently
(modulo a hermitian operation), the set of all possible linear combinations
of the columns of A^H. In some linear algebra texts, the row space of a
matrix A is defined as the couln space of A^H.

____________________________________________________________________________

LEFT NULL SPACE
The left null space of a matrix A is the set of all m-dimensional row
vectors y^H such that y^H A = 0.

In some linear algebra texts, the left null space of A is referred to as
the left kernel of A.

____________________________________________________________________________

Here are some useful facts.

ORTHOGONALITY PROPERTIES
Elements within the row space are orthogonal to elements within the right
null space.
Elements within the column space are orthogonal to elements within the left
null space.

____________________________________________________________________________

The following example illustrates how one can obtain a basis - i.e. a
linearly independent spanning set of vectors - for each of the four
fundamental vector spaces.

EXAMPLE: BASIS FOR FUNDAMENTASL VECTOR SUBSPACES
Consider the matrix

             A = | 1  0  0  0 |
                 | 0  1  0  0 |
                 | 1  0  0  0 |

Column Space
This matrix has rank 2; i.e. 2 linearly independent columns. Given this,
its column (or range) space has dimension given by:

    dimension of column space = number of linearly independent columns
                              = 2. 

A basis for the column space of this matrix is given as follows:

                 | 1 |   | 0 |
                 | 0 |   | 1 |
                 | 1 |   | 0 |

Right Null Space
The dimension of the right null space of A is given by the relationship:

               right nullity of A = (number of columns of A) - (rank of A) 
                                  =  4 - 2 
                                  = 2.

A basis for the right null space of A is as follows:

              [0  0  1  0]^T

              [0  0  0  1]^T

Row Space
Since this matrix has 2 linearly independent columns, it also has 2
linearly independent rows. Given this the dimension of its row spoace is
given by:


    dimension of row space = number of linearly independent rows
                           = 2. 

Note: For any matrix, the number of linearly independent columns and the
number of linearly independent rows are identical. Hence, the column and
row spaces of a matrix always possess the same dimension, namely the rank
of the matrix.

A basis for the row space of A is as follows:

                 [ 1  0  0  0 ]

                 [ 0  1  0  0 ]

Left Null Space
The dimension of the left null space of A is given by the relationship:


              left nullity of A = (number of rows of A) - (rank of A) 
                                =  3 - 2 
                                = 1.

A basis for the left null space of A is given as follows:

                 [1  0  -1] 

____________________________________________________________________________

For a general matrix A, one must use Gaussian elimination techiques to
extract a basis for the each of the fundamental spaces.

____________________________________________________________________________


Hope that the above is helpful.
Keep asking questions.
AAR