**LAB: Generalized Eigenvectors**

Icon: | Item: | Comments: |

Commands entered in MATLAB & resulting output | You should copy relevant input and output from MATLAB and paste it into your Word document. You need only include commands that worked. | |

Plots & graphs | Include all graphs generated in an exercise unless the problem specifically tells you which/how many to include. | |

Full sentence response | Exercise contains a question that you need to use at least one or two complete sentences to answer. Even if you're stuck, write down any reasoning or ideas you've had. | |

Requires work by hand | Do scratch work by hand. Leave space in your Word document and write your scratch work directly on the assignment to turn in. |

*Credits: This design and approach was adapted from work at
University of California at San Diego by Bill Helton and a host of contributors.*

In this lab we investigate how to find a basis for R^{n}
that consists of the eigenvectors of a specified matrix and possibly some other
vectors related to null spaces of the powers of a matrix related to the eigen
problem. We will use MATLAB to do the computations required utilizing the
eig and rref commands.
(MATLAB code using these commands appears in examples but cannot be copied to
paste into MATLAB since it appears as a picture.)

The following terminology is used.

reduced row echelon form | similar matrices | eigenspace | geometric multiplicity |

null space | eigenvalue | diagonalizable | defective matrix |

dimension | eigenvector | algebraic multiplicity | generalized eigenvector |

Let's review some terminology and information about matrices, eigenvalues, and eigenvectors. We state a number of results without proof since linear algebra is a prerequisite for this course.

**Definition: **The
*null space *of a matrix **A**
is the set of all vectors **v** such that **Av** =
**0 **(the zero vector).

We will denote the null space of **A** as
ns**(A)**. The null space of a matrix is a subspace.

**Theorem:** If
**A** is n × n and the rank of **A**
is k, then the *dimension* of the ns(**A**)
is n - k.

**Definition:** The
n × n matrix **A** is said to be
*similar* to the n ×
n matrix **B** provided there exists a nonsingular matrix **P**
such that **B = P**^{-1}**AP**.

Observe that if **A** is similar to **B**,
then **B** is similar to **A** since **A = PBP**^{-1}.

**Definition: **The
scalar λ is called an *
eigenvalue* of n × n matrix **A**
if there exists an n × 1 vector **x**,
**x** ≠ **0**, such that
**Ax** = λ**x**. Every
nonzero vector **x** satisfying **Ax** =
λ**x **is called an
*eigenvector* of **A**
associated with eigenvalue λ.

**Definition:** The
*eigenspace* associated with an eigenvalue λ
of **A** is the set of all corresponding eigenvectors **x**,
such that **Ax** = λ**x**, together with the zero
vector.

The eigenspace of an eigenvalue λ of **A** is
the null space of the matrix **A - λI**. So the eigenspace is a
subspace of n-vectors. Hence it has a basis. The number of vectors in the basis
of the null space is the *dimension* of the
eigenspace and the dimension is always ≥ 1.

**Definition:**
Matrix **A** is *diagonalizable*
provided there exists a nonsingular matrix **P** so that **P**^{-1}**AP
= D**, where **D** is a diagonal matrix.

**Theorem:** An n
× n matrix **A** is
*similar to a
diagonal matrix* if and only if **A** has n linearly
independent eigenvectors.

**Definition:** The
*algebraic multiplicity* of eigenvalue λ of
**A** to be the number of times it is a root of the
*characteristic equation *c(λ) = det(**A
- λI**).

**Definition**: The
*geometric multiplicity* of eigenvalue λ of
**A** is the dimension of the eigenspace associated with λ.

For an eigenvalue λ of **A** the algebraic
multiplicity is always ≥ the geometric multiplicity.

The only way **A** can be similar to a diagonal matrix is if the
algebraic multiplicity is equal to the geometric multiplicity for every
eigenvalue of **A**.

**
Definition:** A matrix **A** is called
*defective* if **A** has an
eigenvalue λ for which the geometric multiplicity <
the algebraic multiplicity.

So if eigenvalue λ of **A**
has algebraic multiplicity m > 1 for which the dimension of the eigenspace
corresponding to λ is less than m, **A**
is defective. Defective matrices are not similar to diagonal matrices using a
matrix **P** consisting of columns which are each eigenvectors of
**A** .

**Theorem:**
If **A** has n distinct eigenvalues, then A is diagonalizable.

So if all the eigenvalues of **A**
have different numerical values then **A** is diagonalizable and is
not defective. In this case we can find n vectors that are eigenvectors of
**A** that form a *basis* for
n-space.

If
A
is diagonalizable then it has n linearly
independent eigenvectors. Hence we have a basis
for
R^{n}
that consists of eigenvectors of
A. |

For defective matrices
A
can we find
a basis for
R^{n}
that includes the linearly independent
eigenvectors of
A
and enough other linearly independent vectors so
that we have n vectors? |

The answer is yes. In such a
case if we construct a matrix **P** whose columns are this set of n
vectors what does **P**^{-1}**AP**
look like?

It won’t be a diagonal matrix with the eigenvalues of **
A** on the diagonal but will it have some other property? The resulting
matrix will be similar to **A**, with possibly a nice structure.

At this point we introduce a new concept.

**Definition: **A
*generalized eigenvector* of **A**
corresponding to eigenvalue λ of **A** is a nonzero vector **
w** such that **(A – λI) ^{k}w= 0** for some
positive integer k.

The smallest value of k is called the
*order *of the generalized eigenvector. Note that a regular
eigenvector is a generalized eigenvector of order 1. Generalized eigenvectors
belong to the null spaces of powers of matrix (**A – λI**). The
null spaces of powers of matrix (**A – λI**) are a set of nested
subspaces which is expressed as \[

ns({\mathbf{A - \lambda I}}) \subseteq ns({\mathbf{A - \lambda I}})^2 \subseteq
ns({\mathbf{{\rm A} - \lambda {\rm I}}})^3 \subseteq \cdots

\]

Let's look at an example.

**Example 1.**
Let \({\mathbf{A}}= \left[ {\begin{array}{*{20}c}1 & { -
1} \\1 & { - 1} \\\end{array} } \right]\). Find a basis for R^{2} that
includes any eigenvectors of **A**. Its characteristic polynomial
is c(λ) = det(**A – λI**) = λ^{2} so it has repeated
eigenvalue λ = 0. The corresponding eigenvectors are \(

{\mathbf{v}}{ = }{\text{r}}\left[ {\begin{array}{*{20}c}

{\text{1}} \\

{\text{1}} \\

\end{array} } \right],\,\,r \ne 0

\) . Thus the eigenspace has basis of only one vector and it follows that
**A** is defective. To find a second vector **w** so
we have a basis for R^{2} we look for a vector in the null space of
**(A – λI)**^{2}** = (A – 0I)**^{2}**
= A**^{2}. Computing **A**^{2}
we get **0**, the zero matrix. So in this example its null space is
all vectors in R^{2}. We need only choose a vector that is not in the
null space of (**A – 0I**) = **A**. Since we are
looking in the null space of **(A – λI)**^{2} we are after
a generalized eigenvector of order 2. We consider several choices.

(i) If we choose \(

{\mathbf{w}} = \left[ {\begin{array}{*{20}c}

1 \\

2 \\

\end{array} } \right]

\) then {**v**, **w**} is a basis for R^{2}.
Let \(

{\mathbf{P}} = \left[ {\begin{array}{*{20}c}

1 & 1 \\

1 & 2 \\

\end{array} } \right]

\) and we have \(

{\mathbf{B}}_{\mathbf{1}} {\mathbf{ = P}}^{{\mathbf{ - 1}}} {\mathbf{AP}} =
\left[ {\begin{array}{*{20}c}

0 & { - 1} \\

0 & 0 \\

\end{array} } \right]

\) which has the eigenvalues as diagonal entries.

(ii) If we choose \(

{\mathbf{w}} = \left[ {\begin{array}{*{20}c}

1 \\

0 \\

\end{array} } \right]

\) then {**v**, **w**} is a basis for R^{2}.
Let \(

{\mathbf{P}} = \left[ {\begin{array}{*{20}c}

1 & 1 \\

1 & 0 \\

\end{array} } \right]

\) and we have \(

{\mathbf{B}}_{\mathbf{2}} {\mathbf{ = P}}^{{\mathbf{ - 1}}} {\mathbf{AP}} =
\left[ {\begin{array}{*{20}c}

0 & { 1} \\

0 & 0 \\

\end{array} } \right]

\) which has the eigenvalues as diagonal entries.

As long as {**v**, **w**} is a
linearly independent set the result of **P**^{-1}**AP
**will be a 2 × 2 non-diagonal matrix with
the eigenvalues on the diagonal.

Exercise 1. If
λ is an eigenvalue of A with algebraic multiplicity 2
but geometric multiplicity 1 and v is an eigenvector
corresponding to λ, show that a nonzero solution w to
linear system (A – λI)w = v satisfies the definition of
a generalized eigenvector for λ. That is, show that (A – λI).^{2}w
= 0Write your solution on paper (with your name, the problem number, etc). Either attach it to the copy of the word file you submit or include it as a portion of the word file. |

Exercise 2. Prove
the following theorem.Theorem: Let A
is a 2 × 2 matrix with repeated eigenvalue λ whose eigenspace is only
one-dimensional and spanned by the eigenvector v. Let
w be a solution to (A – λI)w = v. Let
P = [v w] and let \({\mathbf{B}} = \left[ {\begin{array}{*{20}c} \lambda & 1 \\ 0 & \lambda \\ \end{array} } \right] \) then P^{-1}AP = B
(or equivalently show that AP = PB).Write your solution on paper (with your name, the problem number, etc). Either attach it to the copy of the word file you submit or include it as a portion of the word file. |

Exercise 3. Determine
the eigenvalues of the matrices A and determine if the
Theorem in Exercise 2 applies. If it does, find B and
P and prove that P is nonsingular.
(Choose columns so that P has integer entries.) If the
Theorem in Exercise 2 does not apply, explain why. In general B
will have the form \({\mathbf{B}} = \left[ {\begin{array}{*{20}c} \lambda & a \\ 0 & \lambda \\ \end{array} } \right],\,a \ne 0 \). Use MATLAB to determine the eigenvalues, null spaces, matrices P
and B, and verifying P is nonsingular.
Include your MATLAB code and any responses. Show eigenvectors and
generalized eigenvectors used. |

Next we consider a 3 × 3 matrix which has a repeated eigenvalue and is defective.

**Example 2: **Let
\(

{\mathbf{A}} = \left[ {\begin{array}{*{20}c}

{12} & { - 20} & { - 7} \\

{ - 1} & 4 & 1 \\

{12} & { - 24} & { - 7} \\

\end{array} } \right]

\). Find a basis for R^{3} that includes any eigenvectors of **A**.
The eigenvalues are 5, 2, and 2. The eigen pairs are \(

\left( {5,\,{\mathbf{v}}_{\mathbf{1}} = \left[ {\begin{array}{*{20}c}

1 \\

0 \\

1 \\

\end{array} } \right]} \right)

\) and \(

\left( {2,\,{\mathbf{v}}_{\mathbf{2}} = \left[ {\begin{array}{*{20}c}

2 \\

1 \\

0 \\

\end{array} } \right]} \right)

\). We need a generalized eigenvector **v _{3 }**that
corresponds to λ = 2. Using the null space for

We see that the matrix has rank1so the dimension of the null space is 2. Next we form a homogeneous linear system using the matrix displayed and compute its reduced row echelon form in order to determine vectors in the null space.

From the display of this augmented matrix we have that the
null space consists of all vectors of the form \(

\left[ {\begin{array}{*{20}c}

{2s + 0.75t} \\

s \\

t \\

\end{array} } \right]

\) where s and t are arbitrary. We need only choose
values of s and t so that the resulting vector **
v**_{3}
is linearly independent from **v**_{2}.
For example choosing s = 0, t = 4/3 gives us \(

{\mathbf{v}}_{\mathbf{3}} = \left[ {\begin{array}{*{20}c}

1 \\

0 \\

{4/3} \\

\end{array} } \right]

\) . To check form the matrix \(

{\mathbf{P}} = \left[ {\begin{array}{*{20}c}

1 & 2 & 1 \\

0 & 1 & 0 \\

1 & 0 & {4/3} \\

\end{array} } \right]

\); its determinant is not zero so the columns are linearly independent and a
basis for R^{3}. Next we compute **B = P ^{-1}AP**.

Observe that **B** is
upper triangular with the eigenvalues of **A** on the diagonal. Had
we chosen a different vector **v**_{3} linearly independent
form **v**_{2} the matrix **P **would be
different** **and** **the resulting matrix **B**
would (probably) be a different upper triangular matrix similar to **A**
with diagonal entries the eigenvalues of **A**. The matrices
**P** and **B** are not unique.

Example 3 has an eigenvalue of algebraic multiplicity 3, but geometric multiplicity 1.

**Example 3.** Let
\(

{\mathbf{{\rm A}}} = \left[ {\begin{array}{*{20}c}

2 & 0 & 0 & 0 \\

{ - 3} & 3 & 3 & { - 2} \\

{ - 1} & { - 1} & 6 & { - 1} \\

{ - 1} & { - 1} & 2 & 3 \\

\end{array} } \right]

\). Find a basis for R^{4} that includes any eigenvectors of **A**.
The eigenvalues are 2, 4, 4, and 4. The eigen pairs are \(

\left( {2,\,\,{\mathbf{v}}_{\mathbf{1}} = \left[ {\begin{array}{*{20}c}

1 \\

2 \\

1 \\

1 \\

\end{array} } \right]} \right)

\)

and \(

\left( {4,\,\,{\mathbf{v}}_2 = \left[ {\begin{array}{*{20}c}

0 \\

1 \\

1 \\

1 \\

\end{array} } \right]} \right)

\). We want to find two generalized eigenvectors **v**_{3}
and **v**_{4} corresponding to λ = 4. We start by looking
at powers of (**A - 4I**) and investigating bases for the
corresponding null spaces. Using the null space for **(A - 4I)**^{2}
we compute in MATLAB as follows.

It is not as easy to observe the rank in this case compared to that in Example 2. So let’s compute the reduced row echelon form of the corresponding homogeneous linear system.

Now we see that the rank of **(A – 4I)**^{2
}is 2 so the dimension of the null space is 2. The general solution of the
homogeneous system is has the form \(

\left[ {\begin{array}{*{20}c}

0 \\

s \\

t \\

t \\

\end{array} } \right]

\)where s and t are arbitrary constants. Keep in mind that **v**_{2}
is in the null space of **(A – 4I)**^{2}. Setting s = t = 1
we get **v**_{2}. So we expect one generalized eigenvector
here. One choice is obtained using s = 1 and t = 0. Thus we have \(

{\mathbf{v}}_{\mathbf{3}} = \left[ {\begin{array}{*{20}c}

0 \\

1 \\

0 \\

0 \\

\end{array} } \right]

\). (Of course other choices are possible.)

To find another generalized eigenvector we look at the null
space of **(A - 4I)**^{3} . We have

So the null space
of (**A - 4I**)^{3} is of dimension 3. Since this null
space includes the null spaces of (**A - 4I**)^{2} and (**A
- 4I**) we know that both **v**_{2} and **v**_{3}
are in the null space of (**A - 4I**)^{3} hence we can find
one more generalized eigenvector. The general solution of the homogeneous linear
system with coefficient matrix (**A - 4I**)^{3} has the
form \(

\left[ {\begin{array}{*{20}c}

0 \\

r \\

s \\

t \\

\end{array} } \right]

\) where r, s, and t are arbitrary constants. We need only choose values for r,
s, and t so we obtain a vector linearly independent from both **v**_{2}
and **v**_{3}. One such choice is r = s = 0, t = 1 then \(

{\mathbf{v}}_{\mathbf{4}} = \left[ {\begin{array}{*{20}c}

0 \\

0 \\

0 \\

1 \\

\end{array} } \right]

\). To check form the matrix \(

{\mathbf{P}} = \left[ {\begin{array}{*{20}c}

1 & 0 & 0 & 0 \\

2 & 1 & 1 & 0 \\

1 & 1 & 0 & 0 \\

1 & 1 & 0 & 1 \\

\end{array} } \right]

\); its determinant is not zero so the columns are linearly independent. Next we
compute **B = P ^{-1}AP**.

Observe that
**B** is upper triangular with the eigenvalues of **A**
on the diagonal. Had we chosen a different generalized eigenvectors **v**_{3}
and **v**_{4} linearly independent form **v**_{2}
the matrix **B** would (possibly) be a different upper triangular
matrix similar to **A** with diagonal entries the eigenvalues of
**A**.

Exercise 4.
For each n × n matrix determine a basis for R^{n} consisting of
eigenvectors and generalized eigenvectors with integer entries. Prove
that the set of vectors you construct is a basis for R^{n}. (Use
the procedures in Example 3.)Use MATLAB to determine the eigenvalues, null spaces, matrices P
and B, and verifying P is nonsingular.
Include your MATLAB code and any responses. Show eigenvectors and
generalized eigenvectors used. |

**Last updated 2/10/2013**

**David R. Hill, Temple University**