LAB: Generalized Eigenvectors

 Icon: Item: Comments: Commands entered in MATLAB & resulting output You should copy relevant input and output from MATLAB and paste it into your Word document.  You need only include commands that worked. Plots & graphs Include all graphs generated in an exercise unless the problem specifically tells you which/how many to include. Full sentence response Exercise contains a question that you need to use at least one or two complete sentences to answer.  Even if you're stuck, write down any reasoning or ideas you've had. Requires work by hand Do scratch work by hand.  Leave space in your Word document and write your scratch work directly on the assignment to turn in.

Credits: This design and approach was adapted from work at University of California at San Diego by Bill Helton and a host of contributors.

In this lab we investigate how to find a basis for Rn that consists of the eigenvectors of a specified matrix and possibly some other vectors related to null spaces of the powers of a matrix related to the eigen problem. We will use MATLAB to do the computations required utilizing the eig and rref commands. (MATLAB code using these commands appears in examples but cannot be copied to paste into MATLAB since it appears as a picture.)

The following terminology is used.

 reduced row echelon form similar matrices eigenspace geometric multiplicity null space eigenvalue diagonalizable defective matrix dimension eigenvector algebraic multiplicity generalized eigenvector

Let's review some terminology and information about matrices, eigenvalues, and eigenvectors. We state a number of results without proof since linear algebra is a prerequisite for this course.

Definition: The null space of a matrix A is the set of all vectors v such that Av = 0 (the zero vector).

We will denote the null space of A as ns(A). The null space of a matrix is a subspace.

Theorem: If A is n × n and the rank of A is k, then the dimension of the ns(A) is n - k.

Definition: The n × n matrix A is said to be similar to the n × n matrix B provided there exists a nonsingular matrix P such that B = P-1AP.

Observe that if A is similar to B, then B is similar to A since A = PBP-1.

Definition: The scalar λ is called an eigenvalue of n × n matrix A if there exists an n × 1 vector x, x 0, such that Ax = λx. Every nonzero vector x satisfying  Ax = λx is called an eigenvector of A associated with eigenvalue λ.

Definition: The eigenspace associated with an eigenvalue λ of A is the set of all corresponding eigenvectors x, such that Ax = λx, together with the zero vector.

The eigenspace of an eigenvalue λ of A is the null space of the matrix A - λI. So the eigenspace is a subspace of n-vectors. Hence it has a basis. The number of vectors in the basis of the null space is the dimension of the eigenspace and the dimension is always ≥ 1.

Definition: Matrix A is diagonalizable provided there exists a nonsingular matrix P so that P-1AP = D, where D is a diagonal matrix.

Theorem: An n × n matrix A is similar to a diagonal matrix if and only if A has n linearly independent eigenvectors.

Definition: The algebraic multiplicity of eigenvalue λ of A to be the number of times it is a root of the characteristic equation c(λ) = det(A - λI).

Definition: The geometric multiplicity of eigenvalue λ of A is the dimension of the eigenspace associated with λ.

For an eigenvalue λ of A the algebraic multiplicity is always ≥ the geometric multiplicity.

The only way A can be similar to a diagonal matrix is if the algebraic multiplicity is equal to the geometric multiplicity for every eigenvalue of A.

Definition: A matrix A is called defective if A has an eigenvalue λ for which the geometric multiplicity < the algebraic multiplicity.

So if eigenvalue λ of A has algebraic multiplicity m > 1 for which the dimension of the eigenspace corresponding to λ is less than m, A is defective. Defective matrices are not similar to diagonal matrices using a matrix P consisting of columns which are each eigenvectors of A .

Theorem: If A has n distinct eigenvalues, then A is diagonalizable.

So if all the eigenvalues of A have different numerical values then A is diagonalizable and is not defective. In this case we can find n vectors that are eigenvectors of A that form a basis for n-space.

 If A is diagonalizable then it has n linearly independent eigenvectors. Hence we have a basis for Rn that consists of eigenvectors of A.

 For defective matrices A can we find a basis for Rn that includes the linearly independent eigenvectors of A and enough other linearly independent vectors so that we have n vectors?

The answer is yes. In such a case if we construct a matrix P whose columns are this set of n vectors what does P-1AP look like?

It won’t be a diagonal matrix with the eigenvalues of A on the diagonal but will it have some other property? The resulting matrix will be similar to A, with possibly a nice structure.

At this point we introduce a new concept.

Definition: A generalized eigenvector of A corresponding to eigenvalue λ of A is a nonzero vector w such that (A – λI)kw= 0 for some positive integer k.

The smallest value of k is called the order of the generalized eigenvector. Note that a regular eigenvector is a generalized eigenvector of order 1. Generalized eigenvectors belong to the null spaces of powers of matrix (A – λI). The null spaces of powers of matrix (A – λI) are a set of nested subspaces which is expressed as $ns({\mathbf{A - \lambda I}}) \subseteq ns({\mathbf{A - \lambda I}})^2 \subseteq ns({\mathbf{{\rm A} - \lambda {\rm I}}})^3 \subseteq \cdots$

Let's look at an example.

Example 1.  Let ${\mathbf{A}}= \left[ {\begin{array}{*{20}c}1 & { - 1} \\1 & { - 1} \\\end{array} } \right]$. Find a basis for R2 that includes any eigenvectors of A. Its characteristic polynomial is c(λ) = det(A – λI) = λ2 so it has repeated eigenvalue λ = 0. The corresponding eigenvectors are ${\mathbf{v}}{ = }{\text{r}}\left[ {\begin{array}{*{20}c} {\text{1}} \\ {\text{1}} \\ \end{array} } \right],\,\,r \ne 0$ . Thus the eigenspace has basis of only one vector and it follows that A is defective. To find a second vector w so we have a basis for R2 we look for a vector in the null space of (A – λI)2 = (A – 0I)2 = A2. Computing A2 we get 0, the zero matrix. So in this example its null space is all vectors in R2. We need only choose a vector that is not in the null space of (A – 0I) = A. Since we are looking in the null space of (A – λI)2 we are after a generalized eigenvector of order 2. We consider several choices.

(i) If we choose ${\mathbf{w}} = \left[ {\begin{array}{*{20}c} 1 \\ 2 \\ \end{array} } \right]$ then {v, w} is a basis for R2. Let  ${\mathbf{P}} = \left[ {\begin{array}{*{20}c} 1 & 1 \\ 1 & 2 \\ \end{array} } \right]$
and we have ${\mathbf{B}}_{\mathbf{1}} {\mathbf{ = P}}^{{\mathbf{ - 1}}} {\mathbf{AP}} = \left[ {\begin{array}{*{20}c} 0 & { - 1} \\ 0 & 0 \\ \end{array} } \right]$ which has the eigenvalues as diagonal entries.

(ii) If we choose ${\mathbf{w}} = \left[ {\begin{array}{*{20}c} 1 \\ 0 \\ \end{array} } \right]$ then {v, w} is a basis for R2. Let  ${\mathbf{P}} = \left[ {\begin{array}{*{20}c} 1 & 1 \\ 1 & 0 \\ \end{array} } \right]$
and we have ${\mathbf{B}}_{\mathbf{2}} {\mathbf{ = P}}^{{\mathbf{ - 1}}} {\mathbf{AP}} = \left[ {\begin{array}{*{20}c} 0 & { 1} \\ 0 & 0 \\ \end{array} } \right]$ which has the eigenvalues as diagonal entries.

As long as {v, w} is a linearly independent set the result of P-1AP will be a 2 × 2 non-diagonal matrix with the eigenvalues on the diagonal.

 Exercise 1. If λ is an eigenvalue of A with algebraic multiplicity 2 but geometric multiplicity 1 and v is an eigenvector corresponding to λ, show that a nonzero solution w to linear system (A – λI)w = v satisfies the definition of a generalized eigenvector for λ. That is, show that (A – λI)2w = 0. Write your solution on paper (with your name, the problem number, etc). Either attach it to the copy of the word file you submit or include it as a portion of the word file.
 Exercise 2. Prove the following theorem. Theorem: Let A is a 2 × 2 matrix with repeated eigenvalue λ whose eigenspace is only one-dimensional and spanned by the eigenvector v. Let w be a solution to (A – λI)w = v. Let P = [v w] and let ${\mathbf{B}} = \left[ {\begin{array}{*{20}c} \lambda & 1 \\ 0 & \lambda \\ \end{array} } \right]$ then P-1AP = B (or equivalently show that AP = PB). Write your solution on paper (with your name, the problem number, etc). Either attach it to the copy of the word file you submit or include it as a portion of the word file.

 Exercise 3. Determine the eigenvalues of the matrices A and determine if the Theorem in Exercise 2 applies. If it does, find B and P and prove that P is nonsingular. (Choose columns so that P has integer entries.) If the Theorem in Exercise 2 does not apply, explain why. In general B will have the form ${\mathbf{B}} = \left[ {\begin{array}{*{20}c} \lambda & a \\ 0 & \lambda \\ \end{array} } \right],\,a \ne 0$. Use MATLAB to determine the eigenvalues, null spaces, matrices P and B, and verifying P is nonsingular. Include your MATLAB code and any responses. Show eigenvectors and generalized eigenvectors used.

Next we consider a 3 × 3 matrix which has a repeated eigenvalue and is defective.

Example 2: Let ${\mathbf{A}} = \left[ {\begin{array}{*{20}c} {12} & { - 20} & { - 7} \\ { - 1} & 4 & 1 \\ {12} & { - 24} & { - 7} \\ \end{array} } \right]$. Find a basis for R3 that includes any eigenvectors of A. The eigenvalues are 5, 2, and 2. The eigen pairs are $\left( {5,\,{\mathbf{v}}_{\mathbf{1}} = \left[ {\begin{array}{*{20}c} 1 \\ 0 \\ 1 \\ \end{array} } \right]} \right)$  and $\left( {2,\,{\mathbf{v}}_{\mathbf{2}} = \left[ {\begin{array}{*{20}c} 2 \\ 1 \\ 0 \\ \end{array} } \right]} \right)$. We need a generalized eigenvector v3 that corresponds to λ = 2. Using the null space for (A - 2I)2 we compute in MATLAB as follows.

We see that the matrix has rank1so the dimension of the null space is 2. Next we form a homogeneous linear system using the matrix displayed and compute its reduced row echelon form in order to determine vectors in the null space.

From the display of this augmented matrix we have that the null space consists of all vectors of the form $\left[ {\begin{array}{*{20}c} {2s + 0.75t} \\ s \\ t \\ \end{array} } \right]$  where s and t are arbitrary. We need only choose values of s and t so that the resulting vector v3 is linearly independent from v2. For example choosing s = 0, t = 4/3 gives us ${\mathbf{v}}_{\mathbf{3}} = \left[ {\begin{array}{*{20}c} 1 \\ 0 \\ {4/3} \\ \end{array} } \right]$ . To check form the matrix ${\mathbf{P}} = \left[ {\begin{array}{*{20}c} 1 & 2 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & {4/3} \\ \end{array} } \right]$; its determinant is not zero so the columns are linearly independent and a basis for R3. Next we compute B = P-1AP.

Observe that B is upper triangular with the eigenvalues of A on the diagonal. Had we chosen a different vector v3 linearly independent form v2 the matrix P would be different and the resulting matrix B would (probably) be a different upper triangular matrix similar to A with diagonal entries the eigenvalues of A.  The matrices P and B are not unique.

Example 3 has an eigenvalue of algebraic multiplicity 3, but geometric multiplicity 1.

Example 3. Let ${\mathbf{{\rm A}}} = \left[ {\begin{array}{*{20}c} 2 & 0 & 0 & 0 \\ { - 3} & 3 & 3 & { - 2} \\ { - 1} & { - 1} & 6 & { - 1} \\ { - 1} & { - 1} & 2 & 3 \\ \end{array} } \right]$. Find a basis for R4 that includes any eigenvectors of A. The eigenvalues are 2, 4, 4, and 4. The eigen pairs are $\left( {2,\,\,{\mathbf{v}}_{\mathbf{1}} = \left[ {\begin{array}{*{20}c} 1 \\ 2 \\ 1 \\ 1 \\ \end{array} } \right]} \right)$
and $\left( {4,\,\,{\mathbf{v}}_2 = \left[ {\begin{array}{*{20}c} 0 \\ 1 \\ 1 \\ 1 \\ \end{array} } \right]} \right)$. We want to find two generalized eigenvectors v3 and v4 corresponding to λ = 4. We start by looking at powers of (A - 4I) and investigating bases for the corresponding null spaces. Using the null space for (A - 4I)2 we compute in MATLAB as follows.

It is not as easy to observe the rank in this case compared to that in Example 2. So let’s compute the reduced row echelon form of the corresponding homogeneous linear system.

Now we see that the rank of (A – 4I)2 is 2 so the dimension of the null space is 2. The general solution of the homogeneous system is has the form $\left[ {\begin{array}{*{20}c} 0 \\ s \\ t \\ t \\ \end{array} } \right]$where s and t are arbitrary constants. Keep in mind that v2 is in the null space of (A – 4I)2. Setting s = t = 1 we get v2. So we expect one generalized eigenvector here. One choice is obtained using s = 1 and t = 0. Thus we have ${\mathbf{v}}_{\mathbf{3}} = \left[ {\begin{array}{*{20}c} 0 \\ 1 \\ 0 \\ 0 \\ \end{array} } \right]$. (Of course other choices are possible.)

To find another generalized eigenvector we look at the null space of (A - 4I)3 . We have

So the null space of (A - 4I)3 is of dimension 3. Since this null space includes the null spaces of (A - 4I)2 and (A - 4I) we know that both v2 and v3 are in the null space of (A - 4I)3 hence we can find one more generalized eigenvector. The general solution of the homogeneous linear system with coefficient matrix (A - 4I)3 has the form $\left[ {\begin{array}{*{20}c} 0 \\ r \\ s \\ t \\ \end{array} } \right]$ where r, s, and t are arbitrary constants. We need only choose values for r, s, and t so we obtain a vector linearly independent from both v2 and v3. One such choice is r = s = 0, t = 1 then ${\mathbf{v}}_{\mathbf{4}} = \left[ {\begin{array}{*{20}c} 0 \\ 0 \\ 0 \\ 1 \\ \end{array} } \right]$. To check form the matrix ${\mathbf{P}} = \left[ {\begin{array}{*{20}c} 1 & 0 & 0 & 0 \\ 2 & 1 & 1 & 0 \\ 1 & 1 & 0 & 0 \\ 1 & 1 & 0 & 1 \\ \end{array} } \right]$; its determinant is not zero so the columns are linearly independent. Next we compute
B = P-1AP.

Observe that B is upper triangular with the eigenvalues of A on the diagonal. Had we chosen a different generalized eigenvectors v3 and v4 linearly independent form v2 the matrix B would (possibly) be a different upper triangular matrix similar to A with diagonal entries the eigenvalues of A.

 Exercise 4.  For each n × n matrix determine a basis for Rn consisting of eigenvectors and generalized eigenvectors with integer entries. Prove that the set of vectors you construct is a basis for Rn. (Use the procedures in Example 3.) Use MATLAB to determine the eigenvalues, null spaces, matrices P and B, and verifying P is nonsingular. Include your MATLAB code and any responses. Show eigenvectors and generalized eigenvectors used.

Last updated 2/10/2013

David R. Hill, Temple University