Eigenvalues and Eigenvectors

Definition

1.A linear map, represented by a square matrix $A,$ acts on vectors by moving them in space. It can stretch, compress, rotate, or reflect them, and in general the image of a vector points in a different direction from the original. Among all vectors however there are those for which the action of $A$ is simple. The transformation scales them by a constant factor, leaving their direction unchanged. Such vectors are called eigenvectors of $A,$ and the corresponding scaling factors are called eigenvalues.

2.

Eigenvectors are the directions along which the transformation acts by scaling, and the eigenvalues do not depend on the basis chosen to represent the transformation.

3.Let $A$ be a square matrix of order $n$ with entries in $\mathbb{R}$ or $\mathbb{C}.$ A non-zero vector $\mathbf{v}$ is called an eigenvector of $A$ if there exists a scalar $\lambda$ such that the following equation holds:

$$A\mathbf{v} = \lambda\mathbf{v}$$

4.The scalar $\lambda$ is called the eigenvalue of $A$ associated with $\mathbf{v}.$ The condition requires that $A$ maps $\mathbf{v}$ to a scalar multiple of itself: the vector $\mathbf{v}$ may be stretched or compressed, and its orientation may be reversed if $\lambda$ is negative, but it remains on the same line through the origin. Eigenvectors are the invariant directions of the transformation and eigenvalues are the scaling factors along those directions.

5.

The zero vector is excluded by convention. The equation $A\mathbf{0} = \lambda\mathbf{0}$ is satisfied for every $\lambda$ and carries no information about the matrix.

6.The following diagram illustrates this idea for the square matrix:

$$A = \begin{pmatrix} 2 & 1 \\[6pt] 1 & 2 \end{pmatrix}$$

7.The unit circle is mapped to an ellipse: most vectors change direction under the transformation. The two eigenvectors $\mathbf{v}_1$ and $\mathbf{v}_2$ are the exception. They remain on the same line through the origin, scaled by $\lambda_1 = 3$ and $\lambda_2 = 1$ respectively.

The characteristic equation

8.Rewriting the eigenvalue equation as $(A - \lambda I)\mathbf{v} = \mathbf{0},$ where $I$ is the identity matrix of order $n,$ it is clear that a non-zero solution $\mathbf{v}$ exists precisely when the matrix $A - \lambda I$ is singular. The condition for singularity is that its determinant vanishes. The equation

$$\det(A - \lambda I) = 0$$

9.is called the characteristic equation of $A.$ Expanding the determinant yields a polynomial of degree $n$ in $\lambda,$ known as the characteristic polynomial of $A.$ The eigenvalues of $A$ are the roots of this polynomial, and by the fundamental theorem of algebra there are exactly $n$ of them, counted with multiplicity, in $\mathbb{C}.$

10.

A matrix with real entries has a characteristic polynomial with real coefficients, but this does not prevent complex roots. Complex eigenvalues of a real matrix always appear in conjugate pairs. More generally, for a matrix with entries in a field $F,$ the eigenvalues are the roots of the characteristic polynomial in $F$ or in an algebraic extension of $F,$ so the appropriate ambient field depends on the factorisation properties of that polynomial.

Dependence on the field

11.The eigenvalues of a matrix depend on the field over which the matrix is considered, since they are the roots of the characteristic polynomial and a polynomial with real coefficients need not have real roots. A real matrix can therefore have no real eigenvalues at all. Consider the rotation matrix:

$$A = \begin{pmatrix} 0 & 1 \\[6pt] -1 & 0 \end{pmatrix}$$

12.Its characteristic polynomial is computed from the determinant of $A - \lambda I$:

$$\det(A - \lambda I) = \det\begin{pmatrix} -\lambda & 1 \\[6pt] -1 & -\lambda \end{pmatrix} = \lambda^2 + 1$$

13.Over $\mathbb{R}$ this polynomial has no roots, so $A$ has no real eigenvalues, in agreement with the geometric fact that a rotation fixes no direction. Over $\mathbb{C}$ the roots are $\lambda = i$ and $\lambda = -i.$ Solving $(A - iI)\mathbf{v} = \mathbf{0}$ gives the condition $y = ix,$ so the eigenspace is spanned by $(1, i)^{\mathrm{T}},$ and solving $(A + iI)\mathbf{v} = \mathbf{0}$ gives $(1, -i)^{\mathrm{T}}.$ The two eigenvalues are complex conjugates, and so are the two eigenvectors.

14.A reflection of the plane behaves differently. Its matrix has trace $0$ and determinant $-1,$ so its characteristic polynomial is $\lambda^2 - 1$ and its eigenvalues are the real numbers $1$ and $-1,$ with the axis of the reflection as the eigenspace for $1$ and the perpendicular line as the eigenspace for $-1.$ The rotations and the reflections that preserve a regular polygon are the dihedral groups.

15.

The eigenvalues of a matrix are not in general among its entries, and in particular not the entries on the main diagonal. The diagonal carries the eigenvalues only when the matrix is triangular, since in that case $\det(A - \lambda I)$ is the product of the diagonal differences $a_{ii} - \lambda.$

Eigenspaces

16.For each eigenvalue $\lambda_0,$ the set of all vectors satisfying $A\mathbf{v} = \lambda_0\mathbf{v}$ is a subspace of the vector space $\mathbb{R}^n$ or $\mathbb{C}^n.$ It coincides with the kernel of the linear map $A - \lambda_0 I$ and is called the eigenspace of $A$ associated with $\lambda_0$:

$$E_{\lambda_0} = \ker(A - \lambda_0 I) = \\{\ \mathbf{v} : (A - \lambda_0 I)\mathbf{v} = \mathbf{0} \ \\}$$

17.The dimension of $E_{\lambda_0}$ is called the geometric multiplicity of $\lambda_0.$ Because the eigenspace is the kernel of $A - \lambda_0 I,$ the rank-nullity relation gives $\dim E_{\lambda_0} = n - \mathrm{rank}(A - \lambda_0 I),$ so the geometric multiplicity follows directly from the rank of $A - \lambda_0 I.$ Separately, the multiplicity of $\lambda_0$ as a root of the characteristic polynomial is called the algebraic multiplicity of $\lambda_0.$ It can be shown that the geometric multiplicity never exceeds the algebraic one.

Example 1

18.Consider the following matrix:

$$A = \begin{pmatrix} 3 & 1 \\[6pt] 0 & 2 \end{pmatrix}$$

19.We compute the characteristic polynomial by forming the matrix $A - \lambda I$ and computing its determinant. Since $A - \lambda I$ is upper triangular, its determinant is the product of the diagonal entries:

$$\det(A - \lambda I) = (3 - \lambda)(2 - \lambda)$$

20.Setting this expression equal to zero gives $\lambda_1 = 2$ and $\lambda_2 = 3.$ For $\lambda_1 = 2,$ we solve $(A - 2I)\mathbf{v} = \mathbf{0}.$ The matrix $A - 2I$ reduces to:

$$A - 2I = \begin{pmatrix} 1 & 1 \\[6pt] 0 & 0 \end{pmatrix}$$

21.The system yields the single condition $v_1 + v_2 = 0,$ so $v_1 = -v_2.$ Taking $v_2 = 1,$ the eigenspace $E_2$ is spanned by:

$$\mathbf{v}_1 = \begin{pmatrix} -1 \\[6pt] 1 \end{pmatrix}$$

22.For $\lambda_2 = 3,$ the matrix $A - 3I$ is:

$$A - 3I = \begin{pmatrix} 0 & 1 \\[6pt] 0 & -1 \end{pmatrix}$$

23.Both rows give the condition $v_2 = 0,$ leaving $v_1$ free. Taking $v_1 = 1,$ the eigenspace $E_3$ is spanned by:

$$\mathbf{v}_2 = \begin{pmatrix} 1 \\[6pt] 0 \end{pmatrix}$$

24.The matrix $A$ has therefore eigenvalue $\lambda_1 = 2$ with eigenvector $(-1, 1)^{\mathrm{T}},$ and eigenvalue $\lambda_2 = 3$ with eigenvector $(1, 0)^{\mathrm{T}}.$

Example 2

25.Consider the matrix

$$ A = \begin{pmatrix} 2 & 1 & 0 \\[6pt] 0 & 2 & 0 \\[6pt] 0 & 0 & 3 \end{pmatrix} $$

26.The matrix $A - \lambda I$ is block upper triangular, so its determinant is again the product of the diagonal entries. The characteristic polynomial is the following:

$$p(\lambda) = (2 - \lambda)^2(3 - \lambda)$$

27.Setting $p(\lambda) = 0$ gives two eigenvalues: $\lambda_1 = 2,$ with algebraic multiplicity two, and $\lambda_2 = 3,$ with algebraic multiplicity one.

28.For $\lambda_2 = 3,$ we solve $(A - 3I)\mathbf{v} = \mathbf{0}.$ The matrix $A - 3I$ is:

$$ A - 3I = \begin{pmatrix} -1 & 1 & 0 \\[6pt] 0 & -1 & 0 \\[6pt] 0 & 0 & 0 \end{pmatrix} $$

29.The second row gives $v_2 = 0,$ and the first row then gives $v_1 = 0,$ leaving $v_3$ free. Taking $v_3 = 1,$ the eigenspace $E_3$ is spanned by:

$$\mathbf{v}_1 = \begin{pmatrix} 0 \\[6pt] 0 \\[6pt] 1 \end{pmatrix}$$

30.For $\lambda_1 = 2,$ we solve $(A - 2I)\mathbf{v} = \mathbf{0}.$ The matrix $A - 2I$ is:

$$ A - 2I = \begin{pmatrix} 0 & 1 & 0 \\[6pt] 0 & 0 & 0 \\[6pt] 0 & 0 & 1 \end{pmatrix} $$

31.The first row gives $v_2 = 0$ and the third row gives $v_3 = 0,$ while $v_1$ remains free. Taking $v_1 = 1,$ the eigenspace $E_2$ is one-dimensional, spanned by:

$$\mathbf{v}_2 = \begin{pmatrix} 1 \\[6pt] 0 \\[6pt] 0 \end{pmatrix}$$

32.The geometric multiplicity of $\lambda_1 = 2$ is therefore one, while its algebraic multiplicity is two. Since these two values differ, the matrix $A$ is not diagonalizable. It has only two linearly independent eigenvectors, which is insufficient to form a basis of $\mathbb{R}^3.$

Linear independence of eigenvectors

33.Eigenvectors corresponding to distinct eigenvalues are always linearly independent. More precisely, if $\lambda_1, \ldots, \lambda_k$ are pairwise distinct eigenvalues of $A$ with associated eigenvectors $\mathbf{v}_1, \ldots, \mathbf{v}_k,$ then $\mathbf{v}_1, \ldots, \mathbf{v}_k$ are linearly independent. The proof proceeds by induction on $k$ and uses the fact that each eigenvalue is distinct to derive a contradiction from any supposed linear dependence relation.

34.As a consequence, a square matrix of order $n$ with $n$ distinct eigenvalues always has $n$ linearly independent eigenvectors, and therefore admits a basis of eigenvectors.

Diagonalization

35.A matrix $A$ of order $n$ is called diagonalizable if it can be written in the form

$$A = PDP^{-1}$$

36.where $P$ is an invertible matrix and $D$ is diagonal. The columns of $P$ are eigenvectors of $A,$ and the corresponding diagonal entries of $D$ are the associated eigenvalues. This decomposition, when it exists, simplifies the computation of powers of $A.$ The $k$-th power takes the form:

$$A^k = PD^kP^{-1}$$

37.Since raising a diagonal matrix to a power amounts to raising each diagonal entry to that power, this avoids the need to perform $k$ successive matrix multiplications.

38.A matrix is diagonalizable if and only if, for every eigenvalue, its geometric multiplicity equals its algebraic multiplicity. When this condition fails, the matrix cannot be diagonalized but can be reduced to Jordan canonical form, which is the closest diagonal-like structure available in the general case. A full treatment of the procedure is given in the dedicated entry on matrix diagonalization.

Trace, determinant and eigenvalues

39.Let $\lambda_1, \lambda_2, \ldots, \lambda_n$ be the eigenvalues of $A$ counted with algebraic multiplicity. Two classical identities relate them directly to entries of the matrix. The trace of $A,$ defined as the sum of its diagonal entries, satisfies:

$$\mathrm{tr}(A) = \lambda_1 + \lambda_2 + \cdots + \lambda_n$$

40.The determinant of $A$ satisfies:

$$\det(A) = \lambda_1 \cdot \lambda_2 \cdots \lambda_n$$

41.Both identities follow from the structure of the characteristic polynomial. The second implies that a matrix is singular if and only if zero is one of its eigenvalues. These two relations give a quick consistency check when eigenvalues are computed by hand.