Eigenvalues and eigenvectors

From Wikipedia, the free encyclopedia
  (Redirected from Eigenvalue)
Jump to: navigation, search
In this shear mapping the red arrow changes direction but the blue arrow does not. The blue arrow is an eigenvector of this shear mapping because it doesn't change direction, and since its length is unchanged, its eigenvalue is 1.

In linear algebra, an eigenvector or characteristic vector of a square matrix is a vector that points in a direction which is invariant under the associated linear transformation. In other words, if v is a vector which is not zero, then it is an eigenvector of a square matrix A if Av is a scalar multiple of v. This condition could be written as the equation

A\mathbf{v} = \lambda \mathbf{v},






where λ is a number (also called a scalar) known as the eigenvalue or characteristic value associated with the eigenvector v.

There is a correspondence between n by n square matrices and linear transformation from an n-dimensional vector space to itself. For this reason, it is equivalent to define eigenvalues and eigenvectors using either the language of matrices or the language of linear transformations.


If two-dimensional space is visualized as a rubber sheet, a linear map with two eigenvectors would be a stretching along two directions corresponding to the eigenvectors. For example, the sheet could be stretched by a factor λ1 along the x-axis and λ2 along the y-axis. In two dimensions, there can be two such independent stretching directions, but they do not have to be at right angles to each other. A rotation in two dimensions is a linear map with no eigenvectors, and a shear, as in the photo, has only one eigenvector, with eigenvalue 1. Eigenvectors do not change in direction, and the stretch factor for each is its respective eigenvalue. Other vectors will change in direction, unless the two eigenvalues are equal, in which case all vectors will be eigenvectors with that eigenvalue, yielding a magnification, a linear map which alters neither shape nor direction, but only magnitude. A reflection may be viewed as stretching a line perpendicular to the axis of reflection by a factor of −1 while stretching the axis of reflection by a factor of 1. For 3D rotations, the axis of rotation is an eigenvector of eigenvalue 1.


In abstract linear algebra, these concepts are naturally extended to more general situations, where the set of real scalar factors is replaced by any field of scalars (such as algebraic or complex numbers); the set of Cartesian vectors \mathbb{R}^n is replaced by any vector space (such as the continuous functions, the polynomials or the trigonometric series), and matrix multiplication is replaced by any linear operator that maps vectors to vectors (such as the derivative from calculus). In such cases, the "vector" in "eigenvector" may be replaced by a more specific term, such as "eigenfunction", "eigenmode", "eigenface", or "eigenstate". Thus, for example, the exponential function f(x) = e^{\lambda x} is an eigenfunction of the derivative operator,  ' , with eigenvalue \lambda, since its derivative is f'(x) = \lambda e^{\lambda x} = \lambda f(x).

The set of all eigenvectors of a matrix (or linear operator), each paired with its corresponding eigenvalue, is called the eigensystem of that matrix.[1] Any nonzero scalar multiple of an eigenvector is also an eigenvector corresponding to the same eigenvalue. An eigenspace or characteristic space of a matrix A is the set of all eigenvectors of A corresponding to the same eigenvalue, together with the zero vector.[2][3] An eigenbasis for A is any basis for the set of all vectors that consists of linearly independent eigenvectors of A. Not every matrix has an eigenbasis, but every symmetric matrix does.

The prefix eigen- is adopted from the German word eigen for "own-", "unique to", "peculiar to", or "belonging to" in the sense of "idiosyncratic" in relation to the originating matrix.




In the 18th century Euler studied the rotational motion of a rigid body and discovered the importance of the principal axes. Lagrange realized that the principal axes are the eigenvectors of the inertia matrix.[4] In the early 19th century, Cauchy saw how their work could be used to classify the quadric surfaces, and generalized it to arbitrary dimensions.[5] Cauchy also coined the term racine caractéristique (characteristic root) for what is now called eigenvalue; his term survives in characteristic equation.[6]

Fourier used the work of Laplace and Lagrange to solve the heat equation by separation of variables in his famous 1822 book Théorie analytique de la chaleur.[7] Sturm developed Fourier's ideas further and brought them to the attention of Cauchy, who combined them with his own ideas and arrived at the fact that real symmetric matrices have real eigenvalues.[5] This was extended by Hermite in 1855 to what are now called Hermitian matrices.[6] Around the same time, Brioschi proved that the eigenvalues of orthogonal matrices lie on the unit circle,[5] and Clebsch found the corresponding result for skew-symmetric matrices.[6] Finally, Weierstrass clarified an important aspect in the stability theory started by Laplace by realizing that defective matrices can cause instability.[5]

In the meantime, Liouville studied eigenvalue problems similar to those of Sturm; the discipline that grew out of their work is now called Sturm–Liouville theory.[8] Schwarz studied the first eigenvalue of Laplace's equation on general domains towards the end of the 19th century, while Poincaré studied Poisson's equation a few years later.[9]


The first numerical algorithm for computing eigenvalues and eigenvectors appeared in 1929, when Von Mises published the power method. One of the most popular methods today, the QR algorithm, was proposed independently by John G.F. Francis[12] and Vera Kublanovskaya[13] in 1961.[14]

Real matrices[edit]


Consider n-dimensional vectors that are formed as a list of n real numbers, such as the three dimensional vectors,

\mathbf{u} = \begin{Bmatrix}1\\3\\4\end{Bmatrix}\quad\mbox{and}\quad \mathbf{v} = \begin{Bmatrix}-20\\-60\\-80\end{Bmatrix}.

These vectors are said to be scalar multiples of each other, also parallel or collinear, if there is a scalar λ, such that


In this case λ = −1/20.

Now consider the linear transformation of n-dimensional vectors defined by an n×n matrix A, that is,



\begin{Bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{Bmatrix} = \begin{Bmatrix} w_1 \\ w_2 \\ \vdots \\ w_n \end{Bmatrix}

where, for each index i,


If it occurs that w and v are scalar multiples, that is if


then v is an eigenvector of the linear transformation A and the scale factor λ is the eigenvalue corresponding to that eigenvector.

The transformation matrix A = \bigl[ \begin{smallmatrix} 2 & 1\\ 1 & 2 \end{smallmatrix} \bigr] preserves the direction of vectors parallel to v = (1,−1)T (in purple) and w = (1,1)T (in blue). The vectors in red are not parallel to either eigenvector, so, their directions are changed by the transformation. See also: An extended version, showing all four quadrants.

Two dimensional example[edit]

Consider the transformation matrix A, given by,

A = \begin{bmatrix} 2 & 1\\1 & 2 \end{bmatrix}.

The figure shows the effect of this transformation on point coordinates in the plane. The eigenvectors v of this transformation satisfy the equation,


Rearrange this equation to obtain

(A-\lambda I)\mathbf{v}=0,

which has a solution only when its determinant | AλI | equals zero.

Set the determinant to zero to obtain the polynomial equation,

p(\lambda)=|A-\lambda I| = 3 -4\lambda + \lambda^2 = 0,

known as the characteristic polynomial of the matrix A. In this case, it has the roots λ = 1 and λ = 3.

For λ = 1, the equation becomes,

 (A- I)\mathbf{v}=\begin{bmatrix} 1 & 1\\ 1 & 1\end{bmatrix}\begin{Bmatrix}v_1 \\ v_2\end{Bmatrix} =\begin{Bmatrix}0 \\ 0\end{Bmatrix},

which has the solution,

 \mathbf{v} = \begin{Bmatrix} 1 \\ -1 \end{Bmatrix}.

For λ = 3, the equation becomes,

(A-3 I)\mathbf{w}= \begin{bmatrix} -1 & 1\\ 1 & -1\end{bmatrix}\begin{Bmatrix}w_1 \\ w_2\end{Bmatrix} =\begin{Bmatrix}0 \\ 0\end{Bmatrix},

which has the solution,

 \mathbf{w} = \begin{Bmatrix} 1 \\ 1 \end{Bmatrix}.

Thus, the vectors v and w are eigenvectors of A associated with the eigenvalues λ = 1 and λ = 3, respectively.

Three dimensional example[edit]

The eigenvectors v of the 3×3 matrix A,

A= \begin{bmatrix} 2 & 0 & 1\\0 & 2 & 0\\ 1 & 0 & 2\end{bmatrix},

satisfy the equation

(A-\lambda I)\mathbf{v}=0.

This equation has solutions only if the determinant | AλI | equals zero, which yields the characteristic polynomial,

 p(\lambda)=|A-\lambda I| = 6-11\lambda +6\lambda^2-\lambda^3,

with the roots λ = 1, λ = 2 and λ = 3.

Associated with the roots λ = 1, λ = 2 and λ = 3 are the respective eigenvectors,

 \mathbf{u}=\begin{Bmatrix}1\\0\\-1\end{Bmatrix},\quad \mathbf{v}=\begin{Bmatrix}0\\1\\0\end{Bmatrix}, \quad \mbox{and}\quad \mathbf{w}=\begin{Bmatrix}1\\0\\1\end{Bmatrix},

Diagonal matrices[edit]

Matrices with entries only along the main diagonal are called diagonal matrices. It is easy to see that the eigenvalues of a diagonal matrix are the diagonal elements themselves. Consider the matrix A,


The characteristic polynomial of A is given by

 p(\lambda)=|A-\lambda I| = (1-\lambda)(2-\lambda)(3-\lambda)=0,

which has the roots λ = 1, λ = 2 and λ = 3.

Associated with these roots are the eigenvectors,

 \mathbf{u}=\begin{Bmatrix}1\\0\\0\end{Bmatrix}, \quad \mathbf{v}=\begin{Bmatrix}0\\1\\0\end{Bmatrix},\quad \mbox{and} \quad\mathbf{w}=\begin{Bmatrix}0\\0\\1\end{Bmatrix},


Triangular matrices[edit]

A matrix with elements above the main diagonal that are all zeros is described as a triangular matrix, or in this case, lower triangular. If the elements below the main diagonal are all zeros then the matrix is upper triangular. The eigenvalues of triangular matrices are the elements of the main diagonal, in the same way as for diagonal matrices.

Consider the lower triangular matrix A,

 A=\begin{bmatrix} 1 & 0 & 0\\ 1&2& 0\\2&3&3\end{bmatrix}.

The characteristic polynomial of A is given by



Associated with these roots are the eigenvectors,

 \mathbf{u}=\begin{Bmatrix}1\\-1\\1/2\end{Bmatrix}, \quad \mathbf{v}=\begin{Bmatrix}0\\1\\-3\end{Bmatrix},\quad \mbox{and} \quad\mathbf{w}=\begin{Bmatrix}0\\0\\1\end{Bmatrix},


Eigenvector basis[edit]

In this section, it is shown that a change of coordinates of a matrix A to a basis formed by its eigenvectors results in a diagonal matrix.

Let A be an n×n linear transformation


that has n linearly independent eigenvectors vi, and consider the change of coordinates of A so that it is defined relative to its eigenvector basis.

Recall that the eigenvectors vi of A satisfy the eigenvalue equation,

 A\mathbf{v}_i=\lambda_i \mathbf{v}_i\quad i=1\ldots, n.

Assemble these eigenvectors into the matrix V, which is invertible because these vectors are assumed to be linearly independent. This means the coordinates of x and X relative to the basis vi can be computed to be,

\mathbf{y}=V^{-1}\mathbf{x},\quad \mbox{and}\quad \mathbf{Y}=V^{-1}\mathbf{X}.

This yields the change of coordinates


To see the effect of this change of coordinates on A, introduce I=VV−1 into the eigenvalue equation

AV(V^{-1}\mathbf{v}_i)=\lambda_i V(V^{-1} \mathbf{v}_i),

and multiply both side by V−1 to obtain

V^{-1}AV(V^{-1}\mathbf{v}_i)=\lambda_i (V^{-1} \mathbf{v}_i),

Notice that


which is the natural basis vector. Thus,

V^{-1}AV\mathbf{e}_i=K\mathbf{e}_i=\lambda_i \mathbf{e}_i,

and the matrix K is found to be a diagonal matrix with the eigenvalues λi as its diagonal elements.

This shows that a matrix A with a linearly independent system of eigenvectors is similar to a diagonal matrix formed from its eigenvalues.


Characteristic polynomial[edit]

The eigenvalue equation for a matrix A is

A v - \lambda v = 0,

which is equivalent to

(A-\lambda I)v = 0,


\det(A-\lambda I) = 0


For example, let A be the matrix

A =
2 & 0 & 0 \\
0 & 3 & 4 \\
0 & 4 & 9

The characteristic polynomial of A is

2 & 0 & 0 \\
0 & 3 & 4 \\
0 & 4 & 9
\end{bmatrix} - \lambda
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\det \begin{bmatrix}
2 - \lambda & 0 & 0 \\
0 & 3 - \lambda & 4 \\
0 & 4 & 9 - \lambda

which is



Real domain[edit]


For example, consider the cyclic permutation matrix


This matrix shifts the coordinates of the vector up by one position, and moves the first coordinate to the bottom. Its characteristic polynomial is 1 - \lambda^3 which has one real root \lambda_1 = 1. Any vector with three equal non-zero coordinates is an eigenvector for this eigenvalue. For example,


Complex domain[edit]











Algebraic multiplicity[edit]


Like the geometric multiplicity \gamma_A(\lambda_i), we have 1 \leq \mu_A(\lambda_i) \leq n; and the sum \boldsymbol{\mu}_A of \mu_A(\lambda_i) over all distinct eigenvalues also cannot exceed n. If complex eigenvalues are considered, \boldsymbol{\mu}_A is exactly n.


If \mu_A(\lambda_i) = 1, then \lambda_i is said to be a simple eigenvalue.[16]



For the matrix: A= \begin{bmatrix}
2 & 0 & 0 & 0 \\
1 & 2 & 0 & 0 \\
0 & 1 & 3 & 0  \\
0 & 0 & 1 & 3  

\det \begin{bmatrix}
2- \lambda & 0 & 0 & 0 \\
1 & 2- \lambda & 0 & 0 \\
0 & 1 & 3- \lambda & 0  \\
0 & 0 & 1 & 3- \lambda  
being the product of the diagonal with a lower triangular matrix.


Diagonalization and eigendecomposition[edit]



If A is diagonalizable, the space of all n-coordinate vectors can be decomposed into the direct sum of the eigenspaces of A. This decomposition is called the eigendecomposition of A, and it is preserved under change of coordinates.


Further properties[edit]


  • The trace of A, defined as the sum of its diagonal elements, is also the sum of all eigenvalues:
  • The determinant of A is the product of all eigenvalues:
  • The eigenvalues of the kth power of A, i.e. the eigenvalues of A^k, for any positive integer k, are \lambda_1^k,\lambda_2^k,\dots,\lambda_n^k
  • The matrix A is invertible if and only if all the eigenvalues \lambda_i are nonzero.
  • If A is invertible, then the eigenvalues of A^{-1} are 1/\lambda_1,1/\lambda_2,\dots,1/\lambda_n. Clearly, the geometric multiplicities coincide. Moreover, since the characteristic polynomial of the inverse is the reciprocal polynomial for that of the original, they share the same algebraic multiplicity.
  • If A is equal to its conjugate transpose A^* (in other words, if A is Hermitian), then every eigenvalue is real. The same is true of any a symmetric real matrix. If A is also positive-definite, positive-semidefinite, negative-definite, or negative-semidefinite every eigenvalue is positive, non-negative, negative, or non-positive respectively.
  • Every eigenvalue of a unitary matrix has absolute value |\lambda|=1.

Left and right eigenvectors[edit]


A v = \lambda v.


u A = \lambda u





Variational characterization[edit]

Main article: Min-max theorem

In the Hermitian case, eigenvalues can be given a variational characterization. The largest eigenvalue of  H is the maximum value of the quadratic form  x^\mathsf{T} H x/x^\mathsf{T} x . A value of  x that realizes that maximum, is an eigenvector.

General definition[edit]


T(v)=\lambda v.




Geometric multiplicity[edit]

The geometric multiplicity \gamma_T(\lambda) of an eigenvalue \lambda is the dimension of the eigenspace associated with \lambda, i.e., the maximum number of vectors in any linearly independent set of eigenvectors with that eigenvalue.[15][16] It is clear from the definition of eigenvalue in the eigenvalue equation (1) that we always have \gamma_T(\lambda) \geq 1.

Eigenspace and spectrum[edit]







Dynamic equations[edit]

The simplest difference equations have the form

x_t=a_1x_{t-1} +a_2x_{t-2}+\cdots +a_kx_{t-k}.

The solution of this equation for x in terms of t is found by using its characteristic equation

\lambda^k-a_1\lambda^{k-1} -a_2\lambda^{k-2}-\cdots -a_{k-1}\lambda-a_k=0,

which can be found by stacking into matrix form a set of equations consisting of the above difference equation and the k–1 equations x_{t-1}=x_{t-1}, \dots , x_{t-k+1}=x_{t-k+1}, giving a k-dimensional system of the first order in the stacked variable vector [x_t, \dots , x_{t-k+1}] in terms of its once-lagged value, and taking the characteristic equation of this system's matrix. This equation gives k characteristic roots \lambda_1, \dots , \lambda_k, for use in the solution equation

x_t=c_1\lambda _1^t+\cdots + c_k\lambda _k^t.

A similar procedure is used for solving a differential equation of the form

\frac{d^{k}x}{dt^k}+a_{k-1}\frac{d^{k-1}x}{dt^{k-1}}+\cdots +a_1\frac{dx}{dt} +a_0x=0.


Main article: Eigenvalue algorithm





Efficient, accurate methods to compute eigenvalues and eigenvectors of arbitrary matrices were not known until the advent of the QR algorithm in 1961. [24] Combining the Householder transformation with the LU decomposition results in an algorithm with better convergence than the QR algorithm.[citation needed] For large Hermitian sparse matrices, the Lanczos algorithm is one example of an efficient iterative method to compute eigenvalues and eigenvectors, among several other possibilities.[24]






This matrix equation is equivalent to two linear equations

  \quad\quad\quad that is 




Generalizations to infinite-dimensional spaces[edit]

For more details on this topic, see Spectral theorem.




D f = \lambda f




Spectral theory[edit]



Associative algebras and representation theory[edit]




Eigenvalues of geometric transformations[edit]


scaling unequal scaling rotation horizontal shear hyperbolic rotation
illustration Equal scaling (homothety) Vertical shrink () and horizontal stretch () of a unit square. Rotation by 50 degrees
Horizontal shear mapping
c=\cosh \varphi
s=\sinh \varphi
\lambda_2 = k_2
\lambda_2 = e^{-\varphi},
algebraic multipl.
\mu_1 = 2 \mu_1 = 1
\mu_2 = 1
\mu_1 = 1
\mu_2 = 1
\mu_1 = 2 \mu_1 = 1
\mu_2 = 1
geometric multipl.
\gamma_i = \gamma(\lambda_i)
\gamma_1 = 2 \gamma_1 = 1
\gamma_2 = 1
\gamma_1 = 1
\gamma_2 = 1
\gamma_1 = 1 \gamma_1 = 1
\gamma_2 = 1
eigenvectors All non-zero vectors u_1 = \begin{bmatrix}1\\0\end{bmatrix}


Schrödinger equation[edit]



H\psi_E = E\psi_E \,





where |\Psi_E\rangle is an eigenstate of H and E represents the eigenvalue. It is an observable self adjoint operator, the infinite dimensional analog of Hermitian matrices. As in the matrix case, in the equation above H|\Psi_E\rangle is understood to be the vector obtained by application of the transformation H to |\Psi_E\rangle.

Molecular orbitals[edit]


Geology and glaciology[edit]



Principal components analysis[edit]




Vibration analysis[edit]

1st lateral bending
Main article: Vibration

Eigenvalue problems occur naturally in the vibration analysis of mechanical structures with many degrees of freedom. The eigenvalues are the natural frequencies (or eigenfrequencies) of vibration, and the eigenvectors are the shapes of these vibrational modes. In particular, undamped vibration is governed by

m\ddot x + kx = 0


m\ddot x = -k x



-kx= \omega^2 mx



leads to a so-called quadratic eigenvalue problem,





Eigenfaces as examples of eigenvectors
Main article: Eigenface



Tensor of moment of inertia[edit]


Stress tensor[edit]





Basic reproduction number[edit]


See also[edit]


  2. ^ Wolfram Research, Inc. (2010) Eigenvector. Accessed on 2010-01-29.
  3. ^ Nering (1970, p. 107)
  4. ^ See Hawkins 1975, §2
  5. ^ a b c d See Hawkins 1975, §3
  6. ^ a b c See Kline 1972, pp. 807–808
  7. ^ See Kline 1972, p. 673
  8. ^ See Kline 1972, pp. 715–716
  9. ^ See Kline 1972, pp. 706–707
  10. ^ See Kline 1972, p. 1063
  11. ^ See Aldrich 2006
  15. ^ a b c Nering (1970, p. 107)
  16. ^ a b c Golub & Van Loan (1996, p. 316)
  19. ^ Shilov 1977, p. 109
  20. ^ Lemma for the eigenspace
  21. ^ Nering (1970, p. 107)
  22. ^ Schaum's Easy Outline of Linear Algebra, p. 111
  26. ^ Sneed, E. D.; Folk, R. L. (1958), "Pebbles in the lower Colorado River, Texas, a study of particle morphogenesis", Journal of Geology 66 (2): 114–150, Bibcode:1958JG.....66..114S, doi:10.1086/626490 
  27. ^ Knox-Robinson, C.; Gardoll, Stephen J. (1998), "GIS-stereoplot: an interactive stereonet plotting module for ArcView 3.0 geographic information system", Computers & Geosciences 24 (3): 243, Bibcode:1998CG.....24..243K, doi:10.1016/S0098-3004(97)00122-2 
  28. ^ Stereo32 software



External links[edit]


Online calculators[edit]

Demonstration applets[edit]