search
Search
Login
Unlock 100+ guides
menu
menu
web
search toc
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
What does this mean?
Why is this true?
Give me some examples!
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to
check_circle
Mark as learned
thumb_up
0
thumb_down
0
chat_bubble_outline
0
Comment
auto_stories Bi-column layout
settings

Properties of Eigenvalues and Eigenvectors

schedule Aug 12, 2023
Last updated
local_offer
Linear Algebra
Tags
mode_heat
Master the mathematics behind data science with 100+ top-tier guides
Start your free 7-days trial now!
Theorem.

Eigenvectors corresponding to distinct eigenvalues are linearly independent

If $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ are eigenvectors of a matrix corresponding to distinct eigenvalues $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_k$ respectively, then $\{\boldsymbol{v}_1,\boldsymbol{v}_2\cdots,\boldsymbol{v}_k\}$ is linearly independent.

Solution. We will prove this theorem by contradiction. We assume that $\boldsymbol{A}$ has a linearly dependent set of eigenvectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ each with a distinct eigenvalue $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_k$ respectively.

Even though $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ form a linearly dependent set, we can use the plus-minus theoremlink to construct a linearly independent set $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ where $r\lt{k}$. By definition of eigenvalues and eigenvectors, we have the following equations:

$$\begin{equation}\label{eq:xoVVTLpTXiHndW2VaX6} \begin{gathered} \boldsymbol{A}\boldsymbol{v}_1= \lambda_1\boldsymbol{v}_1\\ \boldsymbol{A}\boldsymbol{v}_2= \lambda_2\boldsymbol{v}_2\\ \vdots\\ \boldsymbol{A}\boldsymbol{v}_r= \lambda_r\boldsymbol{v}_r\\ \end{gathered} \end{equation}$$

Because the eigenvectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ form a linearly dependent set, an arbitrary eigenvector $\boldsymbol{v}_j$ where $j\in[1,k]$ can be expressed as a linear combination of the set of linearly independent eigenvectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ like so:

$$\begin{equation}\label{eq:EeIFGb6TxoAgiAeUdHJ} \boldsymbol{v}_j= \sum^r_{i=1}c_i\boldsymbol{v}_i \end{equation}$$

Let the eigenvalue of $\boldsymbol{v}_j$ be denoted as $\lambda_{j}$. From the definition of eigenvalues and eigenvectors, we have that:

$$\begin{equation}\label{eq:DbHPIz2jQcrqLytsjBb} \boldsymbol{A}\boldsymbol{v}_j= \lambda_{j}\boldsymbol{v}_j \end{equation}$$

Substituting \eqref{eq:EeIFGb6TxoAgiAeUdHJ} into \eqref{eq:DbHPIz2jQcrqLytsjBb} gives:

$$\begin{equation}\label{eq:tfPYGwWIWzWFgoisUah} \boldsymbol{A}\left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right)= \lambda_{j} \left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right) \end{equation}$$

The left-hand side can be written as:

$$\begin{align*} \boldsymbol{A}\left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right) &= \boldsymbol{A}c_1\boldsymbol{v}_1+ \boldsymbol{A}c_2\boldsymbol{v}_2+\cdots+\boldsymbol{A}c_r\boldsymbol{v}_r \\ &=c_1\boldsymbol{A}\boldsymbol{v}_1+ c_2\boldsymbol{A}\boldsymbol{v}_2+\cdots+c_r\boldsymbol{A}\boldsymbol{v}_r \end{align*}$$

Using \eqref{eq:xoVVTLpTXiHndW2VaX6}, we get:

$$\begin{equation}\label{eq:L8BfA4CQDFo5qK3ShZR} \boldsymbol{A}\left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right) =c_1\lambda_1\boldsymbol{v}_1+ c_2\lambda_2\boldsymbol{v}_2+\cdots+ c_r\lambda_r\boldsymbol{v}_r \end{equation}$$

Substituting \eqref{eq:L8BfA4CQDFo5qK3ShZR} into \eqref{eq:tfPYGwWIWzWFgoisUah} gives:

$$\begin{align*} c_1\lambda_1\boldsymbol{v}_1+ c_2\lambda_2\boldsymbol{v}_2+\cdots+ c_r\lambda_r\boldsymbol{v}_r&= \lambda_{j} (c_1\boldsymbol{v}_1+c_2\boldsymbol{v}_2+ \cdots+c_r\boldsymbol{v}_r)\\ c_1\boldsymbol{v}_1(\lambda_1-\lambda_{j})+ c_2\boldsymbol{v}_2(\lambda_2-\lambda_{j})+ \cdots+c_r\boldsymbol{v}_r(\lambda_r-\lambda_{j})&=\boldsymbol{0}\\ \sum^r_{i=1}c_i(\lambda_i-\lambda_j)\boldsymbol{v}_i&=0 \end{align*}$$

By definitionlink of linear independence, because $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ are linearly independent vectors, we have that:

$$\begin{gather*} c_1(\lambda_1-\lambda_j)=0\\ c_2(\lambda_2-\lambda_j)=0\\ \vdots\\ c_r(\lambda_r-\lambda_j)=0\\ \end{gather*}$$

Since $\lambda_i$ and $\lambda_j$ are distinct, $\lambda_i-\lambda_j$ cannot be zero. This means that $c_1=c_2=\cdots=c_r=0$. Now, recall from \eqref{eq:EeIFGb6TxoAgiAeUdHJ} that eigenvector $\boldsymbol{v}_j$ is:

$$\boldsymbol{v}_j= \sum^r_{i=1}c_i\boldsymbol{v}_i$$

Since the scalar coefficients are zero, we have that $\boldsymbol{v}_j=0$. However, eigenvectors are non-zero vectors by definitionlink. We have a contradiction here, which means that our initial assumption that $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ is a linearly dependent set is flawed. In other words, $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ must be linearly independent with distinct eigenvalues.

This completes the proof.

Theorem.

Matrix is invertible if and only if zero is not an eigenvalue.

Let $\boldsymbol{A}$ be a square matrix. $\boldsymbol{A}$ is invertible if and only if $0$ is not an eigenvalue of $\boldsymbol{A}$.

Proof. We first prove the forward proposition. Assume $\boldsymbol{A}$ is invertible. By theoremlink, the only solution to $\boldsymbol{Ax}=\boldsymbol{0}$ is the trivial solution of $\boldsymbol{x}=\boldsymbol{0}$. This also means that the only solution to $\boldsymbol{Ax}=0\boldsymbol{x}$ is $\boldsymbol{x}=0$. In other words, for $0$ to be an eigenvalue of $\boldsymbol{A}$, the corresponding eigenvector must be $\boldsymbol{0}$. However, $\boldsymbol{0}$ cannot be an eigenvector by definitionlink and thus $0$ cannot be an eigenvalue of $\boldsymbol{A}$.

We now prove the converse. We assume that $0$ is not an eigenvalue of $\boldsymbol{A}$. This means that there cannot be any non-zero vector $\boldsymbol{x}$ such that $\boldsymbol{Ax}=0\boldsymbol{x}$ holds. In other words, $\boldsymbol{Ax}=0\boldsymbol{x}$ only holds if $\boldsymbol{x}$ is the zero vector. Because $\boldsymbol{Ax}=0\boldsymbol{x}$ is equivalent to $\boldsymbol{Ax}=\boldsymbol{0}$, we have that the only solution to $\boldsymbol{Ax}=\boldsymbol{0}$ is the trivial solution $\boldsymbol{x}=\boldsymbol{0}$. By theoremlink, we conclude that $\boldsymbol{A}$ is invertible.

This completes the proof.

Theorem.

Positive powers of eigenvalues are eigenvalues of the matrix raised to the same power

Let $\boldsymbol{A}$ be a square matrix. If $\lambda$ is an eigenvalue of $\boldsymbol{A}$, then $\lambda^k$ is an eigenvalue of $\boldsymbol{A}^k$ where $k\ge0$ is an integer.

Proof. We will prove this by induction. Consider the base case when $k=0$. Let $\boldsymbol{x}$ be an eigenvector of a square matrix $\boldsymbol{A}$ corresponding to the eigenvalue $\lambda$. Let's check whether or not the theorem is satisfied for the base case, that is, we want to show that $\lambda^0$ is an eigenvalue of $\boldsymbol{A}^0$.

We use the definition $\boldsymbol{A}^0=\boldsymbol{I}_n$ to get:

$$\begin{align*} \boldsymbol{A}^0\boldsymbol{x} &=\boldsymbol{I}_n\boldsymbol{x}\\ &=\boldsymbol{x}\\ &=1\boldsymbol{x}\\ &=\lambda^0\boldsymbol{x}\\ \end{align*}$$

This proves the base case. We now assume that the theorem is true for $\lambda^{n-1}$, that is:

$$\begin{equation}\label{eq:WBZJ4AFlKyT6k1aT049} \boldsymbol{A}^{n-1}\boldsymbol{x}= \lambda^{n-1}\boldsymbol{x} \end{equation}$$

Now, consider $\boldsymbol{A}^n\boldsymbol{x}$ below:

$$\begin{align*} \boldsymbol{A}^n\boldsymbol{x} &=\boldsymbol{A}\boldsymbol{A}^{n-1}\boldsymbol{x}\\ &=\boldsymbol{A}(\boldsymbol{A}^{n-1}\boldsymbol{x}) \end{align*}$$

We use the inductive assumption \eqref{eq:WBZJ4AFlKyT6k1aT049} to get:

$$\begin{align*} \boldsymbol{A}^n\boldsymbol{x} &=\boldsymbol{A}(\lambda^{n-1}\boldsymbol{x})\\ &=\lambda^{n-1}(\boldsymbol{A}\boldsymbol{x})\\ &=\lambda^{n-1}(\lambda\boldsymbol{x})\\ &=\lambda^{n}\boldsymbol{x} \end{align*}$$

By the principle of mathematical induction, we have that the theorem holds for all integers $k\ge0$. This completes the proof.

Theorem.

Reciprocal of an eigenvalue is an eigenvalue of the matrix inverse

Let $\boldsymbol{A}$ be a square invertible matrix. If $\lambda$ is an eigenvalue of $\boldsymbol{A}$, then $\lambda^{-1}$ is an eigenvalue of $\boldsymbol{A}^{-1}$.

Proof. Let $\boldsymbol{x}\ne\boldsymbol{0}$ be an eigenvector of $\boldsymbol{A}$ corresponding to eigenvalue $\lambda$. To show that $\lambda^{-1}$ is an eigenvalue of $\boldsymbol{A}^{-1}$, we can show that $\boldsymbol{A}^{-1}\boldsymbol{x}=\lambda^{-1}\boldsymbol{x}$. We start with the left-hand side:

$$\begin{align*} \boldsymbol{A}^{-1}\boldsymbol{x} &=\boldsymbol{A}^{-1}(1\boldsymbol{x})\\ &=\boldsymbol{A}^{-1}\Big( \frac{1}{\lambda}\lambda\boldsymbol{x}\Big)\\ &=\frac{1}{\lambda}\boldsymbol{A}^{-1}( \lambda\boldsymbol{x})\\ &=\lambda^{-1}\boldsymbol{A}^{-1}(\lambda\boldsymbol{x})\\ &=\lambda^{-1}\boldsymbol{A}^{-1}(\boldsymbol{Ax})\\ &=\lambda^{-1}\boldsymbol{A}^{-1}\boldsymbol{Ax}\\ &=\lambda^{-1}\boldsymbol{I}\boldsymbol{x}\\ &=\lambda^{-1}\boldsymbol{x}\\ \end{align*}$$

By definition of eigenvalues and eigenvectors, we have that $\lambda^{-1}$ is an eigenvalue of $\boldsymbol{A}^{-1}$. This completes the proof.

Note that because $\boldsymbol{A}$ is invertible, the eigenvalues of $\boldsymbol{A}$ cannot be zero by theoremlink. This prevents the possibility of division by zero for $\lambda^{-1}=1/\lambda$.

Theorem.

Characteristic polynomial of a matrix and its transpose is the same

Let $\boldsymbol{A}$ be a square matrix. $\boldsymbol{A}$ and $\boldsymbol{A}^T$ share the same characteristic polynomiallink.

Proof. The characteristic polynomial of $\boldsymbol{A}$ is:

$$\begin{align*} p_{\boldsymbol{A}}(\lambda) &=\det(\boldsymbol{A}-\lambda\boldsymbol{I}_n)\\ \end{align*}$$

By theoremlink, we have that:

$$\begin{align*} p_{\boldsymbol{A}}(\lambda) &=\det(\boldsymbol{A}-\lambda\boldsymbol{I}_n)\\ &=\det\Big((\boldsymbol{A}-\lambda\boldsymbol{I}_n)^T\Big)\\ \end{align*}$$

Using theoremlink and theoremlink yields:

$$\begin{align*} p_{\boldsymbol{A}}(\lambda) &=\det\Big((\boldsymbol{A}-\lambda\boldsymbol{I}_n)^T\Big)\\ &=\det\Big(\boldsymbol{A}^T-(\lambda\boldsymbol{I}_n)^T\Big)\\ &=\det(\boldsymbol{A}^T-\lambda\boldsymbol{I}_n^T)\\ &=\det(\boldsymbol{A}^T-\lambda\boldsymbol{I}_n)\\ &=p_{\boldsymbol{A}^T}(\lambda) \end{align*}$$

This completes the proof.

Theorem.

Matrix and its transpose share the same eigenvalues

Let $\boldsymbol{A}$ be a square matrix. If $\lambda$ is an eigenvalue of $\boldsymbol{A}$, then $\lambda$ is also an eigenvalue of $\boldsymbol{A}^T$.

Proof. By theoremlink, we know that $\boldsymbol{A}$ and $\boldsymbol{A}^T$ share the same characteristic polynomial. This immediately means that $\boldsymbol{A}$ and $\boldsymbol{A}^T$ share the same eigenvalues. This completes the proof.

Theorem.

Non-zero scalar multiples of eigenvectors are also eigenvectors

Non-zero scalar multiples of an eigenvector of a matrix are also eigenvectors of the matrix.

Proof. Suppose $\boldsymbol{x}^*$ is an eigenvector of a matrix $\boldsymbol{A}$. By definitionlink, the following holds:

$$\boldsymbol{A}\boldsymbol{x}^*=\lambda\boldsymbol{x}^*$$

Where $\lambda$ is the corresponding eigenvalue. Now, consider the vector $k\boldsymbol{x}^*$ where $k$ is some non-zero scalar:

$$\begin{align*} \boldsymbol{A}(k\boldsymbol{x}^*) &=k\boldsymbol{A}\boldsymbol{x}^*\\ &=k\lambda\boldsymbol{x}^*\\ &=\lambda(k\boldsymbol{x}^*) \end{align*}$$

Since $\boldsymbol{A}(k\boldsymbol{x}^*)=\lambda(k\boldsymbol{x}^*)$, we have that $k\boldsymbol{x}^*$ is also an eigenvector of $\boldsymbol{A}$ by definition. This completes the proof.

Theorem.

Degree of characteristic polynomial

The characteristic polynomiallink of an $n\times{n}$ matrix has a degree of $n$.

Proof. Suppose $\boldsymbol{A}$ is an $n\times{n}$ matrix. The characteristic polynomial of $\boldsymbol{A}$ is:

$$\det(\boldsymbol{A}-\lambda\boldsymbol{I})= \begin{vmatrix} a_{11}-\lambda&a_{12}&a_{13}&\cdots&a_{1n}\\ a_{21}&a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ a_{31}&a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\vdots&\smash\ddots&\vdots\\ a_{n1}&a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}$$

Let's compute the determinant using cofactor expansionlink along the first column:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I})= (a_{11}-\lambda)\begin{vmatrix} a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}&+ a_{21}\begin{vmatrix} a_{12}&a_{13}&\cdots&a_{1n}\\ a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}\\ &+a_{31}\begin{vmatrix} a_{12}&a_{13}&\cdots&a_{1n}\\ a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}\\&+ \cdots \end{align*}$$

Observe how we are multiplying the first determinant term by a factor including a $\lambda$ term, but all subsequent determinants do not include a $\lambda$ multiple. For instance, we are multiplying the second and third determinant terms by $a_{21}$ and $a_{31}$ instead of $\lambda$. In this way, we are losing a degree of $\lambda$ for the subsequent determinant terms. Since the degree of the polynomial depends on the highest power, let's ignore the subsequent determinant terms:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I})= (a_{11}-\lambda)\begin{vmatrix} a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda \end{vmatrix}+\cdots \end{align*}$$

Now, we evaluate the determinant recursively by cofactor expansionlink along the first column. The highest power of $\lambda$ will be given by:

$$(a_{11}-\lambda) (a_{22}-\lambda) (a_{33}-\lambda) \cdots (a_{nn}-\lambda)$$

The highest power of $\lambda$ is therefore $n$. This means that the characteristic polynomial has degree $n$. This completes the proof.

Theorem.

Symmetric matrices have real eigenvalues

If $\boldsymbol{A}$ is a symmetric matrixlink, then the eigenvalues of $\boldsymbol{A}$ are real.

Proof. Suppose eigenvalue $\lambda$ is a complex number. The corresponding eigenvector $\boldsymbol{x}$ may be a complex vectorlink as it could have complex entries as well. For this pair of eigenvalue and eigenvector, the following holds by definitionlink:

$$\begin{equation}\label{eq:EwLOklsm4rcvRKwSChk} \boldsymbol{Ax}=\lambda\boldsymbol{x} \end{equation}$$

Here, $\boldsymbol{A}$ is assumed to be real, while $\lambda$ and $\boldsymbol{x}$ are complex. We take the complex conjugatelink of both sides to get:

$$\begin{equation}\label{eq:rUaQmK4SanwoPwdr8yI} \overline{\boldsymbol{Ax}}= \overline{\lambda\boldsymbol{x}} \end{equation}$$

By propertylink of matrix complex conjugates:

$$\begin{equation}\label{eq:lHPypbFUvNh0UQKIxP1} \overline{\boldsymbol{Ax}}= \boldsymbol{A}\overline{\boldsymbol{x}} \end{equation}$$

Also, by another propertylink of complex conjugates, $\overline{\lambda\boldsymbol{x}}= \overline{\lambda} \overline{\boldsymbol{x}}$. We can now express \eqref{eq:rUaQmK4SanwoPwdr8yI} as:

$$\begin{equation}\label{eq:fW1BzFyjPcPXuz6J0jJ} \boldsymbol{A}\overline{\boldsymbol{x}}= \overline{\lambda} \overline{\boldsymbol{x}} \end{equation}$$

Now, consider $\lambda \overline{\boldsymbol{x}}^T\boldsymbol{x}$, which can be evaluated as:

$$\begin{equation}\label{eq:vlvAUm9Kax6if5WAUZi} \begin{aligned}[b] \lambda\overline{\boldsymbol{x}}^T\boldsymbol{x}&= \overline{\boldsymbol{x}}^T(\lambda\boldsymbol{x})\\ &=\overline{\boldsymbol{x}}^T(\boldsymbol{Ax})\\ &=(\overline{\boldsymbol{x}}^T\boldsymbol{A})\boldsymbol{x}\\ &=(\overline{\boldsymbol{A}^T\boldsymbol{x}})^T\boldsymbol{x}\\ &=(\overline{\boldsymbol{A}\boldsymbol{x}})^T\boldsymbol{x}\\ &=(\overline{\lambda}\overline{\boldsymbol{x}})^T\boldsymbol{x}\\ &=\overline{\lambda}\overline{\boldsymbol{x}}^T\boldsymbol{x} \end{aligned} \end{equation}$$

Note the following:

Since $\boldsymbol{x}$ is an eigenvector of $\boldsymbol{A}$, we have that $\boldsymbol{x}$ cannot be the zero vector by definitionlink. By theoremlink then, $\overline{\boldsymbol{x}}^T\boldsymbol{x}\ne0$. The only way for \eqref{eq:vlvAUm9Kax6if5WAUZi} to hold is if $\lambda=\overline{\lambda}$. By theoremlink, this means that $\lambda\in\mathbb{R}$. This completes the proof.

robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Comment
Citation
Ask a question or leave a feedback...