search
Search
Login
Map of Data Science
menu
menu
search toc
Thanks for the thanks!
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to

Properties of Eigenvalues and Eigenvectors

schedule Mar 5, 2023
Last updated
local_offer
Linear Algebra
Tags
map
Check out the interactive map of data science
Theorem.

Eigenvectors corresponding to distinct eigenvalues are linearly independent

If $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ are eigenvectors of a matrix corresponding to distinct eigenvalues $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_k$ respectively, then $\{\boldsymbol{v}_1,\boldsymbol{v}_2\cdots,\boldsymbol{v}_k\}$ is linearly independent.

Solution. We will prove this theorem by contradiction. We assume that $\boldsymbol{A}$ has a linearly dependent set of eigenvectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ each with a distinct eigenvalue $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_k$ respectively.

Even though $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ form a linearly dependent set, we can use the plus-minus theoremlink to construct a linearly independent set $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ where $r\lt{k}$. By definition of eigenvalues and eigenvectors, we have the following equations:

$$\begin{equation}\label{eq:xoVVTLpTXiHndW2VaX6} \begin{gathered} \boldsymbol{A}\boldsymbol{v}_1= \lambda_1\boldsymbol{v}_1\\ \boldsymbol{A}\boldsymbol{v}_2= \lambda_2\boldsymbol{v}_2\\ \vdots\\ \boldsymbol{A}\boldsymbol{v}_r= \lambda_r\boldsymbol{v}_r\\ \end{gathered} \end{equation}$$

Because the eigenvectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_k$ form a linearly dependent set, an arbitrary eigenvector $\boldsymbol{v}_j$ where $j\in[1,k]$ can be expressed as a linear combination of the set of linearly independent eigenvectors $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ like so:

$$\begin{equation}\label{eq:EeIFGb6TxoAgiAeUdHJ} \boldsymbol{v}_j= \sum^r_{i=1}c_i\boldsymbol{v}_i \end{equation}$$

Let the eigenvalue of $\boldsymbol{v}_j$ be denoted as $\lambda_{j}$. From the definition of eigenvalues and eigenvectors, we have that:

$$\begin{equation}\label{eq:DbHPIz2jQcrqLytsjBb} \boldsymbol{A}\boldsymbol{v}_j= \lambda_{j}\boldsymbol{v}_j \end{equation}$$

Substituting \eqref{eq:EeIFGb6TxoAgiAeUdHJ} into \eqref{eq:DbHPIz2jQcrqLytsjBb} gives:

$$\begin{equation}\label{eq:tfPYGwWIWzWFgoisUah} \boldsymbol{A}\left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right)= \lambda_{j} \left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right) \end{equation}$$

The left-hand side can be written as:

$$\begin{align*} \boldsymbol{A}\left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right) &= \boldsymbol{A}c_1\boldsymbol{v}_1+ \boldsymbol{A}c_2\boldsymbol{v}_2+\cdots+\boldsymbol{A}c_r\boldsymbol{v}_r \\ &=c_1\boldsymbol{A}\boldsymbol{v}_1+ c_2\boldsymbol{A}\boldsymbol{v}_2+\cdots+c_r\boldsymbol{A}\boldsymbol{v}_r \end{align*}$$

Using \eqref{eq:xoVVTLpTXiHndW2VaX6}, we get:

$$\begin{equation}\label{eq:L8BfA4CQDFo5qK3ShZR} \boldsymbol{A}\left(\sum^r_{i=1}c_i\boldsymbol{v}_i\right) =c_1\lambda_1\boldsymbol{v}_1+ c_2\lambda_2\boldsymbol{v}_2+\cdots+ c_r\lambda_r\boldsymbol{v}_r \end{equation}$$

Substituting \eqref{eq:L8BfA4CQDFo5qK3ShZR} into \eqref{eq:tfPYGwWIWzWFgoisUah} gives:

$$\begin{align*} c_1\lambda_1\boldsymbol{v}_1+ c_2\lambda_2\boldsymbol{v}_2+\cdots+ c_r\lambda_r\boldsymbol{v}_r&= \lambda_{j} (c_1\boldsymbol{v}_1+c_2\boldsymbol{v}_2+ \cdots+c_r\boldsymbol{v}_r)\\ c_1\boldsymbol{v}_1(\lambda_1-\lambda_{j})+ c_2\boldsymbol{v}_2(\lambda_2-\lambda_{j})+ \cdots+c_r\boldsymbol{v}_r(\lambda_r-\lambda_{j})&=\boldsymbol{0}\\ \sum^r_{i=1}c_i(\lambda_i-\lambda_j)\boldsymbol{v}_i&=0 \end{align*}$$

By definitionlink of linear independence, because $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ are linearly independent vectors, we have that:

$$\begin{gather*} c_1(\lambda_1-\lambda_j)=0\\ c_2(\lambda_2-\lambda_j)=0\\ \vdots\\ c_r(\lambda_r-\lambda_j)=0\\ \end{gather*}$$

Since $\lambda_i$ and $\lambda_j$ are distinct, $\lambda_i-\lambda_j$ cannot be zero. This means that $c_1=c_2=\cdots=c_r=0$. Now, recall from \eqref{eq:EeIFGb6TxoAgiAeUdHJ} that eigenvector $\boldsymbol{v}_j$ is:

$$\boldsymbol{v}_j= \sum^r_{i=1}c_i\boldsymbol{v}_i$$

Since the scalar coefficients are zero, we have that $\boldsymbol{v}_j=0$. However, eigenvectors are non-zero vectors by definitionlink. We have a contradiction here, which means that our initial assumption that $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ is a linearly dependent set is flawed. In other words, $\boldsymbol{v}_1$, $\boldsymbol{v}_2$, $\cdots$, $\boldsymbol{v}_r$ must be linearly independent with distinct eigenvalues.

This completes the proof.

Theorem.

Matrix is invertible if and only if zero is not an eigenvalue.

Let $\boldsymbol{A}$ be a square matrix. $\boldsymbol{A}$ is invertible if and only if $0$ is not an eigenvalue of $\boldsymbol{A}$.

Proof. We first prove the forward proposition. Assume $\boldsymbol{A}$ is invertible. By theoremlink, the only solution to $\boldsymbol{Ax}=\boldsymbol{0}$ is the trivial solution of $\boldsymbol{x}=\boldsymbol{0}$. This also means that the only solution to $\boldsymbol{Ax}=0\boldsymbol{x}$ is $\boldsymbol{x}=0$. In other words, for $0$ to be an eigenvalue of $\boldsymbol{A}$, the corresponding eigenvector must be $\boldsymbol{0}$. However, $\boldsymbol{0}$ cannot be an eigenvector by definitionlink and thus $0$ cannot be an eigenvalue of $\boldsymbol{A}$.

We now prove the converse. We assume that $0$ is not an eigenvalue of $\boldsymbol{A}$. This means that there cannot be any non-zero vector $\boldsymbol{x}$ such that $\boldsymbol{Ax}=0\boldsymbol{x}$ holds. In other words, $\boldsymbol{Ax}=0\boldsymbol{x}$ only holds if $\boldsymbol{x}$ is the zero vector. Because $\boldsymbol{Ax}=0\boldsymbol{x}$ is equivalent to $\boldsymbol{Ax}=\boldsymbol{0}$, we have that the only solution to $\boldsymbol{Ax}=\boldsymbol{0}$ is the trivial solution $\boldsymbol{x}=\boldsymbol{0}$. By theoremlink, we conclude that $\boldsymbol{A}$ is invertible.

This completes the proof.

Theorem.

Positive powers of eigenvalues are eigenvalues of the matrix raised to the same power

Let $\boldsymbol{A}$ be a square matrix. If $\lambda$ is an eigenvalue of $\boldsymbol{A}$, then $\lambda^k$ is an eigenvalue of $\boldsymbol{A}^k$ where $k\ge0$ is an integer.

Proof. We will prove this by induction. Consider the base case when $k=0$. Let $\boldsymbol{x}$ be an eigenvector of a square matrix $\boldsymbol{A}$ corresponding to the eigenvalue $\lambda$. Let's check whether or not the theorem is satisfied for the base case, that is, we want to show that $\lambda^0$ is an eigenvalue of $\boldsymbol{A}^0$.

We use the definition $\boldsymbol{A}^0=\boldsymbol{I}_n$ to get:

$$\begin{align*} \boldsymbol{A}^0\boldsymbol{x} &=\boldsymbol{I}_n\boldsymbol{x}\\ &=\boldsymbol{x}\\ &=1\boldsymbol{x}\\ &=\lambda^0\boldsymbol{x}\\ \end{align*}$$

This proves the base case. We now assume that the theorem is true for $\lambda^{n-1}$, that is:

$$\begin{equation}\label{eq:WBZJ4AFlKyT6k1aT049} \boldsymbol{A}^{n-1}\boldsymbol{x}= \lambda^{n-1}\boldsymbol{x} \end{equation}$$

Now, consider $\boldsymbol{A}^n\boldsymbol{x}$ below:

$$\begin{align*} \boldsymbol{A}^n\boldsymbol{x} &=\boldsymbol{A}\boldsymbol{A}^{n-1}\boldsymbol{x}\\ &=\boldsymbol{A}(\boldsymbol{A}^{n-1}\boldsymbol{x}) \end{align*}$$

We use the inductive assumption \eqref{eq:WBZJ4AFlKyT6k1aT049} to get:

$$\begin{align*} \boldsymbol{A}^n\boldsymbol{x} &=\boldsymbol{A}(\lambda^{n-1}\boldsymbol{x})\\ &=\lambda^{n-1}(\boldsymbol{A}\boldsymbol{x})\\ &=\lambda^{n-1}(\lambda\boldsymbol{x})\\ &=\lambda^{n}\boldsymbol{x} \end{align*}$$

By the principle of mathematical induction, we have that the theorem holds for all integers $k\ge0$. This completes the proof.

Theorem.

Reciprocal of an eigenvalue is an eigenvalue of the matrix inverse

Let $\boldsymbol{A}$ be a square invertible matrix. If $\lambda$ is an eigenvalue of $\boldsymbol{A}$, then $\lambda^{-1}$ is an eigenvalue of $\boldsymbol{A}^{-1}$.

Proof. Let $\boldsymbol{x}\ne\boldsymbol{0}$ be an eigenvector of $\boldsymbol{A}$ corresponding to eigenvalue $\lambda$. To show that $\lambda^{-1}$ is an eigenvalue of $\boldsymbol{A}^{-1}$, we can show that $\boldsymbol{A}^{-1}\boldsymbol{x}=\lambda^{-1}\boldsymbol{x}$. We start with the left-hand side:

$$\begin{align*} \boldsymbol{A}^{-1}\boldsymbol{x} &=\boldsymbol{A}^{-1}(1\boldsymbol{x})\\ &=\boldsymbol{A}^{-1}\Big( \frac{1}{\lambda}\lambda\boldsymbol{x}\Big)\\ &=\frac{1}{\lambda}\boldsymbol{A}^{-1}( \lambda\boldsymbol{x})\\ &=\lambda^{-1}\boldsymbol{A}^{-1}(\lambda\boldsymbol{x})\\ &=\lambda^{-1}\boldsymbol{A}^{-1}(\boldsymbol{Ax})\\ &=\lambda^{-1}\boldsymbol{A}^{-1}\boldsymbol{Ax}\\ &=\lambda^{-1}\boldsymbol{I}\boldsymbol{x}\\ &=\lambda^{-1}\boldsymbol{x}\\ \end{align*}$$

By definition of eigenvalues and eigenvectors, we have that $\lambda^{-1}$ is an eigenvalue of $\boldsymbol{A}^{-1}$. This completes the proof.

Note that because $\boldsymbol{A}$ is invertible, the eigenvalues of $\boldsymbol{A}$ cannot be zero by theoremlink. This prevents the possibility of division by zero for $\lambda^{-1}=1/\lambda$.

Theorem.

Characteristic polynomial of a matrix and its transpose is the same

Let $\boldsymbol{A}$ be a square matrix. $\boldsymbol{A}$ and $\boldsymbol{A}^T$ share the same characteristic polynomiallink.

Proof. The characteristic polynomial of $\boldsymbol{A}$ is:

$$\begin{align*} p_{\boldsymbol{A}}(\lambda) &=\det(\boldsymbol{A}-\lambda\boldsymbol{I}_n)\\ \end{align*}$$

By theoremlink, we have that:

$$\begin{align*} p_{\boldsymbol{A}}(\lambda) &=\det(\boldsymbol{A}-\lambda\boldsymbol{I}_n)\\ &=\det\Big((\boldsymbol{A}-\lambda\boldsymbol{I}_n)^T\Big)\\ \end{align*}$$

Using theoremlink and theoremlink yields:

$$\begin{align*} p_{\boldsymbol{A}}(\lambda) &=\det\Big((\boldsymbol{A}-\lambda\boldsymbol{I}_n)^T\Big)\\ &=\det\Big(\boldsymbol{A}^T-(\lambda\boldsymbol{I}_n)^T\Big)\\ &=\det(\boldsymbol{A}^T-\lambda\boldsymbol{I}_n^T)\\ &=\det(\boldsymbol{A}^T-\lambda\boldsymbol{I}_n)\\ &=p_{\boldsymbol{A}^T}(\lambda) \end{align*}$$

This completes the proof.

Theorem.

Matrix and its transpose share the same eigenvalues

Let $\boldsymbol{A}$ be a square matrix. If $\lambda$ is an eigenvalue of $\boldsymbol{A}$, then $\lambda$ is also an eigenvalue of $\boldsymbol{A}^T$.

Proof. By theoremlink, we know that $\boldsymbol{A}$ and $\boldsymbol{A}^T$ share the same characteristic polynomial. This immediately means that $\boldsymbol{A}$ and $\boldsymbol{A}^T$ share the same eigenvalues. This completes the proof.

Theorem.

Non-zero scalar multiples of eigenvectors are also eigenvectors

Non-zero scalar multiples of an eigenvector of a matrix are also eigenvectors of the matrix.

Proof. Suppose $\boldsymbol{x}^*$ is an eigenvector of a matrix $\boldsymbol{A}$. By definitionlink, the following holds:

$$\boldsymbol{A}\boldsymbol{x}^*=\lambda\boldsymbol{x}^*$$

Where $\lambda$ is the corresponding eigenvalue. Now, consider the vector $k\boldsymbol{x}^*$ where $k$ is some non-zero scalar:

$$\begin{align*} \boldsymbol{A}(k\boldsymbol{x}^*) &=k\boldsymbol{A}\boldsymbol{x}^*\\ &=k\lambda\boldsymbol{x}^*\\ &=\lambda(k\boldsymbol{x}^*) \end{align*}$$

Since $\boldsymbol{A}(k\boldsymbol{x}^*)=\lambda(k\boldsymbol{x}^*)$, we have that $k\boldsymbol{x}^*$ is also an eigenvector of $\boldsymbol{A}$ by definition. This completes the proof.

Theorem.

Degree of characteristic polynomial

The characteristic polynomiallink of an $n\times{n}$ matrix has a degree of $n$.

Proof. Suppose $\boldsymbol{A}$ is an $n\times{n}$ matrix. The characteristic polynomial of $\boldsymbol{A}$ is:

$$\det(\boldsymbol{A}-\lambda\boldsymbol{I})= \begin{vmatrix} a_{11}-\lambda&a_{12}&a_{13}&\cdots&a_{1n}\\ a_{21}&a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ a_{31}&a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\vdots&\smash\ddots&\vdots\\ a_{n1}&a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}$$

Let's compute the determinant using cofactor expansionlink along the first column:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I})= (a_{11}-\lambda)\begin{vmatrix} a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}&+ a_{21}\begin{vmatrix} a_{12}&a_{13}&\cdots&a_{1n}\\ a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}\\ &+a_{31}\begin{vmatrix} a_{12}&a_{13}&\cdots&a_{1n}\\ a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda\\ \end{vmatrix}\\&+ \cdots \end{align*}$$

Observe how we are multiplying the first determinant term by a factor including a $\lambda$ term, but all subsequent determinants do not include a $\lambda$ multiple. For instance, we are multiplying the second and third determinant terms by $a_{21}$ and $a_{31}$ instead of $\lambda$. In this way, we are losing a degree of $\lambda$ for the subsequent determinant terms. Since the degree of the polynomial depends on the highest power, let's ignore the subsequent determinant terms:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I})= (a_{11}-\lambda)\begin{vmatrix} a_{22}-\lambda&a_{23}&\cdots&a_{2n}\\ a_{32}&a_{33}-\lambda&\cdots&a_{3n}\\ \vdots&\vdots&\ddots&\vdots\\ a_{n2}&a_{n3}&\cdots&a_{nn}-\lambda \end{vmatrix}+\cdots \end{align*}$$

Now, we evaluate the determinant recursively by cofactor expansionlink along the first column. The highest power of $\lambda$ will be given by:

$$(a_{11}-\lambda) (a_{22}-\lambda) (a_{33}-\lambda) \cdots (a_{nn}-\lambda)$$

The highest power of $\lambda$ is therefore $n$. This means that the characteristic polynomial has degree $n$. This completes the proof.

Theorem.

Symmetric matrices have real eigenvalues

If $\boldsymbol{A}$ is a symmetric matrixlink, then the eigenvalues of $\boldsymbol{A}$ are real.

Proof. Suppose eigenvalue $\lambda$ is a complex number. The corresponding eigenvector $\boldsymbol{x}$ may be a complex vectorlink as it could have complex entries as well. For this pair of eigenvalue and eigenvector, the following holds by definitionlink:

$$\begin{equation}\label{eq:EwLOklsm4rcvRKwSChk} \boldsymbol{Ax}=\lambda\boldsymbol{x} \end{equation}$$

Here, $\boldsymbol{A}$ is assumed to be real, while $\lambda$ and $\boldsymbol{x}$ are complex. We take the complex conjugatelink of both sides to get:

$$\begin{equation}\label{eq:rUaQmK4SanwoPwdr8yI} \overline{\boldsymbol{Ax}}= \overline{\lambda\boldsymbol{x}} \end{equation}$$

By propertylink of matrix complex conjugates:

$$\begin{equation}\label{eq:lHPypbFUvNh0UQKIxP1} \overline{\boldsymbol{Ax}}= \boldsymbol{A}\overline{\boldsymbol{x}} \end{equation}$$

Also, by another propertylink of complex conjugates, $\overline{\lambda\boldsymbol{x}}= \overline{\lambda} \overline{\boldsymbol{x}}$. We can now express \eqref{eq:rUaQmK4SanwoPwdr8yI} as:

$$\begin{equation}\label{eq:fW1BzFyjPcPXuz6J0jJ} \boldsymbol{A}\overline{\boldsymbol{x}}= \overline{\lambda} \overline{\boldsymbol{x}} \end{equation}$$

Now, consider $\lambda \overline{\boldsymbol{x}}^T\boldsymbol{x}$, which can be evaluated as:

$$\begin{equation}\label{eq:vlvAUm9Kax6if5WAUZi} \begin{aligned}[b] \lambda\overline{\boldsymbol{x}}^T\boldsymbol{x}&= \overline{\boldsymbol{x}}^T(\lambda\boldsymbol{x})\\ &=\overline{\boldsymbol{x}}^T(\boldsymbol{Ax})\\ &=(\overline{\boldsymbol{x}}^T\boldsymbol{A})\boldsymbol{x}\\ &=(\overline{\boldsymbol{A}^T\boldsymbol{x}})^T\boldsymbol{x}\\ &=(\overline{\boldsymbol{A}\boldsymbol{x}})^T\boldsymbol{x}\\ &=(\overline{\lambda}\overline{\boldsymbol{x}})^T\boldsymbol{x}\\ &=\overline{\lambda}\overline{\boldsymbol{x}}^T\boldsymbol{x} \end{aligned} \end{equation}$$

Note the following:

Since $\boldsymbol{x}$ is an eigenvector of $\boldsymbol{A}$, we have that $\boldsymbol{x}$ cannot be the zero vector by definitionlink. By theoremlink then, $\overline{\boldsymbol{x}}^T\boldsymbol{x}\ne0$. The only way for \eqref{eq:vlvAUm9Kax6if5WAUZi} to hold is if $\lambda=\overline{\lambda}$. By theoremlink, this means that $\lambda\in\mathbb{R}$. This completes the proof.

robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Comment
Citation
Ask a question or leave a feedback...