search
Search
Login
Unlock 100+ guides
menu
menu
web
search toc
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
What does this mean?
Why is this true?
Give me some examples!
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to
check_circle
Mark as learned
thumb_up
1
thumb_down
0
chat_bubble_outline
0
Comment
auto_stories Bi-column layout
settings

Guide on eigenspace and eigenbasis

schedule Aug 10, 2023
Last updated
local_offer
Linear Algebra
Tags
mode_heat
Master the mathematics behind data science with 100+ top-tier guides
Start your free 7-days trial now!
Definition.

Eigenspace

Let $\lambda$ be an eigenvaluelink of a square matrix $\boldsymbol{A}$. The eigenspace of $\lambda$ is defined as the null spacelink of $\boldsymbol{A}-\lambda\boldsymbol{I}$, that is:

$$\mathcal{E}_\lambda(\boldsymbol{A})= \mathrm{nullspace}(\boldsymbol{A}-\lambda\boldsymbol{I})$$

Discussion. Recall that once we have obtained the eigenvalues $\lambda$ of a square matrix $\boldsymbol{A}$, we solve the below equation for $\boldsymbol{x}$ to obtain the corresponding eigenvectorslink:

$$\begin{equation}\label{eq:l3BEQJa2h6ZO7W293FK} (\boldsymbol{A}-\lambda\boldsymbol{I} )\boldsymbol{x}=\boldsymbol{0} \end{equation}$$

By definitionlink, the set of vectors $\boldsymbol{x}$ that satisfies such a system is the null space of $\boldsymbol{A}-\lambda\boldsymbol{I}$. The eigenspace associated with $\lambda$ is defined to be this null space.

Note that an eigenspace always contains the zero vector because $\boldsymbol{x}=0$ is a solution to \eqref{eq:l3BEQJa2h6ZO7W293FK}. However, an eigenvector cannot be the zero vector by definitionlink. In other words, all the vectors that reside in the eigenspace are eigenvectors of $\lambda$ except for the zero vector.

Theorem.

Finding the eigenspace of a matrix

Consider the following matrix:

$$\boldsymbol{A}=\begin{pmatrix} 4&3\\1&6 \end{pmatrix}$$

Find the eigenspace of $\boldsymbol{A}$.

Solution. The characteristic polynomiallink of $\boldsymbol{A}$ is:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I}_2)&= \begin{vmatrix}4-\lambda&3\\1&6-\lambda \end{vmatrix}\\ &=(4-\lambda)(6-\lambda)-(3)(1)\\ &=24-4\lambda-6\lambda+\lambda^2-3\\ &=\lambda^2-10\lambda+21\\ &=(\lambda-3)(\lambda-7) \end{align*}$$

Therefore, the eigenvalues of $\boldsymbol{A}$ are $\lambda_1=3$ and $\lambda_2=7$. Next, let's find eigenvectors $\boldsymbol{x}_1$ and $\boldsymbol{x}_2$ corresponding to $\lambda_1$ and $\lambda_2$ respectively.

$$\begin{align*} (\boldsymbol{A}-\lambda_1\boldsymbol{I}_2)\boldsymbol{x}_1&=\boldsymbol{0}\\ \begin{pmatrix}4-3&3\\1&6-3 \end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}1&3\\1&3 \end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}1&3\\0&0 \end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \end{align*}$$

Taking the first row:

$$\begin{align*} x_{11}+3x_{12}&=0\\ x_{11}&=-3x_{12}\\ \end{align*}$$

Let $x_{12}=t$ where $t$ is some scalar. This means that $x_{11}=-3t$. Therefore, an eigenvector corresponding to eigenvalue $\lambda_1$ can be expressed as:

$$\boldsymbol{x}_1 =\begin{pmatrix} -3\\1 \end{pmatrix}t$$

The eigenspace $\mathcal{E}_1$ associated with $\lambda_1$ is the vector space below:

$$\mathcal{E}_1=\left\{\begin{pmatrix} -3\\1 \end{pmatrix}t\;|\;t\in\mathbb{R}\right\}$$

We can think of this eigenspace as a collection of eigenvectors corresponding to $\lambda_1$. Again, keep in mind that any vector in this eigenspace is an eigenvector associated with $\lambda_1$ except the zero vector.

Next, let's find the eigenvectors corresponding to the eigenvalue $\lambda_2=7$. We proceed like so:

$$\begin{align*} (\boldsymbol{A}-\lambda_2\boldsymbol{I}_2)\boldsymbol{x}_2&=\boldsymbol{0}\\ \begin{pmatrix}4-7&3\\1&6-7 \end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}-3&3\\1&-1 \end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}1&-1\\1&-1\end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}1&-1\\0&0\end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \end{align*}$$

Let $x_{22}=t$ where $t\in\mathbb{R}$. This means $x_{21}$ is:

$$\begin{align*} x_{21}-t&=0\\ x_{21}&=t\\ \end{align*}$$

The eigenspace $\mathcal{E}_2$ corresponding to eigenvalue $\lambda_2$ is:

$$\mathcal{E}_2=\left\{\begin{pmatrix}1\\1 \end{pmatrix}t\;|\;t\in\mathbb{R}\right\}$$
Theorem.

Eigenspace is a subspace

Let $\boldsymbol{A}$ be an $n\times{n}$ matrix and let $\lambda$ be an eigenvalue of $\boldsymbol{A}$. The eigenspace associated with $\lambda$ is a subspacelink of $\mathbb{R}^n$.

Proof. By definitionlink, the eigenspace of an eigenvalue $\lambda$ is:

$$\mathcal{E}_\lambda(\boldsymbol{A})= \mathrm{nullspace}(\boldsymbol{A}-\lambda\boldsymbol{I})$$

By theorem, the null space of any $m\times{n}$ matrix is a space of $\mathbb{R}^n$. This completes the proof.

Definition.

Eigenbasis

Let $\boldsymbol{A}$ be an $n\times{n}$ matrix. The eigenbasis of $\boldsymbol{A}$ is a basislink of $\mathbb{R}^n$ consisting of $n$ eigenvectors of $\boldsymbol{A}$. As we shall see later, not all square matrices have an eigenbasis.

Theorem.

Finding the eigenbasis of a matrix

Find the eigenbasis of the following matrix:

$$\boldsymbol{A}= \begin{pmatrix} 1&5\\2&4 \end{pmatrix}$$

Solution. The characteristic polynomial of $\boldsymbol{A}$ is:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I}) &=\begin{vmatrix} 1-\lambda&5\\ 2&4-\lambda \end{vmatrix}\\ &=(1-\lambda)(4-\lambda)-10\\ &=4-5\lambda+\lambda^2-10\\ &=\lambda^2-5\lambda-6\\ &=(\lambda-6)(\lambda+1) \end{align*}$$

Therefore, the eigenvalues of $\boldsymbol{A}$ are $\lambda_1=6$ and $\lambda_2=-1$.

Let's find an eigenvector associated with $\lambda_1=6$ like so:

$$\begin{align*} (\boldsymbol{A}-\lambda_1\boldsymbol{I}_2)\boldsymbol{x}_1&=\boldsymbol{0}\\ \begin{pmatrix}1-6&5\\2&4-6 \end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}-5&5\\2&-2 \end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}1&-1\\1&-1\end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}1&-1\\0&0\end{pmatrix}\begin{pmatrix}x_{11}\\x_{12}\end{pmatrix} &=\boldsymbol{0}\\ \end{align*}$$

Here, $x_{11}$ is a basic variablelink while $x_{12}$ is a free variablelink. Let $x_{12}=t$ where $t$ is some scalar. The solution can now be expressed as:

$$\begin{pmatrix} x_{11}\\x_{12} \end{pmatrix}= \begin{pmatrix} t\\t \end{pmatrix}= \begin{pmatrix} 1\\1 \end{pmatrix}t$$

Therefore, the eigenspace $\mathcal{E}_1$ associated with $\lambda_1$ is:

$$\mathcal{E}_1=\left\{\begin{pmatrix}1\\1 \end{pmatrix}t\;|\;t\in\mathbb{R}\right\}$$

The basis for $\mathcal{E}_1$ is:

$$\mathcal{B}_1=\left\{\begin{pmatrix}1\\1 \end{pmatrix}\right\}$$

Next, let's find the eigenvectors corresponding to $\lambda_2=-1$. We proceed like so:

$$\begin{align*} (\boldsymbol{A}-\lambda_2\boldsymbol{I}_2)\boldsymbol{x}_2&=\boldsymbol{0}\\ \begin{pmatrix}1+1&5\\2&4+1 \end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}2&5\\2&5 \end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \begin{pmatrix}2&5\\0&0 \end{pmatrix}\begin{pmatrix}x_{21}\\x_{22}\end{pmatrix} &=\boldsymbol{0}\\ \end{align*}$$

Let $x_{22}=t$ where $t\in\mathbb{R}$. The solution can be expressed as:

$$\begin{pmatrix} x_{21}\\x_{22} \end{pmatrix}= \begin{pmatrix} -2.5t\\t \end{pmatrix}= \begin{pmatrix} -2.5\\1 \end{pmatrix}t$$

The eigenspace $\mathcal{E}_2$ is:

$$\mathcal{E}_2=\left\{\begin{pmatrix}-2.5\\1 \end{pmatrix}t\;|\;t\in\mathbb{R}\right\}$$

The basis for $\mathcal{E}_2$ is:

$$\mathcal{B}_1=\left\{\begin{pmatrix}-2.5\\1 \end{pmatrix}\right\}$$

The eigenbasis for $\mathbb{R}^2$ is the union of $\mathcal{B}_1$ and $\mathcal{B}_2$ that is:

$$\mathcal{B}= \left\{ \begin{pmatrix}1\\1 \end{pmatrix},\; \begin{pmatrix}-2.5\\1 \end{pmatrix} \right\}$$

Note that the combined basis $\mathcal{B}_1\cup\mathcal{B}_2$ is guaranteed to be linearly independent given that they correspond to distinct eigenvalues. We will prove this laterlink in this guide.

Example.

Showing that a matrix does not have an eigenbasis

Consider the following matrix:

$$\boldsymbol{A}= \begin{pmatrix} 2&2\\ 0&2 \end{pmatrix}$$

Show that an eigenbasis does not exist for $\boldsymbol{A}$.

Solution. The characteristic polynomial of $\boldsymbol{A}$ is:

$$\begin{align*} \det(\boldsymbol{A}-\lambda\boldsymbol{I}) &=\begin{vmatrix} 2-\lambda&2\\ 0&2-\lambda \end{vmatrix}\\ &= (2-\lambda)(2-\lambda) \end{align*}$$

The eigenvalue of $\boldsymbol{A}$ is thus $\lambda=2$. The corresponding eigenvectors are obtained by:

$$\begin{align*} \begin{pmatrix} 0&2\\0&0 \end{pmatrix} \begin{pmatrix} x_1\\x_2 \end{pmatrix} =\begin{pmatrix} 0\\0 \end{pmatrix} \end{align*}$$

Here, $x_1$ is a free variable and $x_2$ is a basic variable. Let $x_1=t$ where $t$ is some scalar. The solution set can now be expressed as:

$$\begin{pmatrix} x_1\\x_2 \end{pmatrix} =\begin{pmatrix} t\\0 \end{pmatrix}= \begin{pmatrix} 1\\0 \end{pmatrix}t$$

Therefore, the eigenspace $\mathcal{E}$ associated with $\lambda=2$ is:

$$\mathcal{E}=\left\{\begin{pmatrix}1\\0 \end{pmatrix}t\;|\;t\in\mathbb{R}\right\}$$

The basis for $\mathcal{E}$ is:

$$\mathcal{B}= \left\{\begin{pmatrix}1\\0\end{pmatrix}\right\}$$

Since the eigenspace $\mathcal{E}$ is spanned by one basis vector, the dimensionlink of $\mathcal{E}$ is $1$. By definitionlink, the eigenbasis must be a basis for $\mathbb{R}^2$ in this case. However, one basis vector cannot span $\mathbb{R}^2$, which means there is no eigenbasis for $\boldsymbol{A}$.

Theorem.

Union of the basis for eigenspace corresponding to distinct eigenvalues is linearly independent

If $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_k$ are distinct eigenvalues of a matrix with corresponding eigenspace $\mathcal{E}_1$, $\mathcal{E}_2$, $\cdots$, $\mathcal{E}_k$ spanned by basis $\mathcal{B}_1$, $\mathcal{B}_2$, $\cdots$, $\mathcal{B}_k$ respectively, then $\mathcal{B}_1\cup \mathcal{B}_2\cup\cdots \cup\mathcal{B}_k$ is linearly independent.

Proof. Let $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_k$ be distinct eigenvalues with corresponding eigenspace $\mathcal{E}_1$, $\mathcal{E}_2$, $\cdots$, $\mathcal{E}_k$ spanned by basis $\mathcal{B}_1$, $\mathcal{B}_2$, $\cdots$, $\mathcal{B}_k$ respectively. Suppose the union of the eigenvectors is:

$$\mathcal{B}_1\cup \mathcal{B}_2\cup\cdots \cup\mathcal{B}_k= \{ \boldsymbol{v}_1, \boldsymbol{v}_2,\cdots, \boldsymbol{v}_n \}$$

Where $n\ge{k}$. Note that $n$ may be greater than $k$ because the dimension of the eigenspace corresponding to an eigenvalue may be greater than $1$.

Now, consider the criterialink for linear independence:

$$\begin{equation}\label{eq:S8hmL8bic98jR6Mwu9A} \boldsymbol{0}= c_1\boldsymbol{v}_1+ c_2\boldsymbol{v}_2+ \cdots+ c_n\boldsymbol{v}_n \end{equation}$$

Where $c_1$, $c_2$, $\cdots$, $c_n$ are some scalar coefficients. Our goal is to show that all of these coefficients must equal zero for \eqref{eq:S8hmL8bic98jR6Mwu9A} to hold - this will imply that $\{\boldsymbol{v}_1,\boldsymbol{v}_2,\cdots,\boldsymbol{v}_n\}$ is linearly independent by definitionlink. Let's group the eigenbasis vectors with the same eigenvalue:

$$\begin{equation}\label{eq:qmIrjxX8JAhopmIRuIR} \boldsymbol{0}= {\color{blue}(c_1\boldsymbol{v}_1+ c_2\boldsymbol{v}_2)}+ {\color{green}(c_3\boldsymbol{v}_3)}+ {\color{red}(c_4\boldsymbol{v}_4+ c_5\boldsymbol{v}_5+ c_6\boldsymbol{v}_6)}+ \cdots+ {\color{purple}(c_n\boldsymbol{v}_n)} \end{equation}$$

Here, we've color-coded the eigenbasis vectors with the same eigenvalue - this is only an example to demonstrate what we mean by the grouping of eigenvectors. For instance, in this case, $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ are the eigenbasis vectors of $\mathcal{B}_1$ corresponding to the first eigenvalue. Let's define the following vectors:

$$\begin{equation}\label{eq:RP57MID6AK8Y0KxDkzh} \begin{aligned} {\color{blue}\boldsymbol{w}_1}&= {\color{blue}(c_1\boldsymbol{v}_1+ c_2\boldsymbol{v}_2)}\\ {\color{green}\boldsymbol{w}_2}&= {\color{green}(c_3\boldsymbol{v}_3)}\\ {\color{red}\boldsymbol{w}_3}&= {\color{red}(c_4\boldsymbol{v}_4+ c_5\boldsymbol{v}_5+ c_6\boldsymbol{v}_6)}\\ &\;\;\vdots\\ {\color{purple}\boldsymbol{w}_k}&= {\color{purple}(c_n\boldsymbol{v}_n)} \end{aligned} \end{equation}$$

We can rewrite \eqref{eq:qmIrjxX8JAhopmIRuIR} as:

$$\begin{equation}\label{eq:OCeQpwpa6e9mJEXYLcg} \begin{aligned} {\color{blue}\boldsymbol{w}_1}+ {\color{green}\boldsymbol{w}_2}+ {\color{red}\boldsymbol{w}_3}+\cdots+ {\color{purple}\boldsymbol{w}_k} &=\boldsymbol{0} \end{aligned} \end{equation}$$

For \eqref{eq:OCeQpwpa6e9mJEXYLcg} to hold, the following must be true:

$$\begin{equation}\label{eq:ZwFNg4Pk0JaWJNtRaGK} \begin{gathered} {\color{blue}\boldsymbol{w}_1}=\boldsymbol{0},\;\; {\color{green}\boldsymbol{w}_2}=\boldsymbol{0},\;\; {\color{red}\boldsymbol{w}_3}=\boldsymbol{0},\;\; \cdots,\;\; {\color{purple}\boldsymbol{w}_k}=\boldsymbol{0} \end{gathered} \end{equation}$$

To understand why, suppose these eigenvectors are not equal to the zero vector:

$$\begin{gather*} {\color{blue}\boldsymbol{w}_1}\ne\boldsymbol{0},\;\; {\color{green}\boldsymbol{w}_2}\ne\boldsymbol{0},\;\; {\color{red}\boldsymbol{w}_3}\ne\boldsymbol{0},\;\; \cdots,\;\; {\color{purple}\boldsymbol{w}_k} \ne\boldsymbol{0} \end{gather*}$$

We can make $\boldsymbol{w}_1$ in \eqref{eq:OCeQpwpa6e9mJEXYLcg} the subject to get:

$$\begin{align*} {\color{blue}\boldsymbol{w}_1}= -{\color{green}\boldsymbol{w}_2} -{\color{red}\boldsymbol{w}_3}-\cdots -{\color{purple}\boldsymbol{w}_k} \end{align*}$$

This means that we can express $\boldsymbol{w}_1$ as a linear combination of the other vectors $\boldsymbol{w}_2$, $\boldsymbol{w}_3$, $\cdots$, $\boldsymbol{w}_k$. In other words, $\boldsymbol{w}_1$ is linearly dependent on the other vectors. Now, recall that all the eigenvectors corresponding to distinct eigenvalues are linearly independent by theoremlink. This means that $\boldsymbol{w}_1$, $\boldsymbol{w}_2$, $\cdots$, $\boldsymbol{w}_k$ cannot be eigenvectors, which implies they must be zero vectors. This is why \eqref{eq:ZwFNg4Pk0JaWJNtRaGK} holds.

Gong back to \eqref{eq:RP57MID6AK8Y0KxDkzh}, we have that:

$$\begin{align*} {\color{blue}(c_1\boldsymbol{v}_1+ c_2\boldsymbol{v}_2)}&=\boldsymbol{0}\\ {\color{green}(c_3\boldsymbol{v}_3)}&=\boldsymbol{0}\\ {\color{red}(c_4\boldsymbol{v}_4+ c_5\boldsymbol{v}_5+ c_6\boldsymbol{v}_6)}&=\boldsymbol{0}\\ \vdots\\ {\color{purple}(c_n\boldsymbol{v}_n)}&=\boldsymbol{0} \end{align*}$$

Because basis vectors are linearly independent by definitionlink, the following is true:

$$c_1=c_2=\cdots=c_n=0$$

Since all the coefficients must be zero for \eqref{eq:qmIrjxX8JAhopmIRuIR} to hold, the union of the basis $\{\boldsymbol{v}_1, \boldsymbol{v}_2, \cdots, \boldsymbol{v}_n\}$ is linearly independent by definitionlink. This completes the proof.

robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Comment
Citation
Ask a question or leave a feedback...