search
Search
Login
Map of Data Science
menu
menu search toc more_vert
Robocat
Guest 0reps
Sign up
Log in
account_circleMy Profile homeAbout paidPricing
emailContact us
exit_to_appLog out
Map of data science
Thanks for the thanks!
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
help Ask a question
Share on Twitter
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to
A
A
brightness_medium
share
arrow_backShare
Twitter
Facebook

Comprehensive Guide on Invertible Matrices and their Properties

Linear Algebra
chevron_right
Matrix Algebra
schedule Dec 21, 2022
Last updated
local_offer Linear Algebra
Tags
map
Check out the interactive map of data science
Definition.

Invertible matrix

Matrix $\boldsymbol{A}$ is said to be invertible if there exists a matrix $\boldsymbol{B}$ such that:

$$\begin{equation}\label{eq:rddCTUGDu2WnoLqjKPO} \boldsymbol{AB}= \boldsymbol{BA}= \boldsymbol{I}_n \end{equation}$$

Where $\boldsymbol{I}_n$ is the identity matrix. Note the following:

  • we call $\boldsymbol{B}$ the inverse matrix of $\boldsymbol{A}$.

  • since $\boldsymbol{AB}$ and $\boldsymbol{BA}$ are equal, if $\boldsymbol{B}$ is an inverse of $\boldsymbol{A}$, then $\boldsymbol{A}$ must also be an inverse of $\boldsymbol{B}$.

  • we typically use the notation $\boldsymbol{A}^{-1}$ for the inverse matrix of $\boldsymbol{A}$.

  • an invertible matrix is also known as a non-singular matrix. A non-invertible matrix is called a singular matrix.

Most textbooks use \eqref{eq:rddCTUGDu2WnoLqjKPO} as the definition of invertible matrices, but we will laterlink prove that:

  • if $\boldsymbol{AB}=\boldsymbol{I}_n$, then $\boldsymbol{BA}=\boldsymbol{I}_n$

  • if $\boldsymbol{BA}=\boldsymbol{I}_n$, then $\boldsymbol{AB}=\boldsymbol{I}_n$.

This means that to prove $\boldsymbol{A}$ is invertible, we only need to show either $\boldsymbol{AB}=\boldsymbol{I}_n$ or $\boldsymbol{BA}=\boldsymbol{I}_n$ - not both as one implies the other.

Example.

Showing that a matrix is invertible

Suppose we have the following two matrices:

$$\boldsymbol{A}= \begin{pmatrix} 2&1\\ 5&3\\ \end{pmatrix},\;\;\;\;\; \boldsymbol{B}= \begin{pmatrix} 3&-1\\ -5&2\\ \end{pmatrix}$$

Show that $\boldsymbol{A}$ and $\boldsymbol{B}$ are inverses of each other.

Solution. The product $\boldsymbol{AB}$ is:

$$\begin{align*} \boldsymbol{AB}&= \begin{pmatrix}2&1\\5&3\end{pmatrix} \begin{pmatrix}3&-1\\-5&2\end{pmatrix}\\ &=\begin{pmatrix}(2)(3)+(1)(-5)&(2)(-1)+(1)(2)\\(5)(3)+(3)(-5)&(5)(-1)+(3)(2)\end{pmatrix}\\ &=\begin{pmatrix}1&0\\0&1\end{pmatrix}\\ \end{align*}$$

The product $\boldsymbol{BA}$ is:

$$\begin{align*} \boldsymbol{BA}&= \begin{pmatrix}3&-1\\-5&2\end{pmatrix} \begin{pmatrix}2&1\\5&3\end{pmatrix}\\ &=\begin{pmatrix}(3)(2)+(-1)(5)&(3)(1)+(-1)(3)\\(-5)(2)+(2)(5)&(-5)(1)+(2)(3)\end{pmatrix}\\ &=\begin{pmatrix}1&0\\0&1\end{pmatrix}\\ \end{align*}$$

Since $\boldsymbol{AB}=\boldsymbol{BA}=\boldsymbol{I}_2$, we conclude that $\boldsymbol{A}$ is an inverse of $\boldsymbol{B}$ and vice versa. This means that both $\boldsymbol{A}$ and $\boldsymbol{B}$ are invertible.

Properties of matrix inverses

Theorem.

Invertible matrices must be square

If $\boldsymbol{A}$ is an invertible matrix, then $\boldsymbol{A}$ and $\boldsymbol{A}^{-1}$ must be square matrices.

Proof. Let $\boldsymbol{A}$ be an $m\times{n}$ invertible matrix. Since $\boldsymbol{A}$ is invertible, there exists an inverse matrix $\boldsymbol{B}$ such that:

$$\boldsymbol{AB}= \boldsymbol{BA}=\boldsymbol{I}_m$$

Since $\boldsymbol{AB}=\boldsymbol{I}_m$, the shape of $\boldsymbol{B}$ must be (in green):

$$(m\times{n})\cdot{\color{green}(n\times{m})} =m\times{m}$$

However, $\boldsymbol{BA}=\boldsymbol{I}_m$, which means that the shape of $\boldsymbol{B}$ is (in green):

$${\color{green}(m\times{n})}\cdot{(n\times{m})} =m\times{m}$$

The only way this is true is when $m=n$, that is, $\boldsymbol{A}$ and $\boldsymbol{B}$ are both square matrices. This completes the proof.

Theorem.

Matrices containing a row or column with all zeros are not invertible

If matrix $\boldsymbol{A}$ has a row or column with all zeros, then $\boldsymbol{A}$ is not invertible.

Proof. Suppose $\boldsymbol{A}$ is a matrix that has a row filled with zeros. For $\boldsymbol{A}$ to be invertible, there must exist some other matrix $\boldsymbol{B}$ such that:

$$\boldsymbol{AB}=\boldsymbol{BA}=\boldsymbol{I}_n$$

Suppose the $i$-th row of $\boldsymbol{A}$ is all zeros. By the nature of matrix multiplication, this means that the $i$-th row of $\boldsymbol{AB}$ will also be all zeros regardless of what $\boldsymbol{B}$ is. Therefore, $\boldsymbol{AB}$ cannot be an identity matrix and so by the definition of invertibility, $\boldsymbol{A}$ is not invertible.

Now, suppose the $j$-th column of $\boldsymbol{A}$ is all zeros. For any matrix $\boldsymbol{B}$, the $j$-th column of the product $\boldsymbol{BA}$ will also be all zeros. Again, this means that $\boldsymbol{BA}\ne\boldsymbol{I}_n$ and thus $\boldsymbol{A}$ is not invertible.

This completes the proof.

Theorem.

Uniqueness of inverses

If matrix $\boldsymbol{A}$ has an inverse $\boldsymbol{A}^{-1}$, then $\boldsymbol{A}^{-1}$ must be unique. In other words, a matrix can never have two or more inverse matrices.

Proof. Suppose $\boldsymbol{B}$ and $\boldsymbol{C}$ are inverses of matrix $\boldsymbol{A}$. Our goal is to show that $\boldsymbol{B}$ and $\boldsymbol{C}$ must be equal:

$$\begin{align*} \boldsymbol{C}&=\boldsymbol{C}\boldsymbol{I}_n\\ &=\boldsymbol{C}(\boldsymbol{AB})\\ &=(\boldsymbol{C}\boldsymbol{A})\boldsymbol{B}\\ &=\boldsymbol{I}_n\boldsymbol{B}\\ &=\boldsymbol{B}\\ \end{align*}$$

This means that the inverses $\boldsymbol{B}$ and $\boldsymbol{C}$ must be equivalent. Therefore, the inverse of a matrix, given that it exists, is unique. This completes the proof.

Theorem.

Taking the inverse of an inverse matrix returns the matrix itself

If $\boldsymbol{A}$ is an invertible $n\times{n}$ matrix, then:

$$(\boldsymbol{A}^{-1})^{-1}=\boldsymbol{A}$$

Proof. From the definition of invertible matrices, we have that:

$$\boldsymbol{A}^{-1}\boldsymbol{A}=\boldsymbol{I}_n$$

The fact that the product of $\boldsymbol{A}^{-1}$ and $\boldsymbol{A}$ is the identity matrix means that $\boldsymbol{A}$ is an inverse matrix of $\boldsymbol{A}^{-1}$, that is:

$$(\boldsymbol{A}^{-1})^{-1}=\boldsymbol{A}$$

This completes the proof.

Theorem.

Inverse of a product of two matrices

If $\boldsymbol{A}$ and $\boldsymbol{B}$ are some $n\times{n}$ invertible matrices, then:

$$(\boldsymbol{AB})^{-1}= \boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$$

Proof. We show that $\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$ is the inverse of $\boldsymbol{AB}$ like so:

$$\begin{align*} (\boldsymbol{AB})(\boldsymbol{B}^{-1}\boldsymbol{A}^{-1})&= \boldsymbol{A}(\boldsymbol{BB}^{-1})\boldsymbol{A}^{-1}\\ &=\boldsymbol{A}\boldsymbol{I}_n\boldsymbol{A}^{-1}\\ &=\boldsymbol{A}\boldsymbol{A}^{-1}\\ &=\boldsymbol{I}_n \end{align*}$$

We have that the product of matrix $\boldsymbol{AB}$ and matrix $\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$ is the identity matrix. This means that $\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$ is the inverse matrix of $\boldsymbol{AB}$ by definition, that is:

$$(\boldsymbol{AB})^{-1}= \boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$$

This completes the proof.

Theorem.

Inverse of a product of three matrices

If $\boldsymbol{A}$ is an $n\times{n}$ invertible matrix, then:

$$(\boldsymbol{ABC})^{-1}= \boldsymbol{C}^{-1}\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$$

Proof. We follow the same logic used in the previous proof:

$$\begin{align*} (\boldsymbol{ABC})(\boldsymbol{C}^{-1}\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}) &=\boldsymbol{ABC}\boldsymbol{C}^{-1}\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}\\ &=\boldsymbol{AB}\boldsymbol{I}_n\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}\\ &=\boldsymbol{AB}\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}\\ &=\boldsymbol{A}\boldsymbol{I}_n\boldsymbol{A}^{-1}\\ &=\boldsymbol{A}\boldsymbol{A}^{-1}\\ &=\boldsymbol{I}_n\\ \end{align*}$$

By definition of an invertible matrix, the inverse of $\boldsymbol{ABC}$ must be $\boldsymbol{C}^{-1}\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$, that is:

$$(\boldsymbol{ABC})^{-1}= \boldsymbol{C}^{-1}\boldsymbol{B}^{-1}\boldsymbol{A}^{-1}$$

This completes the proof.

Theorem.

Interchanging inverse and transpose

If $\boldsymbol{A}$ is an $n\times{n}$ invertible matrix, then:

$$\big(\boldsymbol{A}^T\big)^{-1}= \big(\boldsymbol{A}^{-1}\big)^T$$

Proof. We must show that the product of matrix $\boldsymbol{A}^T$ and $\big(\boldsymbol{A}^{-1}\big)^T$ is the identity matrix, that is:

$$\boldsymbol{A}^T (\boldsymbol{A^{-1}})^T=\boldsymbol{I}_n$$

Let's take the transpose of the left-hand side:

$$\begin{equation}\label{eq:Jn46XXU6f9TSMYBmKnC} \begin{aligned}[b] \Big(\boldsymbol{A}^T (\boldsymbol{A^{-1}})^T\Big)^T&= \Big((\boldsymbol{A^{-1}})^T\Big)^T \Big(\boldsymbol{A}^T\Big)^T\\ &=\boldsymbol{A^{-1}}\boldsymbol{A}\\ &=\boldsymbol{I}_n \end{aligned} \end{equation}$$

Here, we used the property $(\boldsymbol{AB})^T=\boldsymbol{B}^T\boldsymbol{A}^T$ for the first step. Now, we take the transpose of both sides:

$$\begin{align*} \boldsymbol{A}^T (\boldsymbol{A^{-1}})^T &=(\boldsymbol{I}_n)^T\\ &=\boldsymbol{I}_n \end{align*}$$

This means that $(\boldsymbol{A}^{-1})^T$ is the inverse of $\boldsymbol{A}^T$ by definition, that is:

$$\big(\boldsymbol{A}^T\big)^{-1}= \big(\boldsymbol{A}^{-1}\big)^T$$

This completes the proof.

Theorem.

If matrix A is invertible, then A transpose is also invertible

The following statements are all true:

  • if matrix $\boldsymbol{A}$ is invertible, then $\boldsymbol{A}^T$ is also invertible.

  • if matrix $\boldsymbol{A}^T$ is invertible, then $\boldsymbol{A}$ is also invertible.

  • if matrix $\boldsymbol{A}$ is not invertible, then $\boldsymbol{A}^T$ is also not invertible.

Proof. We know from theoremlink that if matrix $\boldsymbol{A}$ is invertible, then:

$$\begin{equation}\label{eq:oK1Iql8RPMvmpM8pPI6} \big(\boldsymbol{A}^T\big)^{-1}= \big(\boldsymbol{A}^{-1}\big)^T \end{equation}$$

Because we can find the inverse of $\boldsymbol{A}^T$, we have that $\boldsymbol{A}^T$ is also invertible by definition:

$$\boldsymbol{A}\text{ is invertible} \;\;\;\;\;{\color{blue}\implies}\;\;\;\;\; \boldsymbol{A}^T\text{ is invertible}$$

Next, let's prove the following:

$$\boldsymbol{A}^T\text{ is invertible} \;\;\;\;\;{\color{blue}\implies}\;\;\;\;\; \boldsymbol{A}\text{ is invertible}$$

Let $\boldsymbol{B}^T$ be an invertible matrix. Substituting $\boldsymbol{B}^T$ into $\boldsymbol{A}$ in \eqref{eq:oK1Iql8RPMvmpM8pPI6} gives:

$$\begin{align*} \big((\boldsymbol{B}^T)^T\big)^{-1}= \big((\boldsymbol{B}^T)^{-1}\big)^T\\ \boldsymbol{B}^{-1}= \big((\boldsymbol{B}^T)^{-1}\big)^T \end{align*}$$

Since $\boldsymbol{B}^T$ is invertible, we know that $(\boldsymbol{B}^T)^{-1}$ exists. The inverse of $\boldsymbol{B}$ is the transpose of $(\boldsymbol{B}^T)^{-1}$. Because we can find the inverse of $\boldsymbol{B}$, we have that $\boldsymbol{B}$ is also invertible. Now, instead of using the letter $\boldsymbol{B}$, we can use the letter $\boldsymbol{A}$ for consistency to conclude:

$$\boldsymbol{A}^T\text{ is invertible} \;\;\;\;\;{\color{blue}\implies}\;\;\;\;\; \boldsymbol{A}\text{ is invertible}$$

Now, recall from our guide on proof by contraposition that this is logically equivalent to the following contrapositive statement:

$$\boldsymbol{A}\text{ is not invertible} \;\;\;\;\;{\color{blue}\implies}\;\;\;\;\; \boldsymbol{A}^T\text{ is not invertible}$$

This completes the proof.

Theorem.

Inverse of a scalar-matrix product

If $\boldsymbol{A}$ is an $n\times{n}$ invertible matrix and $k$ is any non-zero scalar, then:

$$(k\boldsymbol{A})^{-1}= k^{-1}\boldsymbol{A}^{-1}$$

Proof. We use the matrix property of bringing the scalar values to the front:

$$\begin{align*} (k\boldsymbol{A})(k^{-1}\boldsymbol{A}^{-1}) &=kk^{-1}\boldsymbol{A}\boldsymbol{A}^{-1}\\ &=(1)\boldsymbol{I}_n\\ &=\boldsymbol{I}_n\\ \end{align*}$$

Therefore, $k^{-1}\boldsymbol{A}^{-1}$ is the inverse of $k\boldsymbol{A}$ by definition, that is:

$$(k\boldsymbol{A})^{-1}= k^{-1}\boldsymbol{A}^{-1}$$

This completes the proof.

Theorem.

If AB is invertible, then A and B must also be invertible

Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be square matrices of the same shape. If the matrix product $\boldsymbol{AB}$ is invertible, then $\boldsymbol{A}$ and $\boldsymbol{B}$ must also be invertible.

Proof. Because $\boldsymbol{AB}$ is invertible, there exists a matrix $\boldsymbol{C}$ such that:

$$\begin{align*} \boldsymbol{C}(\boldsymbol{AB})&=\boldsymbol{I}_n \end{align*}$$

Using the associativity of matrix multiplication:

$$\begin{align*} (\boldsymbol{C}\boldsymbol{A})\boldsymbol{B}&=\boldsymbol{I}_n \end{align*}$$

By definition, this means that $\boldsymbol{CA}$ is the inverse of $\boldsymbol{B}$, that is, $\boldsymbol{B}^{-1}=\boldsymbol{CA}$. Because there exists an inverse of $\boldsymbol{B}$, then $\boldsymbol{B}$ must be invertible by definition.

We can use the same logic to show that $\boldsymbol{A}$ is also invertible. Because $\boldsymbol{AB}$ is invertible, there exists a matrix $\boldsymbol{D}$ such that:

$$\begin{align*} (\boldsymbol{AB})\boldsymbol{D}&= \boldsymbol{I}_n\\ \boldsymbol{A}(\boldsymbol{BD})&= \boldsymbol{I}_n \end{align*}$$

This means that $\boldsymbol{A}^{-1}=\boldsymbol{BD}$, and thus $\boldsymbol{A}$ is invertible by definition. This completes the proof.

Theorem.

Relaxing the definition of invertible matrices

Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be $n\times{n}$ square matrices. The following statements are true:

  • if $\boldsymbol{BA}=\boldsymbol{I}_n$, then $\boldsymbol{B}=\boldsymbol{A}^{-1}$ and $\boldsymbol{AB}=\boldsymbol{I}_n$.

  • if $\boldsymbol{AB}=\boldsymbol{I}_n$, then $\boldsymbol{B}=\boldsymbol{A}^{-1}$ and $\boldsymbol{BA}=\boldsymbol{I}_n$.

Proof. We will prove the first statement - the proof for the second statement is identical. To follow this proof, we will use a theoremlink covered in the next section.

Let matrices $\boldsymbol{A}$ and $\boldsymbol{B}$ be square matrices of shape $n\times{n}$. Assume $\boldsymbol{BA}=\boldsymbol{I}_n$. To prove that $\boldsymbol{B}=\boldsymbol{A}^{-1}$, we must show that $\boldsymbol{A}$ is invertible, that is, $\boldsymbol{A}^{-1}$ exists. This allows us to manipulate our assumption to obtain the desired results:

$$\begin{equation}\label{eq:CCAaKX8jLF61MiDz8RJ} \begin{aligned} \boldsymbol{BA}&=\boldsymbol{I}_n\\ \boldsymbol{BA}\boldsymbol{A}^{-1}&=\boldsymbol{I}_n\boldsymbol{A}^{-1}\\ \boldsymbol{B}\boldsymbol{I}_n&=\boldsymbol{A}^{-1}\\ \boldsymbol{B}&=\boldsymbol{A}^{-1}\\ \end{aligned} \end{equation}$$

Multiplying $\boldsymbol{A}$ to both sides gives us $\boldsymbol{AB}=\boldsymbol{I}_n$ as well.

We know from theorem that if the homogeneous system $\boldsymbol{Ax}=\boldsymbol{0}$ only has the trivial solution $\boldsymbol{x}=\boldsymbol{0}$, then $\boldsymbol{A}$ is invertible. Let $\boldsymbol{x}_0$ be any solution to $\boldsymbol{Ax}=\boldsymbol{0}$, that is:

$$\boldsymbol{A}\boldsymbol{x}_0=\boldsymbol{0}$$

Multiplying both sides by $\boldsymbol{B}$ gives:

$$\begin{align*} \boldsymbol{B}\boldsymbol{A}\boldsymbol{x}_0 &=\boldsymbol{B}\boldsymbol{0}\\ \boldsymbol{I}_n\boldsymbol{x}_0 &=\boldsymbol{0}\\ \boldsymbol{x}_0 &=\boldsymbol{0}\\ \end{align*}$$

Therefore, the only solution to the homogeneous system is the zero vector, which is the trivial solution. Therefore, $\boldsymbol{A}$ must be invertible - performing steps \eqref{eq:CCAaKX8jLF61MiDz8RJ} completes the proof.

robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Ask a question or leave a feedback...