**Linear Algebra**

# Comprehensive Guide on Rank and Nullity of Matrices

*schedule*Aug 12, 2023

*toc*Table of Contents

*expand_more*

**mathematics behind data science**with 100+ top-tier guides

Start your free 7-days trial now!

# Rank of a matrix

The rank of a matrix $\boldsymbol{A}$ is the dimensionlink of the column spacelink of $\boldsymbol{A}$, that is:

## Finding the rank of a matrix (1)

Consider the following matrix:

Find the rank of $\boldsymbol{A}$.

Solution. The rank of $\boldsymbol{A}$ is defined as the dimensionlink of the column spacelink of $\boldsymbol{A}$ denoted as $\mathrm{col}(\boldsymbol{A})$. The dimension is defined as the number of basis vectorslink for $\mathrm{col}(\boldsymbol{A})$. Therefore, to find the rank of $\boldsymbol{A}$, we need to find the basis for $\mathrm{col}(\boldsymbol{A})$.

The column space of $\boldsymbol{A}$ is defined as the span of its column vectors:

By definition, the two vectors do not form a basis for $\mathrm{col}(\boldsymbol{A})$ because they are linearly dependentlink. Therefore, we can remove the second vector - this will give us a basis for $\mathrm{col}(\boldsymbol{A})$ below:

Since this single vector spans the column space of $\boldsymbol{A}$ and is linearly independent, we have that this vector forms a basis for $\mathrm{col}(\boldsymbol{A})$. Therefore, the rank of $\mathrm{col}(\boldsymbol{A})$ is $1$.

## Finding the rank of a matrix (2)

Consider the following matrix:

Find the rank of $\boldsymbol{A}$.

Solution. To find the rank of $\boldsymbol{A}$, we must find the basis for the column space of $\boldsymbol{A}$. The column space of $\boldsymbol{A}$ is defined as:

We now must check whether or not the three vectors form a basis for $\mathrm{col}(\boldsymbol{A})$. The third vector is double the first vector so the third vector is redundant. The remaining two vectors are linearly independent, and thus the basis for $\mathrm{col}(\boldsymbol{A})$ is:

Since two vectors form a basis for the column space of $\boldsymbol{A}$, the dimension of the column space is $2$. Therefore, we conclude that $\mathrm{rank}(\boldsymbol{A})=2$.

# Nullity of matrix

The nullity of a matrix $\boldsymbol{A}$ is the dimensionlink of the null spacelink of $\boldsymbol{A}$, that is:

## Finding the nullity of a matrix (1)

Consider the following matrix:

Find the nullity of $\boldsymbol{A}$.

Solution. We first need to find the basis for the null space of $\boldsymbol{A}$. The null space of $\boldsymbol{A}$ is the set of all vectors $\boldsymbol{x}$ such that:

Writing this in matrix notation:

Let's solve this linear system. We row-reduce the coefficient matrix:

We have that $x_2$ is a free variablelink, so let's set $x_2=t$ where $t$ is some scalar. Substituting this into the first row gives:

Therefore, the null space of $\boldsymbol{A}$ is:

This means that the basis for the null space of $\boldsymbol{A}$ is:

Since a single vector forms a basis for the null space of $\boldsymbol{A}$, the dimension of the null space is $1$. Therefore, we conclude that $\mathrm{nullity}(\boldsymbol{A})=1$.

## Finding the nullity of a matrix (2)

Consider the following matrix:

Find the nullity of $\boldsymbol{A}$.

Solution. Let's first find the null space of $\boldsymbol{A}$. By definition, the null space of $\boldsymbol{A}$ is the solution set of the following homogeneous linear system:

Let's row-reduce $\boldsymbol{A}$ to get:

We have that $x_2$ and $x_3$ are free variableslink. We set $x_2=r$ and $x_3=t$ where $r$ and $t$ are some scalars. The solution to the homogeneous linear system is therefore:

This means that the null space of $\boldsymbol{A}$ is spanned by the following linearly independent vectors:

These two vectors form a basis for the null space of $\boldsymbol{A}$, which means that the dimension of the null space is $2$. We conclude that $\mathrm{nullity}(\boldsymbol{A})=2$.

# Nullity is equal to the number of non-pivot columns

The nullity or the dimension of the null space of a matrix $\boldsymbol{A}$ is equal to the number of non-pivot columns of the reduced row echelon formlink of $\boldsymbol{A}$, that is:

Proof. Consider the homogeneous linear system $\boldsymbol{Ax}=\boldsymbol{0}$. Suppose the reduced row echelon form of $\boldsymbol{A}$ has $2$ non-pivot columns, say the 3rd and 5th column. This means that the system has $2$ free variableslink and so the general solution can be written as:

Where $x_3=r$ and $x_5=t$ for some $t,r\in\mathbb{R}$. Clearly, the two vectors in \eqref{eq:FkyBOPfLTItKai4Efbh} are linearly independentlink because:

the first vector has a $1$ in slot $3$ whereas the second vector has a $0$ in slot $3$.

the first vector has a $0$ in slot $5$ whereas the second vector has a $1$ in slot $5$.

Next, from \eqref{eq:FkyBOPfLTItKai4Efbh}, we also know that the general solution $\boldsymbol{x}$ can be expressed as a linear combination of these vectors. In other words, these vectors span the null space of $\boldsymbol{A}$.

Because these vectors are linearly independent and span the null space of $\boldsymbol{A}$, we conclude that they form a basis for the null space of $\boldsymbol{A}$ by definitionlink. In general, if there are $k$ non-pivot columns of $\mathrm{rref}(\boldsymbol{A})$, then there will be $k$ free variables, which means that $k$ vectors form a basis for the null space of $\boldsymbol{A}$. The dimension of the null space is $k$ and thus the nullity of $\boldsymbol{A}$ is also $k$.

This completes the proof.

# Rank of matrix A is equal to the number of pivot columns of rref(A)

The rank or the dimension of the column space of a matrix $\boldsymbol{A}$ is equal to the number of pivot columns of the reduced row echelon form of $\boldsymbol{A}$, that is:

Proof. By definitionlink, the dimension of the column space of matrix $\boldsymbol{A}$ is equal to the number of basis vectors for the column space. From theoremlink, we know that the columns of $\boldsymbol{A}$ corresponding to the pivot columns in $\mathrm{rref}(\boldsymbol{A})$ form a basis for the column space of $\boldsymbol{A}$. If there are $k$ pivot columns in $\mathrm{rref}(\boldsymbol{A})$, then there will be $k$ basis vectors for $\mathrm{col}(\boldsymbol{A})$, which means that the dimension of $\mathrm{col}(\boldsymbol{A})$ will be $k$. This completes the proof.

# Rank of matrix is equal to its number of linearly independent columns

The rank of a matrix $\boldsymbol{A}$ is equal to the number of linearly independent columns of $\boldsymbol{A}$, that is:

Proof. From theoremlink, we know that the rank of a matrix $\boldsymbol{A}$ is equal to the number of pivot columns of $\mathrm{rref}(\boldsymbol{A})$. From theoremlink, we also know that the number of pivot columns in $\mathrm{rref}(\boldsymbol{A})$ equals the number of linearly independent columns. Therefore, the rank of $\boldsymbol{A}$ is equal to the number of linearly independent columns of $\boldsymbol{A}$. This completes the proof.

# Elementary row operations preserve linear independence

Suppose we have some matrix $\boldsymbol{A}$ and we perform an elementary row operation on $\boldsymbol{A}$ to get matrix $\boldsymbol{B}$. A given set of column vectors of some matrix $\boldsymbol{A}$ is linearly independent if and only if the corresponding vectors of $\boldsymbol{B}$ are linearly independent.

Proof. Let $\mathrm{rref}(\boldsymbol{A})$ be the reduced row echelon form of $\boldsymbol{A}$ and let matrix $\boldsymbol{B}$ be the result of applying an elementary row operation on $\boldsymbol{A}$.

By definition, the reduced row echelon form of $\boldsymbol{A}$ is obtained by performing a series of elementary row operations on $\boldsymbol{A}$.

We are given that $\boldsymbol{B}$ is obtained after the first elementary row operation so if we apply all the subsequent elementary row operations on $\boldsymbol{B}$, then we would end up with the reduced row echelon form of $\boldsymbol{A}$. In other words, we have $\mathrm{rref}(\boldsymbol{A})=\mathrm{rref}(\boldsymbol{B})$.

This means that $\mathrm{rref}(\boldsymbol{A})$ and $\mathrm{rref}(\boldsymbol{B})$ have the same pivot columns. By theoremlink, the columns of $\boldsymbol{A}$ corresponding to the pivot columns of $\mathrm{rref}(\boldsymbol{A})$ are linearly independent, and the same can be said for $\boldsymbol{B}$. This means that, for instance, if $\boldsymbol{a}_1$, $\boldsymbol{a}_3$ and $\boldsymbol{a}_4$ are linearly independent columns of $\boldsymbol{A}$, then $\boldsymbol{b}_1$, $\boldsymbol{b}_3$ and $\boldsymbol{b}_4$ must also be linearly independent columns of $\boldsymbol{B}$.

The converse is true as well, that is, if $\boldsymbol{b}_1$, $\boldsymbol{b}_3$ and $\boldsymbol{b}_4$ are linearly independent columns of $\boldsymbol{B}$, then $\boldsymbol{a}_1$, $\boldsymbol{a}_3$ and $\boldsymbol{a}_4$ must also be linearly independent columns. This completes the proof.

# Elementary row operations do not affect the matrix rank

Performing elementary row operations on a matrix $\boldsymbol{A}$ preserves the rank of $\boldsymbol{A}$.

Proof. From theoremlink, we know that elementary row operations maintain the linear independence between columns. This means that elementary row operations preserve the number of linearly independent columns. By theoremlink, the rank of a matrix $\boldsymbol{A}$ is equal to the number of linearly independent columns of $\boldsymbol{A}$. Therefore, we conclude that performing an elementary row operation on $\boldsymbol{A}$ preserves the rank of $\boldsymbol{A}$.

This completes the proof.

# Multiplying a vector of a linearly independent set by a scalar preserves linear independence

If $S$ is a linearly independent set of vectors, then multiplying any vector in $S$ by a scalar $k$ will preserve the linear independence of $S$.

Proof. Suppose $S=\{\boldsymbol{v}_1,\boldsymbol{v}_2,\boldsymbol{v}_3\}$ is a linearly independent set of vectors. By definitionlink of linear independence, this means that:

This equation only holds when the coefficients $c_1$, $c_2$ and $c_3$ are all equal to zero. Now, let's multiply one of the vectors in $S$, say $\boldsymbol{v}_2$, by a scalar $k$ to get $k\boldsymbol{v}_2$. We now use the definitionlink of linear dependence to show that $\{\boldsymbol{v}_1,k\boldsymbol{v}_2,\boldsymbol{v}_3\}$ is linearly independent:

Where $d_1$, $d_2$ and $d_3$ are some scalars. Since $d_2k$ is just another scalar, let's rewrite it as $d_4$ like so:

We know from \eqref{eq:AW6mOv2M1B1oKxihH5e} that the only coefficients to make the equation hold are all zeros. This means that the only way for \eqref{eq:i1dQ8X9HQnIWTCRofta} to hold is if $d_1=d_4=d_3=0$. Therefore, we conclude that $\boldsymbol{v}_1$, $k\boldsymbol{v}_2$ and $\boldsymbol{v}_3$ are linearly independent.

This completes the proof.

# Elementary columns operations do not affect the matrix rank

Performing elementary column operations on a matrix $\boldsymbol{A}$ preserves the rank of $\boldsymbol{A}$.

Proof. There are three types of elementary column operations so let's show that the matrix rank is preserved after each type of elementary column operation. By theoremlink, the matrix rank is equal to the number of linearly independent columns of the matrix. Therefore, we need to show that an elementary column operation preserves the number of linearly independent columns.

The first type of elementary column operation is a column swap. Clearly, swapping two columns does not affect the linear dependence between them. For instance, if column vectors $\boldsymbol{a}_1$ and $\boldsymbol{a}_2$ are linearly independent, then $\boldsymbol{a}_2$ and $\boldsymbol{a}_1$ will also be linearly independent - the ordering does not matter.

The second type of elementary column operation is scalar multiplication. By theoremlink, given a linearly independent set of vectors $S$, multiplying any vector in $S$ by a scalar will preserve the linear independence of $S$.

The third type of elementary column operation is multiplying a vector by a scalar $k$ and then adding it to another column vector.

Let $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ be linearly independent columns. By definitionlink, we have that:

Where $c_1$ and $c_2$ are both zero.

Now, our goal is to show that $k\boldsymbol{v}_1+\boldsymbol{v}_2$ and $\boldsymbol{v}_2$ are also linearly independent. We once again use the definitionlink of linear independence to check whether the two vectors are independent:

Where $d_1$ and $d_2$ are both scalars. Simplifying this gives:

Here, $kd_1$ and $d_1+d_2$ are some constants, so let's replace them with some constants $e_1$ and $e_2$ like so:

Equation \eqref{eq:cPUQtzZQGPHIZNcAaiH} becomes:

From \eqref{eq:fDWRXmiM3xwcK95NA6l}, we know that the scalar coefficients of $\boldsymbol{v}_1$ and $\boldsymbol{v}_2$ must be zero. Therefore, we have that $\boldsymbol{e}_1= \boldsymbol{e}_2= 0$ for \eqref{eq:rCbL4BoUSVsNz9yT4RQ} to hold as well. From \eqref{eq:WK9M3CBHt5bKvfJoRrB}, since $k$ is nonzero, $d_1=0$. This also means that $d_2=0$. Because $d_1=d_2=0$ for the equality \eqref{eq:mwy9PyejhBAYxoQ84Pr} to hold, we conclude that $k\boldsymbol{v}_1+\boldsymbol{v}_2$ and $\boldsymbol{v}_2$ are also linearly independent.

We have now shown that all three types of elementary column operations preserve the linear independence of columns. This means that the rank of a matrix is preserved after an elementary column operation. This completes the proof.

# Rank of a matrix transpose

If $\boldsymbol{A}$ is a matrix, then the rank of $\boldsymbol{A}^T$ is:

Proof. Consider the matrix $\boldsymbol{A}$ and its transpose $\boldsymbol{A}^T$ below:

Let $\mathrm{rank}(\boldsymbol{A})=r$ and $\mathrm{rank}(\boldsymbol{A}^T)=t$. Our goal is to show that $r=t$.

Notice that performing an elementary row operation on $\boldsymbol{A}$ is equivalent to performing an elementary column operation on $\boldsymbol{A}^T$. For instance, suppose we perform an elementary row operation of multiplying the second row of $\boldsymbol{A}$ by some scalar $k$. This is equivalent to performing an elementary column operation of multiplying the second column of $\boldsymbol{A}^T$ by $k$ like so:

Now, suppose we reduced $\boldsymbol{A}$ to its reduced row echelon form $\mathrm{rref}(\boldsymbol{A})$. If we perform the equivalent column operations on $\boldsymbol{A}^T$, we may end up with:

Now, let's perform a series of column operations to the left matrix as well as equivalent row operations on the right matrix to get:

We can clearly see that the number of linearly independent columns in the reduced form of $\boldsymbol{A}$ and $\boldsymbol{A}^T$ is the same. By theoremlink, matrix rank is equal to the number of linearly independent columns, and so we have that:

By theoremlink and theoremlink, we know that elementary row/column operations do not affect the rank of a matrix. Since the reduced form of $\boldsymbol{A}$ and $\boldsymbol{A}^T$ is obtained by performing a series of elementary row/column operations, we have that:

This means that \eqref{eq:gRA0TYBQOoKkHqDgLwx} becomes:

This completes the proof.

# Rank-nullity theorem

The sum of the rank and nullity of an $m\times{n}$ matrix $\boldsymbol{A}$ is:

Proof. Let $\boldsymbol{A}$ be an $m\times{n}$ matrix. We know that every column in $\boldsymbol{A}$ must either correspond to a pivot or a non-pivot column in the reduced row echelon form of $\boldsymbol{A}$, which means that:

From theoremlink, the number of pivot columns in $\mathrm{rref}(\boldsymbol{A})$ is equal to the rank of $\boldsymbol{A}$, that is:

From theoremlink, the number of non-pivot columns in $\mathrm{rref}(\boldsymbol{A})$ is equal to the nullity of $\boldsymbol{A}$, that is:

Finally, the number of columns in $\mathrm{rref}(\boldsymbol{A})$ is the same as the number of columns in $\boldsymbol{A}$. Therefore, we conclude that:

This completes the proof.

# Rank of PA and AP where P is invertible

Let $\boldsymbol{A}$ be a square matrix. If $\boldsymbol{P}$ is an invertible matrix, then:

Proof. We will first prove that $\mathrm{rank}(\boldsymbol{A}) =\mathrm{rank}(\boldsymbol{PA})$. Let $\boldsymbol{U}$ be some invertible matrix. Note that we use the notation $\boldsymbol{U}$ instead of $\boldsymbol{P}$ to avoid confusion later. By theoremlink, because $\boldsymbol{U}$ is invertible, we can express $\boldsymbol{U}$ in terms of elementary matrices $\boldsymbol{E}_k$, $\cdots$, $\boldsymbol{E}_2$, $\boldsymbol{E}_1$ like so:

Now, substituting this $\boldsymbol{U}$ into $\mathrm{rank}(\boldsymbol{A}) =\mathrm{rank}(\boldsymbol{UA})$ gives:

By theoremlink, we know that elementary row operations do not affect the matrix rank. Since multiplying an elementary matrix to $\boldsymbol{A}$ is equivalent to performing an elementary row operation on $\boldsymbol{A}$, we have that:

Next, let's show that $\mathrm{rank}(\boldsymbol{AP})=\mathrm{rank}(\boldsymbol{A})$. By theoremlink and theoremlink, we have that:

However, we have just proven that $\mathrm{rank}(\boldsymbol{UA}) = \mathrm{rank}(\boldsymbol{A})$. Therefore, we can reduce \eqref{eq:cEZXdQF3V7ZIIu92tIr} to:

This completes the proof.