search
Search
Login
Unlock 100+ guides
menu
menu
web
search toc
close
Comments
Log in or sign up
Cancel
Post
account_circle
Profile
exit_to_app
Sign out
What does this mean?
Why is this true?
Give me some examples!
search
keyboard_voice
close
Searching Tips
Search for a recipe:
"Creating a table in MySQL"
Search for an API documentation: "@append"
Search for code: "!dataframe"
Apply a tag filter: "#python"
Useful Shortcuts
/ to open search panel
Esc to close search panel
to navigate between search results
d to clear all current filters
Enter to expand content preview
icon_star
Doc Search
icon_star
Code Search Beta
SORRY NOTHING FOUND!
mic
Start speaking...
Voice search is only supported in Safari and Chrome.
Navigate to
check_circle
Mark as learned
thumb_up
0
thumb_down
0
chat_bubble_outline
0
Comment
auto_stories Bi-column layout
settings

Comprehensive Guide on Rank and Nullity of Matrices

schedule Aug 12, 2023
Last updated
local_offer
Linear Algebra
Tags
mode_heat
Master the mathematics behind data science with 100+ top-tier guides
Start your free 7-days trial now!
Definition.

Rank of a matrix

The rank of a matrix A is the dimensionlink of the column spacelink of A, that is:

rank(A)=dim(col(A))
Example.

Finding the rank of a matrix (1)

Consider the following matrix:

A=(1224)

Find the rank of A.

Solution. The rank of A is defined as the dimensionlink of the column spacelink of A denoted as col(A). The dimension is defined as the number of basis vectorslink for col(A). Therefore, to find the rank of A, we need to find the basis for col(A).

The column space of A is defined as the span of its column vectors:

col(A)=span((12),(24))

By definition, the two vectors do not form a basis for col(A) because they are linearly dependentlink. Therefore, we can remove the second vector - this will give us a basis for col(A) below:

{(12)}

Since this single vector spans the column space of A and is linearly independent, we have that this vector forms a basis for col(A). Therefore, the rank of col(A) is 1.

Example.

Finding the rank of a matrix (2)

Consider the following matrix:

A=(122244316)

Find the rank of A.

Solution. To find the rank of A, we must find the basis for the column space of A. The column space of A is defined as:

col(A)=span((123),(241),(246))

We now must check whether or not the three vectors form a basis for col(A). The third vector is double the first vector so the third vector is redundant. The remaining two vectors are linearly independent, and thus the basis for col(A) is:

{(123),(241)}

Since two vectors form a basis for the column space of A, the dimension of the column space is 2. Therefore, we conclude that rank(A)=2.

Definition.

Nullity of matrix

The nullity of a matrix A is the dimensionlink of the null spacelink of A, that is:

nullity(A)=dim(nullspace(A))
Example.

Finding the nullity of a matrix (1)

Consider the following matrix:

A=(1224)

Find the nullity of A.

Solution. We first need to find the basis for the null space of A. The null space of A is the set of all vectors x such that:

Ax=0

Writing this in matrix notation:

(1224)(x1x2)=(00)

Let's solve this linear system. We row-reduce the coefficient matrix:

(1224)(1200)

We have that x2 is a free variablelink, so let's set x2=t where t is some scalar. Substituting this into the first row gives:

x1=2t

Therefore, the null space of A is:

nullspace(A)={(2tt)|tR}

This means that the basis for the null space of A is:

{(21)}

Since a single vector forms a basis for the null space of A, the dimension of the null space is 1. Therefore, we conclude that nullity(A)=1.

Example.

Finding the nullity of a matrix (2)

Consider the following matrix:

A=(132264396)

Find the nullity of A.

Solution. Let's first find the null space of A. By definition, the null space of A is the solution set of the following homogeneous linear system:

(132264396)(x1x2x3)=(000)

Let's row-reduce A to get:

(132264396)(132000000)

We have that x2 and x3 are free variableslink. We set x2=r and x3=t where r and t are some scalars. The solution to the homogeneous linear system is therefore:

(x1x2x3)=(3r2trt)=(310)r+(201)t

This means that the null space of A is spanned by the following linearly independent vectors:

{(310),(201)}

These two vectors form a basis for the null space of A, which means that the dimension of the null space is 2. We conclude that nullity(A)=2.

Theorem.

Nullity is equal to the number of non-pivot columns

The nullity or the dimension of the null space of a matrix A is equal to the number of non-pivot columns of the reduced row echelon formlink of A, that is:

nullity(A)=dim(nullspace(A))=number of non-pivot columns of rref(A)

Proof. Consider the homogeneous linear system Ax=0. Suppose the reduced row echelon form of A has 2 non-pivot columns, say the 3rd and 5th column. This means that the system has 2 free variableslink and so the general solution can be written as:

(1)x=(x1x2x3x4x5)=(rt)=(10)r+(01)t

Where x3=r and x5=t for some t,rR. Clearly, the two vectors in (1) are linearly independentlink because:

  • the first vector has a 1 in slot 3 whereas the second vector has a 0 in slot 3.

  • the first vector has a 0 in slot 5 whereas the second vector has a 1 in slot 5.

Next, from (1), we also know that the general solution x can be expressed as a linear combination of these vectors. In other words, these vectors span the null space of A.

Because these vectors are linearly independent and span the null space of A, we conclude that they form a basis for the null space of A by definitionlink. In general, if there are k non-pivot columns of rref(A), then there will be k free variables, which means that k vectors form a basis for the null space of A. The dimension of the null space is k and thus the nullity of A is also k.

This completes the proof.

Theorem.

Rank of matrix A is equal to the number of pivot columns of rref(A)

The rank or the dimension of the column space of a matrix A is equal to the number of pivot columns of the reduced row echelon form of A, that is:

rank(A)=dim(col(A))=number of pivot columns of rref(A)

Proof. By definitionlink, the dimension of the column space of matrix A is equal to the number of basis vectors for the column space. From theoremlink, we know that the columns of A corresponding to the pivot columns in rref(A) form a basis for the column space of A. If there are k pivot columns in rref(A), then there will be k basis vectors for col(A), which means that the dimension of col(A) will be k. This completes the proof.

Theorem.

Rank of matrix is equal to its number of linearly independent columns

The rank of a matrix A is equal to the number of linearly independent columns of A, that is:

rank(A)=number of linearly independent columns of A

Proof. From theoremlink, we know that the rank of a matrix A is equal to the number of pivot columns of rref(A). From theoremlink, we also know that the number of pivot columns in rref(A) equals the number of linearly independent columns. Therefore, the rank of A is equal to the number of linearly independent columns of A. This completes the proof.

Theorem.

Elementary row operations preserve linear independence

Suppose we have some matrix A and we perform an elementary row operation on A to get matrix B. A given set of column vectors of some matrix A is linearly independent if and only if the corresponding vectors of B are linearly independent.

Proof. Let rref(A) be the reduced row echelon form of A and let matrix B be the result of applying an elementary row operation on A.

By definition, the reduced row echelon form of A is obtained by performing a series of elementary row operations on A.

We are given that B is obtained after the first elementary row operation so if we apply all the subsequent elementary row operations on B, then we would end up with the reduced row echelon form of A. In other words, we have rref(A)=rref(B).

This means that rref(A) and rref(B) have the same pivot columns. By theoremlink, the columns of A corresponding to the pivot columns of rref(A) are linearly independent, and the same can be said for B. This means that, for instance, if a1, a3 and a4 are linearly independent columns of A, then b1, b3 and b4 must also be linearly independent columns of B.

The converse is true as well, that is, if b1, b3 and b4 are linearly independent columns of B, then a1, a3 and a4 must also be linearly independent columns. This completes the proof.

Theorem.

Elementary row operations do not affect the matrix rank

Performing elementary row operations on a matrix A preserves the rank of A.

Proof. From theoremlink, we know that elementary row operations maintain the linear independence between columns. This means that elementary row operations preserve the number of linearly independent columns. By theoremlink, the rank of a matrix A is equal to the number of linearly independent columns of A. Therefore, we conclude that performing an elementary row operation on A preserves the rank of A.

This completes the proof.

Theorem.

Multiplying a vector of a linearly independent set by a scalar preserves linear independence

If S is a linearly independent set of vectors, then multiplying any vector in S by a scalar k will preserve the linear independence of S.

Proof. Suppose S={v1,v2,v3} is a linearly independent set of vectors. By definitionlink of linear independence, this means that:

(2)c1v1+c2v2+c3v3=0

This equation only holds when the coefficients c1, c2 and c3 are all equal to zero. Now, let's multiply one of the vectors in S, say v2, by a scalar k to get kv2. We now use the definitionlink of linear dependence to show that {v1,kv2,v3} is linearly independent:

d1v1+d2(kv2)+d3v3=0

Where d1, d2 and d3 are some scalars. Since d2k is just another scalar, let's rewrite it as d4 like so:

(3)d1v1+d4v2+d3v3=0

We know from (2) that the only coefficients to make the equation hold are all zeros. This means that the only way for (3) to hold is if d1=d4=d3=0. Therefore, we conclude that v1, kv2 and v3 are linearly independent.

This completes the proof.

Theorem.

Elementary columns operations do not affect the matrix rank

Performing elementary column operations on a matrix A preserves the rank of A.

Proof. There are three types of elementary column operations so let's show that the matrix rank is preserved after each type of elementary column operation. By theoremlink, the matrix rank is equal to the number of linearly independent columns of the matrix. Therefore, we need to show that an elementary column operation preserves the number of linearly independent columns.

The first type of elementary column operation is a column swap. Clearly, swapping two columns does not affect the linear dependence between them. For instance, if column vectors a1 and a2 are linearly independent, then a2 and a1 will also be linearly independent - the ordering does not matter.

The second type of elementary column operation is scalar multiplication. By theoremlink, given a linearly independent set of vectors S, multiplying any vector in S by a scalar will preserve the linear independence of S.

The third type of elementary column operation is multiplying a vector by a scalar k and then adding it to another column vector.

Let v1 and v2 be linearly independent columns. By definitionlink, we have that:

(4)c1v1+c2v2=0

Where c1 and c2 are both zero.

Now, our goal is to show that kv1+v2 and v2 are also linearly independent. We once again use the definitionlink of linear independence to check whether the two vectors are independent:

(5)d1(kv1+v2)+d2v2=0

Where d1 and d2 are both scalars. Simplifying this gives:

(6)kd1v1+(d1+d2)v2=0

Here, kd1 and d1+d2 are some constants, so let's replace them with some constants e1 and e2 like so:

(7)e1=kd1e2=d1+d2

Equation (6) becomes:

(8)e1v1+e2v2=0

From (4), we know that the scalar coefficients of v1 and v2 must be zero. Therefore, we have that e1=e2=0 for (8) to hold as well. From (7), since k is nonzero, d1=0. This also means that d2=0. Because d1=d2=0 for the equality (5) to hold, we conclude that kv1+v2 and v2 are also linearly independent.

We have now shown that all three types of elementary column operations preserve the linear independence of columns. This means that the rank of a matrix is preserved after an elementary column operation. This completes the proof.

Theorem.

Rank of a matrix transpose

If A is a matrix, then the rank of AT is:

rank(A)=rank(AT)

Proof. Consider the matrix A and its transpose AT below:

A=(a11a12a13a14a21a22a23a24a31a32a33a34),AT=(a11a21a31a12a22a32a13a23a33a14a24a34)

Let rank(A)=r and rank(AT)=t. Our goal is to show that r=t.

Notice that performing an elementary row operation on A is equivalent to performing an elementary column operation on AT. For instance, suppose we perform an elementary row operation of multiplying the second row of A by some scalar k. This is equivalent to performing an elementary column operation of multiplying the second column of AT by k like so:

(a11a12a13a14ka21ka22ka23ka24a31a32a33a34),(a11ka21a31a12ka22a32a13ka23a33a14ka24a34)

Now, suppose we reduced A to its reduced row echelon form rref(A). If we perform the equivalent column operations on AT, we may end up with:

(10000100000),(10000010000)

Now, let's perform a series of column operations to the left matrix as well as equivalent row operations on the right matrix to get:

(100001000000),(100010000000)

We can clearly see that the number of linearly independent columns in the reduced form of A and AT is the same. By theoremlink, matrix rank is equal to the number of linearly independent columns, and so we have that:

(9)rank(reduced(A))=rank(reduced(AT))

By theoremlink and theoremlink, we know that elementary row/column operations do not affect the rank of a matrix. Since the reduced form of A and AT is obtained by performing a series of elementary row/column operations, we have that:

rank(reduced(A))=rank(A)rank(reduced(AT))=rank(AT)

This means that (9) becomes:

rank(A)=rank(AT)

This completes the proof.

Theorem.

Rank-nullity theorem

The sum of the rank and nullity of an m×n matrix A is:

rank(A)+nullity(A)=n

Proof. Let A be an m×n matrix. We know that every column in A must either correspond to a pivot or a non-pivot column in the reduced row echelon form of A, which means that:

# pivot columns in rref(A)+# non-pivot columns in rref(A)=# columns in rref(A)

From theoremlink, the number of pivot columns in rref(A) is equal to the rank of A, that is:

rank(A)=number of pivot columns of rref(A)

From theoremlink, the number of non-pivot columns in rref(A) is equal to the nullity of A, that is:

nullity(A)=number of non-pivot columns of rref(A)

Finally, the number of columns in rref(A) is the same as the number of columns in A. Therefore, we conclude that:

rank(A)+nullity(A)=n

This completes the proof.

Theorem.

Rank of PA and AP where P is invertible

Let A be a square matrix. If P is an invertible matrix, then:

rank(A)=rank(PA)=rank(AP)

Proof. We will first prove that rank(A)=rank(PA). Let U be some invertible matrix. Note that we use the notation U instead of P to avoid confusion later. By theoremlink, because U is invertible, we can express U in terms of elementary matrices Ek, , E2, E1 like so:

U=EkE2E1In

Now, substituting this U into rank(A)=rank(UA) gives:

rank(UA)=rank(EkE2E1InA)=rank(EkE2E1A)

By theoremlink, we know that elementary row operations do not affect the matrix rank. Since multiplying an elementary matrix to A is equivalent to performing an elementary row operation on A, we have that:

rank(UA)=rank(EkE2E1A)=rank(A)

Next, let's show that rank(AP)=rank(A). By theoremlink and theoremlink, we have that:

(10)rank(AP)=rank((AP)T)=rank(PTAT)

However, we have just proven that rank(UA)=rank(A). Therefore, we can reduce (10) to:

rank(AP)=rank(PTAT)=rank(AT)

This completes the proof.

robocat
Published by Isshin Inada
Edited by 0 others
Did you find this page useful?
thumb_up
thumb_down
Comment
Citation
Ask a question or leave a feedback...
Cookie Policy
close
By using our site, you acknowledge that you agree to our Privacy Policy and Terms and Conditions.