Solution. Firstly, we can see that $\boldsymbol{A}$ is symmetric so we're halfway done already 🙂. Next, let's find the eigenvalues of $\boldsymbol{A}$. The characteristic polynomiallink of $\boldsymbol{A}$ is:

This means that the eigenvalues of $\boldsymbol{A}$ are $\lambda_1=3$ and $\lambda_2=1$, which are both positive. By definitionlink, $\boldsymbol{A}$ is positive definite.

Proof. By definitionlink, a positive definite matrix is symmetric. By theoremlink, all eigenvalues of symmetric matrices are real. Therefore, the eigenvalues of positive definite matrices are real. This completes the proof.

Proof. Let $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_n$ be the eigenvalues of a positive definite matrix $\boldsymbol{A}$ that includes duplicates. By propertylink, these eigenvalues are real. This allows us to apply theoremlink to get:

Since the eigenvalues of a positive definite matrix are positive by definitionlink, we have that $\det(\boldsymbol{A})\gt0$. This completes the proof. Note that the converse does not necessarily hold, that is, even if $\mathrm{det}(\boldsymbol{A})\gt0$, $\boldsymbol{A}$ may not be positive definite.

Proof. Assume $\boldsymbol{A}$ is a positive definite matrix. By definitionlink and propertylink, the eigenvalues $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_n$ of $\boldsymbol{A}$ are all positive and real. By propertylink of eigenvalues, we have that the eigenvalues of $\boldsymbol{A}^{-1}$ are $\lambda_1^{-1}$, $\lambda_2^{-1}$, $\cdots$, $\lambda_n^{-1}$. Since the reciprocal of a positive value is also positive, we conclude that $\boldsymbol{A}^{-1}$ has positive eigenvalues as well.

Next, by propertylink of symmetric matrices, since $\boldsymbol{A}$ is an invertible symmetric matrix, $\boldsymbol{A}^{-1}$ is symmetric. Because $\boldsymbol{A}^{-1}$ is a symmetric matrix with positive eigenvalues, $\boldsymbol{A}^{-1}$ is positive definite. This completes the proof.

The next theorem about positive definite matrices is useful in proving future proofs and has important applications in fields such as optimization. In fact, some textbooks use this theorem as the definition of positive definite matrices!

Proof. Before we prove the forward and backward propositions, let's first establish a lemma that will make the proof easier. Let $\boldsymbol{A}$ be a symmetric $n\times{n}$ matrix. By the principal axis theoremlink, $\boldsymbol{A}$ is orthogonally diagonalizablelink, which means that $\boldsymbol{A}$ can be expressed as:

Where:

• $\boldsymbol{Q}$ is an $n\times{n}$ orthogonal matrixlink.

• $\boldsymbol{D}$ is an $n\times{n}$ diagonal matrixlink whose diagonal entries are the (real) eigenvalues of $\boldsymbol{A}$.

Let $\boldsymbol{x}$ be any vector in $\mathbb{R}^n$. Let $\boldsymbol{y}\in\mathbb{R}^n$ be defined as:

Taking the transpose of both sides and applying propertylink of transpose gives:

Now, using \eqref{eq:taBETZVGlBpj1uIsipL}, \eqref{eq:N4p8gH4ODUpJLGK4WG2} and \eqref{eq:BaN5zBAVObjTuptLDAv}, we express $\boldsymbol{x}^T\boldsymbol{Ax}$ in terms of $\boldsymbol{y}$ and $\boldsymbol{D}$ like so:

By propertylink of diagonal matrices, $\boldsymbol{y}^T\boldsymbol{D}$ can be expressed as:

Where $\lambda_1$, $\lambda_2$, $\cdots$, $\lambda_n$ are the eigenvalues of $\boldsymbol{A}$.

Multiplying both sides of \eqref{eq:hKTc6s9ib53WOKFaa2m} by $\boldsymbol{y}$ gives:

Equating \eqref{eq:oUP1ptSwuIwE7t75b8E} and \eqref{eq:eKWQyABlSBnB2DJeI13} gives:

We will now use \eqref{eq:LEOPQYWBm4fgEUtWnC3} to prove the forward and backward proposition.

Let's first prove the forward proposition. Let $\boldsymbol{A}$ be a positive definite matrix. By definitionlink, the eigenvalues $\lambda_1$, $ \lambda_2$, $\cdots$, $\lambda_n$ of $\boldsymbol{A}$ are all positive. Therefore, we use \eqref{eq:LEOPQYWBm4fgEUtWnC3} to establish the following inequality:

This proves the forward proposition.

Let's now prove the converse. Assume $\boldsymbol{x}^T\boldsymbol{Ax}\gt0$ for any vector $\boldsymbol{x}$ in $\mathbb{R}^n$. To show that $\boldsymbol{A}$ is positive definite, we must show that all of its eigenvalues $\lambda_1$, $\lambda_2$, $ \cdots$, $\lambda_n$ are positive. Since $\boldsymbol{x}$ can be any vector in $\mathbb{R}^n$, we can consider $\boldsymbol{x}$ to be:

Where $\boldsymbol{e}_j\in\mathbb{R}^n$ is the $j$-th column of the identity matrix $\boldsymbol{I}_n$. This means that the $j$-th entry of vector $\boldsymbol{e}_j$ is $1$ while the rest of its entries are $0$. Since $\boldsymbol{y}= \boldsymbol{Q}^T\boldsymbol{x}$ by \eqref{eq:N4p8gH4ODUpJLGK4WG2}, we have that:

Here, $\boldsymbol{Q}^T\boldsymbol{Q}= \boldsymbol{I}_n$ by definitionlink of orthogonal matrices. Substituting $\boldsymbol{y}=\boldsymbol{e}_j$ into \eqref{eq:LEOPQYWBm4fgEUtWnC3} gives:

Since $\boldsymbol{x}^T\boldsymbol{Ax}\gt0$ by assumption, we have that:

This is true for $j=1,2,\cdots,n$, which means that every eigenvalue of $\boldsymbol{A}$ is positive. Since $\boldsymbol{A}$ is also symmetric by assumption, we conclude that $\boldsymbol{A}$ is positive definite. This completes the proof.

Proof. Let $\boldsymbol{A}$ and $\boldsymbol{B}$ be $n\times{n}$ positive definite matrices. By theoremlink, for any $\boldsymbol{x}$ in $\mathbb{R}^n$, the following equations hold:

Adding the two inequalities gives:

This means that $\boldsymbol{A}+\boldsymbol{B}$ is positive definite by theoremlink. This completes the proof.

Proof. Let $\boldsymbol{A}$ be an $n\times{n}$ positive definite matrix. By theoremlink, since $\boldsymbol{A}$ is positive definite, for any $\boldsymbol{x}$ in $\mathbb{R}^n$, the following holds:

Let $\boldsymbol{e}_j$ be the $j$-th column of an $n\times{n}$ identity matrix, which means that the $j$-th entry of $\boldsymbol{e}_j$ is $1$ while the other entries are zero. Since \eqref{eq:DIrJMHKRNQnbnhYD4eM} holds for any $\boldsymbol{x}$ in $\mathbb{R}^n$, we let $\boldsymbol{x}=\boldsymbol{e}_j$ and substitute this into \eqref{eq:DIrJMHKRNQnbnhYD4eM} to get:

This is true for $j=1,2,\cdots,n$, which means that the diagonal entries of $\boldsymbol{A}$ are all positive. This completes the proof.

Solution. Since $\boldsymbol{A}$ is a $3\times3$ matrix, $\boldsymbol{A}$ has $3$ principal sub-matrices:

Proof. Let matrix $\boldsymbol{A}$ be an $n\times{n}$ positive definite matrix, which can be expressed as the following block formlink:

Where $\boldsymbol{A}_{(k)}$ is the $k$-th principal sub-matrix, which has shape $k\times{k}$.

Let $\boldsymbol{y}$ be any non-zero vector in $\mathbb{R}^k$. We define $\boldsymbol{x}\in\mathbb{R}^n$ such that:

Where $\boldsymbol{0}\in\mathbb{R}^{n-k}$ is a zero vector.

Now, by theoremlink, since $\boldsymbol{A}$ is positive definite, the following holds:

Substituting \eqref{eq:mZ9oyoz09LCGYxMnNHw} and \eqref{eq:OcA1HmepVByk8cj3Jz0} into \eqref{eq:VGnbjlccowPDQweR3nd} gives:

Multiplying the block matrices yields:

Since $\boldsymbol{y}$ is any non-zero vector in $\mathbb{R}^k$, we conclude that $\boldsymbol{A}_{(k)}$ is positive definite by theoremlink. This completes the proof.

Proof. We will prove this by induction on the size of the matrix $\boldsymbol{A}$. For the base case, assume $\boldsymbol{A}$ is a symmetric $1\times1$ matrix, which only has a single principal sub-matrix $\boldsymbol{A}_{(1)}=\boldsymbol{A}$. This means that $\boldsymbol{A}_{(1)}$ is symmetric. This proves the base case.

We now assume that the theorem holds for when $\boldsymbol{A}$ is an $(n-1)\times(n-1)$ symmetric matrix, that is, every principal sub-matrix of a symmetric $(n-1)\times(n-1)$ matrix $\boldsymbol{A}$ is symmetric. Our goal is to show that the theorem holds for when $\boldsymbol{A}$ is an $n\times{n}$ symmetric matrix.

Let $\boldsymbol{A}$ be a symmetric $n\times{n}$ matrix. We know that $\boldsymbol{A}_{(k)}$ is symmetric for $k=1,2,\cdots,(n-1)$ by the inductive assumption. $\boldsymbol{A}_{(n)}=\boldsymbol{A}$ is symmetric as well, which means that every principal sub-matrix of $\boldsymbol{A}$ is symmetric. This completes the proof.

Proof. Let $\boldsymbol{A}=\boldsymbol{P}^T\boldsymbol{P}$. If $\boldsymbol{x}$ is any non-zero vector in $\mathbb{R}^n$, then:

Here:

• the 3rd equality holds by propertylink of transpose.

• the last equality holds by propertylink of transpose.

By propertylink, since $\boldsymbol{P}$ is invertible, we have that $\Vert\boldsymbol{Px}\Vert\gt0$. Therefore, we conclude $\boldsymbol{x}^T\boldsymbol{Ax}\gt0$, which means that $\boldsymbol{A}$ is positive definite by theoremlink. This completes the proof.

The next theorem provides more equivalent definitions of positive definite matrices. In particular, the third statement about the so-called Cholesky factorizationlink has important applications in numerical computing.

Proof. Let's prove $(1)\implies(2)\implies(3)\implies(1)$. We shall prove the easy ones first - starting with $(1)\implies(2)$. Assume $\boldsymbol{A}$ is a positive definite $n\times{n}$ matrix. By theoremlink, every principal sub-matrix of $\boldsymbol{A}$ is positive definite. By propertylink, every positive definite matrix has a positive determinant, which means that every principal sub-matrix of $\boldsymbol{A}$ has a positive determinant. This proves $(1)\implies(2)$.

Next, let's show $(3)\implies(1)$. Assume matrix $\boldsymbol{A}$ can be Cholesky-factorized as $\boldsymbol{A}=\boldsymbol{U}^T\boldsymbol{U}$ where $\boldsymbol{U}$ is an upper triangular matrix whose diagonal entries are positive. By propertylink, $\boldsymbol{U}$ is invertiblelink because every diagonal entry of $\boldsymbol{U}$ is non-zero. By theoremlink, $\boldsymbol{A}=\boldsymbol{U}^T\boldsymbol{U}$ is positive definite. This proves $(3)\implies(1)$.

Finally, let's prove $(2)\implies(3)$. Assume $\boldsymbol{A}$ is an $n\times{n}$ symmetric matrix where $\det(\boldsymbol{A}_{(k)})\gt0$ for every $k=1,2,\cdots,n$. We will prove this by induction on the size $n$ of matrix $\boldsymbol{A}$. For the base case, let $\boldsymbol{A}$ be an $1\times{1}$ symmetric matrix with a single entry $a$. Since $\boldsymbol{A}$ is $1\times1$, it only has one principal sub-matrix:

If we let $\boldsymbol{U}= \begin{pmatrix}\sqrt{a}\end{pmatrix}$, then we can factorize $\boldsymbol{A}$ like so:

Since $\boldsymbol{U}$ is trivially an upper triangular matrix by definitionlink with a positive diagonal entry, the base case holds. We now assume that $(2)\implies(3)$ holds for when $\boldsymbol{A}$ is an $(n-1)\times(n-1)$ matrix. This means that the $(n-1)$-th principal sub-matrix can be factorized as:

Where $\boldsymbol{U}$ is an $(n-1)\times(n-1)$ upper triangular matrix. whose diagonal entries are positive.

Our goal now is to show that if $\boldsymbol{A}$ is a symmetric $n\times{n}$ matrix and $\det(\boldsymbol{A}_{(k)})\gt0$ for every $k=1,2,\cdots,n$, then $\boldsymbol{A}= \boldsymbol{U}_n^T\boldsymbol{U}_n$ where $\boldsymbol{U}_n$ is an $n\times{n}$ upper triangular matrix whose diagonal entries are positive. Note that we added a subscript $n$ for $\boldsymbol{U}_n$ to distinguish it from the $\boldsymbol{U}$ in \eqref{eq:ZdAwzHhzyf2Xdy5EbIc}.

By theoremlink, since $\boldsymbol{A}$ is symmetric, $\boldsymbol{A}_{(n-1)}$ is also symmetric. We can express $\boldsymbol{A}$ in block formatlink using $\boldsymbol{A}_{(n-1)}$ like so:

Where:

• $\boldsymbol{b}$ is a vector in $\mathbb{R}^{n-1}$.

• $c$ is a scalar in $\mathbb{R}$.

Substitute \eqref{eq:ZdAwzHhzyf2Xdy5EbIc} into \eqref{eq:vae6XlpmHQMfT3VnFqm} to get:

Now, define $\boldsymbol{x}\in\mathbb{R}^{n-1}$ such that:

Here, we know that the inverse of $\boldsymbol{U}^T$ exists because:

1. the diagonal entries of the upper triangular matrix $\boldsymbol{U}$ are positive by assumption.

2. $\boldsymbol{U}^T$ will be a lower triangular matrix with positive diagonals by propertylink.

3. since the diagonal entries of $\boldsymbol{U}^T$ are non-zero, $\boldsymbol{U}^T$ is invertible by propertylink.

Rearranging \eqref{eq:SUq4ij5TqSwB6phITB2} to make $\boldsymbol{b}$ the subject gives:

By propertylink of transpose, $\boldsymbol{b}^T$ is:

Next, define $k\in\mathbb{R}$ such that:

Rearranging \eqref{eq:Cvb1rlC5GWOMgq3POoW} to make $c$ the subject gives:

Substituting \eqref{eq:Bgkam4eafNYKAW5lXQO}, \eqref{eq:xiZNth1EUQpexBoTzzi} and \eqref{eq:yIkEL9EMACSpc1WQhGN} into \eqref{eq:On7Uo2RmWno2sHuJPvD} gives:

Just as we did in the proof of theoremlink, let's express $\boldsymbol{A}$ as a product of two block matrices:

Now, by propertylink of block matrices, notice how the two matrices on the right-hand side of \eqref{eq:wwopqUA1SZvbGPfr96D} will be transposes of one another if the bottom-right entry matched up. Suppose that the bottom-right entry did match up. In such a case, we can let $\boldsymbol{U}_n$ be the right matrix and $\boldsymbol{U}_n^T$ will be equal to the left matrix, that is, $\boldsymbol{A}=\boldsymbol{U}_n^T\boldsymbol{U}_n$. We also know that $\boldsymbol{U}_n$ is an upper triangular matrix because $\boldsymbol{U}$ in \eqref{eq:wwopqUA1SZvbGPfr96D} is upper triangular, which means our theorem is mostly proven.

One clever way of making the bottom-right entries of \eqref{eq:wwopqUA1SZvbGPfr96D} match is to show that $k\gt0$. This will allow us to write the product \eqref{eq:wwopqUA1SZvbGPfr96D} as:

Note also that the diagonal entries of $\boldsymbol{U}_n$ are positive because $\boldsymbol{U}$ has positive diagonals by our inductive assumption. So, our goal now is to show that $k\gt0$. Let's go 🚀!

Taking the determinantlink of both sides of \eqref{eq:wwopqUA1SZvbGPfr96D} and applying the multiplicative propertylink of determinants gives:

By theoremlink and theoremlink, the determinant terms on the right-hand side evaluate to:

By theoremlink, $\det(\boldsymbol{U}^T)=\det(\boldsymbol{U})$. This gives us:

Now, $\boldsymbol{A}=\boldsymbol{A}_{(n)}$, which means that $\det(\boldsymbol{A})\gt0$ by assumption $(2)$ of our theorem. Therefore, we get the following inequality:

Recall that $\boldsymbol{U}$ is an upper triangular matrix with positive diagonals. This means that $\boldsymbol{U}$ is invertiblelink by propertylink, and thus $\det(\boldsymbol{U})\ne0$ by propertylink. Because $\big[ \mathrm{det}(\boldsymbol{U})\big]^2$ is strictly positive, we have that $k\gt0$. This is what we set out to show - this completes the proof.

Solution. We compute the determinantlink of every principal sub-matrixlink of $\boldsymbol{A}$ below:

The determinant of the first principal sub-matrix is:

The determinant of the second principal sub-matrix is:

The determinant of the third principal sub-matrix is:

Since $\mathrm{det}\big(\boldsymbol{A}_{(k)}\big) \gt0$ for $k=1,2,3$, we conclude that $\boldsymbol{A}$ is positive definite by theoremlink.