In linear algebra, a symmetric n × n real matrix M is said to be positive definite if z^{T}Mz is positive for every nonzero column vector z of n real numbers. Here z^{T} denotes the transpose of z.
More generally, an n × n Hermitian matrix M is said to be positive definite if z*Mz is real and positive for all nonzero column vectors z of n complex numbers. Here z* denotes the conjugate transpose of z.
The negative definite, positive semidefinite, and negative semidefinite matrices are defined in the same way, except that the expression z^{T}Mz or z*Mz is required to be always negative, nonnegative, and nonpositive, respectively.
Positive definite matrices are closely related to positivedefinite symmetric bilinear forms (or sesquilinear forms in the complex case), and to inner products of vector spaces.^{[1]}
Some authors use more general definitions of "positive definite" that include some nonsymmetric real matrices, or nonHermitian complex ones.
Contents

Examples 1

Connections 2

Characterizations 3

Quadratic forms 4

Simultaneous diagonalization 5

Negativedefinite, semidefinite and indefinite matrices 6

Negativedefinite 6.1

Positivesemidefinite 6.2

Negativesemidefinite 6.3

Indefinite 6.4

Further properties 7

Block matrices 8

On the definition 9

Consistency between real and complex definitions 9.1

Extension for non symmetric matrices 9.2

See also 10

Notes 11

References 12

External links 13
Examples

The identity matrix I = \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix} is positive definite. Seen as a real matrix, it is symmetric, and, for any column vector z with real entries a and b, one has z^{\mathrm{T}}I z = \begin{bmatrix} a & b\end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix} \begin{bmatrix} a \\ b\end{bmatrix}= a^2 + b^2. Seen as a complex matrix, for any column vector z with complex entries a and b one has z^*I z = \begin{bmatrix} a^* & b^*\end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 1\end{bmatrix} \begin{bmatrix} a \\ b\end{bmatrix}=a^*a +b^*b = a^2 + b^2. Either way, the result is positive as long as z is not the zero vector (that is, at least one of a and b is not zero).

The real symmetric matrix


M = \begin{bmatrix} 2&1&0\\1&2&1\\0&1&2 \end{bmatrix}

is positive definite since for any nonzero column vector z with entries a, b and c, we have

\begin{align} z^{\mathrm{T}}M z = (z^{\mathrm{T}}M) z &= \begin{bmatrix} (2ab)&(a+2bc)&(b+2c) \end{bmatrix} \begin{bmatrix} a\\b\\c \end{bmatrix} \\ &= 2{a}^2  2ab + 2{b}^2  2bc + 2{c}^2 \\ &= {a}^2+(a  b)^{2} + (b  c)^{2}+{c}^2 \end{align}

This result is a sum of squares, and therefore nonnegative; and is zero only if a = b = c = 0, that is, when z is zero.

The real symmetric matrix


N = \begin{bmatrix} 1 & 2 \\ 2 & 1\end{bmatrix}

is not positive definite. If z is the vector \begin{bmatrix} 1\\ 1\end{bmatrix}, one has z^{\mathrm{T}}N z = \begin{bmatrix} 1 & 1\end{bmatrix} \begin{bmatrix} 1 & 2 \\ 2 & 1\end{bmatrix} \begin{bmatrix} 1 \\ 1\end{bmatrix}=\begin{bmatrix} 1 & 1\end{bmatrix} \begin{bmatrix} 1 \\ 1\end{bmatrix}=2 \not > 0.

For any real nonsingular matrix A, the product A^T A is a positive definite matrix. A simple proof is that for any nonzero vector z , the condition z^T A^T A z = \ A z \_2^2 > 0, since the nonsingularity of matrix A means that Az \neq 0.
The examples M and N above show that a matrix in which some elements are negative may still be positivedefinite, and conversely a matrix whose entries are all positive may not be positive definite.
Connections
The general purely quadratic real function f(z) on n real variables z_{1}, ..., z_{n} can always be written as z^{T}Mz where z is the column vector with those variables, and M is a symmetric real matrix. Therefore, the matrix being positive definite means that f has a unique minimum (zero) when z is zero, and is strictly positive for any other z.
More generally, a twicedifferentiable real function f on n real variables has an isolated local minimum at arguments z_{1}, ..., z_{n} if its gradient is zero and its Hessian (the matrix of all second derivatives) is positive definite at that point. Similar statements can be made for negative definite and semidefinite matrices.
In statistics, the covariance matrix of a multivariate probability distribution is always positive semidefinite; and it is positive definite unless one variable is an exact linear combination of the others. Conversely, every positive semidefinite matrix is the covariance matrix of some multivariate distribution.
Characterizations
Let M be an n × n Hermitian matrix. The following properties are equivalent to M being positive definite:

All its eigenvalues are positive. Let P^{−1}DP be an eigendecomposition of M, where P is a unitary complex matrix whose rows comprise an orthonormal basis of eigenvectors of M, and D is a real diagonal matrix whose main diagonal contains the corresponding eigenvalues. The matrix M may be regarded as a diagonal matrix D that has been reexpressed in coordinates of the basis P. In particular, the onetoone change of variable y = Pz shows that z*Mz is real and positive for any complex vector z if and only if y*Dy is real and positive for any y; in other words, if D is positive definite. For a diagonal matrix, this is true only if each element of the main diagonal—that is, every eigenvalue of M—is positive. Since the spectral theorem guarantees all eigenvalues of a Hermitian matrix to be real, the positivity of eigenvalues can be checked using Descartes' rule of alternating signs when the characteristic polynomial of a real, symmetric matrix M is available.

The associated sesquilinear form is an inner product. The sesquilinear form defined by M is the function \langle \cdot,\cdot\rangle from C^{n} × C^{n} to C such that \langle x,y\rangle := y^*M x for all x and y in C^{n}, where y^{*} is the complex conjugate of y. For any complex matrix M, this form is linear in each argument separately. Therefore the form is an inner product on C^{n} if and only if \langle z,z\rangle is real and positive for all nonzero z; that is if and only if M is positive definite. (In fact, every inner product on C^{n} arises in this fashion from a Hermitian positive definite matrix.)

It is the Gram matrix of linearly independent vectors. Let x_1,\ldots,x_n be a list of n linearly independent vectors of some complex vector space with an inner product \langle \cdot,\cdot\rangle. It can be verified that the Gram matrix M of those vectors, defined by M_{ij} = \langle x_i, x_j\rangle, is always positive definite. Conversely, if M is positive definite, it has an eigendecomposition P^{−1}DP where P is unitary, D diagonal, and all diagonal elements D_{ii} = λ_{i} of D are real and positive. Let E be the real diagonal matrix with entries E_{ii} = \sqrt{\lambda_i} so E^2 = D ; then P^{1}DP = P^*DP = P^*E E P = (EP)^* EP. Now we let x_1,\ldots,x_n be the columns of EP. These vectors are linearly independent, and by the above M is their Gram matrix, under the standard inner product of C^{n}, namely \langle x_i, x_j\rangle = x_i^* x_j

Its leading principal minors are all positive. The kth leading principal minor of a matrix M is the determinant of its upperleft k by k submatrix. It turns out that a matrix is positive definite if and only if all these determinants are positive. This condition is known as Sylvester's criterion, and provides an efficient test of positivedefiniteness of a symmetric real matrix. Namely, the matrix is reduced to an upper triangular matrix by using elementary row operations, as in the first part of the Gaussian elimination method, taking care to preserve the sign of its determinant during pivoting process. Since the kth leading principal minor of a triangular matrix is the product of its diagonal elements up to row k, Sylvester's criterion is equivalent to checking whether its diagonal elements are all positive. This condition can be checked each time a new row k of the triangular matrix is obtained.

It has a unique Cholesky decomposition. The matrix M is positive definite if and only if there exists a unique lower triangular matrix L, with real and strictly positive diagonal elements, such that M = LL*. This factorization is called the Cholesky decomposition of M.
Quadratic forms
The (purely) quadratic form associated with a real matrix M is the function Q : R^{n} → R such that Q(x) = x^{T}Mx for all x. It turns out that the matrix M is positive definite if and only if it is symmetric and its quadratic form is a strictly convex function.
More generally, any quadratic function from R^{n} to R can be written as x^{T}Mx + x^{T}b + c where M is a symmetric n × n matrix, b is a real nvector, and c a real constant. This quadratic function is strictly convex when M is positive definite, and hence has a unique finite global minimum, if and only if M is positive definite. For this reason, positive definite matrices play an important role in optimization problems.
Simultaneous diagonalization
A symmetric, and a symmetric and positivedefinite matrix can be simultaneously diagonalized, although not necessarily via a similarity transformation. This result does not extend to the case of three or more matrices. In this section we write for the real case. Extension to the complex case is immediate.
Let M be a symmetric and N a symmetric and positivedefinite matrix. Write the generalized eigenvalue equation as (M−λN)x = 0 where we impose that x be normalized, i.e. x^{T}Nx = 1. Now we use Cholesky decomposition to write the inverse of N as Q^{T}Q. Multiplying by Q and Q^{T}, we get Q(M−λN)Q^{T}x = 0, which can be rewritten as (QMQ^{T})y = λy where y^{T}y = 1. Manipulation now yields MX = NXΛ where X is a matrix having as columns the generalized eigenvectors and Λ is a diagonal matrix with the generalized eigenvalues. Now premultiplication with X^{T} gives the final result: X^{T}MX = Λ and X^{T}NX = I, but note that this is no longer an orthogonal diagonalization.
Note that this result does not contradict what is said on simultaneous diagonalization in the article Diagonalizable matrix, which refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to a simultaneous diagonalization of two quadratic forms, and is useful for optimization of one form under conditions on the other. For this result see Horn&Johnson, 1985, page 218 and following.
Negativedefinite, semidefinite and indefinite matrices
A Hermitian matrix is negativedefinite, negativesemidefinite, or positivesemidefinite if and only if all of its eigenvalues are negative, nonpositive, or nonnegative, respectively.
Negativedefinite
The n × n Hermitian matrix M is said to be negativedefinite if

x^{*} M x < 0\,
for all nonzero x in C^{n} (or, all nonzero x in R^{n} for the real matrix), where x* is the conjugate transpose of x.
A matrix is negative definite if its kth order leading principal minor is negative when k is odd, and positive when k is even.
Positivesemidefinite
M is called positivesemidefinite (or sometimes nonnegativedefinite) if

x^{*} M x \geq 0
for all x in C^{n} (or, all x in R^{n} for the real matrix).
A matrix M is positivesemidefinite if and only if it arises as the Gram matrix of some set of vectors. In contrast to the positivedefinite case, these vectors need not be linearly independent.
For any matrix A, the matrix A*A is positive semidefinite, and rank(A) = rank(A*A). Conversely, any Hermitian positive semidefinite matrix M can be written as M = LL*, where L is lower triangular; this is the Cholesky decomposition. If M is not positive definite, then some of the diagonal elements of L may be zero.
A Hermitian matrix is positive semidefinite if and only if all of its principal minors are nonnegative. It is however not enough to consider the leading principal minors only, as is checked on the diagonal matrix with entries 0 and 1.
Negativesemidefinite
It is called negativesemidefinite if

x^{*} M x \leq 0
for all x in C^{n} (or, all x in R^{n} for the real matrix).
Indefinite
A Hermitian matrix which is neither positive definite, negative definite, positivesemidefinite, nor negativesemidefinite is called indefinite. Indefinite matrices are also characterized by having both positive and negative eigenvalues.
Further properties
If M is an Hermitian positivesemidefinite matrix, one sometimes writes M ≥ 0 and if M is positivedefinite one writes M > 0.^{[2]} The notion comes from functional analysis where positivesemidefinite matrices define positive operators.
For arbitrary square matrices M, N we write M ≥ N if M − N ≥ 0; i.e., M − N is positive semidefinite. This defines a partial ordering on the set of all square matrices. One can similarly define a strict partial ordering M > N.

Every positive definite matrix is invertible and its inverse is also positive definite.^{[3]} If M ≥ N > 0 then N^{−1} ≥ M^{−1} > 0, and √M > √N > 0.^{[4]} Moreover, by the minmax theorem, the kth largest eigenvalue of M is greater than the kth largest eigenvalue of N

If M is positive definite and r > 0 is a real number, then rM is positive definite.^{[5]} If M and N are positive definite, then the sum M + N^{[5]} and the products MNM and NMN are also positive definite. If MN = NM, then MN is also positive definite.

Every principal submatrix of a positive definite matrix is positive definite.

Q^{T} M Q is nonnegative definite. If Q is invertible, then Q^{T} M Q is positive definite. Note that Q^{1} M Q need not be positive definite.

The determinant of M is bounded by the product of its diagonal elements.

The diagonal entries m_{ii} are real and nonnegative. As a consequence the trace, tr(M) ≥ 0. Furthermore,^{[6]} since every principal sub matrix (in particular, 2by2) is positive definite,


m_{ij} \leq \sqrt{m_{ii}m_{jj}} \leq \frac{m_{ii}+m_{jj}}{2}

and thus

\max m_{ij} \leq \maxm_{ii}

A matrix M is positive semidefinite if and only if there is a positive semidefinite matrix B with B^{2} = M. This matrix B is unique,^{[7]} is called the square root of M, and is denoted with B = M^{1/2} (the square root B is not to be confused with the matrix L in the Cholesky factorization M = LL*, which is also sometimes called the square root of M). If M > N > 0 then M^{1/2} > N^{1/2} > 0.

If M is a symmetric matrix of the form m_{ij} = m(i−j), and the strict inequality holds


\sum\nolimits_{j\neq 0}m(j)< m(0)

then M is strictly positive definite.

Let M > 0 and N Hermitian. If MN + NM ≥ 0 (resp., MN + NM > 0) then N ≥ 0 (resp., N > 0).

If M > 0 is real, then there is a δ > 0 such that M > δI, where I is the identity matrix.

If M_{k} denotes the leading k by k minor, \det(M_k)/\det(M_{k1}) is the kth pivot during LU decomposition.

The set of positive semidefinite symmetric matrices is convex. That is, if M and N are positive semidefinite, then for any α between 0 and 1, αM + (1−α)N is also positive semidefinite. For any vector x:


x^\top (\alpha M + (1\alpha) N ) x = \alpha x^\top M x + (1\alpha) x^\top N x \geq 0.

This property guarantees that semidefinite programming problems converge to a globally optimal solution.

If M,N ≥ 0, although MN is not necessary positivesemidefinite, the Kronecker product M ⊗ N ≥ 0, the Hadamard product M ○ N ≥ 0 (this result is often called the Schur product theorem).,^{[8]} and the Frobenius product M : N ≥ 0 (LancasterTismenetsky, The Theory of Matrices, p. 218).

Regarding the Hadamard product of two positivesemidefinite matrices M = (m_{ij}) ≥ 0, N ≥ 0, there are two notable inequalities:

Oppenheim's inequality: \det(M \circ N) \geq \det (N) \prod\nolimits_{i} m_{ii}.^{[9]}

det(M ○ N) ≥ det(M) det(N).^{[10]}
Block matrices
A positive 2n × 2n matrix may also be defined by blocks:

M = \begin{bmatrix} A & B \\ C & D \end{bmatrix}
where each block is n × n. By applying the positivity condition, it immediately follows that A and D are hermitian, and C = B*.
We have that z*Mz ≥ 0 for all complex z, and in particular for z = ( v, 0)^{T}. Then

\begin{bmatrix} v^* & 0 \end{bmatrix} \begin{bmatrix} A & B \\ B^* & D \end{bmatrix} \begin{bmatrix} v \\ 0 \end{bmatrix} = v^* A v \ge 0.
A similar argument can be applied to D, and thus we conclude that both A and D must be positive definite matrices, as well.
Converse results can be proved with stronger conditions on the blocks, for instance using the Schur complement.
On the definition
Consistency between real and complex definitions
Since every real matrix is also a complex matrix, the definitions of "positive definite" for the two classes must agree.
For complex matrices, the most common definition says that "M is positive definite if and only if z*Mz is real and positive for all nonzero complex column vectors z". This condition implies that M is Hermitian, that is, its transpose is equal to its conjugate. To see this, consider the matrices A = (M+M*)/2 and B = (M−M*)/(2i), so that M = A+iB and z*Mz = z*Az + iz*Bz. The matrices A and B are Hermitian, therefore z*Az and z*Bz are individually real. If z*Mz is real, then z*Bz must be zero for all z. Then B is the zero matrix and M = A, proving that M is Hermitian.
By this definition, a positive definite real matrix M is Hermitian, hence symmetric; and z^{T}Mz is positive for all nonzero real column vectors z". However the last condition alone is not sufficient for M to be positive definite. For example, if

M = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix},
then for any real vector z with entries a and b we have z^{T}Mz = (a−b)a + (a+b)b = a^{2} + b^{2}, which is always positive if z is not zero. However, if z is the complex vector with entries 1 and i, one gets

z*Mz = [1, −i]M[1, i]^{T} = [1+i, 1−i][1, i]^{T} = 2 + 2i,
which is not real. Therefore, M is not positive definite.
On the other hand, for a symmetric real matrix M, the condition "z^{T}Mz > 0 for all nonzero real vectors z" does imply that M is positive definite in the complex sense.
Extension for non symmetric matrices
Some authors choose to say that a complex matrix M is positive definite if Re(z*Mz) > 0 for all nonzero complex vectors z, where Re(c) denotes the real part of a complex number c.^{[11]} This weaker definition encompasses some nonHermitian complex matrices, including some nonsymmetric real ones, such as \bigl[ \begin{smallmatrix} 1 & 1 \\ 1 & 1 \end{smallmatrix} \bigr] .
Indeed, with this definition, a real matrix is positive definite if and only if z^{T}Mz > 0 for all nonzero real vectors z, even if M is not symmetric.
In general, we have Re(z*Mz) > 0 for all complex nonzero vectors z if and only if the Hermitian part (M + M*)/2 of M is positive definite in the narrower sense. Similarly, we have x^{T}Mx > 0 for all real nonzero vectors x if and only if the symmetric part (M + M^{T})/2 of M is positive definite in the narrower sense.
In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on a complex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarization identity. That is no longer true in the real case.
See also
Notes

^ Stewart, J. (1976). Positive definite functions and generalizations, an historical survey. Rocky Mountain J. Math, 6(3).

^ This may be confusing, as sometimes nonnegative matrices are also denoted in this way. A common alternative notation is M \succeq 0 and M \succ 0 for positive semidefinite and positive definite matrices, respectively.

^ Horn & Johnson (1985), p. 397

^ Horn & Johnson (1985), Corollary 7.7.4(a)

^ ^{a} ^{b} Horn & Johnson (1985), Observation 7.1.3

^ Horn & Johnson (1985), p. 398

^ Horn & Johnson (1985), Theorem 7.2.6 with k = 2

^ Horn & Johnson (1985), Theorem 7.5.3

^ Horn & Johnson (1985), Theorem 7.8.6

^ (Styan 1973)

^ Weisstein, Eric W. Positive Definite Matrix. From MathWorldA Wolfram Web Resource. Accessed on 20120726
References

Horn, Roger A.; Johnson, Charles R. (1990), Matrix Analysis, .

Rajendra Bhatia. Positive definite matrices. Princeton Series in Applied Mathematics, 2007. ISBN 9780691129181.
External links

Hazewinkel, Michiel, ed. (2001), "Positivedefinite form",

Wolfram MathWorld: Positive Definite Matrix
This article was sourced from Creative Commons AttributionShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and USA.gov, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for USA.gov and content contributors is made possible from the U.S. Congress, EGovernment Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a nonprofit organization.