Quantum relative entropy

From Wikipedia, the free encyclopedia

In quantum information theory, quantum relative entropy is a measure of distinguishability between two quantum states. It is the quantum mechanical analog of relative entropy.

1 Motivation
2 Definition
- 2.1 Non-finite relative entropy
3 Klein's inequality
- 3.1 Corresponding classical statement
- 3.2 The result
4 An entanglement measure

[edit] Motivation

For simplicity, it will be assumed that all objects in the article are finite dimensional.

We first discuss the classical case. Suppose the probabilities of a finite sequence of events is given by the probability distribution P = {p₁...p_n}, but somehow we mistakenly assumed it to be Q = {q₁...q_n}. For instance, we can mistake an unfair coin for a fair one. According to this erroneous assumption, our uncertainty about the j-th event, or equivalently, the amount of information provided after observing the j-th event, is

$\; - \log q_j.$

The (assumed) average uncertainty of all possible events is then

$\; - \sum_j p_j \log q_j.$

On the other hand, the Shannon entropy of the probability distribution p, defined by

$\; - \sum_j p_j \log p_j,$

is the real amount of uncertainty before observation. Therefore the difference between these two quantities

$\; - \sum_j p_j \log q_j - (- \sum_j p_j \log p_j) = \sum_j p_j \log p_j - \sum_j p_j \log q_j$

is a measure of the distinguishability of the two probability distributions p and q. This is precisely the classical relative entropy, or Kullback–Leibler divergence:

$D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} \!.$

Note

In the definitions above, the convention that 0·log 0 = 0 is assumed, since lim_{x → 0} x log x = 0. Intuitively, one would expect that an event of zero probability to contribute nothing towards entropy.
The relative entropy is not a metric. For example, it is not symmetric. The uncertainty discrepancy in mistaking a fair coin to be unfair is not the same as the opposite situation.

[edit] Definition

As with many other objects in quantum information theory, quantum relative entropy is defined by extending the classical definition from probability distributions to density matrices. Let ρ be a density matrix. The von Neumann entropy of ρ, which is the quantum mechanical analaog of the Shannon entropy, is given by

$S(\rho) = - \operatorname{Tr} \rho \log \rho.$

For two density matrices ρ and σ, the quantum relative entropy of ρ with respect to σ is defined by

$S(\rho \| \sigma) = - \operatorname{Tr} \rho \log \sigma - S(\rho) = \operatorname{Tr} \rho \log \rho - \operatorname{Tr} \rho \log \sigma = \operatorname{Tr}\rho (\log \rho - \log \sigma).$

We see that, when the states are classical, i.e. ρσ = σρ, the definition coincides with the classical case.

[edit] Non-finite relative entropy

In general, the support of a matrix M, denoted by supp(M), is the orthogonal complement of its kernel. When consider the quantum relative entropy, we assume the convention that - s· log 0 = ∞ for any s > 0. This leads to the definition that

$S(\rho \| \sigma) = \infty$

when

$supp(\rho) \cap supp(\sigma)^{\perp} \neq {0}.$

This makes physical sense. Informally, the quantum relative entropy is a measure of our ability to distinguish two quantum states. But orthogonal quantum states can always be distinguished, via projective measurement. In the present context, this is reflected by non-finite quantum relative entropy.

In the interpretation given in the previous section, if we erroneously assume the state ρ has support in supp(ρ)^⊥, this is an error impossible to recover from.

[edit] Klein's inequality

[edit] Corresponding classical statement

For the classical Kullback–Leibler divergence, it can be shown that

$D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} \geq 0,$

and equality holds if and only if P = Q. Colloquially, this means that the uncertainty calculated using erroneous assumptions is always greater than the real amount of uncertainty.

To show the inequality, we rewrite

$D_{\mathrm{KL}}(P\|Q) = \sum_j p_j \log \frac{p_j}{q_j} = \sum_j (- \log \frac{q_j}{p_j})(p_j).$

Notice that log is a concave function. Therefore -log is convex. Applying Jensen's inequality to -log gives

$D_{\mathrm{KL}}(P\|Q) = \sum_j (- \log \frac{q_j}{p_j})(p_j) \geq - \log ( \sum_j \frac{q_j}{p_j} p_j ) = 0.$

Jensen's inequality also states that equality holds if and only if, for all i, q_i = (∑q_j) p_i, i.e. p = q.

[edit] The result

Klein's inequality states that the quantum relative entropy

$S(\rho \| \sigma) = \operatorname{Tr}\rho (\log \rho - \log \sigma).$

is non-negative in general. It is zero if and only ρ = σ.

Proof

Let ρ and σ have spectral decompositions

$\rho = \sum_i p_i v_i v_i ^* \; , \; \sigma = \sum_i q_i w_i w_i ^*.$

$\log \rho = \sum_i (\log p_i) v_i v_i ^* \; , \; \log \sigma = \sum_i (\log q_i)w_i w_i ^*.$

Direct calculation gives

$S(\rho \| \sigma)$

$= \sum_k p_k \log p_k - \sum_{i,j} (p_i \log q_j) | v_i ^* w_j |^2$

$= \sum_i p_i ( \log p_i - \sum_j \log q_j | v_i ^* w_j |^2)$

$\;= \sum_i p_i (\log p_i - \sum_j (\log q_j )P_{ij}),$ where P_{i j} = |v_i*w_j|².

Since the matrix (P_{i j})_{i j} is a doubly stochastic matrix and -log is a concave function, the above expression is

$\geq \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij})$

$\; = \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij}).$

Define r_i = ∑_jq_j P_{i j}. Then {r_i} is a probability distribution. From the non-negativity of classical relative entropy, we have

$S(\rho \| \sigma) \geq \sum_i p_i \log \frac{p_i}{r_i} \geq 0.$

The second part of the claim follows from the fact that, since -log is strictly convex, equality is achieved in

$\sum_i p_i (\log p_i - \sum_j (\log q_j )P_{ij}) \geq \sum_i p_i (\log p_i - \log (\sum_j q_j P_{ij})$

if and only if (P_{i j}) is a permutation matrix, which implies ρ = σ, after a suitable labeling of the eigenvectors {v_i} and {w_i}.

[edit] An entanglement measure

Let a composite quantum system have state space

$H = \otimes _k H_k$

and ρ be a density matrix acting on H.

The relative entropy of entanglement of ρ is defined by

$\; D_{\mathrm{REE}} (\rho) = \min_{\sigma} S(\rho \| \sigma)$

where the minimum is taken over the family of separable states. A physical interpretation of the quantity is the optimal distinguishability of the state ρ from separable states.

Clearly, when ρ is not entangled

$\; D_{\mathrm{REE}} (\rho) = 0$