Quantcast
Channel: Entropy dominance - MathOverflow
Viewing all articles
Browse latest Browse all 2

Answer by Bjørn Kjos-Hanssen for Entropy dominance

$
0
0

Does Shannon entropy of a random variable from distribution $\mathcal{P}_{abc}$ dominate the other three for all $a,b,c\in\Bbb R^+$ such that $0<a<b<c$?

No.

On a sample space $\{A,B,C\}$ we can think of $\mathcal P_{ab}$ as just $\mathcal P_{abc}$ conditioned on the event that $C$ did not happen.

If the probability $x$ of $C$ is very large and the probabilities of $A$ and $B$ are equal and small, then the entropy of $\mathcal P_{ab}$ will be larger than that of $\mathcal P_{abc}$ as you can verify by looking at the graph of$$-2((1-x)/2)\log_2((1-x)/2)-x\log_2(x),\quad x\in [1/3, 1]$$


Viewing all articles
Browse latest Browse all 2

Trending Articles