HGR maximal correlation revisited : a corrected reverse inequality

Sudeep Kamath sent me a note about a recent result he posted on the ArXiV that relates to an earlier post of mine on the HGR maximal correlation and an inequality by Erkip and Cover for Markov chains which I had found interesting:
.
Since learning about this inequality, I’ve seen a few talks which have used the inequality in their proofs, at Allerton in 2011 and at ITA this year. Unfortunately, the stated inequality is not correct!

On Maximal Correlation, Hypercontractivity, and the Data Processing Inequality studied by Erkip and Cover
Venkat Anantharam, Amin Gohari, Sudeep Kamath, Chandra Nair

What this paper shows is that the inequality is not satisfied with , but by another quantity:

where is given by the following definition.

Let and be random variables with joint distribution . We define
,
where denotes the -marginal distribution of and the supremum on the right hand side is over all probability distributions that are different from the probability distribution . If either or is a constant, we define to be 0.

Suppose have joint distribution (I know I am changing notation here but it’s easier to explain). The key to showing their result is through deriving variational characterizations of and in terms of the function

Rather than getting into that in the blog post, I recommend reading the paper.

The implication of this result is that the inequality of Erkip and Cover is not correct : not only is not the supremum of the ratio, there are distributions for which it is not an upper bound. The counterexample in the paper is the following: , and is generated via this asymmetric erasure channel:

Joint distribution counterexample (Fig. 2 of the paper)

So where is the error in the original proof? Anantharam et al. point to an explanation that the Taylor series expansion used in the proof of the inequality with may not be valid at all points.

This seems to just be the start of a longer story, which I look forward to reading in the future!