Htykuut

Consider a family of probability distributions $P_theta$ indexed by $theta in Theta$. The parameter space is endowed with some metric $d$. We assume that there is a true parameter $theta_0$, and we are interested of the convergence of the Bayesian posterior distribution, given a prior $Pi$. We denote the posterior distribution after $n$ samples by $Pi_n(cdot|X^{(n)})$.

The posterior distribution is said to contract at rate $epsilon_n to 0$ at $theta_0$ if $Pi_n(theta:d(theta,theta_0) > M_nepsilon_n | X^{(n)}) to 0$ in $P_{theta_0}^{(n)}$ probability, for every $M_nto infty$ as $ntoinfty$ (i.e., regardless of how slow $M$ goes to infinity).

Now I have to proof the following proposition:

Suppose that the posterior distribution $Pi(cdot|X^{(n)})$ contracts
at rate $epsilon_n$ at $theta_0$. Then $hat{theta}_n$, defined as
the center of a (nearly) samllest ball that contains posterior mass at
least $frac{1}{2}$, satisfies $d(hattheta_n,theta_0) = O_P(epsilon_n)$ under $P_{theta_0}^{(n)}$.

$O_P(epsilon_n)$ is not defined but I suppose it denotes stochastic boundedness (see wikipedia), that is, for $tildeepsilon >0$, there are $M, N in mathbb{N}$ such that for $n>N$, we have $P_{theta_0}^{(n)} ( d(theta_0,hattheta_n)/epsilon_n > M) ) < tildeepsilon$.

The notes state the proof is very similar to a previous proof, which brought me to the following:

Let $B(theta,r)$ be the closed ball of radius $r$ centered at $theta$. Define $hat{r}_n(theta) = inf{r:Pi_n(B(theta,r)|X^{(n)}) geq frac{1}{2}}$.

Then $hat{r}_n(theta_0) < M_nepsilon_n$ with a probability tending to 1, with $M_n$ an arbitrary sequence with $M_n to infty$.

As we have chosen $hattheta_n$ as a nearly smallest ball, certainly $hat{r}_n(hattheta_n) leq hat{r}_n(theta_0) + frac{1}{n} < M_nepsilon_n + frac{1}{n}$.

Now, $B(theta_0,M_nepsilon_n)$ and $B(hattheta_n,hat{r}_n(hattheta_n))$ are not disjoint with probability tending to 1, since they would have a joined probability tending to 1.5. So:

$d(theta_0,hattheta_n) < 2M_nepsilon_n + frac{1}{n} implies d(theta_0,hattheta_n)/epsilon_n < 2M_n + frac{1}{epsilon_n n}$,

with probability tending to one. In other words, for $tildeepsilon >0$ there is some $N$ such that for $n>N$,

$P_{theta_0}( d(theta_0,hattheta_n)/epsilon_n geq 2M_n + frac{1}{epsilon_n n} ) < tildeepsilon$.

However, if I understand stochastic boundedness correctly I need to somehow replace $2M_n + frac{1}{epsilon_n n}$ with a fixed $M$. But since $M_n to infty$, I don't see how this is possible. Any ideas on what I could do?

lecture notes

I think you're almost there. $X_n=O_p(a_n)$ when $P( | frac{X_n}{a_n} | ge M_{delta} ) < delta$. $M_{delta}$ is affected by the choice of $delta$. In other notations you can also write $M_{delta}$ as $M_{n}$ where $n > N(delta)$ s.t $ni P( |frac{X_n}{a_n}| ge M_n ) < delta$.

Bottom line is that in the definition of contraction rate, you could choose your $M_n$ as you please.