Given a probability distribution $(p_1,cdots,p_d)$ and integer $k$.

We can sample the distribution $k$ times. For any obtained sample $(X_1,cdots,X_d)$ with $X_i$ denoting the occurrence of $i$, we sort $(X_1,cdots,X_d)$ into $(Y_1,cdots,Y_d)$ such that $Y_1leq Y_2leq cdotsleq Y_d$. Now we have a distribution $(Y_1/k,cdots,Y_d/k)$.

This means that, by sampling $(p_1,cdots,p_d)$, we can have a distribution of distributions. Let us denote it by $mathcal{Q}$. For any distribution $q’$, $q’_{mathcal{Q}}$ denote the probability of obtaining distribution $q’$.

On the other hand, we can sort $(p_1,cdots,p_d)$ into $(q_1,cdots,q_d)$ with $q_1leq q_2leq cdotsleq q_d$.

We want to compare $q$ with $mathcal{Q}$, i.e., to estimate the following.

$$

mathbb{E}_{q’} d(q,q’)

$$

In the expectation, $q’$ ranges over the distribution of $mathcal{Q}$, and the distance is $L_1$ distance or K-L divergence or $chi^2$.