# probability distributions – Sampling after sorting vs sorting without sampling

Given a probability distribution $$(p_1,cdots,p_d)$$ and integer $$k$$.

We can sample the distribution $$k$$ times. For any obtained sample $$(X_1,cdots,X_d)$$ with $$X_i$$ denoting the occurrence of $$i$$, we sort $$(X_1,cdots,X_d)$$ into $$(Y_1,cdots,Y_d)$$ such that $$Y_1leq Y_2leq cdotsleq Y_d$$. Now we have a distribution $$(Y_1/k,cdots,Y_d/k)$$.

This means that, by sampling $$(p_1,cdots,p_d)$$, we can have a distribution of distributions. Let us denote it by $$mathcal{Q}$$. For any distribution $$q’$$, $$q’_{mathcal{Q}}$$ denote the probability of obtaining distribution $$q’$$.

On the other hand, we can sort $$(p_1,cdots,p_d)$$ into $$(q_1,cdots,q_d)$$ with $$q_1leq q_2leq cdotsleq q_d$$.

We want to compare $$q$$ with $$mathcal{Q}$$, i.e., to estimate the following.
$$mathbb{E}_{q’} d(q,q’)$$
In the expectation, $$q’$$ ranges over the distribution of $$mathcal{Q}$$, and the distance is $$L_1$$ distance or K-L divergence or $$chi^2$$.