Psychology Wiki

Order statistic

Revision as of 15:21, February 14, 2006 by Lifeartist (Talk | contribs)

34,191pages on
this wiki



In statistics, the kth order statistic of a statistical sample is equal its kth-smallest value. Together with rank statistics, order statistics are among the most fundamental tools in non-parametric statistics and inference.


When using probability theory to analyse order statistics of random samples from a continuous distribution, the cumulative distribution function is used to reduce the analysis to the case of order statistics of the uniform distribution.

Notation and examples

For example, suppose that four numbers are observed or recorded, resulting in a sample of size n=4. if the sample values are

6, 9, 3, 8,

they will usually be denoted

x_1=6;\ \ x_2=9;\ \ x_3=3;\ \ x_4=8\,


The order statistics would be denoted

x_{(1)}=3;\ \ x_{(2)}=6;\ \ x_{(3)}=8;\ \ x_{(4)}=9\,











Probabilistic analysis

We will assume that the random variables under consideration are continuous and, where convenient we will also assume that they have a density (that is, they are absolutely continuous). The peculiarities of the analysis of distributions assigning mass to points (in particular, discrete distributions) are discussed at the end.

Distribution of each order statistic of an absolutely continuous distribution

Let X_{1}, X_{2}, \ldots, X_{n} be iid absolutely continuously distributed random variables, and X_{(1)}, X_{(2)}, \ldots, X_{(n)} be the corresponding order statistics. Let f(x) be the probability density function and F(x) be the cumulative distribution function of X_{i}. Then the probability density of the kth statistic can be found as follows.

f_{X_{(k)}}(x)={d \over dx} F_{X_{(k)}}(x)
={d \over dx}P\left(X_{(k)}\leq x\right)={d \over dx}P(\mathrm{at}\ \mathrm{least}\ k\ \mathrm{of}\ \mathrm{the}\ n\ X\mathrm{s}\ \mathrm{are}\leq x)
={d \over dx}P(\geq k\ \mathrm{successes}\ \mathrm{in}\ n\ \mathrm{trials})={d \over dx}\sum_{j=k}^n{n \choose j}P(X_1\leq x)^j(1-P(X_1\leq x))^{n-j}
={d \over dx}\sum_{j=k}^n{n \choose j} F(x)^j (1-F(x))^{n-j}
+F(x)^j (n-j)(1-F(x))^{n-j-1}(-f(x))\right)
=\sum_{j=k}^n\left(n{n-1 \choose j-1}F(x)^{j-1}(1-F(x))^{n-j} - n{n-1 \choose j} F(x)^j(1-F(x))^{n-j-1} \right)f(x)
=nf(x)\left(\sum_{j=k-1}^{n-1} {n-1 \choose j}
F(x)^j (1-F(x))^{(n-1)-j}
- \sum_{j=k}^n {n-1 \choose j}
F(x)^j (1-F(x))^{(n-1)-j}\right)

and the sum above telescopes, so that all terms cancel except the first and the last:

=nf(x)\left({n-1 \choose k-1} F(x)^{k-1} (1-F(x))^{(n-1)-(k-1)}
- \underbrace{{n-1 \choose n}
F(x)^n (1-F(x))^{(n-1)-n}}\right)

and the term over the underbrace is zero, so:

={n! \over (k-1)!(n-k)!} F(x)^{k-1} (1-F(x))^{n-k} f(x).

Probability distributions of order statistics

In this section we show that the order statistics of the uniform distribution on the unit interval have marginal distributions belonging to the Beta family. We also give a simple method to derive the joint distribution of any number of order statistics, and finally translate these results to arbitrary continuous distributions using the cdf.


The order statistics of the uniform distribution

The probability of the order statistic U_{(k)} falling in the interval [u,u+du] is equal to

{n!\over (k-1)!(n-k)!}u^{k-1}(1-u)^{n-k}du+O(du^2),



The proof of these statements is as follows. In order for U_{(k)} to be between u and u+du, it is necessary that exactly k-1 elements of the sample are smaller than u, and that at least one is between u and u+du. The probability that more than one is in this latter interval is already O(du^2), so we have to calculate the probability that exactly k-1, 1 and n-k observations fall in the intervals (0,u), (u,u+du) and (u+du,1) respectively. This equals (refer to multinomial distribution for details)

{n!\over (k-1)!1!(n-k)!}u^k\cdot du\cdot(1-u-du)^{n-k}

and the result follows.

Joint distributions

Similarly, for i < j, the joint probability density function of the two order statistics Ui < Uj can be shown to be




f_{U_{(1)},U_{(2)},\ldots,U_{(n)}}(u_{1},u_{2},\ldots,u_{n})\,du_1\,\cdots\,du_n = n! \, du_1\cdots du_n.


Application: confidence intervals for quantiles


Estimating the median


A small-sample-size example






If the distribution is known to be symmetric and have finite variance (as is the case for the normal distribution) the population mean equals the median, and the sample mean has much better confidence intervals than the sample median. This is an illustration of the relative weakness of distribution-free statistical methods. On the other hand, using methods tailored to the wrong distribution can lead to large systematic errors in estimation.

Large sample sizes

Estimating quantiles

Dealing with discrete variables

Computing order statistics

Main article: Selection algorithm


See also

External links


Around Wikia's network

Random Wiki