Thank you to Patrick Royston,  Robert Cuffe, Simon Bond, Ted Harding,
antony fielding, Roger Newson, A.Bertie and John McKellar for their
answers to my question. I have posted a summary of the responses for the
interested. 

########################################################################
#########################

Question: 
Are p-values uniformly distributed ? Is there a proof/paper/book/website
for or against this and other properties of p-values. Much appreciated. 


########################################################################
#########################

Answers:

From: Patrick Royston [mailto:[log in to unmask]] 
under a suitable null hypothesis of exchangeability of the sample
values, yes.

have a look also for the related, classic Fisher test: sum(-2 ln p_i) is
distributed as chi square on 2n d.f. where p_1,...,p_n are the "p
values" (assumed to be samples from a uniform distribution).

a random sample from a cdf of any continuous distribution whatever is
uniformly distributed on (0,1) - this arises from statistical
distribution theory and may be proved simply by standard calculus
methods.

-----Original Message-----
From: Robert Cuffe [mailto:[log in to unmask]] 

Yes they are. That's why you can generate from any distribution using a
uniform generator and the inverse of the CDF.


-----Original Message-----
From: Simon Bond [mailto:[log in to unmask]] 

Given a random variable (your test statistic) X, consider the
transformation of X,

Y=F(X), where F(  ) is the cumulative distribtuion function (
P(X<x)=F(x) )  .

Considering the probability of the event {Y<y}, and using the monotone
increasing property of F,

P(Y<y)=P(F(X)<y)
        =P(X<F^{-1}(y))
        =F(F^{-1}(y))
        =y

So clearly Y is U(0,1), but also Y is 1-p, where p is the p-value, and
hence the p-value is U(0,1). So the answer to your first question is
yes. Not sure about the second question, but the proof I've just given
would be in an undergrad statistics course. 

-----Original Message-----
From: [log in to unmask] [mailto:[log in to unmask]]


When the test statistic has a continuous distribution, then the P-value
does indeed have a uniform distribution when the null hypothesis is
true.

The proof is quite simple.

Let the test statistic be T, and let F(t) = P(T >= t) when the null
hypothesis holds -- i.e. F(t) gives the P-value corresponding to the
value t of the test statistic T, or p = F(t).

Now look at the distribution of p when the null hypotheis hols: for any
value p0 of p such that 0 <= p0 <= 1, corresponding to the value t0 of T
so that F(t0) = p0,

  P(p <= p0) = P(F(T) <= F(t0)) = P(T >= t0) = F(t0) = p0

In other words, the random variable p on [0,1] is exactly like a random
variable X on [0,1] such that for any x in [0,1] we have

  P(X <= x) = x,

so p is indeed uniformly distributed.

If the test statistic does not have a continuous distribution (as in
discrete-variable tests) then not every value of p in [0,1] can occur,
so clearly p does not have a uniform distribution in such cases.
However, where large-sample approximations to the distribution of T are
continuous, then p also has approximately a uniform distribution.

-----Original Message-----
From: antony fielding [mailto:[log in to unmask]] 

If under an hypothesis a sample statistic ( or any other random
variable) x  has say a sampling distribution with cumulative
distribution function
F(x) then it is well known that F(x)  is uniform . A one sided p-value
in the lower tail is
p=F(x) and hence is uniform. This is standrard statistical theory a
proof of which may be found in most books on elementary statistical
theory.


-----Original Message-----
From: Roger Newson [mailto:[log in to unmask]] 

P-values are uniformly distributed for a continuously distributed test 
statistic, if the null hypothesis being tested if true. A P-value is 
defined formally as the cumulative distribution function (cdf) of the 
likelihood function,, or of the probability density or mass function, or
of 
the absolute value of the test statistic. A cumulative distribution 
function F(.) for a random variable X has a uniform distribution if X is

continuously distributed, and a discrete distribution if X is discretely

distributed, having the property that

F(z) = Pr(X<=z) <= Pr(F(X)<=F(z))

for each z in the range of X.. This follows from the fact that a cdf is 
non-decreasing.

A good book on elementary probability and statistics (including the
uniform 
distribution of a cdf) is:

Mood AM, Graybill FA, Boes DC. Introduction to the Theory of Statistics.

Third edition. Singapore: McGraw Hill; 1974.

-----Original Message-----
From: A.Bertie [mailto:[log in to unmask]] 

See my website at:-
http://www2.open.ac.uk/CES/projects/SUStats/Hypothesistesting/Hypothesis
testingApplet.html

This runs a Java applet that simulates hypothesis testing and plots a
histogram of p-values for various distributions (click on the
"Significance tests" tab). This should demonstrate that p is uniformly
distributed when the null hypothesis is true, but not when it's false.

-----Original Message-----
From: McKellar, John [mailto:[log in to unmask]] 

Yes, I believe that they should appear uniform except is there is an
underlying influence.