A litte experiment to demonstrate p values.

Let’s make some data.

data<-rnorm(1000,mean=3.6,sd=2)

Let’s set a function to do a t.test for H0=m and Ha <> m.

f <- function(l) t.test(data, alternative="two.sided", mu=l)$p.value

Now, let’s do that test over the range 2 to +5 (both ends are sure to be of very low probability…)

mrange<-seq(2,5,0.01)
v<-sapply(mrange, f)

It now looks like this:

plot(mrange, v)
abline(v=3.6)

It is clear that the p-value become very close to 1 around 3.6 (actually, R says 1). This is the true mean.

So, around 3.6, from the sample, there is (very very) weak evidence against H0.

And around, say 2.5, from the sample, there is strong evidence against H0 (which would be mean=2.5 at that point).

It all makes sense.

Re running the thing a couple of times, with different samples, gives a better feel.