The one-sentence definition
A p-value is the probability of getting a result at least as extreme as the one you observed, assuming the null hypothesis is true.
Every word in that sentence is load-bearing, and the italicised part is the one people drop. The p-value is calculated inside a hypothetical world where there is no effect. It asks: if nothing were really going on, how surprising would my data be?
A worked example
A machine is supposed to fill packets with 500 g of cereal. You weigh 16 packets and find a mean of 486 g with a standard deviation of 20 g.
The null hypothesis is H₀: μ = 500 — the machine is fine, and your low reading is just sampling noise. The test statistic is:
t = (486 − 500) ÷ (20 ÷ √16) = −14 ÷ 5 = −2.80
With 15 degrees of freedom, that gives a one-tailed p-value of about 0.0067.
Read that properly: if the machine were filling correctly, you would see a sample this low only about 7 times in 1,000. That is unusual enough to doubt the assumption. You reject the null and conclude the machine is underfilling.
The three misreadings that cost marks
1. “The p-value is the probability the null hypothesis is true.” It is not. The calculation assumes the null is true and works forward from there. It cannot then turn round and tell you the probability of its own assumption.
2. “p = 0.0067 means there is a 0.67% chance the result was due to chance.” Also wrong, and subtly so. The p-value is the probability of the data given no effect — not the probability of chance given the data.
3. “A small p-value means a big effect.” No. It means the effect is distinguishable from noise. With a large enough sample, a completely trivial difference will produce a tiny p-value. Significance and importance are different questions.
What does “significant” actually mean?
You compare the p-value to a threshold, α, chosen before you see the data — almost always 0.05.
- If p < α, you reject the null hypothesis. The result is statistically significant.
- If p ≥ α, you fail to reject it. There is not enough evidence.
Notice the careful phrasing. We never “accept” the null. Absence of evidence is not evidence of absence: a non-significant result may simply mean your sample was too small to detect a real effect.
Why 0.05 is arbitrary
There is nothing magic about 0.05. It is a convention, and treating it as a bright line produces some genuinely silly behaviour — a result at p = 0.049 declared a discovery while p = 0.051 is dismissed as nothing, when the two datasets are practically identical.
Better practice is to report the actual p-value alongside a confidence interval and an effect size, then let readers judge. A p-value tells you whether an effect is real. It says nothing about whether it matters.
The link between p-values and confidence intervals
These are two views of the same evidence. For a two-tailed test at α = 0.05, the test rejects H₀: μ = μ₀ exactly when the 95% confidence interval excludes μ₀.
In the cereal example the 95% interval is roughly (475.3, 496.7). It does not contain 500, which is another way of saying the same thing the p-value said — but with a bonus: the interval also tells you how far off the machine appears to be, which the p-value never does.
One-tailed or two-tailed?
Use a one-tailed test only when you care about a difference in one direction and decided that in advance. The two-tailed p-value is exactly double the one-tailed value, so switching after seeing the data is a way of manufacturing significance. It has a name — p-hacking — and it is the single biggest reason published findings fail to replicate.
A checklist for reading any p-value
- What was the null hypothesis? A p-value is meaningless without it.
- Was the direction chosen in advance? If not, treat a one-tailed result with suspicion.
- How big is the sample? Huge samples make trivial effects significant.
- What is the effect size? Significance is not importance.
- Were the assumptions checked? A p-value from a model whose conditions failed is just a number.
In one line
A p-value answers a narrow question well: would data like mine be surprising if nothing were going on? It does not tell you the probability your hypothesis is true, how large the effect is, or whether it matters. Those questions need confidence intervals, effect sizes, and judgement.