P-value for the Null Hypothesis: When to Reject the Null Hypothesis

Research often starts with a null hypothesis — the idea that any observed variations are due to chance. But how do we decide when to question this hypothesis? How can we decide whether the data supports it?

This is where the p-value for the null hypothesis comes in. It's a statistical tool that helps us judge whether our results are consistent with the null. In this article, we'll answer the following questions:

• What is the p-value?
• What is a null hypothesis?
• When do you reject the null hypothesis?
• What does a small p-value mean?

There's a lot to cover, so let's get started!

To understand the purpose of the p-value, we must first grasp some key concepts.

What is a p-value? The role of p-value in hypothesis testing

The p-value for the null hypothesis is a crucial statistical instrument in hypothesis testing. In research, we test against the null hypothesis, which is a basic assumption that what we are testing (e.g., a new medication or method) has no effect. Therefore, the p-value allows us to determine whether an observed effect is real or random by indicating how likely the obtained results are if the null hypothesis were true. In other words, the p-value answers the question: "If I repeat my test with another sample, what are the chances that I get the same results, assuming that the null is true?"

The p-value doesn't prove or disprove anything; it's more of a guide that tells us how strongly the data contradicts the null hypothesis. Depending on the type of test, the p-value can be derived from different test statistics:

• z-score;
• t-score and degrees of freedom;
• chi-square (χ²) score and degrees of freedom; or
• F-score and degrees of freedom.

You can calculate the p-value and find out how to determine it using our p-value calculator. We have also created a handy p-value table, so make sure to check it out as well!

Thresholds of significance and interpretation

We use thresholds — significance levels — to make decisions regarding hypotheses. If the p-value falls below the chosen significance level, the result is statistically significant, which is grounds for rejecting the null.

Significance thresholds are not rigid rules, however. If our p-value is slightly above our chosen threshold, it doesn't automatically mean that there's no effect — when interpreting results, we need to consider the context of our study and other factors.

As we've seen, the primary goal of hypothesis testing is to assist us in determining whether to reject the null hypothesis. In this regard, the p-value is extremely important. Let's now examine the null hypothesis in greater detail and determine when we should reject it.

When do you reject the null hypothesis?

What's the p-value for rejecting the null hypothesis? Generally speaking, when the p-value ($p$) is below the significance level ($\alpha$) we have selected, the null hypothesis is rejected. For instance, when the likelihood of observing our obtained results under the null hypothesis assumption is less than 5%, we reject the null if we select the most commonly used significance level in research, 0.05.

So, what does a small p-value really tell you?

A p-value below the threshold doesn't mean that an effect must exist — rather, it indicates that our results would be unusual if the null were true. A small p-value constitutes evidence against the hypothesis, but it's not absolute proof.

For a more detailed explanation, take a look at our dedicated article, A p-value Less Than 0.05 — What Does it Mean?.

Example: p-value and null hypothesis in practice

Let’s calculate a two-tailed p-value using a z-score.

1. State the hypotheses:

• $H_0 : \mu=50$
• $H_1 : \mu \neq50$

2. Note down the data. For example:

• Known population standard deviation: $\sigma = 10$
• Sample size: $n = 36$
• Sample mean: $\bar{x} = 54$

3. Choose a significance level, that is, the p-value for rejecting the null hypothesis. Let's take a standard value of $\alpha = 0.05$.

4. Calculate the standard error:

$\rm{SE} = \frac{\sigma}{\sqrt{n}} = \frac{10}{6} = 1.667$

5. Calculate the z-score:

$z = \frac{\bar{x} - \mu_0}{SE} = \frac{54 - 50}{1.667} \approx 2.4$

6. Calculate the p-value under the null hypothesis (two-sided, $H_1 : \mu \neq50$).

$p = 2 \times (1 - \Phi\lvert z \rvert)$

Looking up $\Phi\lvert 2.4 \rvert$ in a z-table (row 2.4, col. 0.00), we find $0.9918$. Let's use this in the formula:

$p = 2 \times (1 - 0.9918) \approx 0.0164$

7. Compare your p-value to the chosen significance level.

Since $p = 0.0164 < 0.05$, the result is statistically significant. Therefore, we reject $H_0$ at $\alpha = 0.05$.

Do you need help calculating a statistic? Why not use our standard deviation calculator or the mean calculator.

We hope that this article has clarified the p-value and, more crucially, the reasons and circumstances in which H₀ ought to be rejected. To sum up:

• If our p-value is greater than our chosen significance level, it means that our results are consistent with the null hypothesis; therefore, we should not reject it.
• If, on the other hand, the p-value is less than the significance level, assuming H₀ is true, the data is unlikely. This is grounds for rejection of the null hypothesis.

And remember, the p-value is only an indication. Ultimately, we should always look at the bigger picture and decide whether the null hypothesis should be rejected.

Happy researching!