McNemar's Test Calculator
Omni's McNemar's test calculator can help you perform different versions of McNemar's test for paired proportions - as many as five versions of this test are available. In particular, this tool can serve as McNemar's exact test calculator.
Not sure what the definition of McNemar's test is? Scroll down to learn everything you need about it (the correct interpretation of McNemar's test included). At the end of this article, you'll find an example of McNemar's test to see how it works in practice.
What is McNemar's test for paired proportions?
Let's first discuss what the assumptions of McNemar's test are. As you might've already heard, this statistical test deals with paired binomial data. More precisely, we call on McNemar's test when we have two paired groups, for instance, patients before and after treatment. Additionally, we have the values of a dichotomous variable (i.e., a variable with only two categories). So, for our patients, we can have a result of some medical test that is either positive or negative.
Other examples of when we can use McNemar's test:
- Maths exam result (pass vs. fail) of a group of students prior to and following a special preparatory curse.
- The will of citizens (yes vs. no) to vote for candidates before and after watching their TV spot.
|💡 Have you heard about the paired t-test? McNemar's test has a similar aim, but it uses a dichotomous dependent variable, while the t-test needs a continuous variable.|
Like in the t-test, the crucial assumption of McNemar's test is that we have a simple random sample (i.e., a set of independent, identically distributed random variables).
When we want to perform McNemar's test, the first step is to prepare the contingency table. We'll discuss this in the next section. And once we have the contingency tables, we'll be able to state the hypotheses of McNemar's test more formally. Let's go!
How do I calculate the contingency table for McNemar's test?
The contingency table that appears in the context of McNemar's test is a 2 x 2 table registering how many subjects fall into a given pair of categories. For instance, for patients subject to a test before and after treatment, we have the following pairs of categories:
- Before treatment and positive test result;
- After treatment and positive test result;
- Before treatment and negative test result; and
- After treatment and negative test result.
The corresponding contingency table looks like this:
As you can see, there are four fields, which we need to fill in with the data we obtained during our experiment. Assume we have:
apeople in the before-and-positive group;
bpeople in the after-and-positive group;
cpeople in the before-and-negative group; and
dpeople in the after-and-negative group.
This translates into the following table:
We can now easily add to our table both column totals and row totals. By
N, we denote the total number of observations.
|positive|| || ||
|negative|| || ||
|column total|| ||
Having discussed the contingency table, we are in a good position to discuss the hypotheses connected to McNemar's test.
McNemar's test definition
Let's denote by
pa, pb, pc, pd the theoretical probability of their respective group:
|positive|| || ||
|negative|| || ||
|column marginal|| |
As probabilities always sum to one, we obviously have:
pa + pb + pc + pd = 1.
Now, the null hypothesis of McNemar's test is a claim about the marginal distributions. Namely, it states that the row marginal and column marginal are equal - people often call this the hypothesis of marginal homogeneity:
pa + pb = pa + pc
pc + pd = pb + pd, but this follows immedately from the previous equation).
The alternative hypothesis is that the marginal distributions are not equal, i.e., that the above equation does not hold:
pa + pb ≠ pa + pc.
Simplifying these equations, we get a slightly simpler form of our hypotheses, which you can often see in textbooks:
pb = pc
pb ≠ pc
McNemar's test interpretation
Let's discuss the interpretation of McNemar's test by stating the hypotheses this test deals with, without any mathematical notation:
- The null hypothesis of McNemar's test is that the treatment (or the preparatory math course or the TV spot) has no effect on the proportion of people with the positive result (or, respectively, the proportion of students passing the exam, or that of citizens willing to vote in favor of our candidate).
- The alternative hypothesis is that these proportions differ significantly, and so the treatment (course/TV spot) does have some effect.
A quick tip: don't focus that much on formulas, but try to remember the examples - they are beneficial when it comes to recalling the correct interpretation of McNemar's test for paired proportions.
How to perform McNemar's test?
When we've got the contingency table ready, we can quickly and easily compute McNemar's test statistic. The formula employs only the off-diagonal fields from the contingency table:
χ² = (b - c)² / (b + c).
Under the null hypothesis and if
c are large enough, then
χ² follows approximately the chi-squared distribution with
1 degree of freedom.
Note that, under the null hypothesis, we expect to observe small test statistic values. Indeed, if
pb = pc, then in our data
b should be roughly equal to
c, which translates into small values of
χ². Hence, by contradiction, big values of
χ² provide evidence in favor of the alternative hypothesis H₁. The next step of McNemar's test depends on the approach you want to use:
p-value approach - To calculate the p-value, you have to compute the probability that a random variable with the chi-squared distribution with
1degree of freedom assumes values greater than or equal to the value of the test statistic
χ²we've got for our sample:
p-value = 1 - cdf(χ²),
cdfis the cumulative distribution function of the chi-squared distribution with
1degree of freedom. If the
p-valueis smaller than the significance level we set for the test, we reject the null hypothesis.
Critical region approach - The critical region is right-sided, i.e., it takes the form
[k, ∞), where the critical value
(1-α)-quantile of the chi-squared distribution with
1degree of freedom and
αis the significance level you set for the test. If
χ²falls into the critical region, i.e., if
χ² ≥ k, we reject the null hypothesis.
Corrections to the test statistics
When we approximate a smooth distribution (in this case, the chi-squared distribution) with a discrete distribution, we are bound to have some inaccuracy, for which we can compensate by using a continuity correction. In the case of McNemar's test, the correction appears in the numerator of the test statistics. It involves:
- Taking first the absolute value of
- Subtracting some constant from it.
- Squaring the resulting number.
The denominator remains unchanged. There are two corrections you can encounter:
χ² = (|b - c| - 1)² / (b + c)
χ² = (|b - c| - ½)² / (b + c)
Edwards correction is more widespread, and some statistical packages use it by default - keep that in mind when they give a different value of
χ² than you expected.
What is McNemar's exact test?
As mentioned earlier, in the standard way of performing McNemar's test, we approximate the distribution of the test statistic with the chi-squared distribution with
1 degree of freedom. However, as always, this approximation fails when our sample contains too few observations.
In such a case, we do not approximate but rather perform the exact binomial test. Here's the formula for its (two-sided) p-value:
After a moment's thought, one can recognize here the cumulative distribution function of the binomial distribution
b(n,p) with parameters
p = 0.5 and
n = b + c. We use the exact McNemar's test if our sample satisfies
b + c < 25.
Alternatively, we can use the mid-p-value McNemar's test, which we can calculate by subtracting the probability of the observed
b from the exact two-sided p-value, that is:
Here's the full formula:
The mid-p-value approach has been recently becoming more and more popular. We won't throw at you any technical details; let us just say that the mid-p-value allows us to avoid the disadvantages of the other methods (especially those related to controlling the type I error) while keeping their advantages to the almost full extent. In Omni's McNemar's exact test calculator, we've implemented both exact-p-value and mid-p-value so that you can easily compare the results these two approaches yield.
How to use this McNemar's test calculator?
To perform McNemar's test with the help of our calculator, follow these steps:
Input your data (that is, the coefficients of the 2x2 contingency table) into the dedicated fields. The filled-in contingency table will appear at the bottom of McNemar's test calculator along with its marginals so that you can easily verify if you've entered data correctly.
Decide on the version of the McNemar's test you want to perform. There are five options available:
- Standard McNemar's test;
- With Edwards correction;
- With Yates correction;
- Exact binomial test; and
- Mid-p binomial test.
If you are not sure which one to choose, go for the standard McNemar's test.
Our McNemar's test calculator will return the p-value, based on which you can decide if your findings are significant at the desired significance level.
Additionally, for tests based on the
χ²test statistic, our McNemar's test calculator displays the value of test statistic so that you can easily verify your own calculations.
McNemar's test example
In this final section, we'll go through an example of McNemar's test for paired proportions together to see how all these calculations work in practice.
Assume we have the following data:
Calculate the value of test statistics:
χ² = (b - c)² / (b + c) = (70 - 50)² / (70 + 50) = 400 / 120 = 3.33
Determine the value of cdf of the chi-squared distribution with 1 degree of freedom at
cdf(3.33) = 0.932
Determine the p-value:
p-value = 1 - cdf(3.33) = 0.068
As we can see, at the standard significance level
0.05, we do not have enough evidence to reject the null hypothesis - our findings are not statistically significant.
That's it, we're done with our example of McNemar's test! We hope you agree that's it's not at all hard to perform this test even if you have to compute it by hand. But, certainly, it's much more fun to use Omni's McNemar's test calculator 😉.
What is McNemar's test?
McNemar's test is a statistical technique for paired binomial data which helps us decide if the contingency table has equal marginal distributions. For instance, we can use it to decide if a particular treatment has any effect on the results of a dichotomous medical test.
How do I calculate McNemar's test?
In McNemar's test, we compute the test statistic using the off-diagonal fields
c from the contingency table:
χ² = (b - c)² / (b + c).
c are large enough, then the p-value is equal to one minus the cdf of the chi-squared distribution with
1 degree of freedom taken at the value of
χ². Alternatively, use the right-sided critical region.
When to use McNemar's exact test?
Use McNemar's exact test (instead of the standard version based on χ² approximation) when your sample is small, that is, the off-diagonal cells
c in the contingency table are such that
b + c < 25.
Why do I need the contingency table in McNemar's test?
Strictly speaking, you don't need the whole contingency table to perform McNemar's test, but only its off-diagonal values, also known as discordant values (as opposed to concordant values at the diagonal). You need these values to compute the test statistic of McNemar's test.