False Positive Paradox Calculator

Medical testing is essential for diagnosing and treating diseases but is not 100% accurate, and, in fact, there's a little-known and counterintuitive phenomenon related to false positive results: the false positive paradox.

Apart from correctly detecting positive results, it's worth avoiding false positive results as much as possible, as they can cause undue anxiety and distress for the patient and lead to unnecessary further medical interventions.

The false positive paradox is the most common example of base rate fallacy, which consists of people's tendency to ignore the base rate (e.g., prevalence) in favor of information pertaining only to a specific case (e.g., sensitivity and specificity).

Keep reading this article, and discover why "accurate" can be highly inaccurate.

To understand the false positive paradox, we must be aware of the following terminology:

• Sensitivity measures the proportion of positive cases correctly identified by a test. Mathematically, it is the ratio of true positive results to the total number of positive cases (i.e., true positives plus false negatives):

  $\mathrm{\!Sensitivity \!\!=\!\! \frac{True\ positives}{True\ positives\ +\ False\ negatives}}$

• Specificity measures the proportion of negative cases correctly identified by the test. We calculate it as the ratio of true negative results to the total number of negative cases (i.e., true negatives plus false positives):

  $\mathrm {\!Specificity \!\!=\!\! \frac{True\ negatives}{True\ negatives\ +\ False\ positives}}$

• The base rate is the proportion of individuals in a population with a particular condition at a given time. In epidemiology, the base rate is the same as the prevalence, which is the proportion of a population affected by a medical condition or disease.

The false positive paradox consists of a high portion of the positive test results being, in fact, false positives, even if the test has a high specificity and sensitivity. This paradox occurs when the base rate for the condition is low, as it happens with rare diseases that affect a tiny percentage of the population.

Visit the false positive calculator to learn more about false positives.

Yes, many positive test results can be false positives, but how do we know the exact proportion? Let's apply the knowledge gained with our Bayes theorem calculator to this problem.

Let A be the event of having the condition (being positive) and B be the event of testing positive. Then, the probability of having a condition given a positive test result is:

where:

• $\mathrm {P(A)}$ — Base rate of the condition in the population;
• $\mathrm {P(B)}$ — Probability of testing positive; and
• $\mathrm {P(B|A)}$ — Probability of testing positive, given that you actually have the condition. In other words, the sensitivity of the test.

We usually know the base rate and sensitivity, but P(B) is not readily available and will depend on other probabilities. Using the law of total probability, we can express P(B) as:

where:

• $\text{P(B|not A)}$ — Probability of testing positive given the actual condition is negative, also known as the false positive rate of the test; and
• $\text{P(not A)}$ — Complement of the base rate, which is the probability of not having the condition.

Now, we can mix the two above equations and obtain a more explicit version of P(A|B):

Let's relate the above terms to the base rate, sensitivity, and specificity parameters. Before, let's remember:

• $\text{P(A) = Base rate}$;
• $\text{P(B|A) = Sensitivity}$;
• $\text{P(B|not A) = 1 - Specificity}$ (as $\mathrm{P(not\, B| \, not A)}$ is the specificity); and
• $\text{P(not A) = 1 - Base rate}$.

Let's input all that in the previous equation to finally obtain the probability of having a condition given a positive test result, in terms of base rate (BR), specificity (SP), and sensitivity (SE):

The probability of having a condition given a positive test result, that is $\mathrm {P(A|B)}$, is also called the positive predictive value.

If you're a TL;DR lover, don't worry. The summary from the previous section is the following formula:

where:

• $\text{PPV}$ — Positive predictive value;
• $\text{SE}$ — Sensitivity of the test;
• $\text{SP}$ — Specificity of the test; and
• $\text{BR}$ — Base rate of the condition, known as "prevalence" in epidemiology.

The PPV is the formal name for that quantity we've been talking about all this article: the probability of having a condition given a positive test result.

Even if we're aware of this paradox, we can also be victims of some misconceptions associated with its counterintuitive nature. Here are some of them:

• "A positive test result is more likely to be incorrect than correct." This affirmation is not necessarily true, as the false positive paradox arises given a low prior probability of the event to test. This low prior probability can increase as we get more information about the actual condition.

• "We can resolve the false positive paradox by increasing the sample size." As we saw in the previous section, the false positive paradox (P(A|B) equation) doesn't depend on the sample size but on the base rate, specificity, and sensitivity. If we increase the sample size, there will be more true positives but also more false positives, and the TP/(TP + FP) proportion will remain the same. The following section explains how to modify the sample to overcome this paradox.

• "Increasing the sensitivity will significantly attenuate the paradox effects". A higher sensitivity will increase the ability to detect positives correctly. Still, more is needed to solve the main issue: the tremendous number of false positives arising from the specificity and the enormous quantity of negative events.

In the previous section, we saw that the sample size and sensitivity have little effect on the false positive paradox, but what can we do? Let's find out.

Increase the specificity

Increasing the specificity will attenuate the FP paradox effects, as it'll decrease the number of false positives without affecting the number of true positives.

The easiest way to increase the specificity is by switching to a test with higher specificity. If that's not possible, you can dig deeper into the biological and technical causes behind that specificity error and try to modify them in your favor. The CDC presents some examples:

• For HIV tests, technical issues include specimen mix-up, mislabeling, improper handling, and misinterpreting of a visually-read rapid test result.
• Biological causes include participation in an HIV vaccine study, autoimmune disorders, and other medical conditions that generate HIV-like antibodies.

Improve your sampling

The false positive paradox assumes the prevalence is the only prior probability we know. We can modify this by improving our sample and only testing those "suspicious" cases.

Suppose you carefully select your sample to mainly include people with a higher prior probability of having the condition. In that case, this can increase the test's positive predictive value and decrease the likelihood of obtaining false positives given a positive result.

For example, suppose you are testing for a rare disease that only affects 1% of the population and select a sample of individuals with a family history of the disease. In that case, the prior probability of having the disease in this sample may be much higher than 1%.

If you live in the US and get an HIV-positive test, the chance of a false positive is about 63%, assuming the US prevalence is the only info about your HIV risk. The probability will be higher if you belong to a high-risk group or know you've been exposed to the virus. As always, follow the guidelines of a health professional.

To calculate sensitivity and specificity:

1. Identify the number of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) cases.

2. Calculate the sensitivity by dividing the number of true positives by the sum of true positives and false negative cases:

   Sensitivity = TP / (TP + FN)

3. Calculate the specificity by dividing the true negative by the sum of the true negative and false positive cases:

   Specificity = TN / (FP + TN)

There are two ways to calculate the PPV:

• Dividing the number of true positives by the total number of positive test results (the sum of the false and true positives):

  PPV = TP / (TP + FP)

• If you know the sensitivity (SE), specificity (SP), and base rate (BR, aka prevalence), use the formula:

  PPV = SE × BR / [SE × BR + (1 - SP) × (1 - BR)]

The most common example of base rate fallacy is the false positive paradox, in which, even if the test accuracy is high, the proportion of false positives will be high because of the condition's low prevalence (base rate).