Methods for FDR correction

Oct 13, 2025 by pynkocean.com 97 views

**p-value correction for multiple models with same sets of predictors** ===========================================================

Introduction

Conducting an exploratory study with multiple outcomes and overlapping sets of predictors can be a challenging task. One of the key concerns in such studies is the issue of multiple comparisons, which can lead to an inflated false discovery rate (FDR). In this article, we will discuss the best approach for FDR correction for p-values in the context of multiple models with the same sets of predictors.

What is False Discovery Rate (FDR)?

False Discovery Rate (FDR) is a measure of the expected proportion of false positives among all significant findings. In other words, it is the probability that a significant result is actually due to chance. FDR is an important concept in multiple comparisons, as it helps to control the number of false positives and avoid over-interpretation of results.

Why is FDR correction necessary?

FDR correction is necessary because of the problem of multiple comparisons. When conducting multiple tests, the probability of obtaining a significant result by chance increases. This can lead to an inflated FDR, where many significant results are actually false positives. FDR correction helps to control the FDR by adjusting the p-values to account for the number of comparisons made.

Methods for FDR correction

There are several methods for FDR correction, including:

Benjamini-Hochberg (BH) method

The BH method is a popular method for FDR correction. It works by ranking the p-values in ascending order and then adjusting the p-values to account for the number of comparisons made. The BH method is simple to implement and is widely used in many fields.

Benjamini-Yekutieli (BY) method

The BY method is another popular method for FDR correction. It is similar to the BH method but uses a different formula to adjust the p-values. The BY method is more conservative than the BH method and is often used when the number of comparisons is large.

Holm-Bonferroni method

The Holm-Bonferroni method is a more conservative method for FDR correction. It works by adjusting the p-values using a step-down procedure, where the p-value with the smallest p-value is adjusted first, and then the next smallest p-value is adjusted, and so on. The Holm-Bonferroni method is more conservative than the BH and BY methods but can be more computationally intensive.

Storey-Tibshirani method

The Storey-Tibshirani method is a more recent method for FDR correction. It works by estimating the FDR using a Bayesian approach and then adjusting the p-values to account for the estimated FDR. The Storey-Tibshirani method is more flexible than the other methods and can be used in a variety of situations.

Choosing the right method

Choosing the right method for FDR correction depends on the specific needs of the study. The BH method is a good choice when the number of comparisons is small, while the BY method is a good choice when the number of comparisons is large. The Holm-Bonferroni method is a good choice when a more conservative approach is desired, while the Storey-Tibshirani method is a good choice when a more flexible approach is desired.

Software implementation

FDR correction can be implemented using a variety of software packages, including R, Python, and MATLAB. The BH method is implemented in the p.adjust function in R, while the BY method is implemented in the p.adjust function with the method="BY" argument. The Holm-Bonferroni method is implemented in the p.adjust function with the method="holm" argument, while the Storey-Tibshirani method is implemented in the p.adjust function with the method="storey" argument.

Example

Suppose we have a dataset with 10 variables and we want to perform a multiple comparison test to determine which variables are significant. We can use the BH method to adjust the p-values as follows:

# Load the data
data <- read.csv("data.csv")
p.values <- lm(y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7 + x8 + x9 + x10, data = data)$p.value
p.adjusted <- p.adjust(p.values, method = "BH")
print(p.adjusted)

Conclusion

FDR correction is an important step in multiple comparison tests to control the number of false positives and avoid over-interpretation of results. There are several methods for FDR correction, including the BH, BY, Holm-Bonferroni, and Storey-Tibshirani methods. Choosing the right method depends on the specific needs of the study, and software implementation can be done using a variety of packages, including R, Python, and MATLAB.

Frequently Asked Questions

Q: What is the difference between FDR and p-value?

A: The p-value is the probability of obtaining a result by chance, while the FDR is the expected proportion of false positives among all significant findings.

Q: Why is FDR correction necessary?

A: FDR correction is necessary because of the problem of multiple comparisons, which can lead to an inflated FDR.

Q: What are the different methods for FDR correction?

A: The different methods for FDR correction include the BH, BY, Holm-Bonferroni, and Storey-Tibshirani methods.

Q: How do I choose the right method for FDR correction?

A: Choosing the right method depends on the specific needs of the study. The BH method is a good choice when the number of comparisons is small, while the BY method is a good choice when the number of comparisons is large.

Q: How do I implement FDR correction in software?

A: FDR correction can be implemented using a variety of software packages, including R, Python, and MATLAB. The BH method is implemented in the p.adjust function in R, while the BY method is implemented in the p.adjust function with the method="BY" argument.