This report is aim to calculate false discovery rate (FRD) in my master capstone project.

Define false discovery rate as the proportion of false discovery among the discoveries. (copy from wikipedia)

Classification of m hypothesis tests

  1. for top n method, FRD will be 1-power, because the numerator (flase positive) remains same as we got in power calculation and the denominator (declared significant number) is equal to defined true signal number .

  2. for type one error method, the numerator (flase positive) is same, but the denominator (declared significant number) become number of signal that larger than cutoff.

  1. reg
##        [,1]   [,2]   [,3]   [,4]   [,5]
## [1,] 0.7189 0.6006 0.5745 0.5460 0.5390
## [2,] 0.5762 0.4804 0.4556 0.4183 0.4054
## [3,] 0.5000 0.4155 0.3852 0.3637 0.3720
## [4,] 0.4493 0.3657 0.3426 0.3382 0.3548
## [5,] 0.4012 0.3397 0.3186 0.3274 0.3293
  1. smo
##        [,1]   [,2]   [,3]   [,4]   [,5]
## [1,] 0.8201 0.6720 0.5520 0.5094 0.5033
## [2,] 0.6360 0.4888 0.3960 0.3524 0.3294
## [3,] 0.5954 0.4161 0.3771 0.3170 0.3170
## [4,] 0.5269 0.4029 0.3515 0.3404 0.3506
## [5,] 0.4718 0.3544 0.3219 0.3362 0.3228
  1. for permutation method, the calculation is similar with part (b).
  1. reg
##        [,1]   [,2]   [,3]   [,4]   [,5]
## [1,] 0.7102 0.5564 0.5370 0.5143 0.5225
## [2,] 0.5717 0.4846 0.4397 0.4116 0.4157
## [3,] 0.5194 0.3923 0.3811 0.4214 0.3565
## [4,] 0.4738 0.3944 0.3614 0.3375 0.3499
## [5,] 0.3857 0.3534 0.3133 0.2996 0.3002
  1. smo
##        [,1]   [,2]    [,3]    [,4]    [,5]
## [1,] 0.8088 0.4889 0.20667 0.08433 0.02033
## [2,] 0.6566 0.2467 0.11567 0.06467 0.01500
## [3,] 0.5006 0.2434 0.07033 0.06433 0.05952
## [4,] 0.4482 0.2162 0.08299 0.13980 0.09014
## [5,] 0.3360 0.1145 0.06277 0.11067 0.06429