CBDA-SL and Knockoff Filter Results on a ‘Fake’ Binomial Dataset (continuous features and binary outcome)
Simeone Marino
December 8, 2016
Some useful information
This is a summary of a set of 30 experiments I ran on Cranium using a single pipe workflow file that performs 3000 independent jobs, each one with the CBDA-SL and the knockoff filter feature mining strategies. Each experiments is uniquely identified by 6 input arguments: # of jobs [M], % of missing values [misValperc], min [Kcol_min] and max [Kcol_max] % for FSR-Feature Sampling Range, min [Nrow_min] and max [Nrow_max] % for SSR-Subject Sampling Range.
This document has the final results, by experiment. See https://drive.google.com/file/d/0B5sz_T_1CNJQWmlsRTZEcjBEOEk/view?ths=true for some general documentation of the CBDA-SL project and github https://github.com/SOCR/CBDA for the code [still in progress]. The test dataset is defined as below:
# Problem parameters
n = 300 # number of observations
p = 100 # number of variables
nonzero=c(1,seq(10,p,10)) # variables with nonzero coefficients (fix location)
k = length(nonzero) # number of variables with nonzero coefficients
amplitude = 3.5 # signal amplitude (for noise level = 1)
X1 = matrix(rnorm(n*p), nrow=n, ncol=p)
beta = amplitude * (1:p %in% nonzero) # setting the nonzero variables to 3.5
ztemp <- function() X1 %*% beta + rnorm(n) # linear combination with a bias
z = ztemp()
pr = 1/(1+exp(-z)) # pass through an inv-logit function
Ytemp = rbinom(n,1,pr) # bernoulli response variable
X2 <- cbind(Ytemp,X1)
# Here I write the data in a text file [not executed]
#write.table(X2,"C:/Users/simeonem/Documents/CBDA-SL/Cranium/Binomial_dataset.txt",sep=",")
# Here I load the dataset [not executed]
#Binomial_dataset = read.csv("C:/Users/simeonem/Documents/CBDA-SL/Cranium/Binomial_dataset.txt",header = TRUE)
# Here the X and Y matrix/vector are set for the CBDA-SL algorithm to proceed [not executed]
#Ytemp <- Binomial_dataset[,1]
#Xtemp <- Binomial_dataset[,-1]Thus, the features that should be extracted by both the knockoff filter and the CBDA-SL algorithms are 1, 10, 20, 30, 40, 50, 60, 70, 80, 90 and 100. That translates into spikes on these locations in the histograms shown below. I list the False Discovery Rates, however that is just an example (the FDRs are based on how many I list for the top features selected, set to 11 now, so we can check if all the 11 non zero features are selected). Overall, the knockoff filter is really really good (this example is ad hoc for that). However, the CBDA-SL seems to perform pretty good as well. The power of CBDA-SL is that we have potentially an infinite list of “learners” that can be gradually built into it, thus eventually returning the best predictions. Now the list is short (with some issues regarding the simultaneous use of GAM and BartMachine). I am working now on generating the same type of results with a random dataset (binomial outcome), and with a continuous (i.e., Gaussian) dataset.
## [1] 1
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 1 3000 0 5 15 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 1 3000 0 5 15 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 1 3000 0 5 15 60 80
## CBDA Frequency Density Knockoff Density
## 1 20 3.787879 1 7.790417
## 50 17 3.219697 80 7.687004
## 70 15 2.840909 70 6.721820
## 90 15 2.840909 50 5.515340
## 80 14 2.651515 90 5.377456
## 10 12 2.272727 40 4.653568
## 30 12 2.272727 100 4.412272
## 34 10 1.893939 10 3.619442
## 100 10 1.893939 24 2.930024
## 11 9 1.704545 30 2.723199
## 2 8 1.515152 20 2.550845
## [1] 27.27273
## [1] 9.090909
## [1] 2
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 2 3000 10 5 15 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 2 3000 10 5 15 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 2 3000 10 5 15 60 80
## CBDA Frequency Density Knockoff Density
## 80 14 2.626642 80 7.675028
## 50 13 2.439024 1 7.001123
## 62 12 2.251407 70 6.327218
## 13 11 2.063790 90 5.466118
## 40 11 2.063790 50 5.204043
## 90 11 2.063790 100 3.706477
## 1 10 1.876173 10 3.556720
## 3 9 1.688555 40 3.369525
## 30 9 1.688555 30 3.182329
## 34 9 1.688555 20 2.470985
## 91 9 1.688555 24 2.171471
## [1] 45.45455
## [1] 9.090909
## [1] 3
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 3 3000 20 5 15 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 3 3000 20 5 15 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 3 3000 20 5 15 60 80
## CBDA Frequency Density Knockoff Density
## 80 19 3.362832 1 6.686930
## 90 15 2.654867 80 6.155015
## 58 14 2.477876 70 5.927052
## 70 13 2.300885 50 5.775076
## 40 12 2.123894 90 4.977204
## 20 11 1.946903 40 4.255319
## 30 11 1.946903 100 3.837386
## 50 11 1.946903 10 3.571429
## 56 11 1.946903 30 2.963526
## 83 10 1.769912 20 2.279635
## 1 9 1.592920 24 2.241641
## [1] 27.27273
## [1] 9.090909
## [1] 4
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 4 3000 30 5 15 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 4 3000 30 5 15 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 4 3000 30 5 15 60 80
## CBDA Frequency Density Knockoff Density
## 80 15 2.901354 80 6.537029
## 20 10 1.934236 70 5.875052
## 50 10 1.934236 1 5.709557
## 1 9 1.740812 50 5.544063
## 30 9 1.740812 90 4.633844
## 70 9 1.740812 40 4.137360
## 82 9 1.740812 100 3.558130
## 90 9 1.740812 10 2.813405
## 10 8 1.547389 24 2.523790
## 14 8 1.547389 30 2.441043
## 22 8 1.547389 20 2.234175
## [1] 27.27273
## [1] 9.090909
## [1] 5
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 5 3000 40 5 15 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 5 3000 40 5 15 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 5 3000 40 5 15 60 80
## CBDA Frequency Density Knockoff Density
## 80 17 3.074141 80 6.101255
## 88 14 2.531646 1 5.495457
## 1 13 2.350814 70 5.062743
## 10 12 2.169982 50 4.759844
## 20 12 2.169982 90 4.500216
## 29 11 1.989150 100 3.548248
## 66 11 1.989150 40 3.202077
## 62 10 1.808318 10 2.423193
## 95 10 1.808318 30 2.120294
## 24 9 1.627486 20 1.774124
## 36 9 1.627486 24 1.730852
## [1] 63.63636
## [1] 9.090909
## [1] 6
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 6 3000 0 5 15 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 6 3000 0 5 15 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 6 3000 0 5 15 100 100
## CBDA Frequency Density Knockoff Density
## 1 17 3.024911 70 9.303882
## 10 14 2.491103 1 9.069612
## 40 13 2.313167 80 8.567604
## 90 13 2.313167 50 7.730924
## 30 12 2.135231 90 7.530120
## 50 12 2.135231 100 6.425703
## 70 11 1.957295 40 5.957162
## 80 11 1.957295 10 5.722892
## 100 10 1.779359 30 3.949130
## 17 9 1.601423 24 3.279786
## 12 8 1.423488 20 2.677376
## [1] 18.18182
## [1] 9.090909
## [1] 7
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 7 3000 10 5 15 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 7 3000 10 5 15 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 7 3000 10 5 15 100 100
## CBDA Frequency Density Knockoff Density
## 70 22 4.066543 70 9.184061
## 1 19 3.512015 80 8.880455
## 80 12 2.218115 1 8.500949
## 10 10 1.848429 50 7.855787
## 40 10 1.848429 90 7.438330
## 50 10 1.848429 40 5.654649
## 30 9 1.663586 100 5.502846
## 32 9 1.663586 10 4.554080
## 39 9 1.663586 30 4.326376
## 51 9 1.663586 20 3.111954
## 71 9 1.663586 24 2.580645
## [1] 36.36364
## [1] 9.090909
## [1] 8
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 8 3000 20 5 15 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 8 3000 20 5 15 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 8 3000 20 5 15 100 100
## CBDA Frequency Density Knockoff Density
## 50 19 3.454545 80 8.693987
## 70 15 2.727273 70 8.579088
## 80 15 2.727273 1 7.813098
## 40 12 2.181818 50 6.893910
## 1 11 2.000000 90 6.779012
## 100 11 2.000000 100 6.089621
## 90 10 1.818182 10 5.247032
## 32 9 1.636364 40 5.132133
## 35 9 1.636364 30 3.178859
## 10 8 1.454545 20 3.063960
## 29 8 1.454545 24 2.795864
## [1] 27.27273
## [1] 9.090909
## [1] 9
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 9 3000 30 5 15 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 9 3000 30 5 15 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 9 3000 30 5 15 100 100
## CBDA Frequency Density Knockoff Density
## 80 16 2.941176 1 8.613445
## 1 13 2.389706 80 8.571429
## 70 13 2.389706 70 7.857143
## 7 11 2.022059 90 6.848739
## 90 11 2.022059 50 6.176471
## 10 10 1.838235 40 4.831933
## 9 9 1.654412 100 4.747899
## 15 9 1.654412 10 3.697479
## 24 9 1.654412 20 3.445378
## 3 8 1.470588 30 3.151261
## 14 8 1.470588 24 2.058824
## [1] 54.54545
## [1] 9.090909
## [1] 10
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 10 3000 40 5 15 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 10 3000 40 5 15 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 10 3000 40 5 15 100 100
## CBDA Frequency Density Knockoff Density
## 70 15 2.777778 80 8.972047
## 80 14 2.592593 1 8.295762
## 1 13 2.407407 70 8.205591
## 10 10 1.851852 90 6.447250
## 24 10 1.851852 50 6.266907
## 50 10 1.851852 100 4.643823
## 65 10 1.851852 40 3.922453
## 90 10 1.851852 10 3.606853
## 20 9 1.666667 30 2.975654
## 48 9 1.666667 24 2.614968
## 84 9 1.666667 20 1.848512
## [1] 36.36364
## [1] 9.090909
## [1] 11
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 11 3000 0 15 30 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 11 3000 0 15 30 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 11 3000 0 15 30 60 80
## CBDA Frequency Density Knockoff Density
## 40 22 1.870748 80 11.225659
## 70 22 1.870748 1 10.395189
## 80 22 1.870748 50 8.906071
## 100 22 1.870748 70 8.734250
## 90 21 1.785714 90 7.703322
## 12 20 1.700680 40 6.357388
## 60 19 1.615646 100 5.326460
## 1 18 1.530612 10 4.410080
## 10 18 1.530612 30 3.808706
## 20 18 1.530612 24 2.691867
## 98 18 1.530612 20 2.290951
## [1] 18.18182
## [1] 9.090909
## [1] 12
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 12 3000 10 15 30 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 12 3000 10 15 30 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 12 3000 10 15 30 60 80
## CBDA Frequency Density Knockoff Density
## 50 24 2.033898 80 10.265002
## 20 21 1.779661 1 9.899482
## 30 21 1.779661 70 9.686263
## 40 21 1.779661 50 7.493147
## 90 21 1.779661 90 6.274749
## 1 20 1.694915 40 5.817850
## 80 19 1.610169 100 5.543710
## 85 19 1.610169 10 4.812671
## 70 18 1.525424 30 3.898873
## 100 18 1.525424 24 2.467256
## 66 17 1.440678 20 2.436796
## [1] 18.18182
## [1] 9.090909
## [1] 13
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 13 3000 20 15 30 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 13 3000 20 15 30 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 13 3000 20 15 30 60 80
## CBDA Frequency Density Knockoff Density
## 70 24 2.018503 80 9.929789
## 80 24 2.018503 1 8.893347
## 50 22 1.850294 70 8.559010
## 90 21 1.766190 50 7.556001
## 10 19 1.597981 90 6.519559
## 43 18 1.513877 100 5.048479
## 100 18 1.513877 40 4.647275
## 26 17 1.429773 10 3.778001
## 41 17 1.429773 30 3.510532
## 34 16 1.345669 24 3.009027
## 40 16 1.345669 20 2.908726
## [1] 36.36364
## [1] 9.090909
## [1] 14
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 14 3000 30 15 30 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 14 3000 30 15 30 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 14 3000 30 15 30 60 80
## CBDA Frequency Density Knockoff Density
## 1 24 2.020202 80 10.765732
## 90 21 1.767677 1 8.339651
## 40 20 1.683502 70 7.619409
## 70 20 1.683502 50 6.823351
## 58 19 1.599327 90 6.330553
## 30 18 1.515152 40 4.738438
## 39 18 1.515152 100 4.435178
## 7 17 1.430976 10 3.184230
## 46 17 1.430976 30 2.691433
## 80 17 1.430976 20 2.388173
## 89 17 1.430976 24 1.933283
## [1] 45.45455
## [1] 9.090909
## [1] 15
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 15 3000 40 15 30 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 15 3000 40 15 30 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 15 3000 40 15 30 60 80
## CBDA Frequency Density Knockoff Density
## 80 24 2.077922 80 9.394572
## 1 23 1.991342 1 7.599165
## 50 19 1.645022 70 6.764092
## 70 18 1.558442 90 6.430063
## 74 18 1.558442 50 5.594990
## 53 17 1.471861 100 4.175365
## 56 17 1.471861 40 3.841336
## 10 16 1.385281 10 3.089770
## 11 16 1.385281 24 3.089770
## 40 16 1.385281 30 2.630480
## 68 16 1.385281 20 2.338205
## [1] 45.45455
## [1] 9.090909
## [1] 16
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 16 3000 0 15 30 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 16 3000 0 15 30 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 16 3000 0 15 30 100 100
## CBDA Frequency Density Knockoff Density
## 80 24 2.068966 80 12.821713
## 90 23 1.982759 1 12.377164
## 27 21 1.810345 70 11.839027
## 1 20 1.724138 50 10.622368
## 40 20 1.724138 90 9.475901
## 34 19 1.637931 40 7.089378
## 2 18 1.551724 100 6.434254
## 50 18 1.551724 10 5.545157
## 96 18 1.551724 30 4.422087
## 38 17 1.465517 20 2.948058
## 70 17 1.465517 24 2.714085
## [1] 45.45455
## [1] 9.090909
## [1] 17
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 17 3000 10 15 30 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 17 3000 10 15 30 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 17 3000 10 15 30 100 100
## CBDA Frequency Density Knockoff Density
## 70 26 2.164863 80 13.245546
## 30 22 1.831807 1 13.116447
## 40 22 1.831807 70 11.567261
## 90 21 1.748543 50 10.147173
## 80 20 1.665279 90 9.450039
## 1 19 1.582015 40 6.945520
## 17 17 1.415487 100 6.687322
## 49 17 1.415487 10 4.570101
## 60 17 1.415487 30 3.743868
## 95 17 1.415487 24 2.788536
## 100 17 1.415487 20 2.762716
## [1] 27.27273
## [1] 9.090909
## [1] 18
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 18 3000 20 15 30 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 18 3000 20 15 30 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 18 3000 20 15 30 100 100
## CBDA Frequency Density Knockoff Density
## 90 25 2.095557 80 13.562282
## 70 24 2.011735 70 12.019790
## 80 23 1.927913 1 11.874272
## 50 22 1.844091 50 9.284051
## 10 20 1.676446 90 9.080326
## 100 20 1.676446 100 6.781141
## 30 19 1.592624 40 6.722934
## 48 18 1.508801 10 4.685681
## 71 17 1.424979 30 3.434226
## 84 17 1.424979 24 3.055879
## 1 16 1.341157 20 2.619325
## [1] 27.27273
## [1] 9.090909
## [1] 19
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 19 3000 30 15 30 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 19 3000 30 15 30 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 19 3000 30 15 30 100 100
## CBDA Frequency Density Knockoff Density
## 80 28 2.295082 80 13.841060
## 70 23 1.885246 1 10.827815
## 90 22 1.803279 70 10.132450
## 40 20 1.639344 90 9.768212
## 1 19 1.557377 50 9.337748
## 11 19 1.557377 100 5.695364
## 73 19 1.557377 40 5.496689
## 24 18 1.475410 10 4.801325
## 30 18 1.475410 30 4.370861
## 67 18 1.475410 24 3.079470
## 78 18 1.475410 20 1.986755
## [1] 45.45455
## [1] 9.090909
## [1] 20
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 20 3000 40 15 30 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 20 3000 40 15 30 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 20 3000 40 15 30 100 100
## CBDA Frequency Density Knockoff Density
## 80 26 2.195946 80 12.844393
## 40 23 1.942568 1 10.787738
## 1 22 1.858108 70 10.399690
## 50 21 1.773649 50 9.390764
## 90 19 1.604730 90 7.411719
## 85 18 1.520270 100 5.820722
## 7 17 1.435811 40 5.355064
## 10 16 1.351351 10 4.307334
## 17 16 1.351351 30 3.065580
## 20 16 1.351351 20 2.444703
## 21 16 1.351351 24 2.405898
## [1] 36.36364
## [1] 9.090909
## [1] 22
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 22 3000 10 30 50 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 22 3000 10 30 50 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 22 3000 10 30 50 60 80
## CBDA Frequency Density Knockoff Density
## 30 36 1.683029 80 13.322231
## 1 32 1.496026 1 12.461837
## 70 32 1.496026 70 11.462670
## 40 31 1.449275 50 8.742714
## 50 30 1.402525 90 7.854566
## 15 29 1.355774 40 5.856231
## 43 28 1.309023 100 5.772967
## 84 28 1.309023 10 4.551762
## 23 27 1.262272 30 3.746878
## 74 27 1.262272 20 2.442409
## 2 26 1.215521 24 2.109353
## [1] 54.54545
## [1] 9.090909
## [1] 23
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 23 3000 20 30 50 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 23 3000 20 30 50 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 23 3000 20 30 50 60 80
## CBDA Frequency Density Knockoff Density
## 10 32 1.551891 80 12.734798
## 50 29 1.406402 1 10.697230
## 52 29 1.406402 70 10.697230
## 90 29 1.406402 50 8.404967
## 70 28 1.357905 90 7.895575
## 1 27 1.309408 40 5.985355
## 21 27 1.309408 100 5.475963
## 80 27 1.309408 10 4.680038
## 8 26 1.260912 30 3.215536
## 23 26 1.260912 24 2.387775
## 37 26 1.260912 20 2.005731
## [1] 45.45455
## [1] 9.090909
## [1] 24
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 24 3000 30 30 50 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 24 3000 30 30 50 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 24 3000 30 30 50 60 80
## CBDA Frequency Density Knockoff Density
## 30 29 1.422266 80 11.796151
## 43 28 1.373222 1 10.620100
## 64 28 1.373222 70 9.550962
## 70 28 1.373222 50 7.982894
## 29 27 1.324179 90 7.483963
## 50 27 1.324179 40 5.416964
## 54 27 1.324179 100 5.167498
## 19 26 1.275135 10 3.706344
## 40 26 1.275135 24 2.957947
## 80 26 1.275135 30 2.851033
## 99 26 1.275135 20 2.102637
## [1] 54.54545
## [1] 9.090909
## [1] 25
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 25 3000 40 30 50 60 80
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 25 3000 40 30 50 60 80
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 25 3000 40 30 50 60 80
## CBDA Frequency Density Knockoff Density
## 1 34 1.666667 80 11.468076
## 50 31 1.519608 1 9.231395
## 81 31 1.519608 70 7.848719
## 95 29 1.421569 50 7.157381
## 44 28 1.372549 90 6.303375
## 80 28 1.372549 100 4.676698
## 36 27 1.323529 40 4.310695
## 70 27 1.323529 10 3.782025
## 13 26 1.274510 30 2.724685
## 56 26 1.274510 24 2.602684
## 63 26 1.274510 20 2.155348
## [1] 63.63636
## [1] 9.090909
## [1] 26
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 26 3000 0 30 50 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 26 3000 0 30 50 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 26 3000 0 30 50 100 100
## CBDA Frequency Density Knockoff Density
## 50 36 1.679104 80 15.511325
## 100 34 1.585821 1 14.396019
## 10 33 1.539179 70 13.435141
## 80 32 1.492537 50 11.547701
## 48 30 1.399254 90 10.586822
## 60 29 1.352612 40 7.223747
## 70 29 1.352612 100 6.691833
## 1 28 1.305970 10 5.404942
## 20 28 1.305970 30 4.306795
## 40 28 1.305970 20 2.264928
## 62 27 1.259328 24 2.093342
## [1] 18.18182
## [1] 9.090909
## [1] 27
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 27 3000 10 30 50 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 27 3000 10 30 50 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 27 3000 10 30 50 100 100
## CBDA Frequency Density Knockoff Density
## 1 32 1.554153 80 15.953545
## 80 32 1.554153 1 14.527302
## 10 30 1.457018 70 13.284434
## 30 30 1.457018 50 11.145069
## 40 30 1.457018 90 9.739201
## 100 30 1.457018 40 7.314588
## 70 29 1.408451 100 6.295844
## 90 29 1.408451 10 5.134474
## 63 28 1.359883 30 3.810106
## 60 27 1.311316 24 2.526487
## 19 26 1.262749 20 2.363488
## [1] 18.18182
## [1] 9.090909
## [1] 28
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 28 3000 20 30 50 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 28 3000 20 30 50 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 28 3000 20 30 50 100 100
## CBDA Frequency Density Knockoff Density
## 80 32 1.566324 80 15.799477
## 50 30 1.468429 1 14.516512
## 78 30 1.468429 70 13.399857
## 10 29 1.419481 50 10.833927
## 100 29 1.419481 90 9.812307
## 70 28 1.370534 100 6.509860
## 1 27 1.321586 40 6.082205
## 3 27 1.321586 10 4.490378
## 37 27 1.321586 30 3.278689
## 97 27 1.321586 24 2.352103
## 20 26 1.272638 20 2.280827
## [1] 36.36364
## [1] 9.090909
## [1] 29
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 29 3000 30 30 50 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 29 3000 30 30 50 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 29 3000 30 30 50 100 100
## CBDA Frequency Density Knockoff Density
## 30 32 1.535509 80 16.713961
## 50 32 1.535509 1 14.188422
## 40 31 1.487524 70 13.166856
## 80 31 1.487524 50 10.130533
## 90 29 1.391555 90 9.108967
## 47 28 1.343570 100 6.044268
## 70 28 1.343570 40 5.703746
## 100 28 1.343570 10 3.830874
## 10 27 1.295585 30 3.405221
## 1 26 1.247601 24 2.383655
## 15 26 1.247601 20 2.241771
## [1] 18.18182
## [1] 9.090909
## [1] 30
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 30 3000 40 30 50 100 100
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 30 3000 40 30 50 100 100
## [1] "CBDA-SL results"
## M misValperc Kcol_min Kcol_max Nrow_min Nrow_max
## 30 3000 40 30 50 100 100
## CBDA Frequency Density Knockoff Density
## 10 31 1.511458 80 15.277778
## 22 30 1.462701 1 12.678063
## 40 30 1.462701 70 12.001425
## 80 30 1.462701 50 9.401709
## 20 29 1.413944 90 9.009972
## 50 29 1.413944 100 6.196581
## 70 29 1.413944 40 5.555556
## 30 27 1.316431 10 4.558405
## 90 27 1.316431 30 3.133903
## 95 27 1.316431 20 2.314815
## 100 27 1.316431 24 2.243590
## [1] 18.18182
## [1] 9.090909