FMG Study 1 CFA model testing FINAL

Design

A total of 84 conditions, with 2000 replications in each.

• n=400, 800, 2000
• Five-factor model with dim=15,30, 60, 100
• distributions: normal, vm1, pl1, ig1, vm2, pl2, ig2

Normal data

Overall

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
std_rls	12	0.5	0.4	0.0	100.0	0
sb_ug_rls	12	0.7	0.5	0.0	100.0	0
peba2_rls	12	0.7	0.6	0.0	100.0	0
sb_rls	12	1.0	0.8	0.0	100.0	0
peba2_ug_rls	12	1.7	1.4	8.3	91.7	0
peba4_rls	12	1.9	1.5	25.0	75.0	0
peba6_rls	12	2.3	1.8	41.7	58.3	0
peba4_ug_rls	12	2.5	2.0	41.7	58.3	0
pall_rls	12	2.6	2.0	41.7	58.3	0
pols2_rls	12	2.6	2.0	33.3	66.7	0
peba6_ug_rls	12	2.7	2.2	41.7	58.3	0
all_ug	12	2.8	2.4	41.7	58.3	0
sf_ug	12	2.8	2.4	41.7	58.3	0
ss_ug	12	2.8	2.4	41.7	50.0	8
pols2_ug_rls	12	2.8	2.3	41.7	58.3	0
pall_ug_rls	12	2.9	2.4	41.7	58.3	0
all	12	3.5	2.7	41.7	50.0	8
sf	12	3.5	2.7	41.7	50.0	8
ss_rls	12	3.8	3.4	66.7	33.3	0
all_rls	12	3.8	3.4	66.7	33.3	0
sf_rls	12	3.8	3.4	66.7	33.3	0
ss_ug_rls	12	3.8	3.5	66.7	33.3	0
all_ug_rls	12	3.9	3.5	66.7	33.3	0
sf_ug_rls	12	3.9	3.5	66.7	33.3	0
ss	12	4.0	2.7	33.3	58.3	8
pols2_ug	12	24.7	11.1	0.0	50.0	50
pall_ug	12	25.7	11.7	0.0	58.3	42
pols2	12	27.5	13.3	0.0	41.7	58
pall	12	28.6	13.9	0.0	41.7	58
peba6_ug	12	29.3	13.7	0.0	58.3	42
peba6	12	30.9	15.6	0.0	41.7	58
peba4_ug	12	31.2	15.3	0.0	58.3	42
peba4	12	32.8	17.2	0.0	41.7	58
peba2_ug	12	34.0	18.2	0.0	41.7	58
peba2	12	36.2	20.3	0.0	41.7	58
sb_ug	12	36.3	20.9	0.0	33.3	67
std_ml	12	37.8	22.4	0.0	33.3	67
sb	12	38.5	23.0	0.0	33.3	67

ML vs RLS

Dimensions in panels, note y-scale is fluid across panels

Non-normal data

dim=15

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba2_ug	18	0.4	0.3	0.0	100.0	0
pols2	18	0.4	0.3	0.0	100.0	0
peba4	18	0.5	0.4	0.0	100.0	0
sb_ug_rls	18	0.5	0.4	0.0	100.0	0
sb_rls	18	0.5	0.4	0.0	100.0	0
peba6	18	0.6	0.5	0.0	100.0	0
peba2	18	0.6	0.5	0.0	100.0	0
pols2_ug	18	0.6	0.5	0.0	100.0	0
peba2_rls	18	0.6	0.5	0.0	100.0	0
peba4_ug	18	0.6	0.5	0.0	100.0	0
peba6_ug	18	0.8	0.7	0.0	100.0	0
peba2_ug_rls	18	0.8	0.8	0.0	100.0	0
pols2_rls	18	1.0	0.8	0.0	100.0	0
sb_ug	18	1.0	0.9	0.0	100.0	0
peba4_rls	18	1.0	0.9	0.0	100.0	0
pall	18	1.1	0.9	0.0	100.0	0
peba6_rls	18	1.2	1.0	0.0	100.0	0
pols2_ug_rls	18	1.3	1.1	0.0	100.0	0
pall_ug	18	1.3	1.1	5.6	94.4	0
peba4_ug_rls	18	1.3	1.1	0.0	100.0	0
sb	18	1.4	1.2	0.0	94.4	6
peba6_ug_rls	18	1.5	1.3	11.1	88.9	0
pall_rls	18	1.7	1.5	11.1	88.9	0
pall_ug_rls	18	1.9	1.7	22.2	77.8	0
ss	18	3.0	2.8	61.1	38.9	0
ss_ug	18	3.1	2.9	66.7	33.3	0
all	18	3.2	3.0	66.7	33.3	0
sf	18	3.3	3.1	66.7	33.3	0
ss_rls	18	3.3	3.1	72.2	27.8	0
all_ug	18	3.4	3.2	72.2	27.8	0
sf_ug	18	3.4	3.2	72.2	27.8	0
ss_ug_rls	18	3.4	3.2	72.2	27.8	0
all_rls	18	3.5	3.3	77.8	22.2	0
sf_rls	18	3.5	3.3	77.8	22.2	0
all_ug_rls	18	3.6	3.4	77.8	22.2	0
sf_ug_rls	18	3.6	3.4	77.8	22.2	0
std_rls	18	22.0	14.0	0.0	11.1	89
std_ml	18	23.2	15.5	0.0	5.6	94

dim=30

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba6_ug	18	1.1	0.7	0.0	94.4	6
peba4_rls	18	1.1	0.9	0.0	94.4	6
peba6_rls	18	1.2	1.0	0.0	100.0	0
pols2_rls	18	1.2	1.0	0.0	94.4	6
peba4_ug_rls	18	1.3	1.1	5.6	94.4	0
pols2_ug_rls	18	1.4	1.1	5.6	88.9	6
pall	18	1.4	1.1	5.6	94.4	0
peba2_ug_rls	18	1.5	1.2	0.0	94.4	6
peba6_ug_rls	18	1.6	1.3	11.1	88.9	0
pall_ug	18	1.8	1.6	22.2	77.8	0
pols2_ug	18	1.8	1.4	0.0	83.3	17
peba4_ug	18	1.9	1.4	0.0	83.3	17
peba6	18	1.9	1.4	0.0	83.3	17
peba2_rls	18	1.9	1.5	0.0	77.8	22
pall_rls	18	2.5	2.2	55.6	44.4	0
sb_ug_rls	18	2.6	2.1	0.0	50.0	50
pols2	18	2.6	2.2	0.0	66.7	33
peba4	18	2.7	2.2	0.0	61.1	39
pall_ug_rls	18	2.8	2.5	55.6	44.4	0
sb_rls	18	3.4	2.9	0.0	50.0	50
peba2_ug	18	3.8	3.1	0.0	50.0	50
ss	18	4.2	4.1	94.4	5.6	0
ss_ug	18	4.3	4.2	94.4	5.6	0
all	18	4.4	4.3	94.4	5.6	0
sf	18	4.4	4.3	94.4	5.6	0
all_ug	18	4.4	4.3	94.4	5.6	0
sf_ug	18	4.4	4.3	94.4	5.6	0
ss_rls	18	4.4	4.3	94.4	5.6	0
ss_ug_rls	18	4.4	4.4	94.4	5.6	0
all_rls	18	4.5	4.4	94.4	5.6	0
sf_rls	18	4.5	4.4	94.4	5.6	0
sf_ug_rls	18	4.5	4.5	94.4	5.6	0
all_ug_rls	18	4.5	4.5	94.4	5.6	0
peba2	18	5.0	4.2	0.0	38.9	61
sb_ug	18	6.1	5.3	0.0	16.7	83
sb	18	7.7	6.6	0.0	16.7	83
std_rls	18	64.3	57.3	0.0	0.0	100
std_ml	18	67.9	62.4	0.0	0.0	100

dim=60

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba4_ug_rls	18	1.9	1.6	27.8	72.2	0
peba6_rls	18	2.0	1.6	27.8	72.2	0
peba4_rls	18	2.2	1.7	0.0	88.9	11
peba6_ug_rls	18	2.5	2.2	33.3	66.7	0
pall_ug	18	2.6	2.1	33.3	66.7	0
pols2_rls	18	2.7	2.4	50.0	50.0	0
peba2_ug_rls	18	2.7	2.1	0.0	72.2	28
pall	18	2.8	2.3	33.3	61.1	6
pols2_ug_rls	18	3.2	2.9	61.1	38.9	0
pall_rls	18	4.2	4.1	94.4	5.6	0
pall_ug_rls	18	4.3	4.2	100.0	0.0	0
peba2_rls	18	4.5	3.0	0.0	61.1	39
sb_ug_rls	18	4.7	3.6	0.0	50.0	50
ss	18	4.9	4.9	100.0	0.0	0
ss_ug	18	4.9	4.9	100.0	0.0	0
all	18	4.9	4.9	100.0	0.0	0
sf	18	4.9	4.9	100.0	0.0	0
sf_ug	18	4.9	4.9	100.0	0.0	0
all_ug	18	4.9	4.9	100.0	0.0	0
ss_rls	18	5.0	4.9	100.0	0.0	0
ss_ug_rls	18	5.0	5.0	100.0	0.0	0
all_rls	18	5.0	5.0	100.0	0.0	0
sf_rls	18	5.0	5.0	100.0	0.0	0
all_ug_rls	18	5.0	5.0	100.0	0.0	0
sf_ug_rls	18	5.0	5.0	100.0	0.0	0
pols2_ug	18	5.5	3.8	0.0	50.0	50
sb_rls	18	7.1	5.8	0.0	22.2	78
pols2	18	8.7	6.2	0.0	44.4	56
peba6_ug	18	10.7	7.3	0.0	44.4	56
peba6	18	14.9	10.4	0.0	27.8	72
peba4_ug	18	15.1	10.8	0.0	27.8	72
peba4	18	19.9	14.3	0.0	27.8	72
peba2_ug	18	22.4	17.1	0.0	5.6	94
sb_ug	18	27.4	21.9	0.0	0.0	100
peba2	18	27.8	21.3	0.0	0.0	100
sb	18	33.2	26.5	0.0	0.0	100
std_rls	18	82.3	78.0	0.0	0.0	100
std_ml	18	87.4	85.7	0.0	0.0	100

dim=100

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba6_ug_rls	18	3.4	3.1	61.1	33.3	6
peba4_ug_rls	18	4.0	3.1	44.4	38.9	17
pols2_rls	18	4.5	4.4	100.0	0.0	0
peba6_rls	18	4.6	3.4	38.9	44.4	17
pols2_ug_rls	18	4.6	4.6	100.0	0.0	0
pall_rls	18	4.8	4.8	100.0	0.0	0
pall_ug_rls	18	4.8	4.8	100.0	0.0	0
ss	18	5.0	5.0	100.0	0.0	0
all	18	5.0	5.0	100.0	0.0	0
sf	18	5.0	5.0	100.0	0.0	0
ss_ug	18	5.0	5.0	100.0	0.0	0
all_ug	18	5.0	5.0	100.0	0.0	0
sf_ug	18	5.0	5.0	100.0	0.0	0
all_rls	18	5.0	5.0	100.0	0.0	0
all_ug_rls	18	5.0	5.0	100.0	0.0	0
sf_rls	18	5.0	5.0	100.0	0.0	0
sf_ug_rls	18	5.0	5.0	100.0	0.0	0
ss_rls	18	5.0	5.0	100.0	0.0	0
ss_ug_rls	18	5.0	5.0	100.0	0.0	0
peba2_ug_rls	18	6.3	3.9	5.6	61.1	33
peba4_rls	18	6.6	3.8	11.1	61.1	28
sb_ug_rls	18	8.7	5.5	0.0	50.0	50
peba2_rls	18	10.8	6.6	0.0	38.9	61
sb_rls	18	13.9	9.8	0.0	16.7	83
pall_ug	18	17.1	9.7	38.9	33.3	28
pall	18	23.3	13.2	27.8	33.3	39
pols2_ug	18	36.5	22.3	22.2	27.8	50
pols2	18	42.4	26.7	11.1	33.3	56
peba6_ug	18	53.6	39.2	0.0	22.2	78
peba6	18	56.6	42.8	0.0	11.1	89
peba4_ug	18	57.0	44.1	0.0	5.6	94
peba4	18	59.8	47.5	0.0	5.6	94
peba2_ug	18	61.1	50.4	0.0	0.0	100
sb_ug	18	63.8	54.6	0.0	0.0	100
peba2	18	63.9	54.0	0.0	0.0	100
sb	18	66.4	57.9	0.0	0.0	100
std_rls	18	83.7	80.1	0.0	0.0	100
std_ml	18	92.4	91.9	0.0	0.0	100

Overall non-normal data

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba6_ug_rls	72	2.4	2.0	29.2	69.4	1
peba4_ug_rls	72	2.4	1.7	19.4	76.4	4
peba6_rls	72	2.6	1.8	16.7	79.2	4
pols2_rls	72	2.7	2.2	37.5	61.1	1
pols2_ug_rls	72	3.0	2.4	41.7	56.9	1
pall_rls	72	3.5	3.1	65.3	34.7	0
peba2_ug_rls	72	3.5	2.0	1.4	81.9	17
peba4_rls	72	3.6	1.8	2.8	86.1	11
pall_ug_rls	72	3.7	3.3	69.4	30.6	0
ss	72	4.3	4.2	88.9	11.1	0
ss_ug	72	4.4	4.3	90.3	9.7	0
all	72	4.4	4.3	90.3	9.7	0
sf	72	4.4	4.3	90.3	9.7	0
ss_rls	72	4.5	4.3	91.7	8.3	0
all_ug	72	4.5	4.4	91.7	8.3	0
sf_ug	72	4.5	4.4	91.7	8.3	0
ss_ug_rls	72	4.5	4.4	91.7	8.3	0
all_rls	72	4.5	4.4	93.1	6.9	0
sf_rls	72	4.5	4.4	93.1	6.9	0
all_ug_rls	72	4.5	4.5	93.1	6.9	0
sf_ug_rls	72	4.5	4.5	93.1	6.9	0
sb_ug_rls	72	5.1	2.9	0.0	62.5	38
peba2_rls	72	5.9	2.9	0.0	69.4	31
sb_rls	72	8.0	4.7	0.0	47.2	53
pall_ug	72	8.7	3.6	25.0	68.1	7
pall	72	11.8	4.4	16.7	72.2	11
pols2_ug	72	18.5	7.0	5.6	65.3	29
pols2	72	21.7	8.8	2.8	61.1	36
peba6_ug	72	27.4	12.0	0.0	65.3	35
peba6	72	29.3	13.8	0.0	55.6	44
peba4_ug	72	29.5	14.2	0.0	54.2	46
peba4	72	31.5	16.1	0.0	48.6	51
peba2_ug	72	32.6	17.7	0.0	38.9	61
sb_ug	72	34.8	20.7	0.0	29.2	71
peba2	72	34.9	20.0	0.0	34.7	65
sb	72	37.3	23.0	0.0	27.8	72
std_rls	72	67.8	57.4	0.0	2.8	97
std_ml	72	73.0	63.9	0.0	1.4	99

By sample size

n=400

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
pols2_rls	28	3.0	2.6	46.4	53.6	0
peba6_ug_rls	28	3.0	2.7	46.4	50.0	4
peba4_ug_rls	28	3.4	2.6	32.1	60.7	7
pols2_ug_rls	28	3.4	3.0	53.6	46.4	0
peba6_rls	28	3.7	2.5	21.4	71.4	7
pall_rls	28	3.9	3.7	78.6	21.4	0
pall_ug_rls	28	4.1	3.9	85.7	14.3	0
ss_ug	28	4.5	4.4	92.9	3.6	4
all_ug	28	4.5	4.5	92.9	7.1	0
sf_ug	28	4.6	4.5	92.9	7.1	0

n=800

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba4_ug_rls	28	2.0	1.6	25.0	71.4	4
peba6_rls	28	2.0	1.6	25.0	71.4	4
peba4_rls	28	2.1	1.5	3.6	89.3	7
peba6_ug_rls	28	2.3	2.0	32.1	67.9	0
peba2_ug_rls	28	2.6	1.7	0.0	82.1	18
pols2_rls	28	2.8	2.2	39.3	60.7	0
pols2_ug_rls	28	3.0	2.4	46.4	53.6	0
pall_rls	28	3.5	3.1	64.3	35.7	0
pall_ug_rls	28	3.6	3.3	64.3	35.7	0
peba2_rls	28	3.8	2.1	0.0	75.0	25

n=2000

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba4_rls	28	1.3	1.0	10.7	85.7	4
peba4_ug_rls	28	1.4	1.1	10.7	89.3	0
peba6_rls	28	1.5	1.2	14.3	85.7	0
peba2_ug_rls	28	1.6	1.2	0.0	89.3	11
peba6_ug_rls	28	1.7	1.3	14.3	85.7	0
peba2_rls	28	1.8	1.2	0.0	82.1	18
pols2_ug	28	2.1	1.5	7.1	78.6	14
pall_ug	28	2.2	1.5	17.9	78.6	4
pols2_rls	28	2.2	1.6	25.0	71.4	4
peba6_ug	28	2.2	1.4	0.0	85.7	14

By distribution

normal

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
std_rls	12	0.5	0.4	0.0	100.0	0
sb_ug_rls	12	0.7	0.5	0.0	100.0	0
peba2_rls	12	0.7	0.6	0.0	100.0	0
sb_rls	12	1.0	0.8	0.0	100.0	0
peba2_ug_rls	12	1.7	1.4	8.3	91.7	0
peba4_rls	12	1.9	1.5	25.0	75.0	0
peba6_rls	12	2.3	1.8	41.7	58.3	0
peba4_ug_rls	12	2.5	2.0	41.7	58.3	0
pall_rls	12	2.6	2.0	41.7	58.3	0
pols2_rls	12	2.6	2.0	33.3	66.7	0

vm1

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba2_ug_rls	12	0.8	0.8	0.0	100.0	0
sb_ug_rls	12	1.1	0.9	0.0	100.0	0
peba4_rls	12	1.4	1.2	8.3	91.7	0
peba2_rls	12	1.6	1.0	0.0	91.7	8
peba6_rls	12	2.2	1.9	33.3	66.7	0
peba4_ug_rls	12	2.2	2.0	41.7	58.3	0
peba6_ug_rls	12	2.8	2.5	41.7	58.3	0
pols2_rls	12	2.9	2.3	41.7	58.3	0
pols2_ug_rls	12	3.1	2.6	41.7	58.3	0
pall_rls	12	3.3	3.0	58.3	41.7	0

vm2

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
pols2_rls	12	2.1	1.8	25.0	75.0	0
peba6_ug_rls	12	2.2	1.6	8.3	83.3	8
pols2_ug_rls	12	2.5	2.1	41.7	58.3	0
pall	12	3.0	2.7	41.7	58.3	0
pall_ug	12	3.5	3.2	66.7	33.3	0
peba4_ug_rls	12	3.7	2.1	0.0	83.3	17
pall_rls	12	3.9	3.6	75.0	25.0	0
pall_ug_rls	12	4.0	3.8	75.0	25.0	0
peba6_rls	12	4.3	2.2	0.0	83.3	17
ss	12	4.7	4.6	100.0	0.0	0

pl1

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
sb_ug_rls	12	0.9	0.8	0.0	100.0	0
peba2_ug_rls	12	0.9	0.8	0.0	100.0	0
peba2_rls	12	1.4	0.9	0.0	91.7	8
peba4_rls	12	1.5	1.2	0.0	100.0	0
peba4_ug_rls	12	2.2	1.9	41.7	58.3	0
peba6_rls	12	2.3	1.9	33.3	66.7	0
peba6_ug_rls	12	2.7	2.3	50.0	50.0	0
pols2_rls	12	2.9	2.3	41.7	58.3	0
sb_rls	12	3.0	2.1	0.0	75.0	25
pols2_ug_rls	12	3.1	2.6	50.0	50.0	0

pl2

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba6_ug_rls	12	1.8	1.7	8.3	91.7	0
peba4_ug_rls	12	2.1	1.4	0.0	91.7	8
peba6_rls	12	2.7	1.7	0.0	91.7	8
pols2_rls	12	2.8	2.3	41.7	50.0	8
pols2_ug_rls	12	3.1	2.7	41.7	50.0	8
pall	12	3.2	2.9	58.3	41.7	0
pall_ug	12	3.5	3.1	75.0	25.0	0
pall_rls	12	3.7	3.3	75.0	25.0	0
pall_ug_rls	12	3.8	3.4	75.0	25.0	0
peba2_ug_rls	12	4.6	3.3	0.0	50.0	50

ig1

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
sb_ug_rls	12	0.7	0.6	0.0	100.0	0
peba2_rls	12	0.7	0.6	0.0	100.0	0
peba2_ug_rls	12	1.2	1.0	8.3	91.7	0
peba4_rls	12	1.6	1.4	8.3	91.7	0
sb_rls	12	2.1	1.7	0.0	66.7	33
peba6_rls	12	2.3	2.0	33.3	66.7	0
peba4_ug_rls	12	2.4	2.1	33.3	66.7	0
peba6_ug_rls	12	2.8	2.5	50.0	50.0	0
pols2_rls	12	2.9	2.3	41.7	58.3	0
pols2_ug_rls	12	3.1	2.6	41.7	58.3	0

ig2

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba4_ug_rls	12	1.0	0.9	0.0	100.0	0
peba6_rls	12	1.0	0.8	0.0	100.0	0
peba4_rls	12	1.3	1.1	0.0	91.7	8
peba6_ug_rls	12	1.6	1.3	16.7	83.3	0
peba2_ug_rls	12	1.9	1.7	0.0	91.7	8
pols2_rls	12	2.6	2.1	33.3	66.7	0
pols2_ug_rls	12	2.7	2.1	33.3	66.7	0
pall_rls	12	3.5	3.2	66.7	33.3	0
pall_ug_rls	12	3.7	3.4	75.0	25.0	0
peba2_rls	12	3.8	3.1	0.0	33.3	67

OVERALL normal + non-normal

By RMSE

method	numcond	RMSE	MAD	BEL2.5	ARR	ABO7.5
peba4_ug_rls	84	2.4	1.8	22.6	73.8	4
peba6_ug_rls	84	2.4	2.0	31.0	67.9	1
peba6_rls	84	2.6	1.8	20.2	76.2	4
pols2_rls	84	2.7	2.1	36.9	61.9	1
pols2_ug_rls	84	2.9	2.4	41.7	57.1	1
peba2_ug_rls	84	3.3	1.9	2.4	83.3	14
peba4_rls	84	3.4	1.8	6.0	84.5	10
pall_rls	84	3.4	3.0	61.9	38.1	0
pall_ug_rls	84	3.6	3.2	65.5	34.5	0
ss_ug	84	4.2	4.0	83.3	15.5	1
all_ug	84	4.3	4.1	84.5	15.5	0
sf_ug	84	4.3	4.1	84.5	15.5	0
ss	84	4.3	4.0	81.0	17.9	1
all	84	4.3	4.1	83.3	15.5	1
sf	84	4.3	4.1	83.3	15.5	1
ss_rls	84	4.4	4.2	88.1	11.9	0
ss_ug_rls	84	4.4	4.3	88.1	11.9	0
all_rls	84	4.4	4.3	89.3	10.7	0
sf_rls	84	4.4	4.3	89.3	10.7	0
all_ug_rls	84	4.5	4.3	89.3	10.7	0
sf_ug_rls	84	4.5	4.3	89.3	10.7	0
sb_ug_rls	84	4.7	2.6	0.0	67.9	32
peba2_rls	84	5.5	2.6	0.0	73.8	26
sb_rls	84	7.4	4.2	0.0	54.8	45
pall_ug	84	12.6	4.8	21.4	66.7	12
pall	84	15.3	5.7	14.3	67.9	18
pols2_ug	84	19.5	7.6	4.8	63.1	32
pols2	84	22.6	9.5	2.4	58.3	39
peba6_ug	84	27.6	12.2	0.0	64.3	36
peba6	84	29.5	14.0	0.0	53.6	46
peba4_ug	84	29.7	14.3	0.0	54.8	45
peba4	84	31.7	16.3	0.0	47.6	52
peba2_ug	84	32.8	17.8	0.0	39.3	61
sb_ug	84	35.1	20.7	0.0	29.8	70
peba2	84	35.1	20.0	0.0	35.7	64
sb	84	37.5	23.0	0.0	28.6	71
std_rls	84	62.8	49.2	0.0	16.7	83
std_ml	84	69.1	58.0	0.0	6.0	94

By MAD

method	numcond	RMSE	MAD	ARR
peba4_ug_rls	84	2.409394	1.765456	73.809524
peba6_rls	84	2.588814	1.771057	76.190476
peba4_rls	84	3.372637	1.788932	84.523809
peba2_ug_rls	84	3.313106	1.907741	83.333333
peba6_ug_rls	84	2.425438	2.016302	67.857143
pols2_rls	84	2.700248	2.132013	61.904762
pols2_ug_rls	84	2.939929	2.415749	57.142857
sb_ug_rls	84	4.731151	2.570904	67.857143
peba2_rls	84	5.514792	2.587945	73.809524
pall_rls	84	3.395651	2.968087	38.095238
pall_ug_rls	84	3.556569	3.169241	34.523809
ss	84	4.300336	3.979575	17.857143
ss_ug	84	4.197583	3.988989	15.476190
all	84	4.308030	4.064183	15.476190
sf	84	4.313563	4.071333	15.476190
all_ug	84	4.268579	4.079556	15.476190
sf_ug	84	4.270302	4.081939	15.476190
sb_rls	84	7.403199	4.163330	54.761905
ss_rls	84	4.372230	4.207541	11.904762
ss_ug_rls	84	4.404471	4.255803	11.904762
all_rls	84	4.423409	4.276658	10.714286
sf_rls	84	4.426184	4.280828	10.714286
all_ug_rls	84	4.456948	4.325516	10.714286
sf_ug_rls	84	4.457125	4.326112	10.714286
pall_ug	84	12.630344	4.774179	66.666667
pall	84	15.348340	5.735497	67.857143
pols2_ug	84	19.492593	7.592116	63.095238
pols2	84	22.598658	9.488179	58.333333
peba6_ug	84	27.646335	12.228776	64.285714
peba6	84	29.530959	14.042257	53.571429
peba4_ug	84	29.739399	14.343034	54.761905
peba4	84	31.708409	16.262095	47.619048
peba2_ug	84	32.789110	17.783736	39.285714
peba2	84	35.125738	20.021569	35.714286
sb_ug	84	35.059006	20.704037	29.761905
sb	84	37.492611	23.036251	28.571429
std_rls	84	62.780180	49.211831	16.666667
std_ml	84	69.079693	57.953430	5.952381

By ARR

method	numcond	RMSE	MAD	ARR
peba4_rls	84	3.372637	1.788932	84.523809
peba2_ug_rls	84	3.313106	1.907741	83.333333
peba6_rls	84	2.588814	1.771057	76.190476
peba2_rls	84	5.514792	2.587945	73.809524
peba4_ug_rls	84	2.409394	1.765456	73.809524
pall	84	15.348340	5.735497	67.857143
peba6_ug_rls	84	2.425438	2.016302	67.857143
sb_ug_rls	84	4.731151	2.570904	67.857143
pall_ug	84	12.630344	4.774179	66.666667
peba6_ug	84	27.646335	12.228776	64.285714
pols2_ug	84	19.492593	7.592116	63.095238
pols2_rls	84	2.700248	2.132013	61.904762
pols2	84	22.598658	9.488179	58.333333
pols2_ug_rls	84	2.939929	2.415749	57.142857
peba4_ug	84	29.739399	14.343034	54.761905
sb_rls	84	7.403199	4.163330	54.761905
peba6	84	29.530959	14.042257	53.571429
peba4	84	31.708409	16.262095	47.619048
peba2_ug	84	32.789110	17.783736	39.285714
pall_rls	84	3.395651	2.968087	38.095238
peba2	84	35.125738	20.021569	35.714286
pall_ug_rls	84	3.556569	3.169241	34.523809
sb_ug	84	35.059006	20.704037	29.761905
sb	84	37.492611	23.036251	28.571429
ss	84	4.300336	3.979575	17.857143
std_rls	84	62.780180	49.211831	16.666667
all	84	4.308030	4.064183	15.476190
all_ug	84	4.268579	4.079556	15.476190
sf	84	4.313563	4.071333	15.476190
sf_ug	84	4.270302	4.081939	15.476190
ss_ug	84	4.197583	3.988989	15.476190
ss_rls	84	4.372230	4.207541	11.904762
ss_ug_rls	84	4.404471	4.255803	11.904762
all_rls	84	4.423409	4.276658	10.714286
all_ug_rls	84	4.456948	4.325516	10.714286
sf_rls	84	4.426184	4.280828	10.714286
sf_ug_rls	84	4.457125	4.326112	10.714286
std_ml	84	69.079693	57.953430	5.952381

postscript on Unbiased Gamma

overall

ub	numcond	RMSE	MAD	ARR
FALSE	1512	17.47909	7.159852	42.39418
TRUE	1512	16.11080	6.462733	43.45238

By sample size

n	ub	numcond	RMSE	MAD	ARR
400	FALSE	504	26.831916	12.502681	31.34921
400	TRUE	504	25.051210	11.204895	32.73810
800	FALSE	504	13.446771	6.218222	41.26984
800	TRUE	504	11.727503	5.529835	41.66667
2000	FALSE	504	3.973501	2.758652	54.56349
2000	TRUE	504	3.684645	2.653467	55.95238

By test

All tests, except ss, seem to benefit from ug?

method	numcond	RMSE	MAD	ARR
peba2	84	35.125738	20.021569	35.71429
peba2_ug	84	32.789110	17.783736	39.28571
peba2_rls	84	5.514792	2.587945	73.80952
peba2_ug_rls	84	3.313106	1.907741	83.33333
peba4	84	31.708409	16.262095	47.61905
peba4_ug	84	29.739399	14.343034	54.76190
peba4_rls	84	3.372637	1.788932	84.52381
peba4_ug_rls	84	2.409394	1.765456	73.80952
peba6	84	29.530959	14.042257	53.57143
peba6_ug	84	27.646335	12.228776	64.28571
peba6_rls	84	2.588814	1.771057	76.19048
peba6_ug_rls	84	2.425438	2.016302	67.85714
pols2	84	22.598658	9.488179	58.33333
pols2_ug	84	19.492593	7.592116	63.09524
pols2_rls	84	2.700248	2.132013	61.90476
pols2_ug_rls	84	2.939929	2.415749	57.14286
sb	84	37.492611	23.036251	28.57143
sb_ug	84	35.059006	20.704037	29.76190
sb_rls	84	7.403199	4.163330	54.76190
sb_ug_rls	84	4.731151	2.570904	67.85714
ss	84	4.300336	3.979575	17.85714
ss_ug	84	4.197583	3.988989	15.47619
ss_rls	84	4.372230	4.207541	11.90476
ss_ug_rls	84	4.404471	4.255803	11.90476
all	84	4.308030	4.064183	15.47619
all_ug	84	4.268579	4.079556	15.47619
pall	84	15.348340	5.735497	67.85714
pall_ug	84	12.630344	4.774179	66.66667
all_rls	84	4.423409	4.276658	10.71429
all_ug_rls	84	4.456948	4.325516	10.71429
pall_rls	84	3.395651	2.968087	38.09524
pall_ug_rls	84	3.556569	3.169241	34.52381
pall	84	15.348340	5.735497	67.85714
pall_ug	84	12.630344	4.774179	66.66667
pall_rls	84	3.395651	2.968087	38.09524
pall_ug_rls	84	3.556569	3.169241	34.52381
sf	84	4.313563	4.071333	15.47619
sf_ug	84	4.270302	4.081939	15.47619
sf_rls	84	4.426184	4.280828	10.71429
sf_ug_rls	84	4.457125	4.326112	10.71429