FMG Study 2 Measurement invariance testing; 2000 and 2001

Design

A total of 189 conditions, with 2000 replications in each.

• n=400, 800, 2000 (group sample sizes)
• ngroups=2,4,8
• One-factor model with dim=5,10, 20
• distributions: normal, vm1, pl1, ig1, vm2, pl2, ig2
• method=2000 or 2001
• 2000 replications

In only 3 of the 378000 calls to pvalues(method=“2001”) was error produced, all for the case small model size, ngroups=8. Error was: lavaan->lav_test_satorra_bentler(): could not invert information matrix needed for UfromUGamma

With method=2000 no errors were produced.

CONCLUSION: 2000 IS UNIVERSALLY BEST, EXCEPT FOR LOW DIMENSIONS. ??

Overall rejection rates

2001 method has lower rejection rates for all tests

test	2001	2000
ss_ug_rls	0.02	0.04
ss_ug	0.02	0.04
ss_rls	0.02	0.04
ss	0.02	0.04
peba6_ug_rls	0.03	0.06
peba4_ug_rls	0.03	0.06
pols2_ug_rls	0.03	0.06
peba6_ug	0.03	0.06
peba6_rls	0.03	0.06
peba4_ug	0.03	0.06
pols2_ug	0.03	0.06
peba4_rls	0.03	0.07
pols2_rls	0.03	0.07
peba6	0.03	0.06
peba2_ug_rls	0.03	0.07
peba4	0.03	0.06
pols2	0.03	0.06
peba2_ug	0.03	0.06
peba2_rls	0.03	0.07
peba2	0.03	0.07
sb_ug_rls	0.03	0.07
sb_ug	0.03	0.07
sb_rls	0.03	0.07
sb	0.03	0.07

Normal data

dim=5

test	method	numcond	RMSE	MAD	ARR
peba2	2000	9	0.803	0.722	100
peba2	2001	9	0.575	0.456	100
peba2_rls	2000	9	0.691	0.600	100
peba2_rls	2001	9	0.518	0.428	100
peba2_ug	2000	9	0.671	0.578	100
peba2_ug	2001	9	0.549	0.456	100
peba2_ug_rls	2000	9	0.579	0.450	100
peba2_ug_rls	2001	9	0.570	0.456	100
peba4	2000	9	0.803	0.722	100
peba4	2001	9	0.572	0.450	100
peba4_rls	2000	9	0.685	0.589	100
peba4_rls	2001	9	0.521	0.428	100
peba4_ug	2000	9	0.671	0.578	100
peba4_ug	2001	9	0.546	0.450	100
peba4_ug_rls	2000	9	0.573	0.444	100
peba4_ug_rls	2001	9	0.572	0.461	100
peba6	2000	9	0.803	0.722	100
peba6	2001	9	0.572	0.450	100
peba6_rls	2000	9	0.685	0.589	100
peba6_rls	2001	9	0.527	0.433	100
peba6_ug	2000	9	0.671	0.578	100
peba6_ug	2001	9	0.546	0.450	100
peba6_ug_rls	2000	9	0.573	0.444	100
peba6_ug_rls	2001	9	0.572	0.461	100
pols2	2000	9	0.803	0.722	100
pols2	2001	9	0.572	0.450	100
pols2_rls	2000	9	0.685	0.589	100
pols2_rls	2001	9	0.527	0.433	100
pols2_ug	2000	9	0.671	0.578	100
pols2_ug	2001	9	0.546	0.450	100
pols2_ug_rls	2000	9	0.573	0.444	100
pols2_ug_rls	2001	9	0.572	0.461	100
sb	2000	9	0.820	0.739	100
sb	2001	9	0.571	0.450	100
sb_rls	2000	9	0.700	0.611	100
sb_rls	2001	9	0.525	0.433	100
sb_ug	2000	9	0.674	0.583	100
sb_ug	2001	9	0.552	0.456	100
sb_ug_rls	2000	9	0.579	0.461	100
sb_ug_rls	2001	9	0.554	0.439	100
ss	2000	9	0.753	0.672	100
ss	2001	9	0.549	0.439	100
ss_rls	2000	9	0.624	0.528	100
ss_rls	2001	9	0.558	0.444	100
ss_ug	2000	9	0.644	0.544	100
ss_ug	2001	9	0.565	0.467	100
ss_ug_rls	2000	9	0.555	0.444	100
ss_ug_rls	2001	9	0.594	0.478	100

dim=10

test	method	numcond	RMSE	MAD	ARR
peba2	2000	9	0.638	0.494	100
peba2	2001	9	0.756	0.656	100
peba2_rls	2000	9	0.290	0.250	100
peba2_rls	2001	9	1.062	0.911	100
peba2_ug	2000	9	0.412	0.328	100
peba2_ug	2001	9	0.926	0.806	100
peba2_ug_rls	2000	9	0.225	0.194	100
peba2_ug_rls	2001	9	1.212	1.044	100
peba4	2000	9	0.621	0.483	100
peba4	2001	9	0.776	0.667	100
peba4_rls	2000	9	0.289	0.250	100
peba4_rls	2001	9	1.070	0.917	100
peba4_ug	2000	9	0.412	0.328	100
peba4_ug	2001	9	0.946	0.817	100
peba4_ug_rls	2000	9	0.223	0.183	100
peba4_ug_rls	2001	9	1.218	1.050	100
peba6	2000	9	0.603	0.472	100
peba6	2001	9	0.776	0.667	100
peba6_rls	2000	9	0.289	0.250	100
peba6_rls	2001	9	1.070	0.917	100
peba6_ug	2000	9	0.412	0.328	100
peba6_ug	2001	9	0.952	0.822	100
peba6_ug_rls	2000	9	0.224	0.189	100
peba6_ug_rls	2001	9	1.218	1.050	100
pols2	2000	9	0.603	0.472	100
pols2	2001	9	0.776	0.667	100
pols2_rls	2000	9	0.289	0.250	100
pols2_rls	2001	9	1.070	0.917	100
pols2_ug	2000	9	0.412	0.328	100
pols2_ug	2001	9	0.952	0.822	100
pols2_ug_rls	2000	9	0.224	0.189	100
pols2_ug_rls	2001	9	1.218	1.050	100
sb	2000	9	0.675	0.528	100
sb	2001	9	0.707	0.617	100
sb_rls	2000	9	0.364	0.306	100
sb_rls	2001	9	1.008	0.867	100
sb_ug	2000	9	0.440	0.350	100
sb_ug	2001	9	0.890	0.772	100
sb_ug_rls	2000	9	0.232	0.206	100
sb_ug_rls	2001	9	1.186	1.022	100
ss	2000	9	0.440	0.350	100
ss	2001	9	0.886	0.767	100
ss_rls	2000	9	0.233	0.211	100
ss_rls	2001	9	1.181	1.017	100
ss_ug	2000	9	0.277	0.206	100
ss_ug	2001	9	1.074	0.933	100
ss_ug_rls	2000	9	0.270	0.256	100
ss_ug_rls	2001	9	1.315	1.133	100

dim=20

test	method	numcond	RMSE	MAD	ARR
peba2	2000	9	0.866	0.689	100.000
peba2	2001	9	2.290	1.939	55.556
peba2_rls	2000	9	0.342	0.278	100.000
peba2_rls	2001	9	2.589	2.189	55.556
peba2_ug	2000	9	0.534	0.450	100.000
peba2_ug	2001	9	2.426	2.089	55.556
peba2_ug_rls	2000	9	0.488	0.411	100.000
peba2_ug_rls	2001	9	2.684	2.300	55.556
peba4	2000	9	0.846	0.667	100.000
peba4	2001	9	2.312	1.961	55.556
peba4_rls	2000	9	0.343	0.278	100.000
peba4_rls	2001	9	2.603	2.206	55.556
peba4_ug	2000	9	0.516	0.433	100.000
peba4_ug	2001	9	2.445	2.111	55.556
peba4_ug_rls	2000	9	0.498	0.417	100.000
peba4_ug_rls	2001	9	2.694	2.311	55.556
peba6	2000	9	0.847	0.672	100.000
peba6	2001	9	2.312	1.961	55.556
peba6_rls	2000	9	0.343	0.278	100.000
peba6_rls	2001	9	2.603	2.206	55.556
peba6_ug	2000	9	0.516	0.433	100.000
peba6_ug	2001	9	2.445	2.111	55.556
peba6_ug_rls	2000	9	0.498	0.417	100.000
peba6_ug_rls	2001	9	2.694	2.311	55.556
pols2	2000	9	0.835	0.667	100.000
pols2	2001	9	2.312	1.961	55.556
pols2_rls	2000	9	0.343	0.278	100.000
pols2_rls	2001	9	2.603	2.206	55.556
pols2_ug	2000	9	0.516	0.433	100.000
pols2_ug	2001	9	2.445	2.111	55.556
pols2_ug_rls	2000	9	0.498	0.417	100.000
pols2_ug_rls	2001	9	2.694	2.311	55.556
sb	2000	9	1.021	0.794	100.000
sb	2001	9	2.220	1.856	66.667
sb_rls	2000	9	0.357	0.300	100.000
sb_rls	2001	9	2.550	2.144	55.556
sb_ug	2000	9	0.619	0.511	100.000
sb_ug	2001	9	2.381	2.033	55.556
sb_ug_rls	2000	9	0.438	0.356	100.000
sb_ug_rls	2001	9	2.645	2.256	55.556
ss	2000	9	0.517	0.433	100.000
ss	2001	9	2.463	2.128	55.556
ss_rls	2000	9	0.543	0.467	100.000
ss_rls	2001	9	2.714	2.339	55.556
ss_ug	2000	9	0.411	0.294	100.000
ss_ug	2001	9	2.587	2.267	55.556
ss_ug_rls	2000	9	0.804	0.683	100.000
ss_ug_rls	2001	9	2.851	2.467	55.556

Overall normal data

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	0.775	0.635	100.000
peba2	2001	27	1.432	1.017	85.185
peba2_rls	2000	27	0.476	0.376	100.000
peba2_rls	2001	27	1.643	1.176	85.185
peba2_ug	2000	27	0.549	0.452	100.000
peba2_ug	2001	27	1.533	1.117	85.185
peba2_ug_rls	2000	27	0.456	0.352	100.000
peba2_ug_rls	2001	27	1.732	1.267	85.185
peba4	2000	27	0.763	0.624	100.000
peba4	2001	27	1.446	1.026	85.185
peba4_rls	2000	27	0.473	0.372	100.000
peba4_rls	2001	27	1.653	1.183	85.185
peba4_ug	2000	27	0.544	0.446	100.000
peba4_ug	2001	27	1.546	1.126	85.185
peba4_ug_rls	2000	27	0.457	0.348	100.000
peba4_ug_rls	2001	27	1.738	1.274	85.185
peba6	2000	27	0.759	0.622	100.000
peba6	2001	27	1.446	1.026	85.185
peba6_rls	2000	27	0.473	0.372	100.000
peba6_rls	2001	27	1.653	1.185	85.185
peba6_ug	2000	27	0.544	0.446	100.000
peba6_ug	2001	27	1.547	1.128	85.185
peba6_ug_rls	2000	27	0.457	0.350	100.000
peba6_ug_rls	2001	27	1.738	1.274	85.185
pols2	2000	27	0.754	0.620	100.000
pols2	2001	27	1.446	1.026	85.185
pols2_rls	2000	27	0.473	0.372	100.000
pols2_rls	2001	27	1.653	1.185	85.185
pols2_ug	2000	27	0.544	0.446	100.000
pols2_ug	2001	27	1.547	1.128	85.185
pols2_ug_rls	2000	27	0.457	0.350	100.000
pols2_ug_rls	2001	27	1.738	1.274	85.185
sb	2000	27	0.850	0.687	100.000
sb	2001	27	1.385	0.974	88.889
sb_rls	2000	27	0.500	0.406	100.000
sb_rls	2001	27	1.612	1.148	85.185
sb_ug	2000	27	0.586	0.481	100.000
sb_ug	2001	27	1.502	1.087	85.185
sb_ug_rls	2000	27	0.440	0.341	100.000
sb_ug_rls	2001	27	1.704	1.239	85.185
ss	2000	27	0.585	0.485	100.000
ss	2001	27	1.544	1.111	85.185
ss_rls	2000	27	0.496	0.402	100.000
ss_rls	2001	27	1.739	1.267	85.185
ss_ug	2000	27	0.469	0.348	100.000
ss_ug	2001	27	1.650	1.222	85.185
ss_ug_rls	2000	27	0.585	0.461	100.000
ss_ug_rls	2001	27	1.845	1.359	85.185

Non-normal data

dim=5

test	method	numcond	RMSE	MAD	ARR
peba2	2000	54	1.299	1.012	94.444
peba2	2001	54	1.447	1.130	88.889
peba2_rls	2000	54	1.204	0.914	94.444
peba2_rls	2001	54	1.428	1.154	90.741
peba2_ug	2000	54	1.077	0.849	98.148
peba2_ug	2001	54	1.537	1.226	87.037
peba2_ug_rls	2000	54	1.035	0.781	98.148
peba2_ug_rls	2001	54	1.529	1.246	87.037
peba4	2000	54	1.067	0.856	100.000
peba4	2001	54	1.525	1.203	87.037
peba4_rls	2000	54	0.953	0.738	98.148
peba4_rls	2001	54	1.525	1.253	87.037
peba4_ug	2000	54	0.889	0.740	100.000
peba4_ug	2001	54	1.620	1.301	87.037
peba4_ug_rls	2000	54	0.832	0.656	100.000
peba4_ug_rls	2001	54	1.638	1.343	87.037
peba6	2000	54	1.009	0.822	100.000
peba6	2001	54	1.541	1.220	87.037
peba6_rls	2000	54	0.902	0.709	100.000
peba6_rls	2001	54	1.558	1.272	87.037
peba6_ug	2000	54	0.854	0.720	100.000
peba6_ug	2001	54	1.646	1.328	87.037
peba6_ug_rls	2000	54	0.787	0.634	100.000
peba6_ug_rls	2001	54	1.671	1.372	85.185
pols2	2000	54	1.096	0.879	98.148
pols2	2001	54	1.521	1.201	87.037
pols2_rls	2000	54	0.991	0.768	98.148
pols2_rls	2001	54	1.514	1.239	87.037
pols2_ug	2000	54	0.919	0.762	100.000
pols2_ug	2001	54	1.609	1.294	87.037
pols2_ug_rls	2000	54	0.853	0.670	100.000
pols2_ug_rls	2001	54	1.623	1.333	87.037
sb	2000	54	1.730	1.344	81.481
sb	2001	54	1.336	1.067	92.593
sb_rls	2000	54	1.641	1.259	87.037
sb_rls	2001	54	1.333	1.100	92.593
sb_ug	2000	54	1.527	1.160	88.889
sb_ug	2001	54	1.423	1.149	90.741
sb_ug_rls	2000	54	1.434	1.069	90.741
sb_ug_rls	2001	54	1.397	1.159	90.741
ss	2000	54	1.055	0.766	96.296
ss	2001	54	2.283	1.945	70.370
ss_rls	2000	54	1.022	0.798	98.148
ss_rls	2001	54	2.313	1.992	72.222
ss_ug	2000	54	1.136	0.832	94.444
ss_ug	2001	54	2.365	2.035	68.519
ss_ug_rls	2000	54	1.099	0.867	96.296
ss_ug_rls	2001	54	2.386	2.067	64.815

dim=10

test	method	numcond	RMSE	MAD	ARR
peba2	2000	54	2.187	1.651	77.778
peba2	2001	54	2.932	2.519	51.852
peba2_rls	2000	54	2.856	1.938	74.074
peba2_rls	2001	54	2.889	2.541	53.704
peba2_ug	2000	54	1.877	1.397	85.185
peba2_ug	2001	54	2.999	2.599	46.296
peba2_ug_rls	2000	54	2.507	1.690	75.926
peba2_ug_rls	2001	54	2.969	2.626	44.444
peba4	2000	54	1.717	1.297	90.741
peba4	2001	54	3.021	2.611	46.296
peba4_rls	2000	54	2.300	1.544	81.481
peba4_rls	2001	54	2.990	2.636	46.296
peba4_ug	2000	54	1.455	1.061	90.741
peba4_ug	2001	54	3.088	2.688	44.444
peba4_ug_rls	2000	54	2.027	1.308	87.037
peba4_ug_rls	2001	54	3.069	2.714	42.593
peba6	2000	54	1.542	1.163	90.741
peba6	2001	54	3.051	2.651	46.296
peba6_rls	2000	54	2.121	1.410	85.185
peba6_rls	2001	54	3.036	2.686	44.444
peba6_ug	2000	54	1.289	0.944	92.593
peba6_ug	2001	54	3.122	2.737	40.741
peba6_ug_rls	2000	54	1.831	1.169	90.741
peba6_ug_rls	2001	54	3.111	2.765	40.741
pols2	2000	54	1.762	1.323	88.889
pols2	2001	54	3.018	2.608	46.296
pols2_rls	2000	54	2.345	1.572	81.481
pols2_rls	2001	54	2.986	2.636	44.444
pols2_ug	2000	54	1.497	1.081	90.741
pols2_ug	2001	54	3.087	2.690	44.444
pols2_ug_rls	2000	54	2.074	1.333	87.037
pols2_ug_rls	2001	54	3.066	2.716	42.593
sb	2000	54	2.913	2.239	64.815
sb	2001	54	2.791	2.385	59.259
sb_rls	2000	54	3.537	2.500	66.667
sb_rls	2001	54	2.755	2.396	59.259
sb_ug	2000	54	2.561	1.942	70.370
sb_ug	2001	54	2.870	2.472	55.556
sb_ug_rls	2000	54	3.210	2.232	70.370
sb_ug_rls	2001	54	2.830	2.479	53.704
ss	2000	54	1.462	1.169	88.889
ss	2001	54	3.692	3.447	22.222
ss_rls	2000	54	1.260	1.052	98.148
ss_rls	2001	54	3.689	3.455	22.222
ss_ug	2000	54	1.617	1.320	87.037
ss_ug	2001	54	3.720	3.483	22.222
ss_ug_rls	2000	54	1.389	1.199	96.296
ss_ug_rls	2001	54	3.741	3.511	20.370

dim=20

test	method	numcond	RMSE	MAD	ARR
peba2	2000	54	3.950	2.794	59.259
peba2	2001	54	3.967	3.643	22.222
peba2_rls	2000	54	7.161	4.032	59.259
peba2_rls	2001	54	3.966	3.672	22.222
peba2_ug	2000	54	3.454	2.419	66.667
peba2_ug	2001	54	4.009	3.701	20.370
peba2_ug_rls	2000	54	6.655	3.666	61.111
peba2_ug_rls	2001	54	3.997	3.718	22.222
peba4	2000	54	3.067	2.152	68.519
peba4	2001	54	4.030	3.729	18.519
peba4_rls	2000	54	6.251	3.329	62.963
peba4_rls	2001	54	4.028	3.751	18.519
peba4_ug	2000	54	2.629	1.797	75.926
peba4_ug	2001	54	4.063	3.777	18.519
peba4_ug_rls	2000	54	5.744	2.979	64.815
peba4_ug_rls	2001	54	4.059	3.787	18.519
peba6	2000	54	2.707	1.889	75.926
peba6	2001	54	4.055	3.781	18.519
peba6_rls	2000	54	5.842	3.059	62.963
peba6_rls	2001	54	4.053	3.785	18.519
peba6_ug	2000	54	2.303	1.574	77.778
peba6_ug	2001	54	4.084	3.817	18.519
peba6_ug_rls	2000	54	5.386	2.736	68.519
peba6_ug_rls	2001	54	4.082	3.820	16.667
pols2	2000	54	3.187	2.225	66.667
pols2	2001	54	4.024	3.722	20.370
pols2_rls	2000	54	6.398	3.419	62.963
pols2_rls	2001	54	4.024	3.748	18.519
pols2_ug	2000	54	2.728	1.874	74.074
pols2_ug	2001	54	4.058	3.765	18.519
pols2_ug_rls	2000	54	5.873	3.056	64.815
pols2_ug_rls	2001	54	4.050	3.778	18.519
sb	2000	54	5.002	3.656	50.000
sb	2001	54	3.891	3.555	24.074
sb_rls	2000	54	8.139	4.842	51.852
sb_rls	2001	54	3.898	3.584	22.222
sb_ug	2000	54	4.459	3.231	55.556
sb_ug	2001	54	3.929	3.601	22.222
sb_ug_rls	2000	54	7.593	4.433	55.556
sb_ug_rls	2001	54	3.930	3.621	22.222
ss	2000	54	2.031	1.743	74.074
ss	2001	54	4.447	4.335	9.259
ss_rls	2000	54	2.005	1.581	90.741
ss_rls	2001	54	4.457	4.358	7.407
ss_ug	2000	54	2.211	1.944	68.519
ss_ug	2001	54	4.467	4.360	7.407
ss_ug_rls	2000	54	2.047	1.697	87.037
ss_ug_rls	2001	54	4.476	4.380	7.407

Overall non-normal data

test	method	numcond	RMSE	MAD	ARR
peba2	2000	162	2.712	1.819	77.160
peba2	2001	162	2.968	2.431	54.321
peba2_rls	2000	162	4.505	2.295	75.926
peba2_rls	2001	162	2.951	2.456	55.556
peba2_ug	2000	162	2.353	1.555	83.333
peba2_ug	2001	162	3.024	2.509	51.235
peba2_ug_rls	2000	162	4.149	2.045	78.395
peba2_ug_rls	2001	162	3.007	2.530	51.235
peba4	2000	162	2.121	1.435	86.420
peba4	2001	162	3.038	2.514	50.617
peba4_rls	2000	162	3.885	1.870	80.864
peba4_rls	2001	162	3.027	2.547	50.617
peba4_ug	2000	162	1.809	1.199	88.889
peba4_ug	2001	162	3.091	2.589	50.000
peba4_ug_rls	2000	162	3.550	1.648	83.951
peba4_ug_rls	2001	162	3.086	2.615	49.383
peba6	2000	162	1.891	1.291	88.889
peba6	2001	162	3.062	2.551	50.617
peba6_rls	2000	162	3.626	1.726	82.716
peba6_rls	2001	162	3.059	2.581	50.000
peba6_ug	2000	162	1.601	1.080	90.123
peba6_ug	2001	162	3.116	2.627	48.765
peba6_ug_rls	2000	162	3.316	1.513	86.420
peba6_ug_rls	2001	162	3.116	2.652	47.531
pols2	2000	162	2.196	1.476	84.568
pols2	2001	162	3.034	2.510	51.235
pols2_rls	2000	162	3.975	1.920	80.864
pols2_rls	2001	162	3.022	2.541	50.000
pols2_ug	2000	162	1.873	1.239	88.272
pols2_ug	2001	162	3.087	2.583	50.000
pols2_ug_rls	2000	162	3.629	1.687	83.951
pols2_ug_rls	2001	162	3.079	2.609	49.383
sb	2000	162	3.488	2.413	65.432
sb	2001	162	2.870	2.335	58.642
sb_rls	2000	162	5.211	2.867	68.519
sb_rls	2001	162	2.861	2.360	58.025
sb_ug	2000	162	3.097	2.111	71.605
sb_ug	2001	162	2.927	2.407	56.173
sb_ug_rls	2000	162	4.831	2.578	72.222
sb_ug_rls	2001	162	2.910	2.420	55.556
ss	2000	162	1.568	1.226	86.420
ss	2001	162	3.588	3.243	33.951
ss_rls	2000	162	1.489	1.144	95.679
ss_rls	2001	162	3.598	3.268	33.951
ss_ug	2000	162	1.712	1.366	83.333
ss_ug	2001	162	3.623	3.293	32.716
ss_ug_rls	2000	162	1.563	1.254	93.210
ss_ug_rls	2001	162	3.639	3.319	30.864

By group sample size

n=400

test	method	numcond	RMSE	MAD	ARR
peba2	2000	63	3.685	2.633	61.905
peba2	2001	63	3.396	2.955	42.857
peba2_rls	2000	63	6.185	3.078	68.254
peba2_rls	2001	63	3.452	3.059	41.270
peba2_ug	2000	63	3.088	2.086	73.016
peba2_ug	2001	63	3.467	3.071	38.095
peba2_ug_rls	2000	63	5.635	2.621	71.429
peba2_ug_rls	2001	63	3.527	3.180	33.333
peba4	2000	63	2.893	2.079	76.190
peba4	2001	63	3.471	3.062	38.095
peba4_rls	2000	63	5.405	2.522	73.016
peba4_rls	2001	63	3.525	3.166	33.333
peba4_ug	2000	63	2.385	1.589	82.540
peba4_ug	2001	63	3.539	3.168	38.095
peba4_ug_rls	2000	63	4.904	2.127	77.778
peba4_ug_rls	2001	63	3.611	3.290	31.746
peba6	2000	63	2.573	1.852	80.952
peba6	2001	63	3.488	3.086	38.095
peba6_rls	2000	63	5.068	2.323	76.190
peba6_rls	2001	63	3.554	3.199	31.746
peba6_ug	2000	63	2.091	1.404	84.127
peba6_ug	2001	63	3.567	3.204	36.508
peba6_ug_rls	2000	63	4.600	1.940	80.952
peba6_ug_rls	2001	63	3.636	3.321	31.746
pols2	2000	63	2.995	2.131	74.603
pols2	2001	63	3.462	3.050	38.095
pols2_rls	2000	63	5.517	2.579	73.016
pols2_rls	2001	63	3.520	3.159	31.746
pols2_ug	2000	63	2.466	1.647	80.952
pols2_ug	2001	63	3.537	3.163	38.095
pols2_ug_rls	2000	63	5.019	2.183	77.778
pols2_ug_rls	2001	63	3.599	3.274	31.746
sb	2000	63	4.690	3.438	47.619
sb	2001	63	3.271	2.793	47.619
sb_rls	2000	63	7.025	3.792	63.492
sb_rls	2001	63	3.346	2.910	44.444
sb_ug	2000	63	4.055	2.846	60.317
sb_ug	2001	63	3.372	2.932	42.857
sb_ug_rls	2000	63	6.476	3.290	68.254
sb_ug_rls	2001	63	3.425	3.027	39.683
ss	2000	63	1.528	1.229	88.889
ss	2001	63	3.929	3.643	23.810
ss_rls	2000	63	1.712	1.220	95.238
ss_rls	2001	63	3.978	3.719	25.397
ss_ug	2000	63	1.744	1.406	82.540
ss_ug	2001	63	3.974	3.717	23.810
ss_ug_rls	2000	63	1.790	1.400	90.476
ss_ug_rls	2001	63	4.039	3.807	20.635

n=800

test	method	numcond	RMSE	MAD	ARR
peba2	2000	63	1.794	1.350	87.302
peba2	2001	63	2.808	2.248	57.143
peba2_rls	2000	63	3.224	1.817	80.952
peba2_rls	2001	63	2.807	2.316	58.730
peba2_ug	2000	63	1.606	1.162	90.476
peba2_ug	2001	63	2.877	2.355	55.556
peba2_ug_rls	2000	63	3.051	1.666	82.540
peba2_ug_rls	2001	63	2.876	2.408	55.556
peba4	2000	63	1.383	1.044	95.238
peba4	2001	63	2.900	2.379	53.968
peba4_rls	2000	63	2.727	1.483	85.714
peba4_rls	2001	63	2.902	2.441	53.968
peba4_ug	2000	63	1.200	0.879	95.238
peba4_ug	2001	63	2.956	2.462	52.381
peba4_ug_rls	2000	63	2.517	1.317	88.889
peba4_ug_rls	2001	63	2.964	2.512	52.381
peba6	2000	63	1.208	0.929	95.238
peba6	2001	63	2.929	2.413	53.968
peba6_rls	2000	63	2.500	1.352	87.302
peba6_rls	2001	63	2.942	2.487	53.968
peba6_ug	2000	63	1.049	0.787	95.238
peba6_ug	2001	63	2.980	2.496	50.794
peba6_ug_rls	2000	63	2.320	1.206	92.063
peba6_ug_rls	2001	63	3.004	2.559	47.619
pols2	2000	63	1.430	1.072	92.063
pols2	2001	63	2.895	2.373	55.556
pols2_rls	2000	63	2.801	1.519	85.714
pols2_rls	2001	63	2.894	2.430	53.968
pols2_ug	2000	63	1.240	0.901	95.238
pols2_ug	2001	63	2.947	2.455	52.381
pols2_ug_rls	2000	63	2.561	1.342	88.889
pols2_ug_rls	2001	63	2.956	2.504	52.381
sb	2000	63	2.398	1.829	77.778
sb	2001	63	2.698	2.102	60.317
sb_rls	2000	63	3.876	2.322	73.016
sb_rls	2001	63	2.697	2.169	60.317
sb_ug	2000	63	2.195	1.625	79.365
sb_ug	2001	63	2.745	2.181	58.730
sb_ug_rls	2000	63	3.603	2.108	74.603
sb_ug_rls	2001	63	2.750	2.240	60.317
ss	2000	63	1.640	1.262	82.540
ss	2001	63	3.450	3.052	36.508
ss_rls	2000	63	1.369	1.125	95.238
ss_rls	2001	63	3.488	3.115	36.508
ss_ug	2000	63	1.747	1.348	80.952
ss_ug	2001	63	3.487	3.108	36.508
ss_ug_rls	2000	63	1.467	1.217	93.651
ss_ug_rls	2001	63	3.527	3.160	33.333

n=2000

test	method	numcond	RMSE	MAD	ARR
peba2	2000	63	1.541	0.967	92.063
peba2	2001	63	2.028	1.482	76.190
peba2_rls	2000	63	1.905	1.167	88.889
peba2_rls	2001	63	1.937	1.444	79.365
peba2_ug	2000	63	1.502	0.944	93.651
peba2_ug	2001	63	2.053	1.503	74.603
peba2_ug_rls	2000	63	1.816	1.123	90.476
peba2_ug_rls	2001	63	1.958	1.460	79.365
peba4	2000	63	1.237	0.835	93.651
peba4	2001	63	2.043	1.463	74.603
peba4_rls	2000	63	1.502	0.964	92.063
peba4_rls	2001	63	1.972	1.448	79.365
peba4_ug	2000	63	1.189	0.808	93.651
peba4_ug	2001	63	2.080	1.509	74.603
peba4_ug_rls	2000	63	1.452	0.941	92.063
peba4_ug_rls	2001	63	1.991	1.467	79.365
peba6	2000	63	1.165	0.806	95.238
peba6	2001	63	2.065	1.500	74.603
peba6_rls	2000	63	1.404	0.923	92.063
peba6_rls	2001	63	1.986	1.459	79.365
peba6_ug	2000	63	1.118	0.776	95.238
peba6_ug	2001	63	2.095	1.539	74.603
peba6_ug_rls	2000	63	1.348	0.895	92.063
peba6_ug_rls	2001	63	2.006	1.486	79.365
pols2	2000	63	1.275	0.857	93.651
pols2	2001	63	2.049	1.472	74.603
pols2_rls	2000	63	1.567	0.998	92.063
pols2_rls	2001	63	1.973	1.452	79.365
pols2_ug	2000	63	1.238	0.830	93.651
pols2_ug	2001	63	2.081	1.508	74.603
pols2_ug_rls	2000	63	1.488	0.962	92.063
pols2_ug_rls	2001	63	1.995	1.477	79.365
sb	2000	63	1.962	1.233	85.714
sb	2001	63	2.007	1.528	80.952
sb_rls	2000	63	2.357	1.432	82.540
sb_rls	2001	63	1.920	1.482	80.952
sb_ug	2000	63	1.881	1.163	87.302
sb_ug	2001	63	2.022	1.543	79.365
sb_ug_rls	2000	63	2.276	1.377	85.714
sb_ug_rls	2001	63	1.930	1.486	79.365
ss	2000	63	1.202	0.868	93.651
ss	2001	63	2.605	2.120	63.492
ss_rls	2000	63	1.001	0.768	98.413
ss_rls	2001	63	2.567	2.113	61.905
ss_ug	2000	63	1.241	0.907	93.651
ss_ug	2001	63	2.640	2.166	60.317
ss_ug_rls	2000	63	1.034	0.806	98.413
ss_ug_rls	2001	63	2.598	2.151	61.905

By distribution

normal

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	0.775	0.635	100.000
peba2	2001	27	1.432	1.017	85.185
peba2_rls	2000	27	0.476	0.376	100.000
peba2_rls	2001	27	1.643	1.176	85.185
peba2_ug	2000	27	0.549	0.452	100.000
peba2_ug	2001	27	1.533	1.117	85.185
peba2_ug_rls	2000	27	0.456	0.352	100.000
peba2_ug_rls	2001	27	1.732	1.267	85.185
peba4	2000	27	0.763	0.624	100.000
peba4	2001	27	1.446	1.026	85.185
peba4_rls	2000	27	0.473	0.372	100.000
peba4_rls	2001	27	1.653	1.183	85.185
peba4_ug	2000	27	0.544	0.446	100.000
peba4_ug	2001	27	1.546	1.126	85.185
peba4_ug_rls	2000	27	0.457	0.348	100.000
peba4_ug_rls	2001	27	1.738	1.274	85.185
peba6	2000	27	0.759	0.622	100.000
peba6	2001	27	1.446	1.026	85.185
peba6_rls	2000	27	0.473	0.372	100.000
peba6_rls	2001	27	1.653	1.185	85.185
peba6_ug	2000	27	0.544	0.446	100.000
peba6_ug	2001	27	1.547	1.128	85.185
peba6_ug_rls	2000	27	0.457	0.350	100.000
peba6_ug_rls	2001	27	1.738	1.274	85.185
pols2	2000	27	0.754	0.620	100.000
pols2	2001	27	1.446	1.026	85.185
pols2_rls	2000	27	0.473	0.372	100.000
pols2_rls	2001	27	1.653	1.185	85.185
pols2_ug	2000	27	0.544	0.446	100.000
pols2_ug	2001	27	1.547	1.128	85.185
pols2_ug_rls	2000	27	0.457	0.350	100.000
pols2_ug_rls	2001	27	1.738	1.274	85.185
sb	2000	27	0.850	0.687	100.000
sb	2001	27	1.385	0.974	88.889
sb_rls	2000	27	0.500	0.406	100.000
sb_rls	2001	27	1.612	1.148	85.185
sb_ug	2000	27	0.586	0.481	100.000
sb_ug	2001	27	1.502	1.087	85.185
sb_ug_rls	2000	27	0.440	0.341	100.000
sb_ug_rls	2001	27	1.704	1.239	85.185
ss	2000	27	0.585	0.485	100.000
ss	2001	27	1.544	1.111	85.185
ss_rls	2000	27	0.496	0.402	100.000
ss_rls	2001	27	1.739	1.267	85.185
ss_ug	2000	27	0.469	0.348	100.000
ss_ug	2001	27	1.650	1.222	85.185
ss_ug_rls	2000	27	0.585	0.461	100.000
ss_ug_rls	2001	27	1.845	1.359	85.185

vm1

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	0.946	0.681	96.296
peba2	2001	27	3.162	2.726	51.852
peba2_rls	2000	27	2.272	1.330	85.185
peba2_rls	2001	27	3.081	2.674	51.852
peba2_ug	2000	27	0.668	0.506	100.000
peba2_ug	2001	27	3.235	2.817	44.444
peba2_ug_rls	2000	27	1.860	1.078	85.185
peba2_ug_rls	2001	27	3.141	2.757	48.148
peba4	2000	27	0.689	0.509	100.000
peba4	2001	27	3.224	2.798	44.444
peba4_rls	2000	27	1.862	1.065	85.185
peba4_rls	2001	27	3.150	2.756	48.148
peba4_ug	2000	27	0.449	0.354	100.000
peba4_ug	2001	27	3.283	2.881	44.444
peba4_ug_rls	2000	27	1.495	0.848	92.593
peba4_ug_rls	2001	27	3.212	2.837	48.148
peba6	2000	27	0.590	0.437	100.000
peba6	2001	27	3.243	2.822	44.444
peba6_rls	2000	27	1.719	0.970	85.185
peba6_rls	2001	27	3.167	2.776	48.148
peba6_ug	2000	27	0.418	0.324	100.000
peba6_ug	2001	27	3.302	2.906	44.444
peba6_ug_rls	2000	27	1.373	0.774	96.296
peba6_ug_rls	2001	27	3.238	2.867	48.148
pols2	2000	27	0.696	0.507	100.000
pols2	2001	27	3.228	2.806	44.444
pols2_rls	2000	27	1.875	1.067	85.185
pols2_rls	2001	27	3.145	2.748	48.148
pols2_ug	2000	27	0.469	0.370	100.000
pols2_ug	2001	27	3.285	2.883	44.444
pols2_ug_rls	2000	27	1.508	0.850	92.593
pols2_ug_rls	2001	27	3.205	2.831	48.148
sb	2000	27	1.387	1.007	88.889
sb	2001	27	3.077	2.609	51.852
sb_rls	2000	27	2.826	1.691	81.481
sb_rls	2001	27	2.972	2.548	55.556
sb_ug	2000	27	1.072	0.770	96.296
sb_ug	2001	27	3.129	2.687	51.852
sb_ug_rls	2000	27	2.405	1.420	81.481
sb_ug_rls	2001	27	3.045	2.631	55.556
ss	2000	27	1.640	1.409	85.185
ss	2001	27	3.630	3.324	25.926
ss_rls	2000	27	1.154	1.061	100.000
ss_rls	2001	27	3.577	3.265	33.333
ss_ug	2000	27	1.812	1.567	81.481
ss_ug	2001	27	3.664	3.369	25.926
ss_ug_rls	2000	27	1.303	1.200	100.000
ss_ug_rls	2001	27	3.619	3.315	25.926

vm2

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	3.519	2.002	74.074
peba2	2001	27	3.996	3.783	18.519
peba2_rls	2000	27	9.360	4.939	59.259
peba2_rls	2001	27	3.832	3.602	25.926
peba2_ug	2000	27	2.975	1.689	81.481
peba2_ug	2001	27	4.021	3.818	18.519
peba2_ug_rls	2000	27	8.776	4.578	59.259
peba2_ug_rls	2001	27	3.874	3.654	22.222
peba4	2000	27	2.619	1.448	85.185
peba4	2001	27	4.036	3.837	18.519
peba4_rls	2000	27	8.324	4.220	62.963
peba4_rls	2001	27	3.903	3.691	22.222
peba4_ug	2000	27	2.232	1.241	88.889
peba4_ug	2001	27	4.059	3.867	18.519
peba4_ug_rls	2000	27	7.730	3.909	62.963
peba4_ug_rls	2001	27	3.948	3.744	22.222
peba6	2000	27	2.288	1.272	88.889
peba6	2001	27	4.044	3.848	18.519
peba6_rls	2000	27	7.835	3.954	62.963
peba6_rls	2001	27	3.936	3.728	22.222
peba6_ug	2000	27	1.944	1.091	88.889
peba6_ug	2001	27	4.073	3.885	18.519
peba6_ug_rls	2000	27	7.305	3.657	66.667
peba6_ug_rls	2001	27	3.970	3.772	18.519
pols2	2000	27	2.727	1.519	81.481
pols2	2001	27	4.036	3.837	18.519
pols2_rls	2000	27	8.496	4.322	62.963
pols2_rls	2001	27	3.899	3.683	22.222
pols2_ug	2000	27	2.313	1.285	85.185
pols2_ug	2001	27	4.058	3.865	18.519
pols2_ug_rls	2000	27	7.884	3.980	62.963
pols2_ug_rls	2001	27	3.943	3.737	22.222
sb	2000	27	4.480	2.759	70.370
sb	2001	27	3.889	3.635	25.926
sb_rls	2000	27	10.508	5.944	48.148
sb_rls	2001	27	3.733	3.455	25.926
sb_ug	2000	27	3.973	2.404	70.370
sb_ug	2001	27	3.913	3.672	25.926
sb_ug_rls	2000	27	9.862	5.493	55.556
sb_ug_rls	2001	27	3.763	3.498	25.926
ss	2000	27	2.483	2.278	55.556
ss	2001	27	4.348	4.231	11.111
ss_rls	2000	27	2.255	1.672	96.296
ss_rls	2001	27	4.324	4.218	7.407
ss_ug	2000	27	2.622	2.426	48.148
ss_ug	2001	27	4.365	4.256	7.407
ss_ug_rls	2000	27	2.161	1.720	92.593
ss_ug_rls	2001	27	4.348	4.246	7.407

pl1

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	1.076	0.850	96.296
peba2	2001	27	2.734	2.185	59.259
peba2_rls	2000	27	0.715	0.570	100.000
peba2_rls	2001	27	2.780	2.298	59.259
peba2_ug	2000	27	0.795	0.631	100.000
peba2_ug	2001	27	2.812	2.298	59.259
peba2_ug_rls	2000	27	0.602	0.485	100.000
peba2_ug_rls	2001	27	2.835	2.374	51.852
peba4	2000	27	0.963	0.767	100.000
peba4	2001	27	2.767	2.230	59.259
peba4_rls	2000	27	0.660	0.533	100.000
peba4_rls	2001	27	2.800	2.324	59.259
peba4_ug	2000	27	0.694	0.552	100.000
peba4_ug	2001	27	2.835	2.324	55.556
peba4_ug_rls	2000	27	0.583	0.454	100.000
peba4_ug_rls	2001	27	2.851	2.393	51.852
peba6	2000	27	0.926	0.741	100.000
peba6	2001	27	2.772	2.233	59.259
peba6_rls	2000	27	0.657	0.528	100.000
peba6_rls	2001	27	2.805	2.330	55.556
peba6_ug	2000	27	0.679	0.544	100.000
peba6_ug	2001	27	2.838	2.328	55.556
peba6_ug_rls	2000	27	0.593	0.463	100.000
peba6_ug_rls	2001	27	2.857	2.404	51.852
pols2	2000	27	0.965	0.769	100.000
pols2	2001	27	2.769	2.231	59.259
pols2_rls	2000	27	0.667	0.537	100.000
pols2_rls	2001	27	2.804	2.328	55.556
pols2_ug	2000	27	0.696	0.554	100.000
pols2_ug	2001	27	2.838	2.326	55.556
pols2_ug_rls	2000	27	0.594	0.465	100.000
pols2_ug_rls	2001	27	2.857	2.404	51.852
sb	2000	27	1.384	1.074	88.889
sb	2001	27	2.662	2.091	59.259
sb_rls	2000	27	0.914	0.698	100.000
sb_rls	2001	27	2.731	2.237	59.259
sb_ug	2000	27	1.045	0.824	96.296
sb_ug	2001	27	2.730	2.202	59.259
sb_ug_rls	2000	27	0.643	0.517	100.000
sb_ug_rls	2001	27	2.779	2.298	55.556
ss	2000	27	0.770	0.628	100.000
ss	2001	27	3.001	2.524	51.852
ss_rls	2000	27	1.055	0.863	100.000
ss_rls	2001	27	3.038	2.607	51.852
ss_ug	2000	27	0.922	0.731	100.000
ss_ug	2001	27	3.052	2.600	51.852
ss_ug_rls	2000	27	1.199	0.967	96.296
ss_ug_rls	2001	27	3.099	2.683	48.148

pl2

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	2.820	1.902	77.778
peba2	2001	27	3.636	3.222	29.630
peba2_rls	2000	27	2.981	1.878	81.481
peba2_rls	2001	27	3.619	3.248	33.333
peba2_ug	2000	27	2.393	1.583	85.185
peba2_ug	2001	27	3.666	3.272	29.630
peba2_ug_rls	2000	27	2.401	1.533	85.185
peba2_ug_rls	2001	27	3.652	3.294	33.333
peba4	2000	27	2.336	1.578	85.185
peba4	2001	27	3.666	3.270	29.630
peba4_rls	2000	27	2.340	1.509	85.185
peba4_rls	2001	27	3.644	3.283	33.333
peba4_ug	2000	27	1.869	1.269	88.889
peba4_ug	2001	27	3.714	3.341	29.630
peba4_ug_rls	2000	27	1.904	1.204	88.889
peba4_ug_rls	2001	27	3.700	3.350	33.333
peba6	2000	27	2.140	1.465	88.889
peba6	2001	27	3.674	3.280	29.630
peba6_rls	2000	27	2.162	1.402	88.889
peba6_rls	2001	27	3.665	3.302	33.333
peba6_ug	2000	27	1.671	1.146	88.889
peba6_ug	2001	27	3.715	3.343	29.630
peba6_ug_rls	2000	27	1.688	1.085	88.889
peba6_ug_rls	2001	27	3.704	3.356	33.333
pols2	2000	27	2.428	1.609	85.185
pols2	2001	27	3.672	3.276	29.630
pols2_rls	2000	27	2.403	1.533	85.185
pols2_rls	2001	27	3.646	3.285	33.333
pols2_ug	2000	27	1.931	1.296	88.889
pols2_ug	2001	27	3.713	3.337	29.630
pols2_ug_rls	2000	27	1.972	1.231	88.889
pols2_ug_rls	2001	27	3.700	3.352	33.333
sb	2000	27	3.787	2.554	66.667
sb	2001	27	3.546	3.102	40.741
sb_rls	2000	27	3.697	2.411	77.778
sb_rls	2001	27	3.548	3.143	40.741
sb_ug	2000	27	3.158	2.161	74.074
sb_ug	2001	27	3.597	3.183	33.333
sb_ug_rls	2000	27	3.274	2.098	81.481
sb_ug_rls	2001	27	3.574	3.187	33.333
ss	2000	27	1.129	0.822	96.296
ss	2001	27	3.938	3.637	25.926
ss_rls	2000	27	0.968	0.741	100.000
ss_rls	2001	27	3.951	3.670	22.222
ss_ug	2000	27	1.296	0.978	88.889
ss_ug	2001	27	3.969	3.687	25.926
ss_ug_rls	2000	27	1.098	0.878	100.000
ss_ug_rls	2001	27	3.977	3.707	22.222

ig1

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	1.911	1.756	85.185
peba2	2001	27	1.745	1.278	85.185
peba2_rls	2000	27	1.432	1.331	100.000
peba2_rls	2001	27	1.875	1.389	81.481
peba2_ug	2000	27	1.652	1.530	92.593
peba2_ug	2001	27	1.819	1.359	77.778
peba2_ug_rls	2000	27	1.258	1.154	100.000
peba2_ug_rls	2001	27	1.979	1.500	77.778
peba4	2000	27	1.622	1.491	92.593
peba4	2001	27	1.839	1.348	77.778
peba4_rls	2000	27	1.202	1.100	100.000
peba4_rls	2001	27	1.988	1.500	74.074
peba4_ug	2000	27	1.338	1.241	100.000
peba4_ug	2001	27	1.903	1.420	77.778
peba4_ug_rls	2000	27	1.050	0.926	100.000
peba4_ug_rls	2001	27	2.071	1.569	74.074
peba6	2000	27	1.504	1.385	96.296
peba6	2001	27	1.866	1.380	77.778
peba6_rls	2000	27	1.138	1.035	100.000
peba6_rls	2001	27	2.020	1.524	74.074
peba6_ug	2000	27	1.271	1.170	100.000
peba6_ug	2001	27	1.951	1.467	77.778
peba6_ug_rls	2000	27	1.002	0.872	100.000
peba6_ug_rls	2001	27	2.113	1.607	70.370
pols2	2000	27	1.640	1.507	92.593
pols2	2001	27	1.823	1.337	77.778
pols2_rls	2000	27	1.214	1.117	100.000
pols2_rls	2001	27	1.980	1.493	74.074
pols2_ug	2000	27	1.369	1.267	100.000
pols2_ug	2001	27	1.901	1.431	77.778
pols2_ug_rls	2000	27	1.063	0.935	100.000
pols2_ug_rls	2001	27	2.062	1.567	74.074
sb	2000	27	2.410	2.239	66.667
sb	2001	27	1.663	1.289	85.185
sb_rls	2000	27	1.887	1.763	88.889
sb_rls	2001	27	1.784	1.337	85.185
sb_ug	2000	27	2.143	1.996	77.778
sb_ug	2001	27	1.743	1.313	85.185
sb_ug_rls	2000	27	1.628	1.524	100.000
sb_ug_rls	2001	27	1.858	1.396	81.481
ss	2000	27	0.866	0.650	100.000
ss	2001	27	2.675	2.261	59.259
ss_rls	2000	27	1.140	0.880	96.296
ss_rls	2001	27	2.758	2.341	59.259
ss_ug	2000	27	1.043	0.789	100.000
ss_ug	2001	27	2.727	2.324	55.556
ss_ug_rls	2000	27	1.301	1.019	92.593
ss_ug_rls	2001	27	2.831	2.417	51.852

ig2

test	method	numcond	RMSE	MAD	ARR
peba2	2000	27	4.254	3.724	33.333
peba2	2001	27	1.777	1.389	81.481
peba2_rls	2000	27	4.190	3.720	29.630
peba2_rls	2001	27	1.929	1.522	81.481
peba2_ug	2000	27	3.852	3.391	40.741
peba2_ug	2001	27	1.888	1.487	77.778
peba2_ug_rls	2000	27	3.887	3.444	40.741
peba2_ug_rls	2001	27	2.025	1.600	74.074
peba4	2000	27	3.262	2.819	55.556
peba4	2001	27	2.052	1.602	74.074
peba4_rls	2000	27	3.231	2.794	51.852
peba4_rls	2001	27	2.179	1.726	66.667
peba4_ug	2000	27	2.947	2.541	55.556
peba4_ug	2001	27	2.150	1.698	74.074
peba4_ug_rls	2000	27	2.923	2.544	59.259
peba4_ug_rls	2001	27	2.269	1.796	66.667
peba6	2000	27	2.858	2.448	59.259
peba6	2001	27	2.173	1.743	74.074
peba6_rls	2000	27	2.853	2.469	59.259
peba6_rls	2001	27	2.288	1.828	66.667
peba6_ug	2000	27	2.562	2.202	62.963
peba6_ug	2001	27	2.260	1.835	66.667
peba6_ug_rls	2000	27	2.552	2.228	66.667
peba6_ug_rls	2001	27	2.382	1.909	62.963
pols2	2000	27	3.389	2.943	48.148
pols2	2001	27	2.011	1.576	77.778
pols2_rls	2000	27	3.382	2.943	51.852
pols2_rls	2001	27	2.152	1.709	66.667
pols2_ug	2000	27	3.066	2.663	55.556
pols2_ug	2001	27	2.108	1.655	74.074
pols2_ug_rls	2000	27	3.038	2.659	59.259
pols2_ug_rls	2001	27	2.225	1.763	66.667
sb	2000	27	5.380	4.844	11.111
sb	2001	27	1.555	1.287	88.889
sb_rls	2000	27	5.141	4.694	14.815
sb_rls	2001	27	1.765	1.441	81.481
sb_ug	2000	27	4.994	4.509	14.815
sb_ug	2001	27	1.693	1.387	81.481
sb_ug_rls	2000	27	4.819	4.417	14.815
sb_ug_rls	2001	27	1.851	1.507	81.481
ss	2000	27	1.812	1.567	81.481
ss	2001	27	3.672	3.478	29.630
ss_rls	2000	27	1.881	1.644	81.481
ss_rls	2001	27	3.704	3.507	29.630
ss_ug	2000	27	1.954	1.704	81.481
ss_ug	2001	27	3.712	3.522	29.630
ss_ug_rls	2000	27	1.987	1.743	77.778
ss_ug_rls	2001	27	3.742	3.546	29.630

By number of groups

2 groups

test	method	numcond	RMSE	MAD	ARR
peba2	2000	63	1.611	1.211	87.302
peba2	2001	63	2.339	1.762	71.429
peba2_rls	2000	63	1.525	1.038	88.889
peba2_rls	2001	63	2.380	1.852	73.016
peba2_ug	2000	63	1.438	1.045	90.476
peba2_ug	2001	63	2.389	1.822	69.841
peba2_ug_rls	2000	63	1.398	0.950	90.476
peba2_ug_rls	2001	63	2.427	1.910	68.254
peba4	2000	63	1.271	0.989	92.063
peba4	2001	63	2.404	1.821	69.841
peba4_rls	2000	63	1.190	0.840	93.651
peba4_rls	2001	63	2.436	1.902	69.841
peba4_ug	2000	63	1.127	0.851	95.238
peba4_ug	2001	63	2.444	1.870	69.841
peba4_ug_rls	2000	63	1.092	0.772	93.651
peba4_ug_rls	2001	63	2.488	1.960	68.254
peba6	2000	63	1.168	0.927	95.238
peba6	2001	63	2.413	1.833	69.841
peba6_rls	2000	63	1.103	0.790	93.651
peba6_rls	2001	63	2.459	1.923	68.254
peba6_ug	2000	63	1.041	0.796	95.238
peba6_ug	2001	63	2.458	1.887	69.841
peba6_ug_rls	2000	63	1.032	0.730	95.238
peba6_ug_rls	2001	63	2.509	1.981	66.667
pols2	2000	63	1.296	1.000	92.063
pols2	2001	63	2.406	1.825	69.841
pols2_rls	2000	63	1.216	0.850	93.651
pols2_rls	2001	63	2.435	1.898	68.254
pols2_ug	2000	63	1.156	0.868	93.651
pols2_ug	2001	63	2.450	1.879	69.841
pols2_ug_rls	2000	63	1.137	0.790	93.651
pols2_ug_rls	2001	63	2.486	1.963	68.254
sb	2000	63	2.074	1.560	82.540
sb	2001	63	2.262	1.679	74.603
sb_rls	2000	63	1.927	1.312	87.302
sb_rls	2001	63	2.305	1.779	73.016
sb_ug	2000	63	1.904	1.390	84.127
sb_ug	2001	63	2.306	1.746	73.016
sb_ug_rls	2000	63	1.800	1.207	87.302
sb_ug_rls	2001	63	2.344	1.826	71.429
ss	2000	63	0.853	0.680	100.000
ss	2001	63	2.847	2.344	57.143
ss_rls	2000	63	0.851	0.705	100.000
ss_rls	2001	63	2.908	2.433	53.968
ss_ug	2000	63	0.948	0.736	98.413
ss_ug	2001	63	2.882	2.394	57.143
ss_ug_rls	2000	63	0.931	0.769	100.000
ss_ug_rls	2001	63	2.938	2.473	53.968

4 groups

test	method	numcond	RMSE	MAD	ARR
peba2	2000	63	2.304	1.541	80.952
peba2	2001	63	2.872	2.305	55.556
peba2_rls	2000	63	3.284	1.847	79.365
peba2_rls	2001	63	2.874	2.341	58.730
peba2_ug	2000	63	2.010	1.341	85.714
peba2_ug	2001	63	2.934	2.395	53.968
peba2_ug_rls	2000	63	3.055	1.679	79.365
peba2_ug_rls	2001	63	2.932	2.429	55.556
peba4	2000	63	1.766	1.219	88.889
peba4	2001	63	2.936	2.405	52.381
peba4_rls	2000	63	2.772	1.500	82.540
peba4_rls	2001	63	2.952	2.464	52.381
peba4_ug	2000	63	1.510	1.038	90.476
peba4_ug	2001	63	2.992	2.481	50.794
peba4_ug_rls	2000	63	2.519	1.333	85.714
peba4_ug_rls	2001	63	3.008	2.533	50.794
peba6	2000	63	1.530	1.087	92.063
peba6	2001	63	2.962	2.445	52.381
peba6_rls	2000	63	2.522	1.367	85.714
peba6_rls	2001	63	2.983	2.503	52.381
peba6_ug	2000	63	1.306	0.937	93.651
peba6_ug	2001	63	3.024	2.531	47.619
peba6_ug_rls	2000	63	2.276	1.212	88.889
peba6_ug_rls	2001	63	3.037	2.572	49.206
pols2	2000	63	1.841	1.260	87.302
pols2	2001	63	2.932	2.396	53.968
pols2_rls	2000	63	2.860	1.557	82.540
pols2_rls	2001	63	2.948	2.457	52.381
pols2_ug	2000	63	1.576	1.076	90.476
pols2_ug	2001	63	2.986	2.473	50.794
pols2_ug_rls	2000	63	2.583	1.367	85.714
pols2_ug_rls	2001	63	2.998	2.520	50.794
sb	2000	63	3.017	2.024	65.079
sb	2001	63	2.774	2.232	61.905
sb_rls	2000	63	3.956	2.360	71.429
sb_rls	2001	63	2.790	2.253	61.905
sb_ug	2000	63	2.684	1.783	73.016
sb_ug	2001	63	2.840	2.303	58.730
sb_ug_rls	2000	63	3.665	2.129	73.016
sb_ug_rls	2001	63	2.836	2.309	60.317
ss	2000	63	1.633	1.267	84.127
ss	2001	63	3.470	3.067	38.095
ss_rls	2000	63	1.387	1.159	96.825
ss_rls	2001	63	3.495	3.112	39.683
ss_ug	2000	63	1.752	1.385	82.540
ss_ug	2001	63	3.506	3.126	36.508
ss_ug_rls	2000	63	1.491	1.273	95.238
ss_ug_rls	2001	63	3.552	3.180	34.921

8 groups

test	method	numcond	RMSE	MAD	ARR
peba2	2000	63	3.357	2.198	73.016
peba2	2001	63	3.133	2.619	49.206
peba2_rls	2000	63	6.259	3.177	69.841
peba2_rls	2001	63	3.102	2.625	47.619
peba2_ug	2000	63	2.874	1.806	80.952
peba2_ug	2001	63	3.194	2.712	44.444
peba2_ug_rls	2000	63	5.750	2.781	74.603
peba2_ug_rls	2001	63	3.170	2.709	44.444
peba4	2000	63	2.661	1.750	84.127
peba4	2001	63	3.198	2.679	44.444
peba4_rls	2000	63	5.459	2.629	74.603
peba4_rls	2001	63	3.176	2.690	44.444
peba4_ug	2000	63	2.234	1.387	85.714
peba4_ug	2001	63	3.266	2.788	44.444
peba4_ug_rls	2000	63	4.995	2.281	79.365
peba4_ug_rls	2001	63	3.248	2.776	44.444
peba6	2000	63	2.394	1.573	84.127
peba6	2001	63	3.226	2.721	44.444
peba6_rls	2000	63	5.131	2.441	76.190
peba6_rls	2001	63	3.208	2.719	44.444
peba6_ug	2000	63	1.983	1.234	85.714
peba6_ug	2001	63	3.287	2.821	44.444
peba6_ug_rls	2000	63	4.703	2.099	80.952
peba6_ug_rls	2001	63	3.279	2.813	42.857
pols2	2000	63	2.752	1.800	80.952
pols2	2001	63	3.191	2.675	44.444
pols2_rls	2000	63	5.574	2.689	74.603
pols2_rls	2001	63	3.168	2.686	44.444
pols2_ug	2000	63	2.309	1.433	85.714
pols2_ug	2001	63	3.257	2.773	44.444
pols2_ug_rls	2000	63	5.099	2.330	79.365
pols2_ug_rls	2001	63	3.241	2.772	44.444
sb	2000	63	4.266	2.916	63.492
sb	2001	63	3.032	2.512	52.381
sb_rls	2000	63	7.111	3.874	60.317
sb_rls	2001	63	3.011	2.529	50.794
sb_ug	2000	63	3.739	2.461	69.841
sb_ug	2001	63	3.100	2.607	49.206
sb_ug_rls	2000	63	6.590	3.440	68.254
sb_ug_rls	2001	63	3.079	2.618	47.619
ss	2000	63	1.754	1.413	80.952
ss	2001	63	3.739	3.402	28.571
ss_rls	2000	63	1.778	1.249	92.063
ss_rls	2001	63	3.728	3.402	30.159
ss_ug	2000	63	1.914	1.540	76.190
ss_ug	2001	63	3.785	3.471	26.984
ss_ug_rls	2000	63	1.827	1.381	87.302
ss_ug_rls	2001	63	3.775	3.464	26.984

OVERALL

test	method	numcond	RMSE	MAD	ARR
peba2	2000	189	2.528	1.650	80.423
peba2	2001	189	2.801	2.229	58.730
peba2_rls	2000	189	4.175	2.021	79.365
peba2_rls	2001	189	2.801	2.273	59.788
peba2_ug	2000	189	2.189	1.397	85.714
peba2_ug	2001	189	2.859	2.310	56.085
peba2_ug_rls	2000	189	3.845	1.803	81.481
peba2_ug_rls	2001	189	2.860	2.349	56.085
peba4	2000	189	1.984	1.319	88.360
peba4	2001	189	2.865	2.302	55.556
peba4_rls	2000	189	3.601	1.656	83.598
peba4_rls	2001	189	2.871	2.352	55.556
peba4_ug	2000	189	1.687	1.092	90.476
peba4_ug	2001	189	2.921	2.380	55.026
peba4_ug_rls	2000	189	3.291	1.462	86.243
peba4_ug_rls	2001	189	2.932	2.423	54.497
peba6	2000	189	1.774	1.196	90.476
peba6	2001	189	2.887	2.333	55.556
peba6_rls	2000	189	3.362	1.533	85.185
peba6_rls	2001	189	2.900	2.382	55.026
peba6_ug	2000	189	1.497	0.989	91.534
peba6_ug	2001	189	2.944	2.413	53.968
peba6_ug_rls	2000	189	3.075	1.347	88.360
peba6_ug_rls	2001	189	2.959	2.456	52.910
pols2	2000	189	2.053	1.353	86.772
pols2	2001	189	2.862	2.298	56.085
pols2_rls	2000	189	3.685	1.699	83.598
pols2_rls	2001	189	2.867	2.347	55.026
pols2_ug	2000	189	1.747	1.126	89.947
pols2_ug	2001	189	2.917	2.375	55.026
pols2_ug_rls	2000	189	3.365	1.496	86.243
pols2_ug_rls	2001	189	2.925	2.418	54.497
sb	2000	189	3.245	2.166	70.370
sb	2001	189	2.709	2.141	62.963
sb_rls	2000	189	4.828	2.515	73.016
sb_rls	2001	189	2.718	2.187	61.905
sb_ug	2000	189	2.875	1.878	75.661
sb_ug	2001	189	2.769	2.219	60.317
sb_ug_rls	2000	189	4.476	2.258	76.190
sb_ug_rls	2001	189	2.770	2.251	59.788
ss	2000	189	1.469	1.120	88.360
ss	2001	189	3.372	2.938	41.270
ss_rls	2000	189	1.391	1.038	96.296
ss_rls	2001	189	3.395	2.982	41.270
ss_ug	2000	189	1.595	1.220	85.714
ss_ug	2001	189	3.412	2.997	40.212
ss_ug_rls	2000	189	1.464	1.141	94.180
ss_ug_rls	2001	189	3.440	3.039	38.624