Comparison of length based idicators and methods

Figure 1. Example of length based indicators and estimators for Sprat.

Calculate true positive rate (sensitivity) and true negative rate (specificity) for identifying \(F/F_{MSY}\)

Then calculate the True Skill Statistic i.e. TSS = TPR + TNR - 1 = TPR - FPR and the area under the ROC curve (AUC)

A Perfect forecasts receive a TSS of 1, while a random forecasts receive a score of 0 and forecasts inferior to random forecasts receive a negative score.

TSS depends on the chosen reference level, in contrast AUC indicates whether a indicator can classifiy i.e. rank status, i.e. AUC=1 if there is a choice of a reference value where TRP=1 & FPR=0. If AUC=0.5 then the estimator is no better than a coin toss.

TSS allows the best estimator for species and its robustness to be evaluated, the AUC identifies whether the indicator can be improved by tuning.

LIME and Haupt estimators and \(L_{mean}\) are biased (as shown by TSS) but can be improved by tuning (as shown by AUC). Tuning of \(L_{mean}\) results in performance equivalent to LBSPR. There are some scenarios, however, where performance is poor, as seen by the outliers, these are for the high CV, and so \(L_{mean}\) can be improved by improved sampling, or combinimng data from multiple years.

Figure 2. True skill score by species

Figure 3. True skill score by Estimator scenario

Figure 4. True skill score by sampling error

Figure 5. Area under the ROC by species

Figure 6. Area under the ROC by scenario

Figure 7. Area under the ROC by sampling error.

Figure 8. Estimates of slope.

Figure 9. P values

MSE

Figure 10. Comparison of Operating Model run for F with HCR run with perfect CPUE and with \(L_{Mean}\)

References

Author information

Laurence Kell. laurie@seaplusplus.es

Acknowledgements

Software Versions

• R version 4.0.3 (2020-10-10)
• FLCore: 2.6.18.9020
• FLBRP: 2.5.8
• FLasher: 0.6.8.9003
• FLife: 3.4.0
• mydas: 1.2.2
• Compiled: Tue May 17 08:40:37 2022