When conducting Management Strategy Evaluation the aim is to select the best performing candidate Management Procedure. To do this it is necessary to evaluate trade-offs between multiple potentially conflicting objectives using performance statistics derived from Operating Model properties.
Ideally, summary statistics should be few and informative and based on the main management objectives, which are to avoid stock collapse with high probability and to secure high and stable long-term yields. It is also necessary to distinguish between technical summary statistics (i.e. those required to evaluate model fits and performance) and those required to evaluate management objectives. The objectives may also be formalised as an objective function, to help automated comparison of multiple models .
Management performance was therefore evaluated using statistics based on SSB, fishing mortality, catch and predator ration. It is essential to distinguish between the roles of Operating Model performance statistics used for setting fishery conservation objectives (e.g. \(F_{MSY}\)) and the operational control points (e.g. Ltrigger) used to define harvest control rules (Cox, Kronlund, and Benson 2013). The objectives of the advice framework applied to stocks in the North East Atlantic is to minimise the risk of depleting a resource to a level where productivity is compromised and maximise surplus production.
Ensuring sustainability requires preventing the stock from becoming overfished, so there is a low probability of compromising productivity, therefore, many fishery management bodies define a limt (Blim) above which the stock should be maintained with high probability. Blim should be based exclusively on the biology of the stock and its resilience to fishing pressure, we therefore defined the biomass limit as the biomass at which recruitment is 30% of that at virgin recruitment. It should not consider economic factors, as the objective is to reduce risks from a biological perspective.
If a limit is inadvertently violated, immediate action should ne taken to return the stock or fishing pressure to the target level. If the objective is to achieve the long-term maximum yield (MSY) by regulating fishing mortality. Then the corresponding fishing mortality (\(F_{MSY}\), defined for a given fishing pattern and current environmental conditions) and biomass (BMSY) can be considered as target levels around which fishing mortality and biomass should fluctuate randomly, e.g. achieve the target level with a 50% probability.
To choose between candidate MPs, it is necessary to summarise the trade-offs between multiple potentially conflicting objectives using summary statistics. These are different to the operational reference points describe above.
Summary statistics, performance statistics or summary metrics are a set of statistics used to evaluate the performance of Candidate MPs against specified pre-agreed management objectives, and the robustness of these MPs to uncertainties in resource and fishery dynamics of concern to stakeholders and managers . These performance indicators are codified as properties of the system, e.g. the ratio of the realised catch to MSY, and the risk of the stock falling below a level where recruitment is impaired. They may be used to test the robustness of assumptions made in a stock assessment or within an MP, for example when \(B_{MSY}\) (based on exploitable biomass) or \(F_{MSY}\) (based on harvest rate) are used as part of the forecast for biomass dynamic models. These may differ from the corresponding quantities in the OM (where, e.g. \(B_{MSY}\) is based on \(SSB\), and \(F_{MSY}\) based on an instantaneous exploitation rate). There are two main ways to derive quantities to be used for performance statistics, namely: (i) using equilibrium assumptions , or (ii) through stochastic simulation e.g. by projecting at F = \(F_{MSY}\) or F = 0 (. The latter approach is preferable where environmental forcing or resonant cohort effects impact on productivity.
The principle metrics for use as summary statistics are safety, stock status, yield and stability. The tuna RFMOs management objectives are mainly articulated through MSY based targets , while LRP in some cases has also been derived from MSY, e.g. the International Commission for the Conservation of Atlantic Tuna (ICCAT) and IOTC. While the Western Central Pacific Fisheries Commission (WCPFC) have used \(SSB_{F=0}\) (the spawning stock biomass in the absence of fishing), since this has the advantage of not depending on the selection pattern of the fleets or the stock recruitment relationship that is often difficult to estimate in practice.
To achieve Marine Stewardship Council (MSC) certification requires a limit reference point to indicate the stock level at which recruitment is impaired (PRI) which the stock should be above, and a biomass target that a stock should be fluctuating around a consistent with achieving MSY. So an additional summary statistic was included that combined safety, status and yield into a single value.
Often variability, measured as the inter-annual variation in catch is also used as a summary statistic . However, in this study catch variability was set in the MP so was not included as a summary statistic.
then### Reference Levels
for performance statistics I think there are 5 things to do i) extract time series as FLQuants, ssb, catch. … ii) calculate ref pts from FLBRP, iii) run a projection without feedback at \(F_{MSY}\) as a reference, i.e. best you can do iv) transform the FLQuants, i.e. SSB to rebuilding time, expected recruitment, “r”, v) summarise the P()’s vi) summarise the variability i.e. should fluctuate around targets, and for limits if you have a 5% P of hitting a limit what if you have 100 iters for 100 years, and either all iters break limit in 5 years, or 5 iters breach limit in all years
The first step when conducting Management Strategy Evaluation is to agree the management objectives, and the last is to identify the strategies that are best able to meet them. To do this management objective need to be agreed and then quantified in the form of statistics known as Performance Metrics. These are used to evaluate the performance of alternative management strategies and their robustness to uncertainties in resource and fishery dynamics.
It is essential to distinguish between Performance Metrics, derived from the Operating Model used for setting fishery conservation objectives and reference points used as control points in harvest control rules (Cox, Kronlund, and Benson 2013). For example, in ICES if stock biomass falls below \(B_{MSY_{Trigger}}\) then fishing mortality is reduced, the objective is to achieve \(MSY\) and maintain the stock at \(B_{MSY}\) but which is not actually defined.
As well as biological objectives, fisheries management has a range of potentially conflicting social, economic, political and ecosystem goals, which may only be loosely defined. These may include minimising the chance of unintended stock depletion, maximising catches over time, enhancing industry stability through low inter-annual variability in catches, and maintaining ecosystems in a healthy, productive, and resilient condition (Hornborg et al. 2019).
ICES estimates Blim based on the stock-recruitment relationship to avoid recruitment overfishing, the undesirable state in which adults of a species are so overfished that they cannot reproduce fast enough to replenish the stock. However, this is problematic since it ignores that there may be no evidence of a stock-recruitment relationship if a stock has not been overfished or if recruitment (as in sprat) is influenced by environmental drivers rather than stock biomass. An alternative is to base Blim as a fraction of virgin biomass (B0), the biomass in the absence of fishing. However, moving towards sustainable ecosystem-based fisheries management also requires ensuring that prey stocks are also sustainable for the consumptive needs of predators and to minimise risks due to a changing environment. ICES defines Blim as the SSB below which recruitment is impaired. However, this requires that the dynamics are stationary, a stock-recruit relationship can be estimated, and that there is a well-defined break-point. We found that if M varied, e.g. due to environmental or species interactions, differently from that assumed in the assessment, then a change in M could be identified as a change in recruitment. However, this is the wrong mechanism, as the relationship is due to the effect of recruitment on stock, not the stock-recruitment relationship. The approach provides a way to evaluate ecosystem trade-offs in relation to environmental variability and the risk of exceeding ecosystem thresholds intended to ensure conservation goals are met, i.e. Bescapment that leaves enough biomass for predator needs.
Objectives should be pre-agreed in a clear and transparent way at the start of the MSE process to ensure that agreement can be achieved at the end. To facilitate the process, Performance Metrics should ideally be few, uncorrelated to avoid redundancy, informative, and based on how well the management objectives are achieved. Performance Metrics are codified as properties of the system, and so are obtained from the Operating Model, e.g. the ratio of realised catch to maximum sustainable yield (\(MSY\)), and the risk of the stock falling below a level where productivity is impaired. The principle metrics for use as Performance Metrics are
The role of pelagic species within the ecosystem needs to be considered as we move towards an ecosytem approach to fisheries (EAF, Kaplan et al. 2021). As an example of how ecosystem considerations can be included in a single species Operating Model, we split natural mortality into background (\(M_1\)) and predation (\(M_2\)) mortality. This allows a consideration of “predator needs”, i.e. the forage (\(R\)) based on the biomass lost to the stock due to \(M_2\). \(R\) can be calculated, equivalent to how yield to the fishery using \(M_2\) rather than \(F\).
\(\sum_{i=0}^{n}W_i N_i \frac{M2_i}{Z_i}(1-e^{-Z_i})\)
Where W is the mass-at-age, N is the numbers-at-age, M2 is predation mortality, and Z is this total mortality.
A reference point then needs to be agreed and an appropriate level of depletionl.
Figure 1
What I tried to do was come up with a new “Blim”, which assumes a good understanding of the stock-recruitment relationship and ignores processes such as growth overfishing.
I dont like the way ICES currently set it, i.e. based on segmented regression etc. This is more related to the stock assessment method and assumptions than the biology.
For example how should you set Blim for SS3 (where the SRR is estimated in the assessment), SAM/a4a (where the SRR is estimated post-hoc), and SPiCT where there is no SRR?
This has a problem of consistency, for example Max developed an ensemble using SS3, but ensembles c/should also include alternative model frameworks. What if you have an ensemble with SS3,SAM & SPiCT?
But if Blim was based on the biomass that would take X% of a generation time to rebuild to an agreed level that would be comparable for SS3,SAM.a4a,SPiCT,…
So my 1st attempt was to deplete a stock to x% of BMSY then set F=0 and estimate how long it took to rebuild
This is the 1st figure, the stock is simulated to be at X% of BMSY from year 0 to 50, then F=0 and the stock rebuilds.
The 2nd figure then plots the time taken for the ABI to equal ABIMSY.
We can do this for ICES stocks and then compare to BLIM, identify inconsistencies etc.,
Figure 2 Recovery from different depletion levels, red horizontal line is \(B_{MSY]\)}, vertical lines are half (white) and one (red) generation time
Figure 3 Time to recover from different depletion levels, vertical lines are half (white) and one (red) generation time
Figure 4 Recovery from different depletion levels, red horizontal line is \(B_{MSY]\)}, vertical lines are half (white) and one (red) generation time
Figure 5 Time to recover from different depletion levels, vertical lines are half (white) and one (red) generation time
Figure 6 Time to recover from different depletion levels, vertical lines are half (white) and one (red) generation time
Figure 7 \(ABI_{MSY}\) for different depletion levels, and years of rebuilding, vertical lines are half (white) and one (red) generation time
While, to allow fishers to plan, yields should not vary too greatly between years .
\(\frac{\sum_{t=1}^n \frac{Y_{t+1} - Y_t}{Y_T}}{n}\).
The adoption of the voluntary Code of Conduct on Responsible Fishing and the United Nations Fish Stocks Agreement [PA, @garcia1996precautionary] requires that reference points and management plans are developed for all stocks-not just targeted commercial stocks, but also by-caught, threatened, endangered, and protected species [@sainsbury2003ref]. Reference points are used in management plans as targets to maximise surplus production and as limits to minimize the risk of depleting a resource to a level where productivity is compromised. Reference points must integrate dynamic processes such as growth, fecundity, recruitment, mortality, and connectivity into indices for exploitation level and spawning reproductive potential. An example of a target reference point is the fishing mortality (F) that will produce the maximum sustainable yield ($F_{MSY}$), commonly defined as the fishing mortality with a given fishing pattern and current environmental conditions that gives the long-term maximum yield. To ensure sustainability requires preventing a stock from becoming overfished, and ensuring that there is a low probability of productivity being compromise many fishery management bodies define a limit reference point, $B_{lim}$, at a biomass at which recruitment or productivity is impaired . When assessing stocks, it is also important to consider trends as well as state since a stock at a target biomass may be declining due to overfishing, while, a depleted stock may be recovering due to management action (Hilborn, 2020).
Reference points are required to prevent growth, recruitment, economic and target overfishing. Growth and recruitment overfishing are generally associated with limit reference points, while economic overfishing may be expressed in terms of either targets or limits. The difference between targets and limits is that indicators may fluctuate around targets but in general limits should not be crossed. Target overfishing occurs when a target is overshot, although variations around a target is not necessarily considered serious unless a consistent bias becomes apparent. In contrast, even a single violation of a limit reference point may indicate the need for immediate action.
An example of a management framework based on reference points is that of ICES’ MSY approach. This has the objective of maintaining the stock above a level at which productivity is impaired (\(B_{lim}\)) with at least a 95% probability, while attaining a fishing mortality rate of the level that provides the Maximum Sustainable Yield (\(F_{MSY}\)). ICES therefore uses \(F_{MSY}\) as a target exploitation level, estimated as the fishing mortality with a given fishing pattern and current environmental conditions that gives the long-term maximum yield.
Traditionally, optimisation methods combine multiple objectives into a single-objective, as a reward or utility function. In economics utility refers to the total satisfaction received from consuming a good or service, and economic theories based on rational choice usually assume that consumers will strive to maximize their utility. Objectives and hence utility, however, are be difficult to measure and quantify in practice. For example to achieve Marine Stewardship Council certification it needs to be shown that it is highly certain that a stock is maintained above a limit reference point and is above or fluctuating around a target level based on Maximum Sustainable Yield (MSY). A benefit of certification is that once achieved there is a price premium. An example of an economic objective is stability of catch between years as this is good for the supply chain and investment planning .
Managers to specify the objectives, and relative risks, scientists to map these to metrics and calculate the probabilities.
A set of statistics used to evaluate the performance of Candidate MPs (CMPs) against specified management objectives, and the robustness of these MPs to important uncertainties in resource and fishery dynamics.
To choose between candidate Management Procedures, it is necessary to evaluate trade-offs between multiple potentially conflicting objectives using Performance Metrics derived from Operating Model properties. Confusingly, these may have the same names as the reference points used in the Management Procedure. It is therefore essential to distinguish between the roles of Operating Model performance metrics used for setting fishery conservation objectives and the Management Procedure reference points, which are operational control points used to define harvest control rules (Cox et al., 2013). The ICES advice framework applied to stocks in the North East Atlantic is based on the Precautionary Approach and achieving the MSY where the objectives are to maximise surplus production and to minimise the risk of depleting a resource to a level where productivity is compromised. Ensuring sustainability requires preventing the stock from becoming overfished, so there is a low probability of compromising productivity. Therefore, many fishery management bodies define that reference points must integrate dynamic processes such as growth, fecundity, recruitment, mortality, and connectivity into indices for exploitation level and spawning reproductive potential. An example of a target reference point is the fishing mortality (F) that will produce the maximum sustainable yield (\(F_{MSY}\)), commonly defined as the fishing mortality with a given fishing pattern and current environmental conditions that gives the long-term maximum yield. An example of a limit is Blim, the biomass at which recruitment or productivity is impaired. Such limits define the point beyond which fishing is no longer considered sustainable. In a well-managed fishery, managers try to avoid this zone with a high degree of certainty. If it is inadvertently violated, immediate action is taken to return the stock or fishing pressure to the target level. Limits should be based exclusively on the biology of the stock and its resilience to fishing pressure. This should not consider economic factors, as the objective is to reduce risks from a biological perspective. ICES defines Blim based on the stock-recruitment relationship to avoid recruitment overfishing, the undesirable state in which adults of a species are so overfished that they cannot reproduce fast enough to replenish the stock. As an example of an ecosystem reference point, we used “predator ration” r forage (R) based on the biomass lost to the stock due to M2. This is equivalent to how yield to the fishery is calculated, i.e. as R=i=0nWiNiM2i/Zi(1-exp(-Zi)) Where W is the mass-at-age, N is the numbers-at-age, M2 is predation mortality, and Z is this total mortality.
Management performance was evaluated using Performance Metrics based on fishing mortality, SSB, catch and forage available for predators. If an objective is to achieve MSY, then \(F_{MSY}\) and MSY can be considered as target levels around the fishing mortality and yield should fluctuate around randomly, e.g. achieve the target level with a 50% probability. While to ensure sustainability, SSB should remain above Blim with a high probability. Not all objectives may be achievable at the same time, and relative importance and probabilities are related to risk, and so are management decisions. Although ICES has defined reference levels and probabilities for SSB, as yet there are no agreed corresponding values for forage or even whether if it should be treated as a limit or a target. Therefore, in our example, we defined it as a limit relative to with a corresponding reference level of 30% of forage without fishing. The reference levels were calculated from a simple projection, i.e. fishing at \(F_{MSY}\).
There are normally four main objectives, namely safety, status, yield and variability. Safety is ensuring that stock productivity is not impaired, while status is to ensure that \(MSY\) is achieved. These are therefore represented by summarising status relative to limit and target reference points respectively. If status is at \(B_MSY\) and exploitation at \(F_{MSY}\) it is implicitly assumed that yield will be at \(MSY\). However, if reference points assume stationary then this may not be the case and so the need for a yield statistic. Management may result in large changes in catches in which case there will be a trade-off between maximising yield and minimising variability, and so inter-annual variability in yields needs to be considered. There is a potential fifth objective for forage species where maximising yield may mean that the predator share is reduced, we therefore have a fifth objective forage.
Fischer et al. (2021) defined a fitness function that included four components: \[\begin{equation}\label{eq:fitness_MSY} \phi_{\text{MSY}}= \phi_{\text{SSB}}+\phi_{\text{Catch}}+\phi_{\text{risk}}+\phi_{\text{ICV}}\text{,} \end{equation}\] where the individual components were
\[\begin{equation}\label{eq:fitness_ssb} \phi_{\text{SSB}}=-\ \bigg|\frac{\text{SSB}}{B_{\text{MSY}}}-1\bigg|\text{,} \end{equation}\] \[\begin{equation}\label{eq:fitness_catch} \phi_{\text{Catch}}=-\ \bigg|\frac{\text{Catch}}{\text{MSY}}-1\bigg|\text{,} \end{equation}\] \[\begin{equation}\label{eq:fitness_risk} \phi_{\text{risk}}=-\ B_{\text{lim}}\ \text{risk}, \text{\ and} \end{equation}\] \[\begin{equation}\label{eq:fitness_ICV} \phi_{\text{ICV}}=-\ \text{ICV}\text{.} \end{equation}\] \[\begin{equation}\label{eq:fitness_ICV} \phi_{\text{forage}}=-\ \text{ICV}\text{.} \end{equation}\]
The Performance Metrics used in these fitness elements were calculated over the 50-year projection and 500 simulation replicates. \(\text{SSB}/B_{\text{MSY}}\) and \(\text{Catch}/\text{MSY}\) were the medians of their respective distributions and \(B_{\text{lim}}\ \text{risk}\) the proportion of the SSB values falling below the biomass limit reference point \(B_{\text{lim}}\) (defined as the SSB corresponding to a recruitment impairment of 30%). The inter-annual catch variability (ICV) was the median of \(|(C_y-C_{y-v})/C_{y-v}|\) (exclusive of undefined values due to division by zero) calculated every \(v\) years, where \(C_y\) is the catch for the year \(y\) and \(v\) the frequency of advice, e.g. \(v=2\) for a biennial advice. Effectively, this fitness function was aimed at reaching MSY reference levels for SSB and catch, while at the same time reducing risk and ICV.
The ICES precautionary criterion generally states that the probability of SSB falling below \(B_\text{lim}\) should not exceed 5%. Therefore, \(\phi_{\text{MSY}}\) is not entirely aligned towards the ICES precautionary approach, and \(\phi_{\text{risk}}\) will need to be changed. Compliance with the precautionary approach can be achieved by including a penalty in the fitness when the risk exceeds 5%, which was implemented by replacing \(\phi_{\text{risk}}\) with a fitness function component for which the fitness value was linked to the \(B_\text{lim}\) risk (\(=P\)) via a penalty function \(\Omega\):
\[\begin{equation}\label{eq:fitness_riskPA} \phi_{\text{risk-PA}}=-\ \Omega(P)\ \text{,} \end{equation}\] and \[\begin{equation}\label{eq:penalty} \Omega(P)= \frac{\tau_{m}}{1+e^{- \left(P - \tau_i \right)\tau_s}}\ \text{.} \end{equation}\] This function has a sigmoid shape (Figure \(\ref{fig:penalty}\)) and is characterised by three parameters; \(\tau_{m}\) defines the maximum penalty, \(\tau_{i}\) the inflection point and \(\tau_{s}\) the steepness of the curve. The three parameters’ values were based on considerations for one example stock (pollack, ). When pollack was projected forward with zero catch, the sum of \(\phi_{\text{SSB}}+\phi_{\text{Catch}}+\phi_{\text{ICV}}\) [Equations (\(\ref{eq:fitness_ssb}\)), (\(\ref{eq:fitness_catch}\)), and (\(\ref{eq:fitness_ICV}\))] had an absolute value of just below 5.
Therefore, the maximum penalty \(\tau_{m}\) was set to 5. This parameterisation had the effect that the \(rfb\)-rule parameterisation leading to zero-catch always had higher fitness than the \(rfb\)-rule parameterisations where \(B_\text{lim}\) risk exceeded 5%. The penalty curve inflection point was set to \(\tau_{i}=0.06\) so that the risk could slightly exceed 5% without immediately incurring the maximum penalty. The penalty steepness was set to \(\tau_{s}=500\) so that the penalty quickly reached its maximum value but avoided a knife-edge which might cause problems during the optimisation.
The final fitness function, which included MSY objectives for catch and SSB and included precautionary considerations for the risk, was defined as: \[\begin{equation}\label{eq:fitness_PA} \phi_{\text{MSY-PA}}= \phi_{\text{SSB}}+\phi_{\text{Catch}}+\phi_{\text{risk-PA}}+\phi_{\text{ICV}}\text{.} \end{equation}\]
Each of the elements in \(\phi_{\text{MSY-PA}}\) is negative because the genetic algorithm maximised the fitness. The fitness value of \(\phi_{\text{MSY-PA}}\) quantifies the management performance of the simulation results (see e.g. Figures \(\ref{fig:pollack_rfb_PA_components}\) and \(\ref{fig:rfb_PA_all_stocks}\), described in the Results section). Fitness values closer to 0 (less negative) indicate better performance. The aim of the optimisation procedure was to provide precautionary management solutions, and options where risk exceeds 5% are clearly indicated by red shading.
Figure 8
The NS1 guidelines provide guidance on determining the minimum (Tmin), maximum (Tmax), and target (Ttarget) time to rebuild a stock to a level that supports MSY (Bmsy). In the past, Councils have had difficulties calculating Tmax based on the original data-intensive method (i.e., Tmin + one generation time) that requires data on life history, natural mortality, age at maturity, fecundity, and maximum age of the stock (Restrepo, et al. 1998). In order to allow Councils to make Tmax calculations despite variable information and data availability amongst stocks, NMFS proposed specifying three methods to calculate Tmax within the guidelines: (1) Tmin plus one mean generation time (status quo); (2) the amount of time the stock is expected to take to rebuild to its Bmsy if fished at 75 percent of the MFMT; or (3) Tmin multiplied by two. Further background and rationale on the proposed revisions to the guidance on the calculation of Tmax was provided on pages 2795-2796 of the proposed rule. See80 FR 2795-2796, January 20, 2015. https://www.federalregister.gov/documents/2016/10/18/2016-24500/magnuson-stevens-act-provisions-national-standard-guidelines
The IUCN Red List of Threatened Species (IUCN, 2017a) is the primary authority for extinction risk assessments. In the Red List assessment, species are categorized as threatened based on criteria related to population decline, geographic range size, fragmentation, and small population size (IUCN, 2017b). In criteria related to population decline, generation time acts as a standardization for time units that allows using the same criteria on species with extremely different life spans (Mace et al., 2008). In criterion A, for example, population size reduction is measured over 10 years or three generations, whichever is longer.
Figure9 Time series of reference points.