Introduction

Landscapes restoration intervention planners are often faced with the challenge of prioritizing among different interventions to optimize outcomes. This is because landscape restoration outcomes are often achieved through complex mechanisms, and the success of restoration actions is rarely guaranteed, with many uncertainties preventing precise impact prediction (Luedeling et al. 2019). Success is even harder to predict, when landscape restoration agencies aim to strengthen restoration efforts indirectly, e.g., by supporting livelihoods and economies of local people as an incentive for them to to restore degraded landscapes (Wafula et al. 2018). However decision support tools that allow for the expression of uncertainty and risks could help to overcome these challenges. In this learning module, we present case study to guide development practitioners on how to holistically evaluate project costs, benefits and risks using decision support tools. We describe how to include variables with uncertain and missing information in the analysis and use a decisionSupport package (Luedeling & Gohring, 2017; Luedeling & Whitney, 2018) to predict outcomes of investing in beekeeping and enrichment planting of trees and management in Desa’a forest in Northern Ethiopia.

Materials and Methods

Study background

To preserve and protect Desa’a forest, a non-profit organization with support from the Ethiopian government launched a long-term FLR program that proposes investments in a portfolio of scalable environmental and socio-economic interventions. The specific objectives of the program are to (i) restore the degraded forest’s biodiversity and enhance ecological integrity (ii) contribute towards meeting the subsistence needs and hence promote economic development and (iii) build the livelihood resilience of communities living within and around the forest. To achieve these objectives, proposed interventions are to be implemented within a zoning framework in a pilot area covering 180 ha of the forest (Tamba et al., 2021). In this module we highlight the impact of investing in beekeeping to reduce forest encroachment, where enrichment planting of indigenous trees is being implemented.

Proposed interventions modeling

To model proposed interventions, we first define the decision question with decision-makers. For instance, what are the short-term and long-term restoration outcomes, who are the targeted beneficiaries and what type of decision is under consideration (prioritizing vs planning) (Luedeling and Shepherd, 2016). Semi-structured interviews with decision-makers can help to contextualize the decision under consideration. The process also helps to identify subject matter experts to engage in the participatory modeling of intervention outcomes.

Decision support package

The decisionSupport function in the R package of the same name requires two inputs:

  1. An input_table (in .csv format) specifying the names and probability distributions for all variables used in the analysis.To generate the input_table, we first describe the interventions and their impact pathways. This helps to identify the emerging cost, benefit and risk variables. For instance, the beekeeping intervention would mostly provide supplemental income to reduce pressure on forest resources. However, project proponents would have to incur the initial investment of establishing the apiary (beehives cost, initial labour cost etc) and recurring costs (hive maintenance cost, harvesting cost, transport cost etc). Also, honey production will be adversely affected by drought, bee disease outbreak or poor apiary management.Enrichment planting and management intervention would mostly increase density of forest cover and sequester C in the process. Expected costs for this intervention include cost of labour, tree seedling cost, cost of pesticides etc. The risks include frost (which mostly affect hardening seedlings), drought, wildfire, encroachment etc. These variables constitute the input_table as shown below.
Monte Carlo inputs
description lower upper distribution variable
General coefficient of variation (CV) 5.00 10.00 posnorm general_CV
Simulation period (years) 25.00 25.00 const n_years
Discount rate (the rate of return expected) 9.00 12.00 posnorm discount_rate
Number of hives per farmer (n) 2.00 3.00 posnorm n_hives_per_economic_unit
Number of honey harvesting seasons/year 1.00 2.00 posnorm n_annual_honey_harvests
Labour for apiary establishment (USD) 20.00 50.00 posnorm apiary_establishment_cost
Bee cost of labour per season (USD) 25.00 60.00 posnorm bee_cost_labour_per_season
Other costs (wax and bee colony costs) (USD) 6.80 13.60 posnorm other_bee_costs
Harvesting costs (USD) 3.40 6.80 posnorm harvest_cost
Transportation costs (USD) 2.04 5.10 posnorm transport_cost
Hive maintenance materials costs (USD) 1.70 3.40 posnorm hive_maint_mat
Cost of beehives (USD) 153.04 187.05 posnorm cost_of_beehive
Cost of honey processing (USD/five beekeepers) 340.09 510.13 posnorm cost_honey_processing_unit
Honey produced (kg/hive) 8.00 20.00 posnorm honey_produced_kg_per_hive
Price of honey (USD/kg) 13.60 17.00 posnorm price_honey_per_kg
Chance of drought affecting bee colonies (on a scale of 0 to 1) 0.10 0.25 tnorm_0_1 drought_risk
Loss due to drought (on a scale of 0 to 1) 0.30 0.50 tnorm_0_1 loss_due_to_drought
Chance of bee diseases (on a scale of 0 to 1) 0.15 0.35 tnorm_0_1 bee_diseases
Loss due to bee diseases (on a scale of 0 to 1) 0.30 0.50 tnorm_0_1 loss_due_bee_diseases
Chance of poor apiary management (on a scale of 0 to 1) 0.20 0.35 tnorm_0_1 poor_apiary_management
Loss due to poor apiary management (on a scale of 0 to 1) 0.15 0.20 tnorm_0_1 loss_due_poor_apiary_mgnt
Maximum volume attained by juniper and olea trees (m3) 3.50 5.00 posnorm maximum_merchantable_volume
Biomass to carbon conversion factor 0.44 0.49 posnorm biomass_conversion_factor
Age of mature tree where growth levels off 18.00 20.00 posnorm stem_maturity_age
Ratio of below ground to above bround biomass 0.27 0.68 posnorm biomass_ratio
Chance of prosopis affecting growth of forest (%) 0.02 0.05 tnorm_0_1 chance_invasive_species
Change in number of trees/hectare due to invading prosopis (%) 1.00 15.00 posnorm percentage_trees_lost_prosopis
Chance that there are fewer trees/ha due to a diease outbreak (0…1) 0.01 0.07 tnorm_0_1 chance_diseases_outbreak
Change in number of trees/hectare due to disease outbreak (%) 3.00 5.00 posnorm percentage_tree_loss_diebark
Chance of wildfires (0…1) 0.10 0.25 tnorm_0_1 chance_fire_outbreaks
Percentage of forest cover lost annually due to wildfire(%) 1.00 10.00 posnorm percentage_trees_lost_fire
Chance of severe frost (0…1) 0.01 0.30 tnorm_0_1 chance_of_frost
Percentage of forest cover lost annually due to frost (%) 0.10 1.00 posnorm tree_loss_due_to_frost
Chance of forest resources extraction (community member ) 0.20 0.40 posnorm chance_extracts_resources
Chance of political conflict (0…1) 0.00 0.01 tnorm_0_1 chance_civil_unrest
Price/ton of carbon sequestered (USD) 40.00 120.00 posnorm carbon_cost
Probability that carbon markets do not materialise? 0.30 0.70 posnorm carbon_market_risk
Percentage increase in biomass/ha due to reduced water stress (%) 0.01 0.50 posnorm percentage_increase_biomass
Change in number of trees/hectare due to invading prosopis (%) 20.00 30.00 posnorm percentage_trees_lost_unrest
Additional trees planted per ha in the core zone (n) 500.00 1000.00 posnorm additional_trees_planted
Core zone total area (ha) 1626.00 1626.00 const core_zone_area
Replanting cost of maintenance pruning, thinning (USD) 40.57 80.86 posnorm replanting_cost_maintenance
Cost of replanting a hectare of forest (USD) 300.00 500.00 posnorm per_ha_cost_planting
Labour costs as a proportion of total enrichment costs(%) 0.10 0.25 tnorm_0_1 enrichment_cost_labour_proportion
Annual rate of deforestation (%) 0.93 2.60 posnorm annual_rate_deforestation
Average biomass produced in Desa’a dry afromontane (Ton/ha/yr) 180.00 250.00 posnorm mean_biomass_per_ha
Mean annual increment in growth of forest biomass (Ton dm /ha/yr) 0.10 0.20 posnorm biomass_mean_annual_increment
Proportion of seedlings that survive to maturity when there are no extra measures taken (%) 0.80 0.90 posnorm seedling_survival_rate

The variable distributions are described by a 90% confidence interval, which are specified by lower (5% quantile) and upper (95% quantile) bounds. To achieve this, project proponents undergo a calibration training (Hubbard, 2014) on how to generate variable estimates, for which they are 90% confident that the actual values lie within the provided ranges. This helps to include variables with missing information in the analysis. The process is usually backstopped by subject matter experts and literature review. For the shapes of variable distribution, const describes variables that are constant throughout, norm describes variables with normal distribution, tnorm_0_1 describes variables with a truncated normal distribution that can only have values between 0 and 1 (useful for probabilities) and posnorm describes variables with normal distribution truncated at 0 (only positive values allowed).

  1. An R_function that predicts decision outcomes based on the variables named in input_table. This R function is customized by the user to address a particular decision problem. For this analysis, the function Desa'a_restoration describing the causal relationships between benefits, costs and risk variables for investing in beekeeping, enrichment planting and management was used.

Loading data in R

We could simply start developing the mathematical decision model now, but since the model function will be designed to make use of variables provided to it externally (random numbers drawn according to the information in the input_table), we’ll need to load the data in R and define sample values for all variables, if we want to test pieces of the function code during the development process. This is accomplished with the following helper function make_variables.

make_variables<-function(est,n=1)
{ x<-random(rho=est, n=n)
    for(i in colnames(x)) assign(i,
     as.numeric(x[1,i]),envir=.GlobalEnv)
make_variables(estimate_read_csv("Desaa_inputs_v2_240620.csv"))
}

This function isn’t included in the decisionSupport package, because it places the desired variables in the global environment. This isn’t allowed for functions included in packages on R’s download servers. Applying make_variables to the data table (with default setting n=1) generates one random number for each variable, which then allows you to easily test code you’re developing:

a) Beekeeping intervention

To model this we first compute the amount of honey that would be produced annually in an ideal case, based on the number of beehives distributed per household. Since the amount of honey produced is expected to vary overtime, we introduce a value varier vv function to account for this. The function produces a time series that contains variation from a specified mean and a desired coefficient of variation. This can be implemented in R, using the decisionSupport package, as shown below

Desaa_restoration <- function(x, varnames){
no_hives_hh<-round(n_hives_per_economic_unit)
amount_honey_produced<-vv(honey_produced_per_hive,general_CV,n=n_years)*no_hives_hh

Projections for amount of honey produced overtime (n_years) would therefore appear as follows

amount_honey_produced
 [1] 41.23289 32.63919 31.77556 35.73969 37.46505 39.50892 40.64377
 [8] 30.00168 34.24214 37.57191 39.43597 41.94117 36.58436 39.87770
[15] 36.57796 32.58756 32.80550 42.54316 38.06027 40.00702 36.23699
[22] 36.26328 34.30495 40.15300 35.80208

The beekeeping risks e.g bee disease outbreak, are modeled using the chance_event function, also contained in the decisionSupport package. The function randomly simulates whether events occur and returns output values accordingly. The outputs can be single values or a series of values, with the option of introducing artificial variation into the dataset. To effectively model emerging risks, we categorize them into the general risks (risks that cut across the board) and intervention risks (risks that are associated with a specific intervention). The general risks in this case include political unrest, wildfire risk and drought. Their chance of occurrence is computed as follows.

drought_event <- rbinom(n_years,1,drought_risk)  
wildfire_event <- rbinom(n_years,1,wildfire_risk)
unrest_event <- rbinom(n_years,1,chance_civil_unrest)

Projections for a drought event for instance, (over a 25 year period) would appear as follows i.e based on the data in the input_table, a drought event is likely to occur in year 12, 18 and 23.

   drought_event
 [1] 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 1 0 0

The respective percentage loss in the amount of honey produced, when a drought event occurs would therefore be as shown below.

 honey_loss_drought
 [1] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000
 [7] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.3172842
[13] 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.3075168
[19] 0.0000000 0.0000000 0.0000000 0.0000000 0.2727146 0.0000000
[25] 0.0000000

Beekeeping intervention risks would therefore be computed as shown below

honey_loss_drought<-drought_event*vv(loss_due_to_drought,
                                     general_CV,n_years)
bee_disease_risk <-chance_event(bee_diseases,
                   value_if=loss_due_bee_diseases,
                   value_if_not=0,
                   n=n_years,
                   CV_if=general_CV,
                   CV_if_not=0,
                   one_draw=FALSE)
poor_apiary_mgnt_risk<-chance_event(poor_apiary_management,
                       value_if=loss_due_poor_apiary_mgnt,
                       value_if_not=0,
                       n=n_years,
                       CV_if=general_CV,
                       CV_if_not=0,
                       one_draw=FALSE)

The cumulative effect of risks on the amount of honey produced would therefore be computed as shown.

effect_of_honey_risks <-sapply(c(bee_disease_risk+
                        honey_loss_drought+poor_apiary_mgnt_risk),
                        function(x) min(x,1))
actual_amount_honey_produced <- amount_honey_produced*
                               (1-effect_of_honey_risks)

Projections for amount of honey produced when risks are considered would therefore be:

[1] 47.835995 36.035621 39.264102 55.961422 26.096744 56.419347
 [7] 51.481976 43.688492 50.330610  3.841762 52.459389 40.562589
[13] 58.787977 19.846078 55.811947 52.123612 34.451764 64.964572
[19] 30.649871 57.881960 48.876053 60.371462 65.784775 27.775082
[25] 56.820374

Returns from sale of honey when risks are considered would therefore be computed as shown.

honey_returns <- actual_amount_honey_produced*vv(price_honey_per_kg,general_CV,n_years)

Since beekeepers incur an initial one-off cost of establishing the apiary (beehives and colony cost, honey processing unit, labour etc) and recurring costs for each honey harvesting season, the computation of beekeeping costs and subsequent profits is as follows.

initial_investment <-c((apiary_establishment_cost),rep(0,n_years-1))
beekeeping_recurring_cost<-(vv(other_bee_costs,general_CV,n_years)+
vv(harvest_cost,general_CV,n_years)+vv(transport_cost,general_CV,
n_years)+vv((bee_cost_labour_per_season),general_CV,n_years)+vv
(hive_maint_mat,general_CV,n_years))*n_annual_honey_harvests
beekeeping_cost <- initial_investment+beekeeping_recurring_cost
beekeeping_profits <- honey_returns-beekeeping_cost

b) Enrichment planting and management

To compute the amount of carbon sequestered overtime, we use the allometric equation (Henry et al. 2013; Cifuentes et al. 2015) as follows

b <- 1:n_years
aboveground_biomass_tons_DM_per_ha<-maximum_merchantable_volume/(1+exp(-(BCEF
*(b-stem_maturity_age))))
total_biomass_DM_per_ha <- aboveground_biomass_tons_DM_per_ha*(1+biomass_ratio)
tree_population <- additional_trees_planted*seedling_survival_rate 
enrichment_biomass_per_ha<-(total_biomass_DM_per_ha)*tree_population

The equation allows for projection of increase in woody biomass that follow the graph shown.

Figure 1: Projected increase in woody biomass from enrichment planting

Improved tree management would also result in natural regeneration of the forest. We compute this benefit by estimating the mean annual increment (MAI) in woody biomass that survives due to improved forest governance and tree management

MAI <- (rep(1,n_years))
MAI[2:n_years] <- vv(biomass_mean_annual_increment,general_CV,n_years-1)
incremental_biomass_per_ha <- vv(mean_biomass_per_ha,general_CV,n_years)*(cumsum(MAI)

This increment overtime follows the trend shown in the graph below.

Figure 2: Projected increase in aboveground biomas from improved forest governance and tree tree management

Since biomass increase overtime will sequester C, amount of carbon sequestered is computed as shown below.

total_biomass_increase <- enrichment_biomass_per_ha+incremental_biomass_per_ha
amount_carbon_sequestered <- total_biomass_increase*biomass_conversion_factor

To compute the effect of risks on enrichment planting and management, we first establish the linkage between improved income from beekeeping and forest management. To do this we determine whether the beekeeping intervention promotes resilience during periods of economic stress i.e positive income during stress periods would indicate that the intervention minimizes encroachment. A negative income shows a high likelihood of encroachment during stress periods.

beekeeping_resilience <- (drought_event+unrest_event)*beekeeping_profits
intervention_resilience <- as.numeric(beekeeping_resilience<0)

The resilience computation helps to estimate the chance of encroachment using the chance_event function and the expected loss. The same applies to natural risks (e.g fire outbreak, tree diseases etc) and market risks (e.g unstable carbon markets).

loss_due_encroachment <- chance_event(chance_encroachment,value_if=loss_encroachment,
                                      value_if_not=0,n=n_years,CV_if=general_CV,CV_if_not=0,
                                      one_draw=FALSE)
loss_due_invasive_species <-  chance_event(chance_invasive_species,
                                           value_if=(percentage_trees_lost_prosopis/100),
                                           value_if_not= 0,n_years,general_CV, CV_if_not = 0,
                                           one_draw = FALSE)
loss_due__tree_disease <- chance_event(chance_diseases_outbreak,
                                       value_if=(percentage_tree_loss_diebark/100),
                                       value_if_not = 0,n_years,general_CV,CV_if_not = 0,
                                       one_draw = FALSE)
tree_loss_due_fire <- wildfire_event*vv(percentage_trees_lost_fire/100,general_CV,n_years)
effect_forest_risks <- sapply(c(loss_due_encroachment+loss_due__tree_disease+
                                loss_due_invasive_species+tree_loss_due_fire),
                                function(x) min(x,1))
actual_amount_carbon_sequestered <- amount_carbon_sequestered*(1-effect_forest_risks)

Since carbon prices are highly volatile, and in some cases don’t materialize, we account for this risk by considering the chance that markets don’t materialize, then introduce variation in the carbon prices as shown.

carbon_market_failure <- chance_event(carbon_market_risk,value_if=1,n=n_years)
price_of_carbon <- vv(carbon_cost,general_CV,n_years)*carbon_market_failure

Revenue from carbon sequestration would therefore be computed as shown below.

carbon_revenue <- actual_amount_carbon_sequestered*price_of_carbon

The cost of enrichment planting and management would also be a one-off cost of tree seedlings and tree planting labour cost. A running cost to manage trees (weeding, pruning, etc) would be added as a recurring cost.

initial_enrichment_cost <- c((tree seedling_cost+tree_planting_labour_cost), rep(0,n_years-1))
enrichment_recurring_cost <- (vv(weeding_cost,general_CV,n_years)+
                              vv(pruning,general_CV,n_years)+
                              vv(pesticide_cost,general_CV,n_years)+
                              vv((bee_cost_labour_per_season),general_CV,n_years)+
                              vv(hive_maint_mat,general_CV,n_years))*n_annual_honey_harvests
enrichment_planting_cost <- initial_enrichment_cost+enrichment_recurring_cost
carbon_profits <- carbon_revenue-enrichment_planting_cost

To calculate net present values (NPV) for interventions, we apply the discount function from the decisionSupport package to discount the expected profits per intervention over a specified period of time (e.g 25 years)

beekeeping_NPV<-discount(beekeeping_profits,discount_rate,calculate_NPV=TRUE)
enrichment_benefit_NPV<-discount(carbon_profits,discount_rate,calculate_NPV=TRUE)
  
return(list(cashflow_beekeeping_NPV=beekeeping_profits,              
            beekeeping_NPV=beekeeping_NPV,
            cashflow_enrichment_benefit_NPV=carbon_profits,              
            enrichment_NPV=enrichment_benefit_NPV))}

Simulation of intervention outcomes

To run the model, the Desaa_restoration function, along with the data from the input_table, were fed into the Monte Carlo simulation (MC) function (Luedeling and Whitney, 2018) to conduct the full analysis. Below is the code we used to perform the Monte Carlo simulation with 10,000 model runs.

Desaa_restoration<-function(x, varnames){

decisionSupport(inputFilePath = input_table, #input file with estimates
                outputPath = results_folder, #output folder
                welfareFunction = Desaa_restoration, #the function created above
                numberOfModelRuns = 1e4, #10,000 model runs
                functionSyntax = "plainNames", 
                write_table = TRUE,)}

Value of information and sensitivity analysis

To identify important knowledge gaps where further measurements efforts could help clarify whether the predicted outcome would have negative or positive impact, We compute the expected value of perfect information (EVPI) (Boncompte, 2018). EVPI represents the opportunity loss that could be incurred by a decision-maker due to lack of information on a specific variable (Hubbard, 2014). Applied in this way, the EVPI computation can help to determine where further measurements may help gain clarity on decision outcomes. We also applied Partial Least Squares (PLS) regression analysis to the MC simulation results to generate the Variable-Importance-in-the-Projection (VIP) statistic for input parameters (Luedeling & Gassner, 2012).The VIP statistic represents the direction and strength of each input variables relationship with the output variable (Wold et al. 2001).

Results

A tabulated summary of the results describing the 90% confidence interval of net present values of the beekeeping intervention and enrichment planting and management intervention are as shown below.

Monte Carlo and VOI outputs
NPV..n.10.000. X X.1 X.2
Intervention Min (5%) Median (50%) Max(95%)
Beekeeping NPV 1594 4517.24 10961
Enrichment planting NPV -492 3211.62 13852

Also shown is the combined graphical representation of the outcomes (beekeeping, enrichment planting and management), value of information analysis, cash flows and sensitivity analysis.

Figure 3: Projected impact of the decision to invest in beekeeping (a), high decision-value variables (b), the respective cashflows (c) and important variables (determined by VIP analysis of PLS regression models) (d). The results were produced through MC simulation (10,000 model runs) of sheep rearing intervention performance over 25 years. In the PLS plot, green bars indicate positive correlations of uncertain variables with the outcome variable, while red bars indicate negative correlations. Blue bars indicate variables that did not meet the threshold for model sensitivity analysis.

Figure 4: Projected impact of the decision to invest in enrichment tree planting and management (a), high decision-value variables (b), the respective cashflows (c) and important variables (determined by VIP analysis of PLS regression models) (d). The results were produced through MC simulation (10,000 model runs) of enrichment tree planting intervention performance over 25 years. In the PLS plot, green bars indicate positive correlations of uncertain variables with the outcome variable, while red bars indicate negative correlations. Blue bars indicate variables that did not meet the threshold for model sensitivity analysis.

Discussion and Conclusion

Clear definition of interventions and expected outcomes can help develop decision impact pathways in complex systems. This enables project proponents to identify plausible cost, benefit and risk variables. The process provides realistic estimates of the plausible ranges of returns of interventions, considering all outcome dimensions that are relevant in a particular context. To realistically value ecosystem benefits, landscape restoration actors should base their predictions on expert knowledge of the local context rather than on benchmark estimates carried over from different contexts (Stalhammar & Pedersen, 2017). The use of distributions when estimating the value of variables rather than best-bet estimates avoids overly hopeful predictions that could misguide planning (Luedeling et al. 2019). Klein’s Pre-mortem (Klein, 2007) and the equivalent-bet technique (Freund, & Jones, 2015), have been proven to measurably improve an expert’s ability to provide accurate estimates (Hubbard, 2014). However, modeling linkages between forest conservation and improved livelihoods is still a challenge. The process requires making spurious assumptions (e.g assuming that increasing income reduces forest encroachment) that my not reflect the situation on the ground. Value of Information analysis can provide indications of what needs to be measured to support intervention decisions. While many uncertainties usually exist in all decisions that affect complex systems, only those uncertainties that are of value to the decision maker should be prioritized for further measurement. This can substantially reduce the cost of data collection aimed at informing decisions.

References

Boncompte, M. (2018). The expected value of perfect information in unrepeatable decision-making. Decision Support Systems. https://doi.org/10.1016/j.dss.2018.03.003

Cifuentes Jara, M., Henry, M., Rejou-Mechain, M., Wayson, C., Zapata-Cuartas, M., Piotto, D., . Westfall, J. (2015). Guidelines for documenting and reporting tree allometric equations. Annals of Forest Science. https://doi.org/10.1007/s13595-014-0415-z

Freund, J., & Jones, J. (2015). Chapter 5 - Measurement. Measuring and Managing Information Risk. https://www.sciencedirect.com/book/9780124202313/measuring-and-managing-information-risk

Henry, M., Bombelli, A., Trotta, C., Alessandrini, A., Birigazzi, L., Sola, G., . Saint-Andre, L. (2013). GlobAllomeTree: International platform for tree allometric equations to support volume, biomass and carbon assessment. IForest. https://doi.org/10.3832/ifor0901-006

Hubbard, D.W., 2014. How to Measure Anything: Finding the Value of Intangibles in Business. https://doi.org/10.1002/9781118983836

Klein, G., 2007. Performing a project premortem.Harv. Bus. Rev., 85 (2007), p. 18 Available online at: https://hbr.org/2007/09/performing-a-project-premortem

Luedeling, E., Whitney, C.W., 2018. R package decisionSupport: Controlled burns in conifer forests. Retrieved from https://cran.r-project.org/web/packages/decisionSupport/vignettes/wildfire_example.html

Luedeling, E. and Gohring, L. (2017). decisionSupport: Quantitative Support of Decision Making under Uncertainty, CRAN archive. Retrieved from https://cran.r-project.org/web/packages/decisionSupport/

Luedeling, E., Gassner, A., 2012. Partial Least Squares Regression for analyzing walnut phenology in California. Agric. For. Meteorol. https://doi.org/10.1016/j.agrformet.2011.10.020

Luedeling, E., & Shepherd, K. (2016). Decision-Focused Agricultural Research. Solutions, 7(5), 46-54. Retrieved from https://www.thesolutionsjournal.com/article/decision-focused-agricultural-research/

Stalhammar, S., & Pedersen, E. (2017). Recreational cultural ecosystem services: How do people describe the value? Ecosystem Services. https://doi.org/10.1016/j.ecoser.2017.05.010

Tamba, Y., Wafula, J., Whitney, C., Luedeling, E., Yigzaw, N., Negussie, A., . Aynekulu, E. (2021). Stochastic simulation of restoration outcomes for a dry afromontane forest landscape in northern Ethiopia. Forest Policy and Economics. https://doi.org/10.1016/j.forpol.2021.102403

Wafula J, Karimjee Y, Tamba Y, Malava G, Muchiri C, Koech G, De Leeuw J, Nyongesa J, Shepherd K and Luedeling E (2018) Probabilistic Assessment of Investment Options in Honey Value Chains in Lamu County, Kenya. Front. Appl. Math. Stat. 4:6. https://doi.org/10.3389/fams.2018.00006

Waliszewski P., Konarski J. (2005) A Mystery of the Gompertz Function. In: Losa G.A., Merlini D., Nonnenmacher T.F., Weibel E.R. (eds) Fractals in Biology and Medicine. Mathematics and Biosciences in Interaction. Birkhäuser Basel. https://doi.org/10.1007/3-7643-7412-8_27

Wold, S., Sjöström, M., Eriksson, L., 2001. PLS-regression: A basic tool of chemometrics, in: Chemometrics and Intelligent Laboratory Systems. pp. 109130. https://doi.org/10.1016/S0169-7439(01)00155-1