| Avoid the use of communication that presents a situation as beyond control and extremely severe, particularly in conjunction with visualizations, to prevent triggering affect heuristics that lead to the overestimation of risks (Harris, Corner, & Hahn 2009 [Study 1, Study 2, Study 3, Study 4, Study 5]; Keller et al. 2006 [Study 3]). |
11201, 11202, 11203, 11204, 11205, 15403 |
3 |
2.75 |
2.17 |
2.64 |
| People often have difficulty correctly interpreting Probability of Precipitation (PoP) forecasts - for example, people often interpret a 30% chance of rain to mean that it will rain 30% of the time. Specifying exactly what forecasts mean in a fluent, accessible, and easily understood way could help to correct this (Abraham et al. 2015; Gigerenzer et al. 2005; Juanchich & Sirota 2018). |
00301, 09301, 14701, 14702, 14703 |
3 |
1.70 |
3.00 |
2.57 |
| In general, it is best to provide an in-text “translation” of verbal statements, preferably right next to the statement in question, in order to anchor the recipients’ interpretation. People do not reliably use guidelines or translation tools unless they are directly in front of them and integrated directly into the communication (Budescu et al. 2009; Budescu et al. 2012; Harris, Por, & Broomell 2017; Wintle et al. 2019). |
03001, 03101, 03102, 11301, 31801 |
3 |
2.50 |
2.00 |
2.50 |
| People often use and interpret words like “improbable” to refer to events that aren’t just unlikely (something like a 10-20% chance, for instance), but nearly impossible (closer to a ~1% chance, for instance). Often, “improbable” is implicitly understood to refer to events that have not happened yet, but have a small chance of happening in the future. Clarifying such terms and providing explicit numeric “translations” could help to clear up these kinds of misunderstandings (Teigen, Juanchich, & Riege 2013). |
29801, 29802, 29803, 29804, 29805 |
3 |
2.25 |
2.20 |
2.48 |
| Use probabilistic information with an awareness that individuals might have difficulty accurately interpreting forecasts, with little difference between experts and members of the general public, depending on a number of different factors, such as ethnic background or numeracy (Kong et al. 1986; Bramwell, West, & Salmon 2006; Harris et al. 2013; Rinne & Mazzocco 2013; Juanchich & Sirota 2016). |
16301, 02601, 11101, 25501, 14801 |
3 |
1.85 |
2.60 |
2.48 |
| Use probabilistic forecasts to better facilitate trust and decision making (Nadav-Greenberg & Joslyn 2009 [Study 1, & 2]; Roulston & Kaplan 2009 [Study 1 & 2]; Joslyn & LeClerc 2012 [Study 1, 2, & 3]; S.L. Joslyn & Grounds 2015 [Study 1, & 2]; Grounds & Joslyn 2018 [Study 1, & 2]). |
21301, 21302, 25701, 25702, 13801, 13802, 13803, 26201, 26202, 09801, 09802 |
3 |
1.98 |
2.45 |
2.48 |
| Use high probability estimates when positively framing a scenario to increase the perceived magnitude of the positive benefits (Flugstad & Windschitl 2003 [Studies 1-3]; Kupor & Laurin 2020 [Study 1, 2, 3, & 4]). |
07001, 07002, 07003, 17201, 17202, 17203, 17204 |
3 |
2.43 |
2.00 |
2.48 |
| Peoples’ thresholds for taking protective action based on a probabilistic forecast appear to be very heterogeneous and highly context-dependent (Kim et al. 2014; Klockow-McClain, McPherson, & Thomas 2020; Morss et al. 2010). |
15701, 15801, 20801 |
2 |
2.42 |
3.00 |
2.47 |
| Use probabilistic forecasts to better facilitate protective action and decision making (Grounds et al. 2017 [Study 1, & 2]; Highhouse 1994; Joslyn et al. 2007; LeClerc & Joslyn 2012; Durbach & Stewart 2011). |
09901, 09902, 11901, 14301, 17401, 06201 |
3 |
1.96 |
2.42 |
2.46 |
| Use visualizations to increase the detection behavior of positive framed forecasts and increase the preventative behavior of negative framed forecasts (Armstrong et al. 2002; Chua et al 2006 [Study 1, & 2]; Garcia-Retamero & Cokely 2011; Harris, Corner, & Hahn 2009 [Study 1, 2, 3, 4, & 5]; Schirillo & Stone 2005). |
00901, 04101, 04102, 07901, 11201, 11202, 11203, 11204, 11205, 26701 |
3 |
2.58 |
1.80 |
2.46 |
| When deciding whether to use a word like “can” (which implies a possibility) or “will” (which implies more certainty), be aware of the “extremity effect”: when shown a probability and asked what “can” happen, people tend to focus on the most extreme possible values, and when asked what “will” happen, they tend to focus on the more likely scenarios. If focusing on more remote possibilities would be counterproductive to the goal of your communication, be wary of what your word choice implies about how certain you are and what risks it draws attention to (Teigen & Filkuková 2013; Teigen, Filkuková, & Hohle 2018; Teigen, Juanchich, & Filkukova 2014). |
29501, 29502, 29503, 29504, 29505, 29601, 29602, 29603, 29604, 29605, 29701, 29702, 29703, 29704 |
3 |
2.21 |
2.07 |
2.43 |
| Use deterministic formats to increase the perceived certainty of the forecast (Joslyn, Savelli, & Nadav-Greenberg 2011 [Study 1, 2, 3 & 4]). |
14601, 14602, 14603, 14604 |
2 |
2.38 |
2.88 |
2.42 |
| Describing small or long-term risks using a “Once in X years” expression can mislead people by creating the false perception that an event is “overdue”, but this also may make it more persuasive than the “annual percent chance” format (Grounds, LeClerc, & Joslyn 2018; Bell & Tobin 2007). |
09801, 09802, 01901 |
2 |
2.25 |
3.00 |
2.42 |
| When comparing between probability and frequency formats, there are no significant benefits to one over the other (Hendrickx et al 1989; Joslyn et. al 2009i [Study 1 & 2]; Neace et. al 2008 [Study 1]; Knapp et. al 2016; Evans et. al 2000 [Study 1, 2 & 3]; Ruiz et al. 2013; Strathie et al. 2017). |
11501, 14401, 14402, 21701, 15901, 06601, 06602, 06603, 26001, 28801 |
3 |
1.83 |
2.35 |
2.39 |
| Forecasts are not interpreted in isolation; non-experts interpret recent forecasts in light of what previous forecasts have said. A “moderate” risk will cause more worry if it has been upgraded from a “low” risk than if it has been downgraded from a “high” risk, for instance (Sigrid, Moyner, Hohle, & Teigen 2015; Løhre 2018 [Studies 2-5]; Windschitl & Weber 1999 [Study 4]). |
27401, 18302, 18303, 18304, 18305, 31604 |
3 |
1.92 |
2.17 |
2.36 |
| The vast majority of people are intuitively aware that even deterministic forecasts are inherently uncertain (Savelli & Joslyn 2012; Joslyn & Savelli 2010; Morss et al. 2008; 2010). |
26401, 14101, 20701, 20801 |
2 |
2.06 |
3.00 |
2.35 |
| Use caution when relying on frequency formats alone to reduce interpretation errors because reference class tends to interact with formats, negating the benefit of using frequencies in place of probabilities (Evans et. al 2000 [Study 1, 2 & 3]; Cuite et al. 2008; Neace et. al 2008 [Study 1, 2, 3, 4 & 5]; Joslyn et. al 2009i [Study 1 & 2]; Knapp et. al 2016). |
06601, 06602, 06603, 04601, 21701, 21702, 21703, 21704, 21705, 14401, 14402, 15901 |
3 |
1.85 |
2.17 |
2.34 |
| Motivated reasoning and general worldview can have a substantial impact on how people interpret uncertainty information, particularly in how they interpret the likelihood of various points within a confidence interval (Dieckmann et al 2017). |
05301, 05302 |
2 |
3.00 |
2.00 |
2.33 |
| Forecasts giving the risk over a long period of time result in higher risk perceptions than forecasts giving the risk in a given year (Keller et al. 2006). |
15401, 15402, 15403 |
2 |
2.00 |
3.00 |
2.33 |
| Use deterministic visualizations to increase protective actions, use probabilistic visualizations to increase understanding in change in risk (Ash et al. 2014; Miran et al. 2019). |
01001, 20601 |
2 |
2.00 |
3.00 |
2.33 |
| Use pictographs to increase understanding of probabilistic information (Dowen et al. 2017; Han et al. 2011 [Study 1, & 2]; Hawley et al. 2008; Leonhardt & Keller 2018). |
06001, 10701, 10702, 11401, 17801 |
3 |
1.90 |
2.10 |
2.33 |
| Bar charts are most useful for reducing cognitive effort when conveying information to people over age 50 or with low-numeracy (Waters et al. 2006; Feldman-Stewart et al. 2007; Garcia-Retamero & Galesic 2010b; Garcia-Retamero, Cokely, & Hoffrage 2015; Shah & Freedman 2011). |
31101, 06701, 08501, 08701, 27001 |
3 |
2.10 |
1.90 |
2.33 |
| Providing additional information to visualizations can reduce common misunderstandings but also lead to lower risk perceptions (Boone, Gunalp & Hegarty 2018 [Study 1, & 2]; Cheong et al. 2016 [Study 1, 2, & 3]; Fraenkel et al. 2018; Gaissmaier et al. 2012). |
02501, 02502, 03601, 03602, 03603, 07201, 07601 |
3 |
1.61 |
2.36 |
2.32 |
| The size of a confidence interval can influence how people interpret a forecast - people understand smaller confidence intervals to mean that a forecast is more technologically advanced and more certain (Teigen, Løhre, & Hohle 2018). |
29901, 29902, 29903, 29904, 29905 |
3 |
2.35 |
1.60 |
2.32 |
| Including a center track line in weather forecasts to increases people’s level of concern (Meyer et al. 2013; Newman & Scholl 2012 [Study 1, 2, 3, 4, & 5]; Padilla, Ruginski, & Creem-Regehr 2017; Ruginski et al. 2016). |
20101, 21801, 21802, 21803, 21804, 21805, 22601, 25901 |
3 |
1.88 |
2.06 |
2.31 |
| Peoples’ interpretations of verbal probability statements in the absence of a translation tool tend to regress towards a 50/50 chance (Budescu et al 2009; Budescu et al 2012; Budescu et al 2014). |
03001, 03101, 03102, 03201 |
2 |
2.88 |
2.00 |
2.29 |
| When verbally communicating forecasts that are vague or have high uncertainty, be aware that people often implicitly understand more uncertain probabilistic information (i.e. information having a larger confidence interval rather than a smaller one) as signalling a higher likelihood and more severe outcomes, independent of the probabilistic information itself (Løhre 2018; Løhre, Juanchich, et al. 2019). |
18301, 18306, 18501, 18502, 18503, 18504, 18505 |
3 |
1.86 |
2.00 |
2.29 |
| In scenarios where underestimating risks could have disastrous consequences, it may be better to provide a number first, then use the verbal expression as a translation (opposite to the way the IPCC reports “translate” their uncertainty communications, for example) in order to minimize the “extremity effect” (Dilla & Stone 1997; Jenkins, Harris, & Lark 2018, Jenkins Harris, & Lark 2019; Patt & Dessai 2005; Windschitl et al. 2017 [Study 1 & 2]). |
05701, 13201, 13301, 22901, 23001, 23002 |
3 |
1.92 |
1.92 |
2.28 |
| Use of 1 in X formats instead of X in NX formats to increase risk understanding (Grimes & Snively 1999; Pighin, et al. 2015; Oudhoff & Timmermans 2015; Denes-Raj, Epstein, & Cole 1995 [Study 1, 2 & 3]; Bell & Tobin 2007; Carey et al. 2018; Pighin et al. 2011). |
09601, 24101, 22301, 04901, 04902, 04903, 01901, 03501, 24201, 24202, 24203, 24204 |
3 |
1.71 |
2.08 |
2.26 |
| Advantageous use of frequency and percentage formats for accuracy of risk comprehension are dependent given the task at hand and the context of the risk (Cuite et. al 2008; Knapp et. al 2009 [Study 1 & 2]; Sinayev et al. 2015; Wallsten et al. 1986). |
04601, 23501, 23502, 27601, 30901 |
3 |
1.75 |
2.00 |
2.25 |
| When using visualizations, women report higher levels of anxiety from visualizations (Lindner & Alsheimer 2019; Zikmund-Fisher, Fagerlin, & Ubel 2008). |
18101, 32401 |
2 |
2.25 |
2.50 |
2.25 |
| Use predictive interval graphics to encourage protective action (S. Joslyn, Nemee, & Savelli 2013 [Study 1, & 2]). |
26101, 26102 |
2 |
1.75 |
3.00 |
2.25 |
| Use deterministic visualizations to increase protective action, and use probabilistic design to better communicate changes in risk (Ash et al. 2014; Baker 1995; Marimo et al. 2015). |
01001, 01301, 19201 |
2 |
2.00 |
2.67 |
2.22 |
| Use PoP forecasts with icons with caution as the type of icon does not appear to improve accuracy of understanding (Joslyn et al 2009ii [Study 1, 2, &3]). |
14501, 14502, 14503 |
2 |
1.67 |
3.00 |
2.22 |
| Use loss-based framing to increase risk perception and encourage detection-related behavior (Grimm et al. n.d.; Banks et al. 1995; Chua et al. 2006 [Study 1]; Garcia-Retamero & Cokely 2011; Teigen & Brun 2003 [Study 1, & 4]). |
09701, 01501, 04101, 07901, 29401, 29402, 29403, 29404 |
3 |
1.72 |
1.94 |
2.22 |
| Avoid including uncertainty and ambiguity with loss-based framing to encourage risk mitigation action (Morton et al. 2011; Levin, Snyder, & Chapman 1998; Windshitl & Weber 1999; Keren & Gerritsen 1999). |
21001, 21002, 17901, 31601, 31602, 31603, 31604, 15601 |
3 |
1.72 |
1.94 |
2.22 |
| Individuals tend to anchor to the severity of warnings regardless of order issued (Losee et al. 2017 [Study 1 & 2]). |
18801, 18802 |
2 |
1.62 |
3.00 |
2.21 |
| Many relevant professionals and forecasters express a desire to want to “boil down” probabilistic information into a deterministic forecast (Kox et al. 2015; Pappenberger et al. 2013). |
16401, 22701 |
2 |
1.62 |
3.00 |
2.21 |
| Don’t assume that a verbal probability term or statement will be interpreted the same way by everyone, as there is strong evidence that people interpret such terms much more ambiguously and heterogeneously than most realize. Verbal expressions describing probabilities close to 0%, 50%, and 100% are the least prone to such misunderstandings (Amer, Hackenbrack, and Nelson 1994; Beyth-Marom 1982; Brun & Teigen 1988; Budescu et al 2009; Budescu et al 2012; Budescu et al 2014; Christopher et al 2010; Christopher and Hotz 2004; Clark et al 1992; Karelitz & Budescu 2004; Kunneman, Stiggelbout, & Pieterse 2020; Rapoport, Wallsten, and Cox 1987; Reagan et al 1989). |
00401, 02101, 02801, 02802, 02803, 03001, 03101, 03102, 03201, 04001, 03901, 04201, 04202, 15201, 15202, 15203, 17001, 24901, 25001 |
3 |
1.93 |
1.68 |
2.21 |
| Very high uncertainty in forecasts leads to lower trust (Joslyn & LeClerc 2012 [Study 1, 2, & 3]). |
13801, 13802, 13803 |
2 |
1.58 |
3.00 |
2.19 |
| Use frequency formats to reduce interpretation errors (Neace et al. 2008 [Study 2, 3, 4, & 5]; Knapp, Raynor, Woolf et. al. 2009; Cuite et al. 2008). |
21702, 21703, 21704, 21705, 16001, 04601 |
3 |
1.75 |
1.83 |
2.19 |
| Icon arrays presented in sequential order can increase understanding and risk avoidance actions (Ancker et al. 2011i; Galesic et al. 2009 [Study 1, &2]; Garcia-Retamero, Galesic, & Gigerenzer 2011; Garcia-Teramero et al. 2010 [Study 1, & 2]; Stone, Yates, & Parker 1997 [Study 1, 2, & 3]; Taylor, Stevenson, & McDowell 2018; Witteman et al. 2014; Zikmund-Fisher et al. 2014; Zikmund-Fisher, Dickson, & Witteman 2011). |
00501, 07801, 07802, 08801, 08901, 08902, 28701, 28702, 28703, 29001, 31901, 32201, 32301 |
3 |
2.02 |
1.50 |
2.17 |
| Verbal analogies can be distracting, lowering probability estimates and leading individuals to underestimate or even overlook risks (Barilli et al. 2010 [Study 1 & 2]). |
01601, 01602 |
2 |
2.50 |
2.00 |
2.17 |
| People generally prefer mixed formats (e.g. a numeric probability and a verbal probability expression together, or a number and a visualization) for probabilistic risk information (Carey et al. 2018; Connelly & Knurth 1998; Dorval et al. 2013; Fortin et al. 2001; Hill et al. 2010; Sink 1995; Zabini et al. 2015). |
03501, 04301, 05901, 07101, 12001, 27701, 32101 |
3 |
1.57 |
1.93 |
2.17 |
| Use probabilistic information to gain higher trust in forecasts (Joslyn & LeClerc (2016) [Study 1, Study 2]; Joslyn and Demnitz (2019) [Study 1, Study 2]. |
13901, 13902, 14201, 14202 |
2 |
2.50 |
2.00 |
2.17 |
| Including a center track line in weather forecasts has no effect on interpretation (Van Pelt et al. 2015; Wu et al 2014). |
30701, 32001 |
2 |
1.50 |
3.00 |
2.17 |
| People often implicitly interpret verbal probability expressions as more likely when they describe more severe or undesirable outcomes (“severity bias”). For instance, someone who interprets a “slight chance” of rain showers to mean a 1-5% chance will likely interpret a “slight chance” of a hurricane to mean something closer to a 10-15% chance (Bonnefon & Villejoubert 2006; Fischer & Jungermann 1996; Harris & Corner 2011; Weber & Hilton 1990). |
02401, 06901, 11001, 11002, 11003, 31201, 31202, 31203 |
3 |
1.50 |
1.94 |
2.15 |
| Avoid the use of uncertainty and/or ambiguity with probabilistic information to reduce the chances of misinterpreting forecasts (Han et al. 2011 [Study 1]; Gibson et al. 2013). |
10701, 09101 |
2 |
2.38 |
2.00 |
2.12 |
| Purely verbal expressions of risk with no “translation” provided should be avoided. If for some reason you must choose between providing purely verbal or purely numeric information, numeric information should be prioritized because it is interpreted more consistently. There is also some evidence that people tend to be more comfortable with and trusting of purely numeric information as opposed to purely verbal information, and that purely verbal statements often lead people to overestimate risks (Budescu, Weinberg, & Wallsten 1988; Gurmankin et al. 2004; Knapp, Raynor, Woolf et. al. 2009; P. Knapp, Raynor, & Berry 2004). |
03401, 03402, 10101, 16001, 22501 |
3 |
1.55 |
1.80 |
2.12 |
| Be mindful of the “reference points” and “directionality” of your verbal probability statements - you could be accidentally implying that your forecasts are higher or lower than “expected” and indirectly influencing how people interpret your communication. Generally, positively directional statements (such as “it is likely that it will rain”) draw attention to the possibility of an event happening and cause people to interpret such statements as more likely, while negatively directional statements (such as “it is unlikely that there will be clear skies”) have the opposite effect. Contextual factors can affect this as well (Honda & Yamagishi 2006; 2009; 2017; McKenzie & Nelson 2003; Teigan & Brun 1995; 1999; 2000; 2003, McKenzie & Nelson 2003; Wallsten et al. 1986; Budescu, Karelitz, & Wallsten 2003). |
12401, 12402, 12501, 12301, 12302, 12303, 12304, 19801, 19802, 19803, 29101, 29102, 29103, 29104, 29201, 29202, 29203, 29301, 29302, 29401, 29402, 29403, 29404, 30901, 03301 |
3 |
1.66 |
1.68 |
2.11 |
| Use foreground displays to increase risk perceptions but with the knowledge that effects are contingent on depicted probability sizes, labels, and risk reduction level associated with protective action (Okan et al. 2020; Okan, Stone, & Bruine de Bruin 2018; Stone et al. 2017). |
22001, 22101, 28401 |
2 |
2.50 |
1.83 |
2.11 |
| Avoid bar charts as they are associated with increased misunderstandings (Correll & Gleicher 2014 [Study 1, 2, & 3]; Dieckman et al. 2015[Study 3]; Schapira, Nattinger, & McAuliffe 2006; Newman & Scholl 2012[Study 6]). |
04401, 04402, 04403, 05203, 26601, 21806 |
3 |
1.83 |
1.50 |
2.11 |
| Avoid the use of negative framing with less numerate audiences and in situations involving specific risks with high uncertainty in order to mitigate its detrimental effect on decision making (Kuhn 1997 [Study 1 & 2]; Morton et al. 2011 [Study 1 & 2]; Peters & Levin 2008; Armstrong et al. 2002; Peters et al. 2006). |
16801, 16802, 21001, 21002, 23801, 00901, 23901 |
3 |
1.75 |
1.57 |
2.11 |
| Use icon array visualizations to increase risk comprehension and understanding of probabilistic information (Tubau et. al 2019 [Study 1 & 2]; Galesic et. al 2009 [Study 1 & 2]; Garcia-Retamero et. al 2010 [Study 1 & 2]; Garcia-Retamero 2009; Keller & Siegrist 2009; Schirillo & Stone 2005; Garcia-Retamero & Cokely 2014; Garcia-Retamero & Galesic 2010a). |
30301, 30302, 07801, 07802, 08901, 08902, 15301, 26701, 08001, 08401 |
3 |
1.95 |
1.35 |
2.10 |
| Avoid reliance on any single format as there is no “best visualization format” for conveying risk (Barnes et al. 2016; Bisantz, Marsiglio, & Munch 2005 [Study 1, 2, 3, & 4]; Etnel et al. 2020; Garcia-Retamero & Dhami 2013; Kreye et al.2012; Lorenz et al. 2015; Sanyal et al. 2009). |
01701, 02201, 02202, 02203, 02204, 06501, 08201, 16501, 18701, 26301 |
3 |
1.35 |
1.95 |
2.10 |
| Including probabilistic / uncertainty information increases trust in expert forecasts. (Nakayachi, Johnson, & Koketsu 2018; Howe et al 2019). |
21501, 12701 |
2 |
2.50 |
1.75 |
2.08 |
| While interpretations of verbal probability statements vary substantially from person to person, they appear to be largely stable over time for each individual (Budescu & Wallsten 1985; Clarke et al 1992; Karelitz & Budescu 2004; Rapoport, Wallsten & Cox 1987). |
02901, 04201, 04202, 15201, 15202, 15203, 24901 |
3 |
1.64 |
1.57 |
2.07 |
| Purely verbal probability statements regarding politically or socially sensitive issues (Climate Change, for instance) are more prone to being misinterpreted due to motivated reasoning on the part of the listeners due to their ambiguity (Budescu et a l 2012; Piercey 2009; Smits & Hoorens 2005). |
03101, 03102, 24001, 28101 |
2 |
2.69 |
1.50 |
2.06 |
| Both experts and non-experts’ interpretations of forecasts tend to be skewed towards the end of the forecast period, even in short-term forecasts (i.e. if there were an X% chance that a given event would occur sometime in a given week, people will, on average, perceive that the event is more likely to happen on Friday than on Monday) (Doyle et al. 2014; McClure, H. Doyle, & Velluppillai 2015; Morss et al. 2016). |
06101, 19601, 20901 |
2 |
1.83 |
2.33 |
2.06 |
| Use gain-based framing to encourage risk avoidance and preventive behavior (Garcia-Retamero & Cokely, 2011; Wernsted et al. 2019). |
07901, 31401 |
2 |
2.12 |
2.00 |
2.04 |
| People do not always recognize uncertainty, and often express a preference for visualizations to aid understanding (Johnson & Slovic 1995; 1998). |
13401, 13501 |
2 |
1.88 |
2.25 |
2.04 |
| Use Red / Orange / Green (from most to least uncertain) to most intuitively convey risk (Elmqvist, Hlawitschka, & Kennedy n.d.; Miran et al. 2016; Retchless & Brewer 2016; Sherman-Morris, Antonelli, & Williams 2015). |
06301, 20501, 25101, 27201 |
2 |
1.56 |
2.50 |
2.02 |
| If combined expressions are used, it remains important to use the correct verbal risk descriptor that is interpreted by people in the same way as the numerical expression that is associated with it. Verbal risk descriptors as a whole mislead rather than inform, leading readers to greatly overestimate their risk of side effects (Webster, Weinman, and Rubin 2017). |
31301 |
1 |
3.00 |
2.00 |
2.00 |
| People generally prefer probabilistic forecasts to deterministic ones (Morss et al. 2008). |
20701 |
1 |
2.00 |
3.00 |
2.00 |
| Use probabilistic visualizations for people familiar with them to lead to informed decisions (Fundel et al. 2019). |
07501 |
1 |
2.00 |
3.00 |
2.00 |
| Use probabilistic information to foster higher levels of confidence in forecasts (Hanrahan & Sweeney 2013). |
10901 |
1 |
2.00 |
3.00 |
2.00 |
| Use a margin of error chart to best convey uncertainty (Nadav-Greenberg, Joslyn, & Taing 2008 [Study 1, & 2]). |
21401, 21402 |
2 |
1.00 |
3.00 |
2.00 |
| Display uncertainty as a uniform circle to reduce miscommunications (McKenzie et al. 2016; Mulder et al 2019; Severston 2015). |
19901, 21101, 26901 |
2 |
2.25 |
1.67 |
1.97 |
| Using positively directional verbal probability statements might be more effective at spurring people to take precautionary actions (Teigen and Brun 1999). |
29201, 29202, 29203 |
2 |
2.08 |
1.83 |
1.97 |
| People intuitively seek out usable comparisons when interpreting probabilistic information (e.g. when asked to estimate the prevalence of a particular disease among women, people intuitively base their judgements on the given prevalence of the disease in men, even if the two are explicitly said to be unrelated). These comparisons combine with what we typically think of as objective probabilistic reasoning to shape feelings and attitudes about risks (P.D. Windschitl, Martin, & Flugstad 2002). |
22401, 22402, 22403, 22404 |
2 |
2.12 |
1.75 |
1.96 |
| Use messaging that communicates positive emotions to produce higher risk perceptions when expressed in frequency formats (L. Wu, Zeng, & Wu 2018 [Study 2, Study 3]). |
17302, 17303 |
2 |
1.88 |
2.00 |
1.96 |
| Giving risk probability statements a positive affect and a frequency format, leads to increased risk perceptions (Wu, Zeng, & Wu 2018 [Study 1, 2 & 3]). |
17301, 17302, 17303 |
2 |
1.83 |
2.00 |
1.94 |
| Though confidence intervals are usually meant to denote that probabilities are normally distributed around a mean, most likely value, a plurality of people understand confidence intervals to mean that all values within the range were equally likely, an interpretation which is often incorrect. Clarifying how confidence intervals should be interpreted could help to offset this misunderstanding (Dieckmann et al 2015). |
05201, 05202, 05203 |
2 |
1.83 |
2.00 |
1.94 |
| Use interactive formats to improve understanding of probabilistic inferences (Hogarth and Soyer 2015; Natter and Berry 2005 [Study 1 & 2]). |
12201, 21601, 21602 |
2 |
2.17 |
1.67 |
1.94 |
| The contextual effects of base-rate and severity bias are still present even when a precise, numeric version of the forecast is given as well (Windschitl & Weber 1999). |
31601, 31602, 31603 |
2 |
1.58 |
2.17 |
1.92 |
| Use experience-based formats to reduce the tendency for individuals to overrate rare risks and rely on emotions for decision-making (Tyszka & Sawicki 2011 [Study 1 & 2]). |
30401, 30402 |
2 |
2.25 |
1.50 |
1.92 |
| Providing context-rich and easily understood risk comparisons (e.g. “X risky activity is about as risky as smoking a pack of cigarettes every two days”) can help facilitate understanding, particularly with small or long-term risks (Kunreuther, Novemsky, & Kahneman 2001). |
17101 |
1 |
2.75 |
2.00 |
1.92 |
| Individuals who believe that science is a debate rather than a search for truth tend to be more willing to act on uncertain information (Rabinovich & Morton 2012 [Study 1 & 2]). |
24601, 24602 |
2 |
1.25 |
2.50 |
1.92 |
| The increase in trust associated with including uncertainty information may be offset / eliminated by the inclusion of a statement acknowledging the inherent epistemological uncertainty of scientific forecasts. In general, giving people usable information about forecast uncertainty seems to increase trust, but general or open-ended acknowledgements of uncertainty can undermine trust (Howe et al. 2019). |
12701 |
1 |
2.75 |
2.00 |
1.92 |
| Use experienced-based probability formats, especially frequency formats, to lower risk perceptions and worry, thereby reducing emotional effects on understanding and the tendency for individuals to overestimate rare risks (Tyszka & Sawicki 2011 [Study 1, Study 2]). |
30401, 30402 |
2 |
2.25 |
1.50 |
1.92 |
| Use probabilistic information to encourage individuals to have a greater willingness to take risks, decrease the likelihood they default to the “status quo” action, avoid one that would “prevail absent a forecast,” and prevent a “cry wolf” effect commonly found with deterministic information (Bolton & Katok 2018, p. 1453 [Study 1, Study 2]). |
02301, 02302 |
2 |
1.75 |
2.00 |
1.92 |
| Use lighter and finer textures to denote information to improve understanding of visualizations (Leitner & Buttenfield 2000; Miran et al. 2017). |
17701, 20301 |
2 |
1.25 |
2.50 |
1.92 |
| When depicting uncertainty ranges, provide context to avoid heuristic misinterpretation based on perceived probabilities (Tak, Toet, & van Erp 2015). |
28901 |
1 |
1.75 |
3.00 |
1.92 |
| Use simplified maps (e.g. legends, contour lines, white space) to improve understanding of probabilistic information (Gerst et al. 2020). |
09101 |
1 |
2.75 |
2.00 |
1.92 |
| Use of single target probabilities increases perceptions of forecast reliability in comparison to multi-target probability frames (Koehler 2001 [Study 1, 2 & 3]). |
16201, 16202, 16203 |
2 |
1.67 |
2.00 |
1.89 |
| Present both key words and percentages in forecast messages (Coventry & Dalgleish 2015 [Study 1, 2, 3]). |
04501, 04502, 04503 |
2 |
1.17 |
2.50 |
1.89 |
| Less numerate people tend to focus on narrative evidence when evaluating risk communications (the context, their perceptions regarding the likelihood of comparable events, etc.), while more numerate people tend to focus on the stated probability of the risk. It is important to consider both elements in order to effectively reach everyone (Dieckmann, Slovic, & Peters 2009). |
05601, 05602 |
2 |
1.62 |
2.00 |
1.88 |
| Both experts and non-experts’ interpretations of long-term forecasts tend to be skewed towards the end of the forecast period (i.e. if there were an X% chance that a given event would occur sometime in a fifty year period, people will, on average, perceive that the event is more likely to happen in the forty-ninth year than in the first) (Doyle et al. 2014; McClure, H. Doyle, & Velluppillai 2015). |
06101, 19601 |
2 |
1.62 |
2.00 |
1.88 |
| Use probabilistic data to reduce overestimation of risk (Berry, Knapp, and Raynor 2002; Dhami & Wallsten 2005). |
02001, 05101 |
2 |
1.88 |
1.75 |
1.88 |
| Visual aids can help to lower and improve the accuracy of peoples’ perceptions of small or long-term risks, especially when the risks are less extremely small (e.g. a 2% chance vs. a 0.2% chance) and among less numerate people (Fraenkel et al. 2018; Gurmankin et al. 2005). |
07201, 10201, 10202, 10203 |
2 |
2.31 |
1.25 |
1.85 |
| Present forecasts as more severe to increase the perceived likelihood of the risk (A. G. Patt & Schrag 2003). |
00101 |
1 |
1.50 |
3.00 |
1.83 |
| Peoples’ understanding of uncertainty information often depends on what aspects of the forecast are emphasized by a forecaster (Wilson et al. 2019). |
31501 |
1 |
1.50 |
3.00 |
1.83 |
| Both probabilistic and deterministic forecasts are equally effective at minimizing risk exposure (Roulston et al 2006). |
25801 |
1 |
1.50 |
3.00 |
1.83 |
| Include descriptive labels with graphical displays to improve risk understanding in people with low numeracy (Okan et al. 2015). |
21901 |
1 |
2.50 |
2.00 |
1.83 |
| Use of fact box formats are equally effective at facilitating comprehension and short-term knowledge recall as visualizations (McDowell et al 2019 [Study 1, & 2]). |
19701, 19702 |
2 |
1.50 |
2.00 |
1.83 |
| Do not use user self-expressed preferences as a reliable basis for probabilistic information formatting, given that expressed preferences for particular formats over another do not relate to increased risk comprehension (Barnes et. al 2016; Erev & Cohen 1990). |
01701, 06401 |
2 |
1.50 |
1.75 |
1.75 |
| Present data in relative terms rather than in absolute terms to increase risk perception (Malenka et al. 1993; Baron 1997 [Study 1]; Stone, Yates, & Parker 1994). |
19001, 01801, 28601, 28602 |
2 |
1.75 |
1.50 |
1.75 |
| Improve accuracy of multiple, overlapping risk estimates with probabilistic information used in combination with mechanism information (Dawson, Johnson, & Luke 2013). |
04701 |
1 |
2.25 |
2.00 |
1.75 |
| Presenting long-term risks as “time uncertain” (e.g. “X will happen within the next 10-30 years”) rather than “outcome uncertain” (e.g. “There is a 40% chance that X will happen within the next 20 years”) significantly increases risk perceptions and intent to take protective action (Ballard and Lewandowsky 2015). |
01401 |
1 |
2.25 |
2.00 |
1.75 |
| Use probabilistic visualizations to increase protective action (Miran et al. 2018). |
20401 |
1 |
1.25 |
3.00 |
1.75 |
| Use probabilistic hazard indicators to increase protective actions in the short term (Miran et al. 2018). |
20401 |
1 |
1.25 |
3.00 |
1.75 |
| Color and size-based visualizations allow for quicker but less thought out decisions, while texture and icon-based depictions required more time and deliberation (Cheong et al. 2019; Seipel & Lim 2017). |
03701, 26801 |
2 |
1.25 |
2.00 |
1.75 |
| People often implicitly interpret verbal probability expressions as more likely when they describe events with higher base rates (“base rate bias”). Additionally, interpretations of verbal probability statements will be more homogeneous when they describe events with higher base rates; for instance, all else being equal there will be more agreement about what “it is likely that you will give birth to a child with blond hair” means than what “it is likely that you will give birth to twins” means, since the former is a higher-frequency event than the latter (Pepper & Prytulak 1974; Weber & Hilton 1990). |
23101, 31201, 31202, 31203 |
2 |
1.19 |
2.00 |
1.73 |
| The framing of a verbal probability statement affects how people interpret it as well; independent of the directionality of the phrase, people generally interpret verbal probability statements as more likely when they describe the chance of failure (or an undesirable outcome) as opposed to the chance of success (Juanchich et al 2013; Mandel 2015). |
15001, 19101 |
2 |
1.62 |
1.50 |
1.71 |
| Present information as representative to increase trust in forecasts (Windschitl & Weber 1999 [Study 2]; McKenzie & Nelson 2003 [Study 1, 2, & 3]). |
31602, 19801, 19802, 19803 |
2 |
1.38 |
1.75 |
1.71 |
| Use absolute risk formats when aiming to communicate higher levels of urgency and risk perceptions (Galesic et al. 2009 [Study 1]; Zikmund-Fisher, Fagerlin, et al. 2008 [Study 1 & 2]; Ilic, Murphy, & Green 2012). |
07801, 32501, 32502, 12901 |
2 |
1.88 |
1.25 |
1.71 |
| Use percentages instead of fractions to improve understanding of probabilistic information among elderly individuals and encourage protective action (Fuller, Dudley, and Blacktop 2001; Fuller, Dudley, and Blacktop 2002). |
25301, 24501 |
2 |
1.38 |
1.75 |
1.71 |
| Caution is needed in interpreting the differences in membership functions between the selection and evaluation tasks. Comprehension usually involves a receiver who hears some expression and tries to figure out what it might mean and to what it might refer. Vague probability terms (likely, probable, possible, unlikely, and improbable) can cause difficulty in communication due to interpretation (Fillenbaum et al. 1991; Reyna 1981). |
06801, 25201 |
2 |
1.00 |
2.00 |
1.67 |
| Different situations can be perceived differently to participants even when using the same wording when using verbal answer scales (very likely, not as likely). To combat this, frequency formats should be used. The presentation and communication of statistical information in the form of frequencies is more intuitive and understandable than alternative forms of presentation such as probabilities and percentages (Krumpal et al.). |
16601 |
1 |
2.00 |
2.00 |
1.67 |
| The perceived meaning of verbal probability statements varies depending on the profession and credibility of the communicator (hearing “possible” from a doctor denotes a lower chance than hearing it from a news reporter, for instance), but it appears that these context-dependent effects are largely inconsistent from person to person and thus difficult to predict or characterize in aggregate (Brun & Teigen 1988). |
02801, 02802, 02803 |
2 |
1.50 |
1.50 |
1.67 |
| Use of visuals is more effective in communicating probabilistic information with younger audiences (Ulph, Townsend, & Glazebrook 2009). |
30501 |
1 |
2.00 |
2.00 |
1.67 |
| Often, probabilities themselves are easily understood but the events to which they refer are unclear to members of the public. Being as specific as possible about what events are being described should help facilitate better understanding, particularly for Probability of Precipitation (PoP) forecasts (Murphy et al. 1980). |
21201 |
1 |
1.00 |
3.00 |
1.67 |
| Forecasts that are more uncertain tend to elicit more risk-averse behavior (Ramos, van Andel, & Pappenberger 2013). |
24801 |
1 |
1.50 |
2.50 |
1.67 |
| When presented with a choice between two risky options, one of which has a higher absolute number of “good” outcomes (e.g. a 3/50 chance) and one of which has a higher proportion of “good” outcomes (e.g. a 1/10 chance), people will often neglect the denominator and choose the latter, even when they admit to knowing that the odds were better with the former (Denes-Raj & Epstein 1994). |
05001, 05002 |
2 |
2.00 |
1.00 |
1.67 |
| More numerate people tend to have more accurate understandings of risk information, regardless of format (Gardner et al. 2011). |
09001 |
1 |
2.00 |
2.00 |
1.67 |
| Risk ladders and other visualizations can help to improve people’s comprehension of information about very small or long-term risks (Keller et al. 2009). |
15501 |
1 |
2.00 |
2.00 |
1.67 |
| Use probabilistic assessments to decrease support for risky actions and encourage information seeking behavior (Friedman, Lerner, and Zeckhauser (2017) [Experiment 1]). |
07401 |
1 |
3.00 |
1.00 |
1.67 |
| Higher forecast probabilities are viewed as more accurate, regardless of visualization format (Bagchi & Ince 2016 [Study 5]). |
01205 |
1 |
2.00 |
2.00 |
1.67 |
| People generally find verbal probability information more intuitive and easier to use. Numeric probability information, on the other hand, is seen as more precise and elicits more deliberative and reasoned responses (Wallsten et al. 1993; Windschitl & Wells 1996). |
31001, 31701 |
2 |
1.88 |
1.00 |
1.62 |
| Avoid the use of survival curves with audiences unfamiliar with these representations as they tend to lead to misinterpreting probabilistic information (Armstrong et al. 2001; Mazur and Hickam 1990). |
00801, 19401 |
2 |
1.88 |
1.00 |
1.62 |
| Use externally focused expressions “It is X% certain” instead of internally focused expressions “ I am X% certain” to increase perceived accuracy (Lohre & Teigen 2016 [Study 1, 2, & 4]). |
18401, 18402, 18404 |
2 |
1.42 |
1.33 |
1.58 |
| Use ensemble plots to reduce overestimation of probabilities (Toet et al. 2019). |
30201 |
1 |
1.75 |
2.00 |
1.58 |
| Provide a clear definition of PoP in light of confusion over its meaning among both experts and the general public (Stewart et al. 2015). |
28201 |
1 |
1.00 |
2.50 |
1.50 |
| Use interactive visualizations to achieve greater affective impact when communicating probabilistic information (Ancker, Chan, & Kukafka 2009). |
00701 |
1 |
1.50 |
2.00 |
1.50 |
| Avoid the use of negative phrasing because it has a deleterious effect on lay judgment and interpretation of probabilistic expressions (Smithson et al. 2012). |
28001 |
1 |
1.50 |
2.00 |
1.50 |
| Use of pictograms can be helpful when aiming to lower risk perceptions (Keller & Siegrist 2009). |
15301 |
1 |
2.25 |
1.00 |
1.42 |
| Negative verbal modifiers (i.e. “not likely”, “improbable”, etc.) are interpreted as more qualifying and less definitive than positive verbal modifiers (i.e. “very possible”, “quite likely”, etc.) (Reyna 1981). |
25202 |
1 |
1.00 |
2.00 |
1.33 |
| Keep in mind that while absolute risk formats can produce greater accuracy, users among the general public might find such presentation of information overwhelming or even confusing (Peter Knapp et al. 2010). |
23601 |
1 |
2.00 |
1.00 |
1.33 |
| Members of the public generally prefer a “cajoling” tone to a “commanding” tone (Connelly & Knurth 1998). |
04301 |
1 |
2.00 |
1.00 |
1.33 |
| Using a scale format can help to reduce additivity neglect (i.e. the tendency to provide responses totaling to over 100% when asked to estimate the probabilities of an exhaustive list of multiple events) (Riege & Teigen 2013). |
25401 |
1 |
1.75 |
1.00 |
1.25 |
| Differences in individual linguistics must be addressed to properly communicate probabilistic risk (Franic & Pathak 2000). |
07301 |
1 |
1.50 |
1.00 |
1.17 |
| Use visualization of uncertainty to influence decision making among nonexperts (Kübler, Richter, & Fabrikant 2019). |
16701 |
1 |
1.50 |
1.00 |
1.17 |
| Upward revisions to forecasts generally lead individuals to perceive events as closer than downward changes (Maglio & Polman 2016 [Study 10]). |
18901 |
1 |
1.00 |
1.00 |
1.00 |
| Use simplified probabilistic forecast formats rather than full range probabilistic forecasts or deterministic to reduce cognitive overload and confusion and improve decision-making and facilitate increased forecast trust (Durbach & Stewart 2011; Nadav-Greenberg & Joslyn 2009 [Study 1 & 2]; Savelli & Joslyn 2013 [Study 1, 2, & 3]; Joslyn, Nemec, & Savelli 2013 [Study 1 & 2]; Joslyn & Demnitz 2019 [Study 1 & 2]; LeClerc & Joslyn 2012). |
06201, 21301, 21302, 26501, 26502, 26503, 26101, 26102, 14101, 14102, 17401 |
3 |
NA |
NA |
NA |
| Use higher probabilities to increase perceived accuracy and confidence in forecasts (Bagchi & Ince 2016 [Study 1, 2, 3, 4 & 5]; Dieckman, Mauro, & Slovic 2010 [Study 1]; Lohre, Sobkow, et al. 2019 [Study 1, 2, 3, & 4]; Juanchich & Sirota 2017 [Study 1, & 2]). |
01201, 01202, 01203, 01204, 01205, 05501, 18601, 18602, 18603, 18604, 14801, 14802 |
3 |
NA |
NA |
NA |
| Use time uncertain formats for presenting future event probabilities to increase risk perceptions and endorsement of protective decisions (Ballard & Lewandowsky 2015). |
1401 |
1 |
NA |
NA |
NA |
| Use relative risk formats to reduce risk perceptions (Zikmund-Fisher, Ubel, et al. 2008; Hux & Naylor 2016; Malenka et al. 1993; Brase 2002 [Study 1, 2, 3 & 4]; de Bruin et al. 2000). |
32601, 12801, 19001, 02701, 02702, 20703, 02704, 04801 |
3 |
NA |
NA |
NA |
| Use risk ladders with caution because individuals with low subjective numeracy tend to have difficulty understanding the graph without comparative information (Hess, Visschers, & Siegrist 2011 [Study 1, 2, & 3]; Keller et al. 2009). |
11801, 11802, 11803, 15301 |
2 |
NA |
NA |
NA |
| When using a polygonal projection, protective actions are greater in the centroid of the polygon but often require context of radar images for appropriate judgements (Lindell et al. 2016; Jon, Huang, & Lindell 2018; Jon, Huang, & Lindell 2019). |
18001, 13701 |
2 |
NA |
NA |
NA |
| Use maps to communicate risk to audiences with experience using maps in geographic proximity to the threat to encourage higher risk perceptions (Roth 2009; H.-C. Wu, Lindell, & Prater 2015; Sherman-Morris & Del Valle-Martinez 2017). |
25601, 10501, 27101 |
2 |
NA |
NA |
NA |