Relative Change Paradox Studies 1-3
Nathaniel Phillips
4 Oct 2015
Summary
Imagine that you’ve invested some money into a stock market over the course of two years. In the first year, the market increases by 20%. In the second year, the market decreases by 20%. How well did your stock do overall?
The problem
Let \(I\) be the intial investment, \(p_{a}\) be the percentage change in year a and \(p_{b}\) be the percentage change in year b. The final investment value \(J\) is defined as:
\[J = I \times (1 + p_{a}) \times (1 + p_{b})\]
Rearranging terms, we find that the final value J is greater than the initial investment value I if and only if
\[p_{a} > \frac{-p_{b}}{1+p_{b}}\]
Because multiplication is commutative, we can swap the \(p_{a}\) and \(p_{b}\) terms:
\[p_{b} > \frac{-p_{a}}{1+p_{a}}\]
For the rest of this analyses, we will assume that one of the yearly changes is negative and one is positive. We’ll call the loss percengate \(p_{l}\) and the gain percentage \(p_{g}\)
We can graph this relationship as follows. Stimuli in the red region result in cumulative losses (J < I), while those in the blue region result in gains (J > I). Points on the black line indicate no change (J = I). Additionally, we’ve added points for the three stimuli in study 1
Materials and Procedure
Participants read the following instructions:
The main portion of this HIT consists of 4 separate stock market scenarios.
In each stock market scenario, you will imagine that you are investing a fixed initial amount of money in a new stock market for a period of two years.
We will tell you how each market performed in the form of a percentage increase or decrease in the first year followed by the second year and ask you to estimate how well your investment in each stock market changed overall from your initial investment until the end of the second year.
Please only use the information we explicitly mention in each market. Do not take into account any additional financial information (such as inflation, fees, transaction costs, opportunity costs, etc.).
Each market scenario is completely independent of the others and has exactly one true answer. For each correct answer you will earn a bonus of 10 cents.
After reading the instrucitons, the first stimulus screen was displayed. Each stimulus screen contained the text “Assume you invested [INVESTMENT] in a stock market. In the 1st year, the stock market [YEAR1]. In the 2nd year, the stock market [YEAR2]. At the end of the 2nd year, you sell all your stocks.” The values in the open brackets changed depending on the stimulus and the experimental condition. A screenshot of an example stimulus is displayed here:
In studies 1 and 2, all text was black. In study 3, the text for stock market increases (e.g.; “increased by 30%”) was blue, while the text for stock market decreases (e.g.; “decreased by 30%”) was red.
After reading the main stimulus text, participants had one or two responses depending on their experimental condition. All participants gave a categorical response where they indicated whether their hypothetical investment lost money, gained money, or neither lost nor gained money. Participants in the Numerical Estimate conditions were subsequently asked to indicate how much money their hypothetical investment gained or lost.
After completing all four stimuli, participants completed a dynamic version of the Berlin Advanced Numeracy Test (ANT) and a brief demographic survey.
Study 1
Stimuli
All participants started with a $100 investment in each scenario. We originally had 4 stimuli in the experiment (see following table). However, we decided to exclude stimuli 4 from our analyses. The reason for this is because stimulus 4 was designed to have a true change of 0% - however, because the intended value of 33.33% was rounded to 33%, the true change was actually a very small loss of -0.25%. Because this very small change could lead to confusion, we ignored this stimulus.
| Stimuli | Percentage Up | Percentage Down | True Change |
|---|---|---|---|
| 1 | 20% | 10% | +8% |
| 2 | 50% | 40% | -10% |
| 3 | 30% | 30% | -9% |
| 4 (deleted) | 33% | 25% | -0.25% (~0%) |
8 Conditions
We had 2 between-participants independent variables resuling in 4 experimental conditions: 2 (Change order: Up-Down, Down-Up) x 2 (Numerical estimate: Yes, No).
Aggregate results from Study 1 are presented in Table XX
| start_amount | numEstCond | change.cond | s1.down | s1.same | s1.up | s2.down | s2.same | s2.up | s3.down | s3.same | s3.up | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 100 | No | DU | 0.03 | 0.00 | 0.97 | 0.77 | 0.00 | 0.23 | 0.61 | 0.32 | 0.06 |
| 2 | 100 | No | UD | 0.00 | 0.06 | 0.94 | 0.43 | 0.00 | 0.57 | 0.46 | 0.34 | 0.20 |
| 3 | 100 | Yes | DU | 0.00 | 0.03 | 0.97 | 0.71 | 0.00 | 0.29 | 0.68 | 0.29 | 0.03 |
| 4 | 100 | Yes | UD | 0.06 | 0.03 | 0.91 | 0.65 | 0.03 | 0.32 | 0.65 | 0.32 | 0.03 |
Percentages of correct choices are presented in Figure XX:
| numEstCond | change.cond | s1.cor | s2.cor | s3.cor | |
|---|---|---|---|---|---|
| 1 | No | DU | 0.97 [0.83, 0.99] | 0.77 [0.57, 0.9] | 0.61 [0.39, 0.8] |
| 2 | No | UD | 0.94 [0.81, 0.98] | 0.43 [0.22, 0.67] | 0.46 [0.24, 0.69] |
| 3 | Yes | DU | 0.97 [0.86, 0.99] | 0.71 [0.52, 0.85] | 0.68 [0.49, 0.83] |
| 4 | Yes | UD | 0.91 [0.76, 0.97] | 0.65 [0.44, 0.81] | 0.65 [0.44, 0.81] |
To see which variables affected choice quality, we conducted a Bayesian binary logistic regression analysis on each stimulus
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 118.56 | -80.55 | 375.92 | 54.85 | 0.26 |
| changeorder_cond | -37.16 | -148.52 | 43.48 | 87.28 | 0.39 |
| numEstCondYes | 2.68 | -81.00 | 84.37 | 125.12 | 0.95 |
| ant.score | 65.39 | 9.17 | 122.02 | 16.38 | 0.01 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 42.30 | -134.21 | 225.85 | 802.35 | 0.63 |
| changeorder_cond | -85.68 | -189.77 | -3.27 | 325.69 | 0.04 |
| numEstCondYes | 58.31 | -22.30 | 149.21 | 597.07 | 0.15 |
| ant.score | 56.64 | 11.94 | 98.48 | 239.73 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | -124.93 | -325.35 | 59.30 | 483.70 | 0.18 |
| changeorder_cond | -29.45 | -128.90 | 45.64 | 669.06 | 0.52 |
| numEstCondYes | 97.62 | 13.63 | 201.79 | 302.20 | 0.03 |
| ant.score | 77.70 | 29.93 | 129.82 | 177.94 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 132.24 | 22.63 | 229.24 | 10.57 | 0.00 |
| change.condUD | -35.61 | -82.23 | 3.70 | 33.41 | 0.05 |
| numEstCondYes | 31.79 | -14.79 | 80.37 | 71.51 | 0.12 |
| s2 | -147.46 | -227.57 | -40.66 | 5.99 | 0.00 |
| s3 | -159.23 | -248.87 | -42.71 | 6.53 | 0.00 |
| s4 | -280.40 | -417.84 | -80.00 | 3.37 | 0.00 |
| ant.score | 28.23 | 3.55 | 54.22 | 11.01 | 0.00 |
Study 1 Conclusions
Almost all participants got s1 correct (95%), while much fewer got s2 (64%) and s3 (60%) correct.
In s2 and s3, being required to give a numeric estimate seemed to improve decisions: those who did not give numeric estimates got (56%) correct, while those who did give numeric estimates got (67%) correct.
Participants with higher ANT scores had much better performance than those with lower ANT scores (kendall-r = 0.28, p = 210^{-4})
Study 2
Stimuli
Study 2 was identical to study 1 with one added condition. In addition to the $100 starting investment, we included a $137 starting investment
8 Conditions
We had 3 between-participants independent variables resuling in 8 experimental conditions: 2 (Starting Investment: $100 vs. $137) 2 (Change order: Up-Down, Down-Up) x 2 (Numerical estimate: Yes, No).
| start_amount | numEstCond | changeorder_cond | s1.down | s1.same | s1.up | s2.down | s2.same | s2.up | s3.down | s3.same | s3.up | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 100 | No | DU | 0.04 | 0.17 | 0.78 | 0.59 | 0.04 | 0.37 | 0.59 | 0.41 | 0.00 |
| 2 | 100 | No | UD | 0.00 | 0.00 | 1.00 | 0.57 | 0.04 | 0.39 | 0.61 | 0.36 | 0.04 |
| 3 | 100 | Yes | DU | 0.11 | 0.04 | 0.82 | 0.68 | 0.04 | 0.25 | 0.61 | 0.36 | 0.00 |
| 4 | 100 | Yes | UD | 0.02 | 0.02 | 0.95 | 0.72 | 0.00 | 0.28 | 0.67 | 0.33 | 0.00 |
| 5 | 137 | No | DU | 0.05 | 0.14 | 0.78 | 0.57 | 0.08 | 0.32 | 0.57 | 0.38 | 0.03 |
| 6 | 137 | No | UD | 0.03 | 0.03 | 0.94 | 0.34 | 0.06 | 0.60 | 0.46 | 0.43 | 0.11 |
| 7 | 137 | Yes | DU | 0.03 | 0.08 | 0.89 | 0.65 | 0.05 | 0.30 | 0.57 | 0.43 | 0.00 |
| 8 | 137 | Yes | UD | 0.03 | 0.06 | 0.91 | 0.40 | 0.03 | 0.57 | 0.43 | 0.54 | 0.03 |
| start_amount | numEstCond | changeorder_cond | s1.cor | s2.cor | s3.cor | |
|---|---|---|---|---|---|---|
| 1 | 100 | No | DU | 0.78 [0.62, 0.89] | 0.59 [0.4, 0.75] | 0.59 [0.4, 0.75] |
| 2 | 100 | No | UD | 1 [0.88, 1] | 0.57 [0.34, 0.78] | 0.61 [0.37, 0.8] |
| 3 | 100 | Yes | DU | 0.82 [0.62, 0.93] | 0.68 [0.45, 0.84] | 0.61 [0.37, 0.8] |
| 4 | 100 | Yes | UD | 0.95 [0.84, 0.99] | 0.72 [0.54, 0.85] | 0.67 [0.49, 0.82] |
| 5 | 137 | No | DU | 0.78 [0.6, 0.89] | 0.57 [0.36, 0.75] | 0.57 [0.36, 0.75] |
| 6 | 137 | No | UD | 0.94 [0.81, 0.98] | 0.34 [0.14, 0.62] | 0.46 [0.24, 0.69] |
| 7 | 137 | Yes | DU | 0.89 [0.74, 0.96] | 0.65 [0.45, 0.81] | 0.57 [0.36, 0.75] |
| 8 | 137 | Yes | UD | 0.91 [0.77, 0.97] | 0.4 [0.19, 0.65] | 0.43 [0.22, 0.67] |
To see which variables affected choice quality, we conducted a binary logistic regression analysis on each stimulus
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 13.01 | -30.52 | 56.61 | 52.30 | 0.54 |
| changeorder_condUD | 21.63 | 2.19 | 40.08 | 9.05 | 0.00 |
| numEstCondYes | 3.02 | -8.41 | 14.90 | 68.33 | 0.61 |
| start_amount | -0.05 | -0.40 | 0.25 | 86.67 | 0.81 |
| ant.score | 9.16 | 1.05 | 17.57 | 7.93 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 149.92 | -32.36 | 374.18 | 763.51 | 0.12 |
| changeorder_condUD | -59.94 | -118.40 | -2.74 | 402.09 | 0.03 |
| numEstCondYes | 55.60 | 0.31 | 120.57 | 513.98 | 0.04 |
| start_amount | -2.10 | -3.76 | -0.53 | 284.34 | 0.01 |
| ant.score | 60.48 | 29.23 | 89.53 | 171.91 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 80.00 | -105.55 | 278.97 | 341.19 | 0.34 |
| changeorder_condUD | -20.00 | -79.89 | 31.37 | 773.57 | 0.43 |
| numEstCondYes | 9.33 | -50.26 | 58.44 | 779.52 | 0.72 |
| start_amount | -1.32 | -2.85 | 0.19 | 92.89 | 0.06 |
| ant.score | 49.18 | 14.25 | 84.35 | 25.94 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 18.42 | 3.31 | 36.17 | 4.60 | 0.00 |
| changeorder_condUD | -0.20 | -8.21 | 7.60 | 254.88 | 0.95 |
| numEstCondYes | 5.26 | -2.47 | 14.46 | 21.72 | 0.12 |
| start_amount137 | -8.59 | -18.76 | -0.29 | 10.70 | 0.01 |
| s2 | -31.66 | -54.89 | -9.57 | 2.18 | 0.00 |
| s3 | -31.83 | -53.03 | -7.09 | 2.37 | 0.00 |
| ant.score | 9.52 | 2.26 | 17.50 | 2.85 | 0.00 |
Study 2 Conclusions
Replicating study 1, almost all participants got s1 correct (88%), while much fewer got s2 (57%) and s3 (56%) correct.
In s2 and s3, being required to give a numeric estimate seemed only improved decisions slightly: those who did not give numeric estimates got (54%) correct, while those who did give numeric estimates got (59%) correct.
In s2 and s3, having a starting investment of $137 seemed to hurt performance relative to a starting investment of $100: those with a starting investment of $100 got (63%) correct, while those who did give numeric estimates got (50%) correct.
Participants with higher ANT scores had much better performance than those with lower ANT scores (kendall-r = 0.29, p = 0)
Study 3
Materials and Procedure
| Stimuli | Percentage Up | Percentage Down | True Change |
|---|---|---|---|
| 1 | 20% | 10% | +8% |
| 2 | 50% | 40% | -10% |
| 3 | 30% | 30% | -9% |
| 4 | 25% | 20% | 0% |
| start.amount | numEstCond | changeorder_cond | s1.down | s1.same | s1.up | s2.down | s2.same | s2.up | s3.down | s3.same | s3.up | s4.down | s4.same | s4.up | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 100 | No | DU | 0.02 | 0.04 | 0.94 | 0.53 | 0.01 | 0.46 | 0.56 | 0.43 | 0.01 | 0.05 | 0.47 | 0.48 |
| 2 | 100 | Yes | DU | 0.00 | 0.02 | 0.98 | 0.70 | 0.01 | 0.29 | 0.68 | 0.32 | 0.00 | 0.03 | 0.70 | 0.27 |
| 3 | x | No | DU | 0.00 | 0.05 | 0.95 | 0.51 | 0.03 | 0.46 | 0.55 | 0.43 | 0.02 | 0.11 | 0.44 | 0.44 |
| cond.all.text | s1.cor | s2.cor | s3.cor | s4.cor | |
|---|---|---|---|---|---|
| 1 | 100.nonum | 0.94 [0.87, 0.97] | 0.53 [0.4, 0.66] | 0.56 [0.43, 0.69] | 0.47 [0.33, 0.61] |
| 2 | 100.num | 0.98 [0.93, 0.99] | 0.7 [0.58, 0.79] | 0.68 [0.56, 0.78] | 0.7 [0.58, 0.79] |
| 3 | x | 0.95 [0.88, 0.98] | 0.51 [0.37, 0.64] | 0.55 [0.41, 0.67] | 0.44 [0.31, 0.59] |
To see which variables affected choice quality, we conducted a binary logistic regression analysis on each stimulus
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 3.15 | 2.12 | 4.21 | 6.80 | 0.00 |
| cond.all.text100.num | 1.12 | 0.32 | 1.90 | 10.65 | 0.00 |
| cond.all.textx | -0.08 | -1.30 | 1.42 | 4.20 | 0.82 |
| ant.score | 0.05 | -0.41 | 0.49 | 5.59 | 0.91 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | -150.38 | -241.34 | -65.49 | 145.15 | 0.00 |
| cond.all.text100.num | 95.96 | 23.50 | 172.71 | 184.19 | 0.01 |
| cond.all.textx | -18.62 | -87.27 | 47.54 | 760.58 | 0.58 |
| ant.score | 69.20 | 36.96 | 103.34 | 113.68 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | -129.84 | -214.00 | -48.96 | 147.41 | 0.00 |
| cond.all.text100.num | 63.67 | 0.18 | 138.36 | 380.84 | 0.05 |
| cond.all.textx | -17.28 | -78.61 | 52.66 | 748.41 | 0.59 |
| ant.score | 68.28 | 34.08 | 97.55 | 82.49 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | -1.85 | -2.54 | -1.30 | 8.12 | 0.00 |
| cond.all.text100.num | 1.00 | 0.57 | 1.78 | 15.24 | 0.00 |
| cond.all.textx | -0.22 | -0.61 | 0.44 | 37.63 | 0.40 |
| ant.score | 0.78 | 0.59 | 1.01 | 24.00 | 0.00 |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 2.51 | 1.73 | 3.26 | 194.24 | 0.00 |
| cond.all.text100.num | 0.88 | 0.25 | 1.56 | 709.50 | 0.01 |
| cond.all.textx | -0.42 | -1.04 | 0.30 | 712.74 | 0.21 |
| s2 | -4.23 | -4.33 | -4.07 | 6.02 | 0.00 |
| s3 | -4.06 | -4.25 | -3.92 | 3.60 | 0.00 |
| s4 | -4.57 | -4.70 | -4.45 | 3.84 | 0.00 |
| ant.score | 0.85 | 0.60 | 1.10 | 46.97 | 0.00 |
| stimulus | X100.num.hdi | x.hdi | ant.hdi | |
|---|---|---|---|---|
| 1 | 1.00 | 1.12 [0.32, 1.9] | -0.08 [-1.3, 1.42] | 0.05 [-0.41, 0.49] |
| 2 | 2.00 | 95.96 [23.5, 172.71] | -18.62 [-87.27, 47.54] | 69.2 [36.96, 103.34] |
| 3 | 3.00 | 63.67 [0.18, 138.36] | -17.28 [-78.61, 52.66] | 68.28 [34.08, 97.55] |
| 4 | 4.00 | 1 [0.57, 1.78] | -0.22 [-0.61, 0.44] | 0.78 [0.59, 1.01] |
| post.mean | l-95% CI | u-95% CI | eff.samp | pMCMC | |
|---|---|---|---|---|---|
| (Intercept) | 2.51 | 1.73 | 3.26 | 194.24 | 0.00 |
| cond.all.text100.num | 0.88 | 0.25 | 1.56 | 709.50 | 0.01 |
| cond.all.textx | -0.42 | -1.04 | 0.30 | 712.74 | 0.21 |
| s2 | -4.23 | -4.33 | -4.07 | 6.02 | 0.00 |
| s3 | -4.06 | -4.25 | -3.92 | 3.60 | 0.00 |
| s4 | -4.57 | -4.70 | -4.45 | 3.84 | 0.00 |
| ant.score | 0.85 | 0.60 | 1.10 | 46.97 | 0.00 |
Study 3 Conclusions
Replicating studies 1 and 2, almost all participants got s1 correct (96%), while much fewer got s2 (58%) and s3 (60%) correct.
New to study 3, we found that most participants got stimulus 3
In s2 and s3, being required to give a numeric estimate seemed only improved decisions slightly: those who did not give numeric estimates got (54%) correct, while those who did give numeric estimates got (59%) correct.
In s2 and s3, having a starting investment of $137 seemed to hurt performance relative to a starting investment of $100: those with a starting investment of $100 got (63%) correct, while those who did give numeric estimates got (50%) correct.
Participants with higher ANT scores had much better performance than those with lower ANT scores (kendall-r = 0.29, p = 0)