##Introduction

The selected Lee, Hall, & Wood, 2018 article reappraised the finding that money spent on experiences yielded higher happiness than that allocated to material goods. Given resource distribution varies widely across the population, the authors posited that socio-economic status (IV) would play a significant role in individuals’ self-evaluations of subjective happiness (DV) when reflecting on apportioned financial resouces. The authors’ define experiential advantage as the phenomenon of recalled experiences generating more happiness than material acquisitions. Findings from Study 1 (to be reproduced) indicate differential effects of SES, such that participants from lower-income backgrounds were happier with material purchases (as compared to investing in experiences) while individuals from higher-income backgrounds reported experiential advantage.

Results are of value to my research program along two key dimensions. First, methodologically, the selected study is a replication with alterations to the original measure–I’m keenly interested in getting a deeper understanding of how more nuanced scale design (particularly of extant measures) can lead to increasingly useful and accurate insights. Second, theoretically, through this process I’m excited to understand how subjective assessment measures can be combined with straight-forward survey design and applied broadly to examine racialized experiences. For example, a future research question this might be succesfully mapped onto is whether people assess targets that label certain behaviors or beliefs as “racism” or “racist” as more racially biased than those who avoid this language, and whether that’s influenced by participant race.

###Procedures

The selected study was an attempt to replicate a Van Boven and Gilovich (2003) design; however, Lee et al. (2018), incorporated some important modifications, which will be noted throughout the procedures write-up where appropriate.

Participants were first given a computer-based survey using Amazon’s Mechanical Turk (AMT) and prompted to recall a recent* “experiential purchase and object purchase” that lead to an increase in their happiness. Next, participants were asked which purchase made them happier. The presented scale was: -3 = Definitely experiential purchase, 3 = Definitely object purchase.**

Afterward, using the MacArthur Scale of Subjective Social Status, participants were presented with a visual of a ladder consisting of 10 steps. Subjects were instructed to think of the ladder as representing their position in society, with the most educated and wealthy at the top and people with the least respected jobs and education at the bottom.

One challenge to reproducing this finding might be that I am relatively new to AMT, RStudio and programming more generally. Therefore, I anticipate there will likely be some time-management concerns, insofar as it will likely be difficult to accurately predict how long certain tasks should take to complete.

*Instead of having participants compare experiential purchases versus material acquistion from a global perspective (across the lifespan) as in the original, Lee et al. had participants compare more local (recent) spending.

**The current study employs a continuous scale for comparative purchase happiness rather than the dichotomous scale used in the original.

##Methods

###Power Analysis

Although it wasn’t reported in the original publication, I calculated the Pearson’s correlation for the original data set and found .2294087, which I believe is the effect size I should use to calculate the number of participants required at various power levels; my procedure is below–this week I’ll verify with a TA.

Based on this value, the study should include 146 participants (80% power), 195 participants (90% power), or 241 participants (95% power). These values were calculated using G*Power.

Participants in the pilot took 2 minutes on average and so it should be about 25 cents per participant, so if we’d like power of 80%, it’ll require us to spend about $40.

###Planned Sample

Planned sample should have approximately the same demographic characteristics as that in the original study because of AMT recruitment, which was as follows: “…209 adult U.S. residents participated on Amazon Mechanical Turk (52% women; age: M = 38.39 years, SD = 12.83). The target sample size (N = 200) was determined before data collection began, and a total of 209 participants actually completed the study.” No participants were excluded, however, our total sample may ultimately be slightly different based on power analyses.

###Materials

Study materials were made publicly available on the Open Science Framework (OSF), thus, stimuli used in this reproduction are expected to be the same as the original study (see ladder visual below).

Measure of Comparative Purchase Happiness: “Comparative purchase happiness was assessed with the question, “Between the two purchases, which made you happier?” Responses were reported on a 7-point scale from −3 (definitely experiential purchase) to 3 (definitely object purchase)."

Measure of Social Class: “Participants were shown a ladder with 10 rungs and given the following instructions: Think of this ladder as representing where people stand in the U.S. At the top of the ladder are the people who are the best off—those who have the most money, the most education, and the most respected jobs. At the bottom are the people who are the worst off—who have the least money, least education, and the least respected jobs or no job. The higher up you are on this ladder, the closer you are to the people at the very top; the lower you are, the closer you are to the people at the very bottom.”

###Procedure

Procedures were followed precisely and are outlined, largely verbatim, below:

“Participants were asked to ‘think about a recent experiential purchase and object purchase that you made to increase your happiness.’ No further information about the definition of these purchases was given.”

“Next, participants reported their social class using the MacArthur Scale of Subjective Social Status…In the past, researchers have measured social class through objective indicators of income, education, or occupation or have used the MacArthur Scale to capture subjective assessment of all three aspects.”

Next, participants were presented with eight demographics questions, including:
1. About how much is your yearly family income?
2. About how much is your yearly personal income?
3. What is the highest level of education you have completed?
4. What would you describe your family and yourself?
5. Currently, what is the number of your household? (If a father, a mother, and a child are living together, then the answer would be 3.)
6. What is your gender?
7. How old are you? (in years)
8. What is your race?

Link to Qualtrics Replication

https://stanforduniversity.qualtrics.com/jfe/form/SV_cvyQcGW8h07edXn

###Analysis Plan

My analyses will attempt to reproduce the regression from the original paper, in addition to reporting the Pearson correlation for the original data (per Mike’s suggestion as this was absent the results) and the new data set. I may also plot their data and create a regression table in the final paper.

Key analysis of interest

Key confirmatory analysis will be “a regression analysis predicting comparative purchase happiness from social class,” as stated in original publication, which “revealed that social class positively predicted happiness.” Second, I’ll conduct an analysis that examines where comparative purchase happiness differs significantly from zero (the midpoint of the scale) using confidence intervals for the fitted values of the regression line. My analytic approach should result in a figure (and correlation) that emulates that presented in the article. My current analytic plan does not include standardizing the beta. Both the IV (SES measure) and DV (subjective happiness rating) require reverse coding.

Strategy

  1. Exclusions: Currently intend to include all participants, as the original study had no exclusions.

  2. Cleaning: Tidy data by removing unnecessary columns (e.g., start date, end date, etc.), convert from wide to long, and ensure item values are accurately coded and that question labels from the build-phase translate to being intuitive for data analysis.

  3. Analysis: Run a linear regression to see whether SES predicts subjective happiness rating. The original figure indicates that the authors found values where “comparative purchase happiness was significantly different from 0.” After speaking with some stats graduate students, I believe this was determined by using confidence intervals around the fixed values of the regression line and finding the values at which (out to two decimal places) the intervals are no longer inclusive of zero. This is the approach I am going to take in my analysis.

###Differences from Original Study

Given data will be collected using the same platform (AMT), with the same study materials, including visual stimuli (downloaded from the OSF), we can reasonably anticipate no significant differences between sample size and demographic composition, setting or procedure.

We can expect some differences will occur (e.g., participant setting or time of study participation, two experimental variables that are not within our control) but, as this study is itself a replication which confirmed the findings of the original, it is not predicted that these differences will have large effects.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

##Results

Data preparation

Data preparation following the analysis plan.

#load library 
library("tidyverse")
## ── Attaching packages ────────────────────────────────────── tidyverse 1.2.1 ──
## ✔ ggplot2 3.2.1     ✔ purrr   0.3.2
## ✔ tibble  2.1.3     ✔ dplyr   0.8.3
## ✔ tidyr   1.0.0     ✔ stringr 1.4.0
## ✔ readr   1.3.1     ✔ forcats 0.4.0
## ── Conflicts ───────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag()    masks stats::lag()
#import data
 lee <- read_csv("251_Reproduce_October 26, 2019_19.10.csv")
## Parsed with column specification:
## cols(
##   .default = col_character()
## )
## See spec(...) for full column specifications.

###Data Preparation

#Remove first two rows

lee2 <- lee %>% 
  filter(!((IPAddress == "IP Address") |
           IPAddress == "{\"ImportId\":\"ipAddress\"}")) %>% 
  
   mutate(subid = row_number()) %>%                     #Insert subject ID row
  select(contains("subid"), CompHap, MacArthurSES)       #Select relevant columns
#Recoding IV and DV measures and renaming
 d <- lee2 %>%         
 mutate(ComparativeHappiness=recode(lee2$CompHap, 
                     `2` = -2, `3` = -3, 
                     `1` = -1, `-3` = 3, 
                     `-1` = 1, `-2` = 2)) %>% 
mutate(MacArthurScale=recode(lee2$MacArthurSES, 
                  `1` = 10, `2` = 9, `3` = 8,
                  `4` = 7, `5` = 6, 
                  `6` = 5, `7` = 4, 
                  `8` = 3, `9` = 2, `10` = 1))

#Note: data is in long format currently, but may change if I do exploratory analyses, at which point I'll convert to long.

Confirmatory analysis

Getting Pearson correlation

cor(d$ComparativeHappiness, d$MacArthurScale)
## [1] 0.4004947

Finding the confidence intervals for fitted values in a linear regression

l <- lm(ComparativeHappiness~MacArthurScale,data=d)      #run regresssion & assign variable

#Generate a list of CIs for each IV value--locate values that have CIs not intersecting 0

predict(l,newdata = data.frame
        (MacArthurScale=d$MacArthurScale),
        interval="confidence",level = 0.95)
##    fit       lwr      upr
## 1  0.6 -1.456959 2.656959
## 2 -0.3 -2.819250 2.219250
## 3 -1.2 -5.313918 2.913918
## 4  1.5 -1.752338 4.752338
## 5  1.5 -1.752338 4.752338
## 6  0.6 -1.456959 2.656959
## 7 -0.3 -2.819250 2.219250
## 8  0.6 -1.456959 2.656959

The sample from pilot a is small (n=8) making the CIs listed above broad. As such, all current CIs encompass 0; identifying significant values will happen after a larger sample is collected (pilot b).

#CIs will give interger values that can be manually reviewed to find significant values
  #the 7.02 in the code below is as an example

ci = predict(l,newdata = data.frame
             (MacArthurScale=7.02), 
             interval="confidence",level = 0.95)
ci
##     fit       lwr      upr
## 1 1.518 -1.766958 4.802958

Looking at the p-value

 summary(lm(ComparativeHappiness~MacArthurScale,data=d))
## 
## Call:
## lm(formula = ComparativeHappiness ~ MacArthurScale, data = d)
## 
## Residuals:
##    Min     1Q Median     3Q    Max 
## -3.500 -1.725  0.450  1.625  2.400 
## 
## Coefficients:
##                Estimate Std. Error t value Pr(>|t|)
## (Intercept)     -4.8000     4.9017  -0.979    0.365
## MacArthurScale   0.9000     0.8406   1.071    0.326
## 
## Residual standard error: 2.302 on 6 degrees of freedom
## Multiple R-squared:  0.1604, Adjusted R-squared:  0.02046 
## F-statistic: 1.146 on 1 and 6 DF,  p-value: 0.3255

Plotting the model

figure <- d %>%
 ggplot(aes(x = MacArthurScale, y = ComparativeHappiness)) +
 geom_smooth(method = "lm", fill = NA)

figure + geom_hline(yintercept=0, color = "black") +
  geom_vline(xintercept = 0, color = "black") #+

#Dashed lines (coded below) will be added during pilot b once sample is large enough to evalute
#  geom_vline(xintercept = XX, linetype="dashed", color = "blue") +
 # geom_vline(xintercept = XX, linetype="dashed", color = "blue")

#Note: My data is fairly skewed but hopefully will even out with bigger n.

Side-by-side graph with original graph is ideal here Original Figure from Lee et al. paper

###Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.