##Introduction

The selected Lee, Hall, & Wood, 2018 article reappraised the finding that money spent on experiences yielded higher happiness than that allocated to material goods. Given resource distribution varies widely across the population, the authors posited that socio-economic status (IV) would play a significant role in individuals’ self-evaluations of subjective happiness (DV) when reflecting on apportioned financial resouces. The authors’ define experiential advantage as the phenomenon of recalled experiences generating more happiness than material acquisitions. Findings from Study 1 (to be reproduced) indicate differential effects of SES, such that participants from lower-income backgrounds were happier with material purchases (as compared to investing in experiences) while individuals from higher-income backgrounds reported experiential advantage.

Results are of value to my research program along two key dimensions. First, methodologically, the selected study is a replication with alterations to the original measure–I’m keenly interested in getting a deeper understanding of how more nuanced scale design (particularly of extant measures) can lead to increasingly useful and accurate insights. Second, theoretically, through this process I’m excited to understand how subjective assessment measures can be combined with straight-forward survey design and applied broadly to examine racialized experiences. For example, a future research question this might be succesfully mapped onto is whether people assess targets that label certain behaviors or beliefs as “racism” or “racist” as more racially biased than those who avoid this language, and whether that’s influenced by participant race.

###Procedures

The selected study was an attempt to replicate a Van Boven and Gilovich (2003) design; however, Lee et al. (2018), incorporated some important modifications, which will be noted throughout the procedures write-up where appropriate.

Participants were first given a computer-based survey using Amazon’s Mechanical Turk (AMT) and prompted to recall a recent* “experiential purchase and object purchase” that lead to an increase in their happiness. Next, participants were asked which purchase made them happier. The presented scale was: -3 = Definitely experiential purchase, 3 = Definitely object purchase.**

Afterward, using the MacArthur Scale of Subjective Social Status, participants were presented with a visual of a ladder consisting of 10 steps. Subjects were instructed to think of the ladder as representing their position in society, with the most educated and wealthy at the top and people with the least respected jobs and education at the bottom.

One challenge to reproducing this finding might be that I am relatively new to AMT, RStudio and programming more generally. Therefore, I anticipate there will likely be some time-management concerns, insofar as it will likely be difficult to accurately predict how long certain tasks should take to complete.

*Instead of having participants compare experiential purchases versus material acquistion from a global perspective (across the lifespan) as in the original, Lee et al. had participants compare more local (recent) spending.

**The current study employs a continuous scale for comparative purchase happiness rather than the dichotomous scale used in the original.

##Methods

###Power Analysis

Original effect size, power analysis for samples to achieve 80%, 90%, 95% power to detect that effect size. Considerations of feasibility for selecting planned sample size.

###Planned Sample

Planned sample should have approximately the same demographic characteristics as that in the original study because of AMT recruitment, which was as follows: “…209 adult U.S. residents participated on Amazon Mechanical Turk (52% women; age: M = 38.39 years, SD = 12.83). The target sample size (N = 200) was determined before data collection began, and a total of 209 participants actually completed the study.” No participants were excluded, however, our total sample may ultimately be slightly different based on power analyses.

###Materials

Study materials were made publicly available on the Open Science Framework (OSF), thus, stimuli used in this reproduction are expected to be the same as the original study (see ladder visual below).

Measure of Comparative Purchase Happiness: “Comparative purchase happiness was assessed with the question, “Between the two purchases, which made you happier?” Responses were reported on a 7-point scale from −3 (definitely experiential purchase) to 3 (definitely object purchase)."

Measure of Social Class: “Participants were shown a ladder with 10 rungs and given the following instructions: Think of this ladder as representing where people stand in the U.S. At the top of the ladder are the people who are the best off—those who have the most money, the most education, and the most respected jobs. At the bottom are the people who are the worst off—who have the least money, least education, and the least respected jobs or no job. The higher up you are on this ladder, the closer you are to the people at the very top; the lower you are, the closer you are to the people at the very bottom.”

###Procedure

Procedures were followed precisely and are outlined, largely verbatim, below:

“Participants were asked to ‘think about a recent experiential purchase and object purchase that you made to increase your happiness.’ No further information about the definition of these purchases was given.”

“Next, participants reported their social class using the MacArthur Scale of Subjective Social Status…In the past, researchers have measured social class through objective indicators of income, education, or occupation or have used the MacArthur Scale to capture subjective assessment of all three aspects.”

Next, participants were presented with eight demographics questions, including:
1. About how much is your yearly family income?
2. About how much is your yearly personal income?
3. What is the highest level of education you have completed?
4. What would you describe your family and yourself?
5. Currently, what is the number of your household? (If a father, a mother, and a child are living together, then the answer would be 3.)
6. What is your gender?
7. How old are you? (in years)
8. What is your race?

###Analysis Plan

My analyses will attempt to reproduce the regression from the original paper, in addition to reporting the Pearson correlation for the original data (per Mike’s suggestion as this was absent the results) and the new data set. I may also plot their data and create a regression table in the final paper.

Key analysis of interest

Key analysis will be “a regression analysis predicting comparative purchase happiness from social class,” as stated in original publication, which “revealed that social class positively predicted happiness.” My plan is to reproduce this finding by running a linear regression, which appears to me the most appropriate method for predicting the DV from the IV and generating a correlation similar to that presented in the study: regression type was not given in the publication, but please let me know if there is a better approach.

After visiting the stats deptartment, I’m under the impression that if the beta was standardized, this would have been reported in the paper. As Julie noted, because it wasn’t reported in the original manuscript, my current analytic plan does not include standardizing the beta, but please let me know if it would be worth asking the original authors to clarify.

Strategy

  1. Exclusions: Currently intend to include all participants, as the original study had no exclusions.

  2. Cleaning: Using mockup Qualtrics survey, run test data (non-pilot) to ensure values are accurately coded, question labels from the build-phase translate to being intuitive for data analysis. Data will need to be converted from wide to long in order to make it tidyverse compatible. Several unnecessary, Qualtrics-generated columns will be removed as well (e.g., start date, end date, etc.).

  3. Analysis: Run a linear regression to see whether SES predicts subjective happiness rating. The figure also indicates that the authors found values where “comparative purchase happiness was significantly different from 0,” which I believe was done using a t-test. To the best of my knowledge these values weren’t generated from the regression, making this an additional statistical test, and from the outline provided on Canvas it seems we are intended to find a single analysis. While this might technically be outside the scope of the work, I will consult with someone from the statistics department this week to determine if the values can be found with a single analytical approach. If not, I plan to run the analyses in order to get a strong approximation of the figure in the paper after the pilot data is collected. Both the IV (SES measure) and DV (subjective happiness rating) require reverse coding.

###Differences from Original Study

Given data will be collected using the same platform (AMT), with the same study materials, including visual stimuli (downloaded from the OSF), we can reasonably anticipate no significant differences between sample size and demographic composition, setting or procedure.

We can expect some differences will occur (e.g., participant setting or time of study participation, two experimental variables that are not within our control) but, as this study is itself a replication which confirmed the findings of the original, it is not predicted that these differences will have large effects.

Methods Addendum (Post Data Collection)

You can comment this section out prior to final report with data collection.

Actual Sample

Sample size, demographics, data exclusions based on rules spelled out in analysis plan

Differences from pre-data collection methods plan

Any differences from what was described as the original plan, or “none”.

##Results

Data preparation

Data preparation following the analysis plan.

####Load Relevant Libraries and Functions
install.packages("tidyverse")
library("tidyverse")
###Import data
 lee <- read_csv("SampleFileName.csv")

###Data Preparation

#Creating subject ID row and removing unnecessary columns
lee2 <- lee %>% 
  select(18:27) %>% 
 filter(CompHap %in% 
          c("-1", "-2", "-3", "0", "1", "2", "3")) %>% 
 mutate(subid = row_number())
  
#Recoding IV and DV measures and renaming
 lee3 <- lee2 %>% 
 mutate(happy=recode(lee2$CompHap, 
                     `2` = -2, `3` = -3, 
                     `1` = -1, `-3` = 3, 
                     `-1` = 1, `-2` = 2)) %>% 
mutate(SES=recode(lee2$MacArthurSES, 
                  `1` = 10, `2` = 9, `3` = 8,
                  `4` = 7, `5` = 6, 
                  `6` = 5, `7` = 4, 
                  `8` = 3, `9` = 2, `10` = 1))
 View(lee3)
 
 #Converting wide to long format  
d <- lee3 %>% 
 gather(Measures, Values, 1:10)
 View(d)

Confirmatory analysis

A little data visualization

d %>%
ggplot(aes(x = SES, y = happy)) +
 geom_point(colour = "blue")

Getting Pearson correlation

cor(d$happy, d$SES)

Plotting the model

figure <- d %>%
 ggplot(aes(x = SES, y = happy)) +
 geom_point(colour = "red") +
 geom_smooth(method = "lm", fill = NA)

figure + geom_hline(yintercept=0, color = "black") +
  geom_vline(xintercept = 0, color = "black") +
  geom_vline(xintercept = XX, linetype="dashed", color = "blue") +
  geom_vline(xintercept = XX, linetype="dashed", color = "blue")

Side-by-side graph with original graph is ideal here

###Exploratory analyses

Any follow-up analyses desired (not required).

Discussion

Summary of Replication Attempt

Open the discussion section with a paragraph summarizing the primary result from the confirmatory analysis and the assessment of whether it replicated, partially replicated, or failed to replicate the original result.

Commentary

Add open-ended commentary (if any) reflecting (a) insights from follow-up exploratory analysis, (b) assessment of the meaning of the replication (or not) - e.g., for a failure to replicate, are the differences between original and present study ones that definitely, plausibly, or are unlikely to have been moderators of the result, and (c) discussion of any objections or challenges raised by the current and original authors about the replication attempt. None of these need to be long.