This is a minimal demonstration of a mismatch between P-curve effect size and what seems like a reasonable meta-analytic effect size.

P-Curve estimation functions from http://www.p-curve.com/Supplement/Rcode_paper2/9%20-%20Loss%20Function%20and%20Estimation.R

Here is a minimal working example of using the word segmentation data for p-curve.

Note, converting Fs to \(t\)s via square root.

df <- filter(all_data, dataset=="Word segmentation") %>%
  mutate(df2 = ifelse(participant_design == "between", (n_1 + n_2)-2, n_1-1),
         t =  ifelse(is.na(t), sqrt(F), t), # turn ts into Fs by squaring them
         df1 = 1)

Here’s the ES distribution histogram.

qplot(d_calc, data=df)

And here’s the developmental trend on effect sizes.

ggplot(aes(x = mean_age_1/30.3, y = d_calc), data=df) + 
  geom_point(aes(size = n)) + 
  geom_smooth(span = 1) + 
  ylab("Effect size (d)") + xlab("Mean Age (months)")

As you can see, the estimated effect size appears to be close to zero, maybe around .25. That’s confirmed by meta analysis.

But P-curve thinks it’s closer to 1, which seems really surprising.

t_obs <- df$t[!is.na(df$t)]
df_obs <- df$df2[!is.na(df$t)]

plotloss(t_obs=t_obs,df_obs=df_obs,dmin=-.5,dmax=2)

## [1] 0.9709729