Tests of Significance and Measures of Association

Learning Objectives

• Understand how relationships are affected by sampling error
• Use informal and formal tests of significance
• Interpret confidence intervals
• Distinguish between difference of means and proportions tests
• Measure strength of relationships with association statistics

Why Test for Significance?

• Sample result: Women mean = 54, Men mean = 49
• Is the 5-point gap real or just noise?
• We use statistical significance testing to decide

The Null Hypothesis

• H₀: No relationship exists in the population
  • Any observed relationship is due to chance
• Hₐ: A relationship exists
  • Use logic of contradiction: reject H₀ only if evidence is strong

Confidence Intervals (CIs)

• CI shows a range where the true population parameter likely falls
• If CI includes H₀ value → fail to reject H₀

Formula: \[ SE = \frac{s}{\sqrt{n}} \\ CI_{95\%} = \bar{x} \pm 1.96 \cdot SE \]

One-Sample Significance Test

• Compare sample statistic to fixed value (mean or proportion)
• Use CI or compute a P-value

Example: Is average quiz score > 15? If CI includes 15 → no significant difference

Two-Sample Tests

We now examine whether differences between groups are statistically significant.

Difference of Means

• Use when DV is continuous (e.g., feeling thermometer)
• Compare group means

Formula: \[ SE_{\text{diff}} = \sqrt{SE_1^2 + SE_2^2} \]

Significant if: \[ \frac{\text{Difference}}{SE_{\text{diff}}} \geq 2 \]

Difference of Proportions

• Use when DV is binary (e.g., support/oppose)
• Compare group proportions
• Test statistic: Z-score

Formulas: \[ SE = \sqrt{\frac{pq}{n}}, \quad q = 1 - p \\ SE_{\text{diff}} = \sqrt{SE_1^2 + SE_2^2} \]

Key Distinction

The Chi-Square Test (\(\chi^2\))

• Use with two categorical variables
• Tests if distributions differ across categories

Formula: \[ \chi^2 = \sum \frac{(f_o - f_e)^2}{f_e} \] Reject H₀ if \(\chi^2\) is large enough.

Measures of Association

• Significance = whether a relationship exists
• Association = how strong that relationship is

Preferred qualities: - Based on PRE - Asymmetric (cause → effect)

Lambda (\(\lambda\))

Use for nominal variables

Formula: \[ \lambda = \frac{E_1 - E_2}{E_1} \]

( \(E_1\) ): prediction errors without IV
( \(E_2\) ): prediction errors with IV

Somers’ dyx

Use for ordinal variables

Formula: \[ d_{yx} = \frac{C - D}{C + D + T_y} \]

• ( C ): Concordant pairs
  
• ( D ): Discordant pairs
  
• ( T_y ): Ties on DV

Ranges from -1 to +1

Recap

• Test whether differences are real using t, Z, or ( ^2 )
• Use confidence intervals to assess uncertainty
• Match the test to your variable types
• Use lambda and Somers’ dyx to assess strength

Next time you see a gap, ask: > Is it real? Is it strong? Or just noise?