1 Introduction

The core problem

The survey has varying numbers of left, center, and right-leaning respondents across the six waves.

As part of the study we track political orientation groups over time with events between measurements. If the sample composition (percentage or left/center/right population) change it can confound the results. The problem is different percentages of respondents of a specific political orientation BETWEEN waves.

Because the composition change occurs at the same time as your event, you can’t separate:

Event effects (theoretical interest)
Composition effects (methodological artifact)

Weighting to consistent group sizes across waves removes this confounding by creating artificial stability in group representation, allowing you to isolate genuine event effects from sampling fluctuations.

community_variable <- "pe_left_center_right"
community_order <- c("left", "center", "right")

dimensions_order <- c("Overall", "Cognitive", "Behavioral", "Social")
wave_order <- c("First", "Second", "Third", "Fourth", "Fifth", "Sixth")

# Read the data from indices_table.txt
df <- as.data.frame(readRDS("Israel Survey/data/il_pe.RDS"))

2 Test Signficance to Political Orientation groups

The Pearson’s Chi-squared test examines whether the distribution of political groups (right/center/left) is independent across all survey waves.

Null Hypothesis (H₀): Group composition is the same across all waves
Alternative Hypothesis (H₁): Group composition differs significantly across waves

Cramer’s V is a measure of effect size for chi-squared tests that indicates the strength of association between categorical variables. In our case it is the asscociation between the waves and each of the variables: political orientation, gender and age_group. It ranges from 0 (no association) to 1 (perfect association) and is calculated by normalizing the chi-squared statistic by sample size and degrees of freedom. Common interpretations are: small effect (0.1), medium effect (0.3), and large effect (0.5), though these thresholds can vary by field and context.

χ² is the chi-squared test statistic, which measures how much the observed frequencies in your contingency table deviate from what you’d expect if there were no association between the variables (i.e., if the null hypothesis were true).

For example: in our gender case, the combination of a low chi-squared value (2.433), non-significant p-value (0.787), and very small Cramer’s V (0.018) for gender tells a consistent story: there’s essentially no meaningful relationship between survey wave and gender composition in your data.

results <- af_test_composition_differences(data = df, wave_var = "Wave", group_var = "pe_left_center_right")

results$interpretation

Chi-square Test Results

The Pearson's Chi-squared test examines whether group composition (pe_left_center_right) is independent across waves.

Results:

Chi-squared: 25.622 (df = 10)
P-value: 0.004283
Cramer's V: 0.042 (small effect size)

Interpretation:

✓ There are significant differences in group composition across waves. The effect size is small (Cramer's V = 0.042), indicating that while statistically meaningful, the practical differences in composition are relatively modest.

Methodological Implication:

Equal weighting or population weighting should be considered to control for composition differences in your analyses.

results$proportions_table

Wave	right	center	left
Group Proportions by Wave
First	56.0%	28.9%	15.1%
Second	55.2%	29.6%	15.2%
Third	61.2%	26.9%	12.0%
Fourth	60.1%	27.1%	12.8%
Fifth	58.3%	30.0%	11.7%
Sixth	59.4%	29.4%	11.1%

results$raw_counts_table

Wave	right	center	left
Raw Counts by Wave
First	901	464	243
Second	887	475	245
Third	542	238	106
Fourth	419	189	89
Fifth	889	457	178
Sixth	662	328	124


## Quick Analysis of Your Extremism Ratios
if(results$chi_square$p.value < 0.05) {
  cat("Significant composition differences detected - weighting recommended\n")
} else {
  cat("No significant composition differences - unweighted analysis appropriate\n")
}

Significant composition differences detected - weighting recommended

3 Test Signficance to Gender groups

The Pearson’s Chi-squared test examines whether the distribution of political groups (right/center/left) is independent across all survey waves.

Null Hypothesis (H₀): Group composition is the same across all waves
Alternative Hypothesis (H₁): Group composition differs significantly across waves

results <- af_test_composition_differences(data = df, wave_var = "Wave", group_var = "gender")

results$interpretation

Chi-square Test Results

The Pearson's Chi-squared test examines whether group composition (gender) is independent across waves.

Results:

Chi-squared: 2.433 (df = 5)
P-value: 0.786624
Cramer's V: 0.018 (small effect size)

Interpretation:

✗ There are no non-significant differences in group composition across waves. The effect size is small (Cramer's V = 0.018), indicating that while not statistically meaningful, the practical differences in composition are relatively modest.

Methodological Implication:

Raw unweighted analyses should be appropriate given the non-significant composition differences.

results$proportions_table

Wave	Male	Female
Group Proportions by Wave
First	49.3%	50.7%
Second	49.8%	50.2%
Third	48.4%	51.6%
Fourth	46.5%	53.5%
Fifth	49.1%	50.9%
Sixth	48.6%	51.4%

results$raw_counts_table

Wave	Male	Female
Raw Counts by Wave
First	792	816
Second	801	806
Third	429	457
Fourth	324	373
Fifth	748	776
Sixth	541	573

4 Test Signficance to Age group

The Pearson’s Chi-squared test examines whether the distribution of political groups (right/center/left) is independent across all survey waves.

Null Hypothesis (H₀): Group composition is the same across all waves
Alternative Hypothesis (H₁): Group composition differs significantly across waves

results <- af_test_composition_differences(data = df, wave_var = "Wave", group_var = "age_group")

results$interpretation

Chi-square Test Results

The Pearson's Chi-squared test examines whether group composition (age_group) is independent across waves.

Results:

Chi-squared: 193.351 (df = 15)
P-value: 0.000000
Cramer's V: 0.093 (small effect size)

Interpretation:

✓ There are significant differences in group composition across waves. The effect size is small (Cramer's V = 0.093), indicating that while statistically meaningful, the practical differences in composition are relatively modest.

Methodological Implication:

Equal weighting or population weighting should be considered to control for composition differences in your analyses.

results$proportions_table

Wave	18–30	31–45	46–60	60plus
Group Proportions by Wave
First	28.8%	32.3%	25.9%	13.0%
Second	24.8%	30.2%	23.8%	21.2%
Third	27.7%	33.3%	24.5%	14.6%
Fourth	23.8%	33.3%	25.7%	17.2%
Fifth	29.5%	30.6%	29.6%	10.4%
Sixth	18.1%	29.4%	25.9%	26.6%

results$raw_counts_table

Wave	18–30	31–45	46–60	60plus
Raw Counts by Wave
First	463	519	417	209
Second	398	485	383	341
Third	245	295	217	129
Fourth	166	232	179	120
Fifth	449	466	451	158
Sixth	202	327	289	296

5 Create Equal Weights

Same respondents percentage (33%) for political orientation groups within each wave

# Create equal weights  
df_equal_weighted <- af_create_equal_weights(data = df, wave_var = "Wave", group_var = "pe_left_center_right")
results <- af_summarize_weights (data = df_equal_weighted, wave_var = "Wave", group_var = "pe_left_center_right",
                                 weight_var = "equal_weight")
# Display the table
results$weight_table

Wave	Group	Original N	Effective N	Weight
Weighting Summary by Wave and Group
First	right	901	243.0	0.270
First	center	464	243.0	0.524
First	left	243	243.0	1.000
Second	right	887	245.0	0.276
Second	center	475	245.0	0.516
Second	left	245	245.0	1.000
Third	right	542	106.0	0.196
Third	center	238	106.0	0.445
Third	left	106	106.0	1.000
Fourth	right	419	89.0	0.212
Fourth	center	189	89.0	0.471
Fourth	left	89	89.0	1.000
Fifth	right	889	178.0	0.200
Fifth	center	457	178.0	0.389
Fifth	left	178	178.0	1.000
Sixth	right	662	124.0	0.187
Sixth	center	328	124.0	0.378
Sixth	left	124	124.0	1.000


# Display the interpretation 
results$interpretation

Weighting Summary

The weighting scheme transforms unequal group sizes into equal effective sample sizes within each wave.

Overall Statistics:

Total original sample: 7436
Total effective sample: 2955
Weight range: 0.187 to 1.000

Color Coding:

Blue (weight = 1.000): Smallest groups (no adjustment needed)
Red (weight < 1.000): Down-weighted groups (were over-represented)
Green (weight > 1.000): Up-weighted groups (were under-represented)

The 'Effective N' shows what each group contributes to analyses after weighting.

5.1 compare equal weighting

Use the equal weighting weights

results <- af_compare_regression_weighting(df_equal_weighted, "pe_violence", "pe_left_center_right", 
                                           "Wave", "equal_weight")
results$coefficient_table

Term	Unweighted Coef	Sig	Weighted Coef	Sig	Difference	% Change	Significance Change
Regression Weighting Impact: Equal_weight vs. Unweighted
(Intercept)	1.5893	***	1.5893	***	0.0000	0.0	Both significant
pe_left_center_rightcenter	−0.0907	.	−0.0907	.	0.0000	0.0	Both non-significant
pe_left_center_rightleft	−0.2025	**	−0.2025	***	0.0000	0.0	Both significant
WaveSecond	0.1565	***	0.1565	**	0.0000	0.0	Both significant
WaveThird	0.0715		0.0715		0.0000	0.0	Both non-significant
WaveFourth	−0.0622		−0.0622		0.0000	0.0	Both non-significant
WaveFifth	0.0045		0.0045		0.0000	0.0	Both non-significant
WaveSixth	−0.0399		−0.0399		0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveSecond	−0.1689	*	−0.1689	*	0.0000	0.0	Both significant
pe_left_center_rightleft:WaveSecond	−0.2026	*	−0.2026	**	0.0000	0.0	Both significant
pe_left_center_rightcenter:WaveThird	−0.0749		−0.0749		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveThird	−0.1934		−0.1934	*	0.0000	0.0	Gained significance
pe_left_center_rightcenter:WaveFourth	0.1381		0.1381		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveFourth	0.1276		0.1276		0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveFifth	0.0774		0.0774		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveFifth	0.1552		0.1552	.	0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveSixth	0.2025	*	0.2025	*	0.0000	0.0	Both significant
pe_left_center_rightleft:WaveSixth	0.5259	***	0.5259	***	0.0000	0.0	Both significant

results$interpretation

Regression Weighting Impact Analysis

Comparing regression results with and without equal_weight weighting for outcome: pe_violence

Model: pe_violence ~ pe_left_center_right * Wave

Summary Statistics:

Maximum absolute coefficient difference: 0.0000
Maximum absolute percent change: 0.0%
Significance changes: 1 coefficients
Impact magnitude: small
R-squared unweighted: 0.012
R-squared weighted: 0.021

Color Coding:

Yellow: Moderate coefficient changes (0.05-0.10)
Red: Large coefficient changes (≥0.10) or significance changes

Interpretation:

Equal_weight weighting has a small impact on regression estimates. Your regression conclusions should be robust to weighting choice.

Methodological Implication:

Either weighted or unweighted regression would lead to similar substantive conclusions.


# With additional controls
results <- af_compare_regression_weighting(df_equal_weighted, "pe_violence", "pe_left_center_right", 
                                           "Wave", "equal_weight", additional_vars = c("gender", "age_group"))
results$coefficient_table

Term	Unweighted Coef	Sig	Weighted Coef	Sig	Difference	% Change	Significance Change
Regression Weighting Impact: Equal_weight vs. Unweighted
(Intercept)	1.8116	***	1.7963	***	−0.0153	−0.8	Both significant
pe_left_center_rightcenter	−0.0538		−0.0578		−0.0040	−7.4	Both non-significant
pe_left_center_rightleft	−0.1591	*	−0.1631	**	−0.0040	−2.5	Both significant
WaveSecond	0.1616	***	0.1617	**	0.0001	0.0	Both significant
WaveThird	0.0827		0.0820		−0.0007	−0.9	Both non-significant
WaveFourth	−0.0364		−0.0378		−0.0015	−4.0	Both non-significant
WaveFifth	−0.0009		−0.0015		−0.0006	NA	Both non-significant
WaveSixth	−0.0069		−0.0085		−0.0016	−23.3	Both non-significant
genderFemale	−0.2216	***	−0.2130	***	0.0086	3.9	Both significant
age_group31–45	−0.0919	**	−0.0935	**	−0.0016	−1.7	Both significant
age_group46–60	−0.2382	***	−0.1928	***	0.0454	19.1	Both significant
age_group60plus	−0.2738	***	−0.2626	***	0.0112	4.1	Both significant
pe_left_center_rightcenter:WaveSecond	−0.1444	.	−0.1441	.	0.0003	0.2	Both non-significant
pe_left_center_rightleft:WaveSecond	−0.1922	*	−0.1927	**	−0.0005	−0.3	Both significant
pe_left_center_rightcenter:WaveThird	−0.0748		−0.0730		0.0018	2.4	Both non-significant
pe_left_center_rightleft:WaveThird	−0.2389	.	−0.2349	*	0.0040	1.7	Gained significance
pe_left_center_rightcenter:WaveFourth	0.1412		0.1411		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveFourth	0.0728		0.0786		0.0058	7.9	Both non-significant
pe_left_center_rightcenter:WaveFifth	0.0961		0.0937		−0.0025	−2.6	Both non-significant
pe_left_center_rightleft:WaveFifth	0.1662		0.1647	*	−0.0014	−0.9	Gained significance
pe_left_center_rightcenter:WaveSixth	0.2242	**	0.2228	*	−0.0015	−0.6	Both significant
pe_left_center_rightleft:WaveSixth	0.5087	***	0.5111	***	0.0024	0.5	Both significant

results$interpretation

Regression Weighting Impact Analysis

Comparing regression results with and without equal_weight weighting for outcome: pe_violence

Model: pe_violence ~ pe_left_center_right * Wave + gender + age_group

Summary Statistics:

Maximum absolute coefficient difference: 0.0454
Maximum absolute percent change: 23.3%
Significance changes: 2 coefficients
Impact magnitude: moderate
R-squared unweighted: 0.036
R-squared weighted: 0.041

Color Coding:

Yellow: Moderate coefficient changes (0.05-0.10)
Red: Large coefficient changes (≥0.10) or significance changes

Interpretation:

Equal_weight weighting has a moderate impact on regression estimates. Weighting meaningfully changes your regression estimates and/or significance patterns.

Methodological Implication:

Weighting choice affects your regression conclusions and should be carefully justified based on your research design.

6 Create Population weights

Fixed population ratio of Left(20%)/Center(35%)/Right(45%) applied to each wave

pop_props <- c("right" = 0.45, "center" = 0.35, "left" = 0.20)
df_pop_weighted <- af_create_pop_weights(data = df, wave_var = "Wave", group_var = "pe_left_center_right", 
                                         pop_props = pop_props)
Results <- af_summarize_weights (data = df_pop_weighted, wave_var = "Wave", group_var = "pe_left_center_right", 
                                 weight_var = "pop_weight")
# Display the table
results$weight_table

NULL


# Display the interpretation 
results$interpretation

Regression Weighting Impact Analysis

Comparing regression results with and without equal_weight weighting for outcome: pe_violence

Model: pe_violence ~ pe_left_center_right * Wave + gender + age_group

Summary Statistics:

Maximum absolute coefficient difference: 0.0454
Maximum absolute percent change: 23.3%
Significance changes: 2 coefficients
Impact magnitude: moderate
R-squared unweighted: 0.036
R-squared weighted: 0.041

Color Coding:

Yellow: Moderate coefficient changes (0.05-0.10)
Red: Large coefficient changes (≥0.10) or significance changes

Interpretation:

Equal_weight weighting has a moderate impact on regression estimates. Weighting meaningfully changes your regression estimates and/or significance patterns.

Methodological Implication:

Weighting choice affects your regression conclusions and should be carefully justified based on your research design.

6.1 compare population weighting

Use the equal weighting weights

results <- af_compare_regression_weighting(df_pop_weighted, "pe_violence", "pe_left_center_right", 
                                           "Wave", "pop_weight")
results$coefficient_table

Term	Unweighted Coef	Sig	Weighted Coef	Sig	Difference	% Change	Significance Change
Regression Weighting Impact: Pop_weight vs. Unweighted
(Intercept)	1.5893	***	1.5893	***	0.0000	0.0	Both significant
pe_left_center_rightcenter	−0.0907	.	−0.0907	.	0.0000	0.0	Both non-significant
pe_left_center_rightleft	−0.2025	**	−0.2025	**	0.0000	0.0	Both significant
WaveSecond	0.1565	***	0.1565	**	0.0000	0.0	Both significant
WaveThird	0.0715		0.0715		0.0000	0.0	Both non-significant
WaveFourth	−0.0622		−0.0622		0.0000	0.0	Both non-significant
WaveFifth	0.0045		0.0045		0.0000	0.0	Both non-significant
WaveSixth	−0.0399		−0.0399		0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveSecond	−0.1689	*	−0.1689	*	0.0000	0.0	Both significant
pe_left_center_rightleft:WaveSecond	−0.2026	*	−0.2026	*	0.0000	0.0	Both significant
pe_left_center_rightcenter:WaveThird	−0.0749		−0.0749		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveThird	−0.1934		−0.1934	.	0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveFourth	0.1381		0.1381		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveFourth	0.1276		0.1276		0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveFifth	0.0774		0.0774		0.0000	0.0	Both non-significant
pe_left_center_rightleft:WaveFifth	0.1552		0.1552	.	0.0000	0.0	Both non-significant
pe_left_center_rightcenter:WaveSixth	0.2025	*	0.2025	*	0.0000	0.0	Both significant
pe_left_center_rightleft:WaveSixth	0.5259	***	0.5259	***	0.0000	0.0	Both significant

results$interpretation

Regression Weighting Impact Analysis

Comparing regression results with and without pop_weight weighting for outcome: pe_violence

Model: pe_violence ~ pe_left_center_right * Wave

Summary Statistics:

Maximum absolute coefficient difference: 0.0000
Maximum absolute percent change: 0.0%
Significance changes: 0 coefficients
Impact magnitude: minimal
R-squared unweighted: 0.012
R-squared weighted: 0.015

Color Coding:

Yellow: Moderate coefficient changes (0.05-0.10)
Red: Large coefficient changes (≥0.10) or significance changes

Interpretation:

Pop_weight weighting has a minimal impact on regression estimates. Your regression conclusions should be robust to weighting choice.

Methodological Implication:

Either weighted or unweighted regression would lead to similar substantive conclusions.


# With additional controls
results <- af_compare_regression_weighting(df_pop_weighted, "pe_violence", "pe_left_center_right", 
                                           "Wave", "pop_weight", additional_vars = c("gender", "age_group"))
results$coefficient_table

Term	Unweighted Coef	Sig	Weighted Coef	Sig	Difference	% Change	Significance Change
Regression Weighting Impact: Pop_weight vs. Unweighted
(Intercept)	1.8116	***	1.8106	***	−0.0010	−0.1	Both significant
pe_left_center_rightcenter	−0.0538		−0.0545		−0.0007	−1.4	Both non-significant
pe_left_center_rightleft	−0.1591	*	−0.1598	*	−0.0007	−0.4	Both significant
WaveSecond	0.1616	***	0.1617	**	0.0001	0.1	Both significant
WaveThird	0.0827		0.0826		−0.0001	−0.1	Both non-significant
WaveFourth	−0.0364		−0.0364		0.0000	−0.1	Both non-significant
WaveFifth	−0.0009		−0.0012		−0.0002	NA	Both non-significant
WaveSixth	−0.0069		−0.0069		−0.0001	−0.8	Both non-significant
genderFemale	−0.2216	***	−0.2211	***	0.0005	0.2	Both significant
age_group31–45	−0.0919	**	−0.0968	**	−0.0048	−5.2	Both significant
age_group46–60	−0.2382	***	−0.2279	***	0.0103	4.3	Both significant
age_group60plus	−0.2738	***	−0.2741	***	−0.0003	−0.1	Both significant
pe_left_center_rightcenter:WaveSecond	−0.1444	.	−0.1442	.	0.0002	0.2	Both non-significant
pe_left_center_rightleft:WaveSecond	−0.1922	*	−0.1924	*	−0.0002	−0.1	Both significant
pe_left_center_rightcenter:WaveThird	−0.0748		−0.0744		0.0005	0.6	Both non-significant
pe_left_center_rightleft:WaveThird	−0.2389	.	−0.2381	*	0.0008	0.3	Gained significance
pe_left_center_rightcenter:WaveFourth	0.1412		0.1409		−0.0003	−0.2	Both non-significant
pe_left_center_rightleft:WaveFourth	0.0728		0.0740		0.0012	1.6	Both non-significant
pe_left_center_rightcenter:WaveFifth	0.0961		0.0955		−0.0007	−0.7	Both non-significant
pe_left_center_rightleft:WaveFifth	0.1662		0.1656	.	−0.0005	−0.3	Both non-significant
pe_left_center_rightcenter:WaveSixth	0.2242	**	0.2239	**	−0.0004	−0.2	Both significant
pe_left_center_rightleft:WaveSixth	0.5087	***	0.5091	***	0.0004	0.1	Both significant

results$interpretation

Regression Weighting Impact Analysis

Comparing regression results with and without pop_weight weighting for outcome: pe_violence

Model: pe_violence ~ pe_left_center_right * Wave + gender + age_group

Summary Statistics:

Maximum absolute coefficient difference: 0.0103
Maximum absolute percent change: 5.2%
Significance changes: 1 coefficients
Impact magnitude: small
R-squared unweighted: 0.036
R-squared weighted: 0.038

Color Coding:

Yellow: Moderate coefficient changes (0.05-0.10)
Red: Large coefficient changes (≥0.10) or significance changes

Interpretation:

Pop_weight weighting has a small impact on regression estimates. Your regression conclusions should be robust to weighting choice.

Methodological Implication:

Either weighted or unweighted regression would lead to similar substantive conclusions.

IL LCR - Weights

IL Political Extremism

Amir Freund

2025-09-02

1 Introduction

2 Test Signficance to Political Orientation groups

Chi-square Test Results

Results:

Interpretation:

Methodological Implication:

3 Test Signficance to Gender groups

Chi-square Test Results

Results:

Interpretation:

Methodological Implication:

4 Test Signficance to Age group

Chi-square Test Results

Results:

Interpretation:

Methodological Implication:

5 Create Equal Weights

Weighting Summary

Overall Statistics:

Color Coding:

5.1 compare equal weighting

Regression Weighting Impact Analysis

Summary Statistics:

Color Coding:

Interpretation:

Methodological Implication:

Regression Weighting Impact Analysis

Summary Statistics:

Color Coding:

Interpretation:

Methodological Implication:

6 Create Population weights

Regression Weighting Impact Analysis

Summary Statistics:

Color Coding:

Interpretation:

Methodological Implication:

6.1 compare population weighting

Regression Weighting Impact Analysis

Summary Statistics:

Color Coding:

Interpretation:

Methodological Implication:

Regression Weighting Impact Analysis

Summary Statistics:

Color Coding:

Interpretation:

Methodological Implication: