Section 1 - Before-and-after comparisons of retail prices of stores in Berkeley, California.

## # A tibble: 6 × 12
##   store_id type  store_type type2   size price price_per_oz price_per_oz_c taxed
##      <dbl> <chr>      <dbl> <chr>  <dbl> <dbl>        <dbl>          <dbl> <dbl>
## 1       16 WATER          2 <NA>    33.8  1.69       0.05             5        0
## 2       16 TEA            2 <NA>    23    0.99       0.0430           4.30     1
## 3       16 TEA            2 <NA>    23    0.99       0.0430           4.30     1
## 4       16 WATER          2 <NA>    33.8  1.69       0.05             5        0
## 5       16 MILK           2 LOW F… 128    3.79       0.0296           2.96     0
## 6       16 MILK           2 LOW F…  64    2.79       0.0436           4.36     0
## # ℹ 3 more variables: supp <dbl>, time <chr>, product_id <dbl>

1. Read ‘S1 Text’, from the journal paper’s supporting information, which explains how the Store Price Survey data was collected.

a. In your own words, explain how the product information was recorded, and the measures that researchers took to ensure that the data was accurate and representative of the treatment group. What were some of the data collection issues that they encountered?

The product information was recorded by trained data collectors who entered the pre-sales tax and pre-bottle fee price information into a database using a tablet computer and paper forms (December 2014) or only paper forms (June 2015 and March 2016). The researchers took several measures to ensure that the data was accurate and representative of the treatment group, such as:

  • Selecting a targeted sample of stores based on the top six stores where participants most frequently shopped, as reported in the Dietary and Shopping Behavior (DSB) telephone survey, and adding additional stores from random selection within their categories and neighborhoods to ensure diversity and coverage.

  • Using a standard panel of 70 beverage items based on information on top selling beverages in the Bay Area and nationally and representing the range of beverage categories, and collecting additional beverages in a supplemental panel to account for variations in store inventory.

  • Using a systematic, standardized process to collect the prices, and double-entering the data collected on paper forms by trained research project assistants into a database and comparing the results.

  • Limiting the analysis to items matched as to product and store in all three rounds, which comprised 55 products and 313 prices across stores. Some of the data collection issues that they encountered were:

  • Some stores refused to allow the data collectors to enter prices, and had to be replaced by other stores of the same type and neighborhood.

  • None of the stores sold all the beverages in the panel, and some stores did not stock any beverages in the standard panel, so the data collectors had to collect prices for beverages in the supplemental panel.

  • The tablet computers were not used during June 2015 and March 2016 because the data collection was more efficient and conducted more quickly using paper forms.

b. Each product was given a unique ID number. How many different products are in the dataset?

Product_ID Frequency
1 3
2 3
3 3
4 3
5 10
6 6
7 3
8 6
9 3
10 2
11 4
12 2
13 2
14 2
15 2
16 15
17 2
18 5
19 3
20 3
21 2
22 3
23 3
24 2
25 3
26 2
27 2
28 1
29 22
30 2
31 1
32 32
33 30
34 1
35 1
36 1
37 2
38 16
39 1
40 17
41 22
42 28
43 36
44 12
45 1
46 1
47 7
48 3
49 2
50 18
51 4
52 2
53 1
54 1
55 1
56 1
57 19
58 39
59 55
60 43
61 38
62 3
63 2
64 2
65 1
66 1
67 2
68 3
69 2
70 1
71 2
72 6
73 2
74 1
75 1
76 18
77 16
78 1
79 18
80 1
81 18
82 4
83 1
84 2
85 7
86 3
87 3
88 3
89 3
90 3
91 14
92 3
93 2
94 1
95 1
96 1
97 42
98 14
99 6
100 30
101 5
102 8
103 8
104 28
105 51
106 10
107 1
108 40
109 3
110 19
111 17
112 44
113 49
114 47
115 2
116 2
117 11
118 2
119 8
120 3
121 1
122 3
123 2
124 2
125 21
126 17
127 1
128 3
129 9
130 23
131 1
132 1
133 10
134 1
135 1
136 2
137 4
138 1
139 1
140 34
141 1
142 2
143 1
144 9
145 2
146 16
147 3
148 3
149 3
150 1
151 3
152 1
153 9
154 1
155 3
156 2
157 2
158 1
159 1
160 22
161 1
162 49
163 9
164 3
165 1
166 1
167 1
168 3
169 3
170 15
171 14
172 42
173 2
174 2
175 3
176 2
177 3
178 1
179 4
180 1
181 15
182 3
183 1
184 1
185 4
186 1
187 4
188 26
189 49
190 35
191 4
192 3
193 53
194 2
195 52
196 35
197 1
198 1
199 1
200 1
201 1
202 1
203 2
204 1
205 1
206 11
207 26
208 2
209 1
210 1
211 2
212 1
213 29
214 43
215 35
216 1
217 1
218 3
219 40
220 8
221 1
222 1
223 2
224 1
225 2
226 1
227 1
228 3
229 3
230 20
231 21
232 40
233 3
234 3
235 3
236 2
237 1
238 1
239 3
240 3
241 1
242 1
243 3
244 4
245 1
246 1
247 1

Following the approach described in ‘S1 Text’, we will compare the variable price per ounce in US$ cents (price_per_oz_c). We will look at what happened to prices in the two treatment groups before the tax (time = DEC2014) and after the tax (time = JUN2015):

treatment group one: large supermarkets (store_type = 1)

treatment group two: pharmacies (store_type = 3)

Before doing this analysis, we will use summary measures to see how many observations are in the treatment and control group, and how the two groups differ across some variables of interest. For example, if there are very few observations in a group, we might be concerned about the precision of our estimates and will need to interpret our results in light of this fact.

We will now create frequency tables containing the summary measures that we are interested in.

2. Create the following tables:

a. A frequency table showing the number (count) of store observations (store type) in December 2014 and June 2015, with ‘store type’ as the row variable and ‘time period’ as the column variable. For each store type, is the number of observations similar in each time period?

##    
##     DEC2014 JUN2015
##   1     177     209
##   3      87     102

While the observations are not similar, they are also not far apart (or far apart in this case due to the small data size).

b. A frequency table showing the number of taxed and non-taxed beverages in December 2014 and June 2015, with ‘store type’ as the row variable and ‘taxed’ as the column variable. (‘Taxed’ equals 1 if the sugar tax applied to that product, and 0 if the tax did not apply). For each store type, is the number of taxed and non-taxed beverages similar?

##    
##       0   1
##   1 291 253
##   3 132 130

In store 3, yes, the number of taxed and non-taxed beverages are similar, while, in store 1, no, the number of taxed and non-taxed beverages are not similar.

c. A frequency table showing the number of each product type (type), with ‘product type’ as the row variable and ‘time period’ as the column variable. Which product types have the highest number of observations and which have the lowest number of observations? Why might some products have more observations than others?

##              
##               DEC2014 JUN2015
##   ENERGY           56      58
##   ENERGY-DIET      49      54
##   JUICE            70      64
##   JUICE DRINK      19      17
##   MILK             63      61
##   SODA            239     262
##   SODA-DIET       128     174
##   SPORT            11      16
##   SPORT-DIET        2       2
##   TEA              52      45
##   TEA-DIET          6       6
##   WATER            48      38
##   WATER-SWEET       1       1

  • Soda and Soda-Diet have the highest number of observations in both the time period.

  • The products in the standard panel were based on the top selling beverages in the Bay Area and nationally, which means that some products might be more popular and widely available than others. Secondly, the products in the supplemental panel were collected to account for variations in store inventory, which means that some products might be more specific and rare than others. For example, water-sweet had only one observation in both rounds, which suggests that it was a very uncommon beverage in the stores. Additionally, the data collection process involved replacing some stores that refused to allow the data collectors to enter prices, which means that some products might have different availability and prices in the new stores. Lastly, the data analysis process involved limiting the analysis to items matched as to product and store in both the rounds (DEC2014 and JUN2015), which means that some products might have fewer observations than others due to missing data or unmatched items.

Besides counting the number of observations in a particular group, we can also calculate the mean by only using observations that satisfy certain conditions (known as the conditional mean). In this case, we are interested in comparing the mean price of taxed and untaxed beverages, before and after the tax.

3. Calculate and compare conditional means:

a. Create a table showing the average price per ounce (in cents) for taxed and untaxed beverages separately, with ‘store type’ as the row variable, and ‘taxed’ and ‘time’ as the column variables. To follow the methodology used in the journal paper, make sure to only include products that are present in all time periods, and non-supplementary products (supp = 0).

Untaxed
Taxed
Store type Total DEC2014 JUN2015 DEC2014 JUN2015
1 36 11.19195 11.4804 15.61744 16.92966
3 18 15.19575 16.0754 18.18177 19.07878

b. Without doing any calculations, summarize any differences or general patterns between December 2014 and June 2015 that you find in the table.

The average price per ounce (in cents) for both taxed and untaxed beverages increased for store type 1 but increased at a decreasing number for store type 3, with taxed beverages being more expensive and having a larger price gap than untaxed beverages for both store types.

c. Would we be able to assess the effect of sugar taxes on product prices by comparing the average price of untaxed goods with that of taxed goods in any given period? Why or why not?

No, because the tax is not the only factor that affects the average price of the products in each group. The groups are not the same to begin with, as shown by the difference in average price before the tax was introduced in December 2014. The difference in June 2015 could reflect the pre-existing differences between the groups, not the impact of the tax. One of the obvious examples could be owner’s will to earn extra profits.

In order to make a before-and-after comparison, we will make a chart similar to Figure 2 in the journal paper, to show the change in prices for store type 1 and 3.

4. Using your table from Question 3:

a. Calculate the change in the mean price after the tax (price in June 2015 minus price in December 2014) for taxed and untaxed beverages, by store type 1 and 3.

Untaxed
Taxed
Store type DEC2014 JUN2015 DEC2014 JUN2015 Difference
Large Supermarket 11.19195 11.4804 NA NA 0.2884493
Pharmacy 15.19575 16.0754 NA NA 0.8796482
Large Supermarket NA NA 15.61744 16.92966 1.3122212
Pharmacy NA NA 18.18177 19.07878 0.8970093

b. Using the values you calculated in Question 4(a), plot a column chart to show this information (as done in Figure 2 of the journal paper) with store type on the horizontal axis and price change on the vertical axis. Label each axis and data series appropriately. You should get the same values as shown in Figure 2.

To assess whether the difference in mean prices before and after the tax could have happened by chance due to the samples chosen (and there are no differences in the population means), we could calculate the p-value. (Here, ‘population means’ refer to the mean prices before/after the tax that we would calculate if we had all prices for all stores in Berkeley.) The authors of the journal article calculate p-values, and use the idea of statistical significance to interpret them. Whenever they get a p-value of less than 5%, they conclude that the assumption of no differences in the population is unlikely to be true: they say that the price difference is statistically significant. If they get a p-value higher than 5%, they say that the difference is not statistically significant, meaning that they think it could be due to chance variation in prices.

Using a particular cutoff level for the p-value, and concluding that a result is only statistically significant if the p-value is below the cutoff, is common in statistical studies, and 5% is often used as the cutoff level. But this approach has been criticized recently by statisticians and social scientists. The main criticisms raised are that any cutoffs are arbitrary. Instead of using a cutoff, we prefer to calculate p-values and use them to assess the strength of the evidence against our assumption that there are no differences in the population means. Whether the statistical evidence is strong enough for us to draw a conclusion about a policy, such as a sugar tax, will always be a matter of judgement.

According to the journal paper, the p-value is 0.02 for large supermarkets, and 0.99 for pharmacies.

5. Based on these p-values and your chart from Question 4, what can you conclude about the difference in means?

The p-value for large supermarkets is quite small, indicating we can reject the null hypotheses showing that there is enough evidence against our assumption that there are no differences in the populations mean (before- and after-tax prices), as long as other assumptions about the data (e.g. stores were really sampled at random) were correct. Thus it is likely that the sugar tax had some effect on prices.

On the other hand, the p-value for pharmacies is quite large, showing that the the populations mean (before- and after-tax price differences) is not statistically significant, thus, we do not have strong evidence against our assumption. In this case, it is unlikely, that the sugar tax had some effect on prices.

Section 3.2 Before-and-after comparisons with prices in other areas

When looking for any price patterns, it is possible that the observed changes in Berkeley were not solely due to the tax, but instead were also influenced by other events that happened in Berkeley and in neighbouring areas. To investigate whether this is the case, the researchers conducted another differences-in-differences analysis, using a different treatment and control group:

- The treatment group: Beverages in Berkeley

- The control group: Beverages in surrounding areas.

The researchers collected price data from stores in the surrounding areas and compared them with prices in Berkeley. If prices changed in a similar way in nearby areas (which were not subject to the tax), then what we observed in Berkeley may not be primarily a result of the tax. We will be using the data the researchers collected to make our own comparisons.

Download the following files:

‘S5 Table’ (https://tinyco.re/7724734) comparing the neighbourhood characteristics of the Berkeley and non-Berkeley stores.

1. Based on ‘S5 Table’, do you think the researchers chose suitable comparison stores? Why or why not?

In doing a difference-in-difference analysis, the characteristics of both the locations should be relative. In other words, it is safe to compare two locations with similar characteristics to see if one event (sugar tax) in a location, Berkeley in this case, (and not the other) would be the major reason for the change in the beverage prices or not.

Looking at the table, yes, the researchers chose suitable comparison stores, as the stores’ characteristics are very similar to those in Berkeley.

2. Assess the effects of a tax on prices:

a. Create a table to show the average price in each month for taxed and non-taxed beverages, according to location. Use ‘year and month’ as the row variables, and ‘tax’ and ‘location’ as the column variables.

Untaxed
Taxed
Year Month Berkeley Non-Berkeley Berkeley Non-Berkeley
2013 1 5.722480 5.348640 8.692803 7.991574
2013 2 5.806468 5.363863 8.654572 8.180877
2013 3 5.858251 5.424512 8.822691 8.186872
2013 4 5.858342 5.643459 9.022075 8.246447
2013 5 5.791333 5.181730 8.678542 7.756793
2013 6 5.761845 5.031972 8.573449 7.426397
2013 7 5.902819 5.098175 8.233288 7.193616
2013 8 5.831730 5.080378 8.821654 7.488764
2013 9 5.834511 5.082787 9.015901 7.804120
2013 10 5.800953 5.176406 8.982419 7.753430
2013 11 5.983980 5.292364 9.090955 7.951408
2013 12 6.016732 5.273657 8.950652 7.798748
2014 1 6.029485 5.296528 8.850165 7.879612
2014 2 6.081301 5.427840 9.027708 7.797533
2014 3 6.359673 5.740176 9.370788 8.028207
2014 4 6.247272 5.681016 9.439623 8.172451
2014 5 6.415236 5.639642 9.512144 8.368728
2014 6 6.470002 5.718528 9.058231 8.000090
2014 7 6.398847 5.526962 9.181642 8.144422
2014 8 6.289867 5.587100 9.224782 8.000210
2014 9 6.577316 5.775283 9.222504 8.507376
2014 10 6.338737 5.559348 9.416063 8.520110
2014 11 5.839390 5.641029 8.023078 8.462292
2014 12 6.164794 5.493931 9.618825 8.484427
2015 1 6.373576 5.824950 9.968138 8.778852
2015 2 6.380764 5.654277 9.222594 8.471740
2015 3 6.482227 5.688276 9.973454 8.776122
2015 4 6.461740 5.810564 10.353428 8.733491
2015 5 6.623755 5.806210 10.310571 8.918695
2015 6 6.630742 5.818193 10.410781 8.682467
2015 7 6.371787 5.735532 10.588301 8.910685
2015 8 6.454268 5.701793 11.109456 9.040408
2015 9 6.515334 5.680509 10.452208 8.717005
2015 10 6.562519 5.691359 10.730807 8.818358
2015 11 6.659933 5.838232 10.697508 8.967660
2015 12 6.551349 5.750560 10.521530 8.571704
2016 1 6.555559 5.848632 10.525640 8.926457
2016 2 6.546415 5.764717 10.815509 8.729508
2016 3 NA NA NA NA

b. Plot the four columns of your table on the same line chart, with average price on the vertical axis and time (months) on the horizontal axis. Describe any differences you see between the prices of non-taxed goods in Berkeley and those outside Berkeley, both before the tax (January 2013 to December 2014) and after the tax (March 2015 onwards). Do the same for prices of taxed goods.

The prices of beverages that are not taxed are higher in Berkeley than in other areas. This shows that there are some obivous factors that make Berkeley different from its neighbours, regardless of the tax policy. The tax does not seem to affect the prices of non-taxed beverages, as the gap between Berkeley and other areas remains stable after the tax.

The prices of beverages that are taxed are also higher in Berkeley than in other areas. The tax makes this gap bigger, as the prices of taxed beverages in Berkeley go up after the tax is introduced (from March 2015 onwards).

The prices of both taxed and non-taxed beverages in other areas follow a similar pattern over time, and this pattern is also similar to the one in Berkeley for non-taxed beverages. However, if we remove the effect of time, the prices of both types of beverages in other areas do not change much before and after the tax. This implies that nothing else happened during that period that influenced the prices in Berkeley and its surroundings, except for the tax that affected the prices of taxed beverages in Berkeley.

c. Based on your chart, is it reasonable to conclude that the sugar tax had an effect on prices?

Yes, it is reasonable to conclude that the sugar tax had an effect on prices. The chart shows that the prices of taxed goods in Berkeley increased more than the prices of taxed goods outside Berkeley after the tax was introduced. This means that the tax made the goods more expensive in Berkeley than in other areas. However, the chart also shows that the prices of all goods in Berkeley were higher than the prices of all goods outside Berkeley before the tax was implemented. This means that there were some other factors that made Berkeley different from other areas, regardless of the tax policy. To isolate the effect of the tax, we need to subtract the difference in prices before the tax from the difference in prices after the tax. This gives us the change in the difference in prices due to the tax. The chart shows that this change is positive, which suggests that the tax has a positive effect on prices.

The aim of the sugar tax was to decrease consumption of sugary beverages. The table below shows the mean number of calories consumed and the mean volume consumed before and after the tax. The researchers reported the p-values for the difference in means before and after the tax in the last column.

Usual intake Pre-tax (Nov–Dec 2014) Post-tax (Nov–Dec 2015) Pre-tax–post-tax difference
Caloric intake (kilocalories/capita/day) NA, n = 623 NA, n = 613 NA
Taxed beverages 45.1 38.7 -6.4, p = 0.56
Non-taxed beverages 115.7 147.6 31.9, p = 0.04
Volume of intake (grams/capita/day) NA, n = 623 NA, n = 613 NA
Taxed beverages 121 97 -24.0, p = 0.24
Non-taxed beverages 1839.4 1896.5 57.1, p = 0.22

3. Based on Figure table above, what can you say about consumption behaviour in Berkeley after the tax? Suggest some explanations for the evidence.

The p-value for the caloric intake of non-taxed beverages is below 0.05, while the p-values for the other groups are fairly large. This implies that we can reject the null hypothesis that the tax did not change their consumption behavior from sugary to non-sugary beverages for the non-taxed beverages group, but we cannot reject it for the other groups, at the 5% significance level. The difference in the mean caloric intake of sugary beverages between the treatment and control groups is not very large.

It would be interesting to explore what other factors might have contributed to the reduction in the intake of sugary beverages after the sugar tax, even though the difference is not very large. One possible explanation is that water and sugar are essential for biological survival, making the demand for beverages relatively inelastic. Moreover, some people may have developed a habit of consuming certain beverages as part of their lifestyle and routine, such as sugary drinks from Starbucks. However, since people take time to change their consumption habits, which is beyond the scope of this project, a longer-term analysis might provide a stronger basis for the reasoning of the change in consumption of both types of drinks.

4. Read the ‘Limitations’ in the ‘Discussions’ section of the paper and discuss the strengths and limitations of this study. How could future studies on the sugar tax in Berkeley address these problems? (Some issues you may want to discuss are: the number of stores observed, number of people surveyed, and the reliability of the price data collected.)

The study has several strengths, such as using multiple data sources and methods, comparing Berkeley with nearby non-Berkeley areas, and adjusting for inflation and seasonality. However, the study also has some limitations that may affect the validity and generalizability of the findings. For example, the study only observed 26 stores in Berkeley, which may not represent the diversity of the retail environment and consumer behavior. The study also relied on self-reported consumption data from a telephone survey with a low response rate, which may introduce measurement error and selection bias including calorie and gram intakes (as opposed to frequency). Moreover, the study did not account for the possible substitution effects of the tax, such as cross-border shopping, online purchasing, or switching to other unhealthy products.

Future studies on the SSB tax in Berkeley could address these problems by expanding the sample size and scope of the data collection, using more objective and reliable measures of consumption, and controlling for the potential confounding factors and behavioral responses of the tax. For example, future studies could use scanner data from a larger and more representative sample of stores in Berkeley and other areas, including convenience stores, restaurants, and vending machines. Future studies could also use technologies, such as urinary sucrose or fructose, to measure the actual intake of SSBs and other sugars (if they are still going to use grams/calories as a measurement tool), rather than relying on self-reports. Furthermore, future studies could use an additional econometric method (synthetic control in DiD), to estimate the causal effect of the tax, while accounting for the possible spillover effects, price endogeneity, and heterogeneity of the impact. Nevertheless, as mentioned, this study was tight on time and thus a perfect foundation for future studies.

5. Suppose that you have the authority to conduct your own sugar tax natural experiment in two neighbouring towns, Town A and Town B. Outline how you would conduct the experiment to ensure that any changes in outcomes (prices, consumption of sugary beverages) are due to the tax and not due to other factors. (Hint: think about what factors you need to hold constant.)

I would still continue with the DiD method:

  1. I would choose two towns that have similar characteristics as this study did (population size, income level, demographic composition, and consumption patterns of sugary and non-sugary beverages) for a comparable baseline and that are not affected by different factors that could influence the outcomes of interest. The selection of these two towns will consider the fact that this study would not want the population to cross-border shop instead.

  2. I will then randomly assign one town to be the treatment group and the other town to be the control group. The treatment group will have a sugar tax imposed on sugary beverages, while the control group will not. This will ensure that the assignment of the tax is independent of any other variables that could affect the outcomes of interest.

  3. Next I would collect data on the prices and consumption of sugary and non-sugary beverages in both towns, both before and after the tax is implemented. The data should be collected from a representative sample of diverse stores and consumers (as many as time permits and that is considered good enough) in both towns, making sure that we have the tools and methods we need, such as scanner data (which was unavailable in some stores as part of the previous study) and bio markers (calorie/gram intake records). The reason why I chose these two methods is because we can have a much more insightful and accurate representation of what the outcome would be. For example, the demand of non-sugary drinks was substituted with sugary drinks, which corresponds with the decrease in general sugar level content by the sample population. With that said, the data should also include information on other sources of sugar, such as food or other drinks, and other factors that could not only affect the consumption of beverages (income, preferences, beliefs, and habits), but also help to identify the unique sugar content from such beverages as close as we can. Furthermore, it is also possible that, just like in the previous study, campaign-related activities may happen even if we decide to unexpectedly impose the sugar tax on a chosen classified date (for the purpose of reducing the effect of events on our data). One way to tackle this is to account for this extreme event by calibrating day-to-day data using the pre and post campaign data, either by modelling a baseline that considers extreme variations or by imputing results using pre- and post- information.

  4. Finally, I will compare the difference in the outcomes of interest between the treatment and control group before the tax with the difference in the outcomes after the tax. This is the DiD method, which can estimate the causal effect of the tax, while controlling for any pre-existing differences or common trends between the two towns. The effect of the tax can be measured by the change in the difference in prices or consumption of sugary and non-sugary beverages between the two towns. If the tax has a negative effect on the consumption of sugary beverages, then the difference in consumption between the two towns should decrease after the tax. If the tax has a positive effect on the prices of sugary beverages, then the difference in prices between the two towns should increase after the tax.

Reference

  1. Lynn D. Silver, Shu Wen Ng, Suzanne Ryan-Ibarra, Lindsey Smith Taillie, Marta Induni, Donna R. Miles, Jennifer M. Poti, and Barry M. Popkin. 2017. S3 Table in ‘Changes in prices, sales, consumer spending, and beverage consumption one year after a tax on sugar-sweetened beverages in Berkeley, California, US: A before-and-after study’. PLoS Med 14 (4): e1002283.