Abstract

This analysis considers how DV-related 911 calls in Douglas County, Nebraska have behaved leading up to and throughout the onset of COVID-19. More specifically, we consider to what extent DV-related 911 calls might have been impacted by the onset of COVID-19. We employ a quasi-experimental approach. We rely on time series modeling to predict how DV-related 911 calls would be expected to have behaved in the absence of COVID-19 (i.e. as a quasi-control group). We then compare projected rates of DV-related 911 calls in the quasi-control group to actual observed rates of calls (i.e. the treatment group). Comparisons between the quasi-control and treatment groups indicate [summary of findings].

This analysis includes the following sections:

1. Summary of Problem
2. Initial Data Exploration
3. Model Parameters
4. Model Summary
5. Conclusions

1. Summary of Problem


Motivation for research. Quick summary of recent articles exploring this same topic in other contexts.

2. Initial Data Exploration


Offer a brief description of the data. What is it. Where does it come from. Motivation for using calls as a proxy measurment of DV (as opposed to say, arrests).

As a first step, we conducted initial exploration of the data. Monthly number of calls ranged from X to Y, with a mean of Z The data exhibited strong seasonality, with monthly number of calls peaking in the summer months and subsiding in the winter months. Trends involving overall increases or decline in rates of calls over time were not apparent.

3. Model Parameters


Explain CasualImpact package (Bayesian structural time-series model). Factual vs. counter factual. Explain justification for treatment and control groups. Explain justification for inclusion of seasonal component of model. Note any other priors.

Offer the snippet that defines the model:

model <- CausalImpact(data_for_impact, pre_period, post_period, model.args = list(nseasons = 4, season.duration = 3, prior.level.sd=.05))

4. Model Summary

Convey and explain the top-level insights learned from the model. Then get into details.

Offer a general interpretation of the model's visual summary.

  • The first panel shows the data and a counterfactual prediction for the post-treatment period

  • The second panel shows the difference between observed data and counterfactual predictions.

  • The third panel adds up the pointwise contributions from the second panel, resulting in a plot of the cumulative effect of the intervention.

Offer a general interpretation of the model's numerical summary.

## Posterior inference {CausalImpact}
## 
##                          Average        Cumulative    
## Actual                   1477           14769         
## Prediction (s.d.)        1439 (37)      14393 (374)   
## 95% CI                   [1361, 1509]   [13612, 15090]
##                                                       
## Absolute effect (s.d.)   38 (37)        376 (374)     
## 95% CI                   [-32, 116]     [-321, 1157]  
##                                                       
## Relative effect (s.d.)   2.6% (2.6%)    2.6% (2.6%)   
## 95% CI                   [-2.2%, 8%]    [-2.2%, 8%]   
## 
## Posterior tail-area probability p:   0.14458
## Posterior prob. of a causal effect:  86%
## 
## For more details, type: summary(impact, "report")

During the post-intervention period, the response variable had an average value of approx. 1.48K. In the absence of an intervention, we would have expected an average response of 1.44K. The 95% interval of this counterfactual prediction is [1.37K, 1.51K]. Subtracting this prediction from the observed response yields an estimate of the causal effect the intervention had on the response variable. This effect is 0.04K with a 95% interval of [-0.03K, 0.11K]. For a discussion of the significance of this effect, see below.

Summing up the individual data points during the post-intervention period, the response variable had an overall value of 14.77K. Had the intervention not taken place, we would have expected a sum of 14.39K. The 95% interval of this prediction is [13.71K, 15.10K].

The above results are given in terms of absolute numbers. In relative terms, the response variable showed an increase of +3%. The 95% interval of this percentage is [-2%, +7%].

This means that, although the intervention appears to have caused a positive effect, this effect is not statistically significant when considering the entire post-intervention period as a whole. Individual days or shorter stretches within the intervention period may of course still have had a significant effect, as indicated whenever the lower limit of the impact time series (lower plot) was above zero. The apparent effect could be the result of random fluctuations that are unrelated to the intervention. This is often the case when the intervention period is very long and includes much of the time when the effect has already worn off. It can also be the case when the intervention period is too short to distinguish the signal from the noise. Finally, failing to find a significant effect can happen when there are not enough control variables or when these variables do not correlate well with the response variable during the learning period.

The probability of obtaining this effect by chance is p = 0.146. This means the effect may be spurious and would generally not be considered statistically significant.

5 Conclusions


Offer non-technical summary of what was learned.