Right to Carry Law Effects - Assessing Publication Bias in RAND’s Analysis
Setup
library(pacman); p_load(ggplot2, DT, meta, metafor, dmetar, DescTools, tools, metasens)
datatable(data, extensions = c("Buttons", "FixedColumns"), options = list(dom = 'Bfrtip', buttons = c('copy', 'csv', 'print'), scrollX = T, fixedColumns = list(leftColumns = 3)))GoodData <- subset(data, Quality == "Good")
BadData <- subset(data, Quality == "Bad")
ViolentCrimeData <- subset(data, VCR == "Yes")Rationale
RAND produced a meta-analysis of the effects of concealed carry laws on violent crime in 2018 and last updated that analysis in 2020. The key findings from the report were that concealed carry law effects on total and firearm homicides, robberies, assaults, and rapes were uncertain, whereas there was some limited evidence that they might increase total violent crime rates. This analysis was a relatively standard meta-analysis, but it did not include assessment of publication bias. Publication bias is a prominent concern with this sort of analysis because this topic is very politically charged, and thus it is crucial to ensure that the results are interpreted with potential researcher biases towards, for example, publishing results suggesting concealed carry laws increase or decrease violence, in mind.
I aimed to redo their meta-analysis in order to incorporate publication bias checks. First, I will run the meta-analysis for the IRRs, which must be logged before they are backtransferred to the correct effect size for interpretability. Second, I will test for publication bias with (1) Egger’s regression test, (2) p-curve, (3) trim-and-fill, and (4) a limit meta-analysis. If anyone would like me to do subgroup analyses, they can email me or run them themselves because I have provided the data above. The only subgroups I looked at were studies that RAND labeled methodologically problematic versus unproblematic and studies that labeled as violent crime versus homicide or assault rate on law enforcement officers. When they classified homicides under violent crime, I did not classify them as violent crime, with the rest of the homicides.
Analysis
Meta-analysis
IRDMeta <- metagen(
log(ES),
SE,
sm = "IRR",
studlab = Study,
backtrans = T,
data = data,
hakn = T,
method.tau = "SJ",
title = "Effects of Concealed-Carry Laws on Violent Crime Rates"); summary(IRDMeta)## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## IRR 95%-CI %W(common)
## Donohue, Aneja & Weber (2019a) 1.0200 [0.9230; 1.1272] 0.0
## Hamill et al. (2019a) 1.0000 [0.9228; 1.0837] 0.0
## Aneja, Donohue & Zhang (2014a) 1.0300 [0.9050; 1.1722] 0.0
## Kendall & Tamura (2010a) 1.0000 [0.9941; 1.0059] 4.8
## Hepburn et al. (2004) 1.0100 [0.9320; 1.0945] 0.0
## Donohue, Aneja & Weber (2019b) 1.0300 [0.9050; 1.1722] 0.0
## Hamill et al. (2019b) 1.0700 [0.9682; 1.1825] 0.0
## French & Heagerty (2008) 1.0600 [0.9975; 1.1264] 0.0
## Donohue, Aneja & Weber (2019c) 1.0200 [0.9561; 1.0882] 0.0
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 0.0
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 0.4
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 0.2
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 0.0
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 4.8
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 0.1
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 0.0
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 43.3
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 0.4
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 0.0
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 43.3
## Crifasi, Pollack & Webster (2016a) 1.0200 [0.8031; 1.2955] 0.0
## Crifasi, Pollack & Webster (2016b) 0.9200 [0.7131; 1.1870] 0.0
## Crifasi, Pollack & Webster (2016c) 1.2700 [0.7585; 2.1265] 0.0
## Crifasi, Pollack & Webster (2016d) 0.7200 [0.5252; 0.9871] 0.0
## Crifasi, Pollack & Webster (2016e) 0.7400 [0.4669; 1.1729] 0.0
## Crifasi, Pollack & Webster (2016f) 0.7400 [0.4770; 1.1479] 0.0
## La Valle (2013a) 0.8700 [0.7826; 0.9671] 0.0
## La Valle (2013b) 0.8500 [0.7498; 0.9636] 0.0
## La Valle & Glover (2012a) 1.2300 [1.0131; 1.4934] 0.0
## Luca, Malhotra & Poliquin (2017a) 1.0600 [0.8730; 1.2870] 0.0
## La Valle & Glover (2012b) 1.3200 [1.0915; 1.5964] 0.0
## Luca, Malhotra & Poliquin (2017b) 1.0800 [0.8404; 1.3880] 0.0
## Luca, Malhotra & Poliquin (2017c) 1.0500 [0.9262; 1.1903] 0.0
## La Valle & Glover (2012c) 0.8100 [0.7287; 0.9004] 0.0
## Luca, Malhotra & Poliquin (2017d) 1.0600 [0.8628; 1.3022] 0.0
## La Valle & Glover (2012d) 0.7700 [0.6832; 0.8678] 0.0
## Luca, Malhotra & Poliquin (2017e) 1.0500 [0.8218; 1.3415] 0.0
## Luca, Malhotra & Poliquin (2017f) 1.1300 [0.9622; 1.3270] 0.0
## Siegal et al. (2017a) 1.0600 [1.0233; 1.0981] 0.1
## Webster, Crifasi & Vernick (2014a) 1.0600 [0.9878; 1.1375] 0.0
## Grambsch (2008) 1.0100 [0.9846; 1.0361] 0.3
## Rosengart et al. (2005a) 1.0700 [0.9739; 1.1756] 0.0
## Martin & Legault (2005a) 0.9500 [0.8993; 1.0036] 0.1
## Kovandzic, Marvell & Vieraitis (2005a) 1.0000 [0.9374; 1.0668] 0.0
## Siegal et al. (2017b) 1.0900 [1.0481; 1.1336] 0.1
## Webster, Crifasi & Vernick (2014b) 1.0600 [0.9592; 1.1714] 0.0
## Crifasi et al. (2018a) 1.0400 [1.0198; 1.0606] 0.4
## Rosengart et al. (2005b) 1.1100 [0.9791; 1.2583] 0.0
## Siegal et al. (2017c) 1.0100 [0.9561; 1.0670] 0.1
## Webster, Crifasi & Vernick (2014c) 1.1000 [0.9857; 1.2276] 0.0
## Crifasi et al. (2018b) 1.0300 [1.0002; 1.0607] 0.2
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 0.1
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 0.1
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 0.1
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 0.0
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 0.1
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 0.1
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 0.1
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 0.0
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 0.0
## Barati (2016a) 1.0200 [0.9655; 1.0775] 0.1
## Barati (2016b) 1.0500 [0.9557; 1.1536] 0.0
## Barati (2016c) 1.0500 [0.9501; 1.1604] 0.0
## Barati (2016d) 0.9400 [0.8456; 1.0449] 0.0
## Barati (2016e) 0.9300 [0.8633; 1.0019] 0.0
## Barati (2016f) 1.0400 [0.9503; 1.1381] 0.0
## Gius (2014) 1.1100 [1.0507; 1.1726] 0.1
## Roberts (2009a) 1.7100 [1.1176; 2.6164] 0.0
## Roberts (2009b) 1.1200 [0.8715; 1.4394] 0.0
## Roberts (2009c) 0.9600 [0.6189; 1.4892] 0.0
## Roberts (2009d) 0.8600 [0.5857; 1.2628] 0.0
## %W(random)
## Donohue, Aneja & Weber (2019a) 1.6
## Hamill et al. (2019a) 1.7
## Aneja, Donohue & Zhang (2014a) 1.4
## Kendall & Tamura (2010a) 1.9
## Hepburn et al. (2004) 1.7
## Donohue, Aneja & Weber (2019b) 1.4
## Hamill et al. (2019b) 1.6
## French & Heagerty (2008) 1.8
## Donohue, Aneja & Weber (2019c) 1.7
## Donohue, Aneja & Weber (2019d) 1.8
## Hamill et al. (2019c) 1.9
## Hamill et al. (2019d) 1.9
## Aneja, Donohue & Zhang (2014b) 1.4
## Kendall & Tamura (2010b) 1.9
## Hamill et al. (2019c) 1.9
## Aneja, Donohue & Zhang (2014c) 1.1
## Kendall & Tamura (2010c) 1.9
## Hamill et al. (2019d) 1.9
## Aneja, Donohue & Zhang (2014d) 1.6
## Kendall & Tamura (2010d) 1.9
## Crifasi, Pollack & Webster (2016a) 0.8
## Crifasi, Pollack & Webster (2016b) 0.8
## Crifasi, Pollack & Webster (2016c) 0.3
## Crifasi, Pollack & Webster (2016d) 0.6
## Crifasi, Pollack & Webster (2016e) 0.3
## Crifasi, Pollack & Webster (2016f) 0.4
## La Valle (2013a) 1.5
## La Valle (2013b) 1.4
## La Valle & Glover (2012a) 1.0
## Luca, Malhotra & Poliquin (2017a) 1.0
## La Valle & Glover (2012b) 1.1
## Luca, Malhotra & Poliquin (2017b) 0.8
## Luca, Malhotra & Poliquin (2017c) 1.4
## La Valle & Glover (2012c) 1.5
## Luca, Malhotra & Poliquin (2017d) 1.0
## La Valle & Glover (2012d) 1.4
## Luca, Malhotra & Poliquin (2017e) 0.8
## Luca, Malhotra & Poliquin (2017f) 1.2
## Siegal et al. (2017a) 1.9
## Webster, Crifasi & Vernick (2014a) 1.7
## Grambsch (2008) 1.9
## Rosengart et al. (2005a) 1.6
## Martin & Legault (2005a) 1.8
## Kovandzic, Marvell & Vieraitis (2005a) 1.7
## Siegal et al. (2017b) 1.8
## Webster, Crifasi & Vernick (2014b) 1.6
## Crifasi et al. (2018a) 1.9
## Rosengart et al. (2005b) 1.4
## Siegal et al. (2017c) 1.8
## Webster, Crifasi & Vernick (2014c) 1.5
## Crifasi et al. (2018b) 1.9
## Martin & Legault (2005b) 1.9
## Kovandzic, Marvell & Vieraitis (2005b) 1.8
## Martin & Legault (2005c) 1.8
## Kovandzic, Marvell & Vieraitis (2005c) 1.8
## Martin & Legault (2005d) 1.8
## Kovandzic, Marvell & Vieraitis (2005d) 1.8
## Martin & Legault (2005e) 1.8
## DeSimone, Markowitz & Xu (2013a) 0.0
## DeSimone, Markowitz & Xu (2013b) 0.1
## Barati (2016a) 1.8
## Barati (2016b) 1.6
## Barati (2016c) 1.6
## Barati (2016d) 1.5
## Barati (2016e) 1.7
## Barati (2016f) 1.6
## Gius (2014) 1.8
## Roberts (2009a) 0.4
## Roberts (2009b) 0.8
## Roberts (2009c) 0.4
## Roberts (2009d) 0.4
##
## Number of studies combined: k = 71
##
## IRR 95%-CI z|t p-value
## Common effect model 1.0003 [0.9990; 1.0016] 0.41 0.6789
## Random effects model 1.0134 [0.9902; 1.0372] 1.15 0.2550
##
## Quantifying heterogeneity:
## tau^2 = 0.0117 [0.0021; 0.0090]; tau = 0.1083 [0.0455; 0.0948]
## I^2 = 68.3% [59.5%; 75.2%]; H = 1.78 [1.57; 2.01]
##
## Test of heterogeneity:
## Q d.f. p-value
## 220.74 70 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Hartung-Knapp adjustment for random effects modelforest.meta(
IRDMeta,
layout = "JAMA",
sortvar = TE,
predict = T,
print.tau2 = T)
Meta-analysis for Violent Crime Only
VCRMeta <- metagen(
log(ES),
SE,
sm = "IRR",
studlab = Study,
backtrans = T,
data = ViolentCrimeData,
hakn = T,
method.tau = "SJ",
title = "Effects of Concealed-Carry Laws on Violent Crime Rates"); summary(VCRMeta)## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## IRR 95%-CI %W(common)
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 0.0
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 0.5
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 0.2
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 0.0
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 5.2
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 0.1
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 0.0
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 46.4
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 0.5
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 0.0
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 46.4
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 0.1
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 0.1
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 0.1
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 0.0
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 0.1
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 0.1
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 0.1
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 0.0
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 0.0
## Barati (2016b) 1.0500 [0.9557; 1.1536] 0.0
## Barati (2016c) 1.0500 [0.9501; 1.1604] 0.0
## Barati (2016e) 0.9300 [0.8633; 1.0019] 0.0
## Barati (2016f) 1.0400 [0.9503; 1.1381] 0.0
## %W(random)
## Donohue, Aneja & Weber (2019d) 4.6
## Hamill et al. (2019c) 5.0
## Hamill et al. (2019d) 4.9
## Aneja, Donohue & Zhang (2014b) 3.5
## Kendall & Tamura (2010b) 5.0
## Hamill et al. (2019c) 4.9
## Aneja, Donohue & Zhang (2014c) 2.8
## Kendall & Tamura (2010c) 5.0
## Hamill et al. (2019d) 5.0
## Aneja, Donohue & Zhang (2014d) 4.1
## Kendall & Tamura (2010d) 5.0
## Martin & Legault (2005b) 4.9
## Kovandzic, Marvell & Vieraitis (2005b) 4.8
## Martin & Legault (2005c) 4.8
## Kovandzic, Marvell & Vieraitis (2005c) 4.6
## Martin & Legault (2005d) 4.7
## Kovandzic, Marvell & Vieraitis (2005d) 4.8
## Martin & Legault (2005e) 4.8
## DeSimone, Markowitz & Xu (2013a) 0.0
## DeSimone, Markowitz & Xu (2013b) 0.1
## Barati (2016b) 4.1
## Barati (2016c) 4.0
## Barati (2016e) 4.4
## Barati (2016f) 4.2
##
## Number of studies combined: k = 24
##
## IRR 95%-CI z|t p-value
## Common effect model 0.9997 [0.9984; 1.0011] -0.39 0.6944
## Random effects model 1.0066 [0.9790; 1.0350] 0.49 0.6270
##
## Quantifying heterogeneity:
## tau^2 = 0.0105 [0.0003; 0.0051]; tau = 0.1025 [0.0159; 0.0711]
## I^2 = 58.4% [34.7%; 73.5%]; H = 1.55 [1.24; 1.94]
##
## Test of heterogeneity:
## Q d.f. p-value
## 55.32 23 0.0002
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Hartung-Knapp adjustment for random effects modelforest.meta(
VCRMeta,
layout = "JAMA",
sortvar = TE,
predict = T,
print.tau2 = T)
Publication Bias
metabias(IRDMeta, method.bias = "linreg")## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## Linear regression test of funnel plot asymmetry
##
## Test result: t = 1.39, df = 69, p-value = 0.1698
##
## Sample estimates:
## bias se.bias intercept se.intercept
## 0.3102 0.2236 -0.0003 0.0012
##
## Details:
## - multiplicative residual heterogeneity variance (tau^2 = 3.1123)
## - predictor: standard error
## - weight: inverse variance
## - reference: Egger et al. (1997), BMJTF <- trimfill(IRDMeta); summary(TF)## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## IRR 95%-CI %W(random)
## Donohue, Aneja & Weber (2019a) 1.0200 [0.9230; 1.1272] 1.4
## Hamill et al. (2019a) 1.0000 [0.9228; 1.0837] 1.5
## Aneja, Donohue & Zhang (2014a) 1.0300 [0.9050; 1.1722] 1.3
## Kendall & Tamura (2010a) 1.0000 [0.9941; 1.0059] 1.6
## Hepburn et al. (2004) 1.0100 [0.9320; 1.0945] 1.5
## Donohue, Aneja & Weber (2019b) 1.0300 [0.9050; 1.1722] 1.3
## Hamill et al. (2019b) 1.0700 [0.9682; 1.1825] 1.4
## French & Heagerty (2008) 1.0600 [0.9975; 1.1264] 1.6
## Donohue, Aneja & Weber (2019c) 1.0200 [0.9561; 1.0882] 1.6
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 1.6
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 1.6
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 1.6
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 1.3
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 1.6
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 1.6
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 1.1
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 1.6
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 1.6
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 1.5
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 1.6
## Crifasi, Pollack & Webster (2016a) 1.0200 [0.8031; 1.2955] 0.9
## Crifasi, Pollack & Webster (2016b) 0.9200 [0.7131; 1.1870] 0.9
## Crifasi, Pollack & Webster (2016c) 1.2700 [0.7585; 2.1265] 0.4
## Crifasi, Pollack & Webster (2016d) 0.7200 [0.5252; 0.9871] 0.7
## Crifasi, Pollack & Webster (2016e) 0.7400 [0.4669; 1.1729] 0.4
## Crifasi, Pollack & Webster (2016f) 0.7400 [0.4770; 1.1479] 0.4
## La Valle (2013a) 0.8700 [0.7826; 0.9671] 1.4
## La Valle (2013b) 0.8500 [0.7498; 0.9636] 1.4
## La Valle & Glover (2012a) 1.2300 [1.0131; 1.4934] 1.1
## Luca, Malhotra & Poliquin (2017a) 1.0600 [0.8730; 1.2870] 1.1
## La Valle & Glover (2012b) 1.3200 [1.0915; 1.5964] 1.1
## Luca, Malhotra & Poliquin (2017b) 1.0800 [0.8404; 1.3880] 0.9
## Luca, Malhotra & Poliquin (2017c) 1.0500 [0.9262; 1.1903] 1.4
## La Valle & Glover (2012c) 0.8100 [0.7287; 0.9004] 1.4
## Luca, Malhotra & Poliquin (2017d) 1.0600 [0.8628; 1.3022] 1.0
## La Valle & Glover (2012d) 0.7700 [0.6832; 0.8678] 1.4
## Luca, Malhotra & Poliquin (2017e) 1.0500 [0.8218; 1.3415] 0.9
## Luca, Malhotra & Poliquin (2017f) 1.1300 [0.9622; 1.3270] 1.2
## Siegal et al. (2017a) 1.0600 [1.0233; 1.0981] 1.6
## Webster, Crifasi & Vernick (2014a) 1.0600 [0.9878; 1.1375] 1.5
## Grambsch (2008) 1.0100 [0.9846; 1.0361] 1.6
## Rosengart et al. (2005a) 1.0700 [0.9739; 1.1756] 1.5
## Martin & Legault (2005a) 0.9500 [0.8993; 1.0036] 1.6
## Kovandzic, Marvell & Vieraitis (2005a) 1.0000 [0.9374; 1.0668] 1.6
## Siegal et al. (2017b) 1.0900 [1.0481; 1.1336] 1.6
## Webster, Crifasi & Vernick (2014b) 1.0600 [0.9592; 1.1714] 1.4
## Crifasi et al. (2018a) 1.0400 [1.0198; 1.0606] 1.6
## Rosengart et al. (2005b) 1.1100 [0.9791; 1.2583] 1.4
## Siegal et al. (2017c) 1.0100 [0.9561; 1.0670] 1.6
## Webster, Crifasi & Vernick (2014c) 1.1000 [0.9857; 1.2276] 1.4
## Crifasi et al. (2018b) 1.0300 [1.0002; 1.0607] 1.6
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 1.6
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 1.6
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 1.6
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 1.6
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 1.6
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 1.6
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 1.6
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 0.0
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 0.1
## Barati (2016a) 1.0200 [0.9655; 1.0775] 1.6
## Barati (2016b) 1.0500 [0.9557; 1.1536] 1.5
## Barati (2016c) 1.0500 [0.9501; 1.1604] 1.4
## Barati (2016d) 0.9400 [0.8456; 1.0449] 1.4
## Barati (2016e) 0.9300 [0.8633; 1.0019] 1.5
## Barati (2016f) 1.0400 [0.9503; 1.1381] 1.5
## Gius (2014) 1.1100 [1.0507; 1.1726] 1.6
## Roberts (2009a) 1.7100 [1.1176; 2.6164] 0.5
## Roberts (2009b) 1.1200 [0.8715; 1.4394] 0.9
## Roberts (2009c) 0.9600 [0.6189; 1.4892] 0.4
## Roberts (2009d) 0.8600 [0.5857; 1.2628] 0.5
## Filled: Aneja, Donohue & Zhang (2014b) 0.8932 [0.7849; 1.0166] 1.3
## Filled: Roberts (2009b) 0.8932 [0.6950; 1.1479] 0.9
## Filled: Luca, Malhotra & Poliquin (2017f) 0.8853 [0.7539; 1.0397] 1.2
## Filled: Aneja, Donohue & Zhang (2014c) 0.8699 [0.7264; 1.0418] 1.1
## Filled: La Valle & Glover (2012a) 0.8134 [0.6699; 0.9875] 1.1
## Filled: Crifasi, Pollack & Webster (2016c) 0.7877 [0.4705; 1.3190] 0.4
## Filled: La Valle & Glover (2012b) 0.7579 [0.6267; 0.9166] 1.1
## Filled: Roberts (2009a) 0.5850 [0.3824; 0.8952] 0.5
## Filled: DeSimone, Markowitz & Xu (2013a) 0.4018 [0.0320; 5.0453] 0.0
## Filled: DeSimone, Markowitz & Xu (2013b) 0.3678 [0.1043; 1.2970] 0.1
##
## Number of studies combined: k = 81 (with 10 added studies)
##
## IRR 95%-CI t p-value
## Random effects model 0.9980 [0.9715; 1.0253] -0.15 0.8846
##
## Quantifying heterogeneity:
## tau^2 = 0.0189 [0.0031; 0.0130]; tau = 0.1375 [0.0557; 0.1139]
## I^2 = 68.2% [60.0%; 74.7%]; H = 1.77 [1.58; 1.99]
##
## Test of heterogeneity:
## Q d.f. p-value
## 251.37 80 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Hartung-Knapp adjustment for random effects model
## - Trim-and-fill method to adjust for funnel plot asymmetrylimit <- limitmeta(IRDMeta); summary(limit)## Results for individual studies
## (left: original data; right: shrunken estimates)
##
## IRR 95%-CI IRR
## Donohue, Aneja & Weber (2019a) 1.0200 [0.9230; 1.1272] 1.0130
## Hamill et al. (2019a) 1.0000 [0.9228; 1.0837] 0.9966
## Aneja, Donohue & Zhang (2014a) 1.0300 [0.9050; 1.1722] 1.0178
## Kendall & Tamura (2010a) 1.0000 [0.9941; 1.0059] 1.0000
## Hepburn et al. (2004) 1.0100 [0.9320; 1.0945] 1.0059
## Donohue, Aneja & Weber (2019b) 1.0300 [0.9050; 1.1722] 1.0178
## Hamill et al. (2019b) 1.0700 [0.9682; 1.1825] 1.0579
## French & Heagerty (2008) 1.0600 [0.9975; 1.1264] 1.0555
## Donohue, Aneja & Weber (2019c) 1.0200 [0.9561; 1.0882] 1.0168
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 1.0842
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 0.9898
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 0.9995
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 1.0933
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 1.0000
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 0.9697
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 1.0987
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 1.0000
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 0.9898
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 1.0681
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 1.0000
## Crifasi, Pollack & Webster (2016a) 1.0200 [0.8031; 1.2955] 0.9956
## Crifasi, Pollack & Webster (2016b) 0.9200 [0.7131; 1.1870] 0.9304
## Crifasi, Pollack & Webster (2016c) 1.2700 [0.7585; 2.1265] 1.0605
## Crifasi, Pollack & Webster (2016d) 0.7200 [0.5252; 0.9871] 0.8135
## Crifasi, Pollack & Webster (2016e) 0.7400 [0.4669; 1.1729] 0.8553
## Crifasi, Pollack & Webster (2016f) 0.7400 [0.4770; 1.1479] 0.8517
## La Valle (2013a) 0.8700 [0.7826; 0.9671] 0.8780
## La Valle (2013b) 0.8500 [0.7498; 0.9636] 0.8632
## La Valle & Glover (2012a) 1.2300 [1.0131; 1.4934] 1.1493
## Luca, Malhotra & Poliquin (2017a) 1.0600 [0.8730; 1.2870] 1.0298
## La Valle & Glover (2012b) 1.3200 [1.0915; 1.5964] 1.2135
## Luca, Malhotra & Poliquin (2017b) 1.0800 [0.8404; 1.3880] 1.0318
## Luca, Malhotra & Poliquin (2017c) 1.0500 [0.9262; 1.1903] 1.0354
## La Valle & Glover (2012c) 0.8100 [0.7287; 0.9004] 0.8236
## Luca, Malhotra & Poliquin (2017d) 1.0600 [0.8628; 1.3022] 1.0275
## La Valle & Glover (2012d) 0.7700 [0.6832; 0.8678] 0.7910
## Luca, Malhotra & Poliquin (2017e) 1.0500 [0.8218; 1.3415] 1.0141
## Luca, Malhotra & Poliquin (2017f) 1.1300 [0.9622; 1.3270] 1.0908
## Siegal et al. (2017a) 1.0600 [1.0233; 1.0981] 1.0584
## Webster, Crifasi & Vernick (2014a) 1.0600 [0.9878; 1.1375] 1.0540
## Grambsch (2008) 1.0100 [0.9846; 1.0361] 1.0096
## Rosengart et al. (2005a) 1.0700 [0.9739; 1.1756] 1.0591
## Martin & Legault (2005a) 0.9500 [0.8993; 1.0036] 0.9500
## Kovandzic, Marvell & Vieraitis (2005a) 1.0000 [0.9374; 1.0668] 0.9977
## Siegal et al. (2017b) 1.0900 [1.0481; 1.1336] 1.0875
## Webster, Crifasi & Vernick (2014b) 1.0600 [0.9592; 1.1714] 1.0489
## Crifasi et al. (2018a) 1.0400 [1.0198; 1.0606] 1.0396
## Rosengart et al. (2005b) 1.1100 [0.9791; 1.2583] 1.0861
## Siegal et al. (2017c) 1.0100 [0.9561; 1.0670] 1.0080
## Webster, Crifasi & Vernick (2014c) 1.1000 [0.9857; 1.2276] 1.0820
## Crifasi et al. (2018b) 1.0300 [1.0002; 1.0607] 1.0292
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 0.9401
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 0.9989
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 0.9793
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 1.0076
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 0.9597
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 0.9795
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 0.9304
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 1.0289
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 1.1306
## Barati (2016a) 1.0200 [0.9655; 1.0775] 1.0177
## Barati (2016b) 1.0500 [0.9557; 1.1536] 1.0410
## Barati (2016c) 1.0500 [0.9501; 1.1604] 1.0400
## Barati (2016d) 0.9400 [0.8456; 1.0449] 0.9410
## Barati (2016e) 0.9300 [0.8633; 1.0019] 0.9311
## Barati (2016f) 1.0400 [0.9503; 1.1381] 1.0325
## Gius (2014) 1.1100 [1.0507; 1.1726] 1.1045
## Roberts (2009a) 1.7100 [1.1176; 2.6164] 1.2346
## Roberts (2009b) 1.1200 [0.8715; 1.4394] 1.0563
## Roberts (2009c) 0.9600 [0.6189; 1.4892] 0.9539
## Roberts (2009d) 0.8600 [0.5857; 1.2628] 0.9050
## 95%-CI
## Donohue, Aneja & Weber (2019a) [0.9167; 1.1195]
## Hamill et al. (2019a) [0.9197; 1.0800]
## Aneja, Donohue & Zhang (2014a) [0.8943; 1.1583]
## Kendall & Tamura (2010a) [0.9941; 1.0059]
## Hepburn et al. (2004) [0.9283; 1.0901]
## Donohue, Aneja & Weber (2019b) [0.8943; 1.1583]
## Hamill et al. (2019b) [0.9572; 1.1691]
## French & Heagerty (2008) [0.9933; 1.1216]
## Donohue, Aneja & Weber (2019c) [0.9531; 1.0848]
## Donohue, Aneja & Weber (2019d) [1.0203; 1.1521]
## Hamill et al. (2019c) [0.9706; 1.0094]
## Hamill et al. (2019d) [0.9706; 1.0293]
## Aneja, Donohue & Zhang (2014b) [0.9606; 1.2442]
## Kendall & Tamura (2010b) [0.9941; 1.0059]
## Hamill et al. (2019c) [0.9361; 1.0045]
## Aneja, Donohue & Zhang (2014c) [0.9174; 1.3158]
## Kendall & Tamura (2010c) [0.9980; 1.0020]
## Hamill et al. (2019d) [0.9706; 1.0094]
## Aneja, Donohue & Zhang (2014d) [0.9722; 1.1735]
## Kendall & Tamura (2010d) [0.9980; 1.0020]
## Crifasi, Pollack & Webster (2016a) [0.7839; 1.2646]
## Crifasi, Pollack & Webster (2016b) [0.7211; 1.2004]
## Crifasi, Pollack & Webster (2016c) [0.6334; 1.7758]
## Crifasi, Pollack & Webster (2016d) [0.5934; 1.1153]
## Crifasi, Pollack & Webster (2016e) [0.5396; 1.3556]
## Crifasi, Pollack & Webster (2016f) [0.5491; 1.3212]
## La Valle (2013a) [0.7898; 0.9760]
## La Valle (2013b) [0.7614; 0.9785]
## La Valle & Glover (2012a) [0.9466; 1.3954]
## Luca, Malhotra & Poliquin (2017a) [0.8482; 1.2503]
## La Valle & Glover (2012b) [1.0034; 1.4676]
## Luca, Malhotra & Poliquin (2017b) [0.8028; 1.3260]
## Luca, Malhotra & Poliquin (2017c) [0.9133; 1.1737]
## La Valle & Glover (2012c) [0.7409; 0.9156]
## Luca, Malhotra & Poliquin (2017d) [0.8364; 1.2623]
## La Valle & Glover (2012d) [0.7019; 0.8915]
## Luca, Malhotra & Poliquin (2017e) [0.7937; 1.2956]
## Luca, Malhotra & Poliquin (2017f) [0.9288; 1.2809]
## Siegal et al. (2017a) [1.0217; 1.0964]
## Webster, Crifasi & Vernick (2014a) [0.9822; 1.1311]
## Grambsch (2008) [0.9842; 1.0356]
## Rosengart et al. (2005a) [0.9640; 1.1635]
## Martin & Legault (2005a) [0.8992; 1.0036]
## Kovandzic, Marvell & Vieraitis (2005a) [0.9353; 1.0644]
## Siegal et al. (2017b) [1.0457; 1.1310]
## Webster, Crifasi & Vernick (2014b) [0.9491; 1.1592]
## Crifasi et al. (2018a) [1.0194; 1.0602]
## Rosengart et al. (2005b) [0.9581; 1.2313]
## Siegal et al. (2017c) [0.9542; 1.0649]
## Webster, Crifasi & Vernick (2014c) [0.9696; 1.2076]
## Crifasi et al. (2018b) [0.9994; 1.0599]
## Martin & Legault (2005b) [0.9075; 0.9739]
## Kovandzic, Marvell & Vieraitis (2005b) [0.9548; 1.0449]
## Martin & Legault (2005c) [0.9362; 1.0245]
## Kovandzic, Marvell & Vieraitis (2005c) [0.9482; 1.0707]
## Martin & Legault (2005d) [0.9084; 1.0138]
## Kovandzic, Marvell & Vieraitis (2005d) [0.9418; 1.0186]
## Martin & Legault (2005e) [0.8894; 0.9733]
## DeSimone, Markowitz & Xu (2013a) [0.0819; 12.9209]
## DeSimone, Markowitz & Xu (2013b) [0.3206; 3.9870]
## Barati (2016a) [0.9633; 1.0751]
## Barati (2016b) [0.9475; 1.1436]
## Barati (2016c) [0.9410; 1.1493]
## Barati (2016d) [0.8465; 1.0460]
## Barati (2016e) [0.8642; 1.0031]
## Barati (2016f) [0.9435; 1.1299]
## Gius (2014) [1.0455; 1.1668]
## Roberts (2009a) [0.8069; 1.8891]
## Roberts (2009b) [0.8219; 1.3575]
## Roberts (2009c) [0.6149; 1.4797]
## Roberts (2009d) [0.6163; 1.3288]
##
## Result of limit meta-analysis:
##
## Random effects model IRR 95%-CI z pval
## Adjusted estimate 1.0058 [0.9738; 1.0389] 0.35 0.7251
## Unadjusted estimate 1.0134 [0.9902; 1.0372] 1.15 0.2550
##
## Quantifying heterogeneity:
## tau^2 = 0.0117; I^2 = 68.3% [59.5%; 75.2%]; G^2 = 100.0%
##
## Test of heterogeneity:
## Q d.f. p-value
## 220.74 70 0
##
## Test of small-study effects:
## Q-Q' d.f. p-value
## 5.99 1 0.0144
##
## Test of residual heterogeneity beyond small-study effects:
## Q' d.f. p-value
## 214.75 69 < 0.0001
##
## Details on adjustment method:
## - expectation (beta0)pcurve(IRDMeta)
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 71
## - Total number of p<0.05 studies included into the analysis: k = 16 (22.54%)
## - Total number of studies with p<0.025: k = 13 (18.31%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.011 -5.668 0.000 -5.935 0
## Flatness test 0.878 2.614 0.996 5.861 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 73% (49.1%-87.8%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: nofunnel(TF, legend = T)
funnel.limitmeta(limit, shrunken = T)
metabias(VCRMeta, method.bias = "linreg")## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## Linear regression test of funnel plot asymmetry
##
## Test result: t = -0.24, df = 22, p-value = 0.8116
##
## Sample estimates:
## bias se.bias intercept se.intercept
## -0.0866 0.3590 -0.0001 0.0012
##
## Details:
## - multiplicative residual heterogeneity variance (tau^2 = 2.5079)
## - predictor: standard error
## - weight: inverse variance
## - reference: Egger et al. (1997), BMJTF <- trimfill(VCRMeta); summary(TF)## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## IRR 95%-CI %W(random)
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 4.6
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 5.0
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 4.9
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 3.5
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 5.0
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 4.9
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 2.8
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 5.0
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 5.0
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 4.1
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 5.0
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 4.9
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 4.8
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 4.8
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 4.6
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 4.7
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 4.8
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 4.8
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 0.0
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 0.1
## Barati (2016b) 1.0500 [0.9557; 1.1536] 4.1
## Barati (2016c) 1.0500 [0.9501; 1.1604] 4.0
## Barati (2016e) 0.9300 [0.8633; 1.0019] 4.4
## Barati (2016f) 1.0400 [0.9503; 1.1381] 4.2
##
## Number of studies combined: k = 24 (with 0 added studies)
##
## IRR 95%-CI t p-value
## Random effects model 1.0066 [0.9790; 1.0350] 0.49 0.6270
##
## Quantifying heterogeneity:
## tau^2 = 0.0105 [0.0003; 0.0051]; tau = 0.1025 [0.0159; 0.0711]
## I^2 = 58.4% [34.7%; 73.5%]; H = 1.55 [1.24; 1.94]
##
## Test of heterogeneity:
## Q d.f. p-value
## 55.32 23 0.0002
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Hartung-Knapp adjustment for random effects model
## - Trim-and-fill method to adjust for funnel plot asymmetrylimit <- limitmeta(VCRMeta); summary(limit)## Results for individual studies
## (left: original data; right: shrunken estimates)
##
## IRR 95%-CI IRR
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 1.0789
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 0.9893
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 0.9984
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 1.0734
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 0.9999
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 0.9682
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 1.0668
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 1.0000
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 0.9893
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 1.0567
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 1.0000
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 0.9387
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 0.9963
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 0.9768
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 1.0029
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 0.9562
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 0.9776
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 0.9282
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 0.9331
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 1.0285
## Barati (2016b) 1.0500 [0.9557; 1.1536] 1.0301
## Barati (2016c) 1.0500 [0.9501; 1.1604] 1.0280
## Barati (2016e) 0.9300 [0.8633; 1.0019] 0.9253
## Barati (2016f) 1.0400 [0.9503; 1.1381] 1.0226
## 95%-CI
## Donohue, Aneja & Weber (2019d) [1.0153; 1.1464]
## Hamill et al. (2019c) [0.9701; 1.0089]
## Hamill et al. (2019d) [0.9695; 1.0282]
## Aneja, Donohue & Zhang (2014b) [0.9431; 1.2216]
## Kendall & Tamura (2010b) [0.9941; 1.0058]
## Hamill et al. (2019c) [0.9346; 1.0030]
## Aneja, Donohue & Zhang (2014c) [0.8908; 1.2776]
## Kendall & Tamura (2010c) [0.9980; 1.0020]
## Hamill et al. (2019d) [0.9701; 1.0089]
## Aneja, Donohue & Zhang (2014d) [0.9619; 1.1610]
## Kendall & Tamura (2010d) [0.9980; 1.0020]
## Martin & Legault (2005b) [0.9062; 0.9724]
## Kovandzic, Marvell & Vieraitis (2005b) [0.9524; 1.0422]
## Martin & Legault (2005c) [0.9338; 1.0219]
## Kovandzic, Marvell & Vieraitis (2005c) [0.9438; 1.0658]
## Martin & Legault (2005d) [0.9051; 1.0101]
## Kovandzic, Marvell & Vieraitis (2005d) [0.9400; 1.0167]
## Martin & Legault (2005e) [0.8873; 0.9710]
## DeSimone, Markowitz & Xu (2013a) [0.0743; 11.7173]
## DeSimone, Markowitz & Xu (2013b) [0.2917; 3.6267]
## Barati (2016b) [0.9376; 1.1317]
## Barati (2016c) [0.9302; 1.1360]
## Barati (2016e) [0.8589; 0.9969]
## Barati (2016f) [0.9344; 1.1190]
##
## Result of limit meta-analysis:
##
## Random effects model IRR 95%-CI z pval
## Adjusted estimate 0.9976 [0.9528; 1.0445] -0.10 0.9185
## Unadjusted estimate 1.0066 [0.9790; 1.0350] 0.49 0.6270
##
## Quantifying heterogeneity:
## tau^2 = 0.0105; I^2 = 58.4% [34.7%; 73.5%]; G^2 = 99.6%
##
## Test of heterogeneity:
## Q d.f. p-value
## 55.32 23 0.0002
##
## Test of small-study effects:
## Q-Q' d.f. p-value
## 0.15 1 0.7025
##
## Test of residual heterogeneity beyond small-study effects:
## Q' d.f. p-value
## 55.17 22 0.0001
##
## Details on adjustment method:
## - expectation (beta0)pcurve(VCRMeta)
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 24
## - Total number of p<0.05 studies included into the analysis: k = 3 (12.5%)
## - Total number of studies with p<0.025: k = 3 (12.5%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.125 -3.089 0.001 -2.476 0.007
## Flatness test 1.000 1.655 0.951 2.460 0.993
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 82% (33.7%-97.9%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: nofunnel(TF, legend = T)
funnel.limitmeta(limit, shrunken = T)
Is Publication Bias Worse in Methodologically High versus Low-quality Studies?
MetaQual <- subgroup.analysis.mixed.effects(IRDMeta, data$Quality); MetaQual## Warning: Use argument 'fixed' instead of 'comb.fixed' (deprecated).## Warning: Use argument 'random' instead of 'comb.random' (deprecated).## Warning: Use argument 'subgroup' instead of 'byvar' (deprecated).## Warning: Use argument 'fixed' instead of 'comb.fixed' (deprecated).## Warning: Use argument 'subgroup' instead of 'byvar' (deprecated).## Subgroup Results:
## --------------
## k TE seTE IRR LLCI ULCI p Q I2
## Bad 33 0.021850725 0.01438725 1.022091 0.994 1.051 0.1288238 109.94959 0.71
## Good 38 0.005426795 0.01794515 1.005442 0.971 1.041 0.7623395 98.78847 0.63
## I2.lower I2.upper
## Bad 0.59 0.80
## Good 0.47 0.74
##
## Test for subgroup differences (mixed/fixed-effects (plural) model):
## --------------
## Q df p
## Between groups 0.5098952 1 0.4751842
##
## - Total number of studies included in subgroup analysis: 71
## - Tau estimator used for within-group pooling: SJforest(MetaQual)
GoodMeta <- metagen(
log(ES),
SE,
sm = "IRR",
studlab = Study,
backtrans = T,
data = GoodData,
hakn = T,
method.tau = "SJ",
title = "Effects of Concealed-Carry Laws on Violent Crime Rates")
BadMeta <- metagen(
log(ES),
SE,
sm = "IRR",
studlab = Study,
backtrans = T,
data = BadData,
hakn = T,
method.tau = "SJ",
title = "Effects of Concealed-Carry Laws on Violent Crime Rates")
metabias(GoodMeta, method.bias = "linreg")## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## Linear regression test of funnel plot asymmetry
##
## Test result: t = 0.14, df = 36, p-value = 0.8932
##
## Sample estimates:
## bias se.bias intercept se.intercept
## 0.0391 0.2893 -0.0001 0.0012
##
## Details:
## - multiplicative residual heterogeneity variance (tau^2 = 2.7427)
## - predictor: standard error
## - weight: inverse variance
## - reference: Egger et al. (1997), BMJmetabias(BadMeta, method.bias = "linreg")## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## Linear regression test of funnel plot asymmetry
##
## Test result: t = 0.49, df = 31, p-value = 0.6282
##
## Sample estimates:
## bias se.bias intercept se.intercept
## 0.2815 0.5755 0.0096 0.0148
##
## Details:
## - multiplicative residual heterogeneity variance (tau^2 = 3.5196)
## - predictor: standard error
## - weight: inverse variance
## - reference: Egger et al. (1997), BMJpcurve(GoodMeta)
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 38
## - Total number of p<0.05 studies included into the analysis: k = 8 (21.05%)
## - Total number of studies with p<0.025: k = 6 (15.79%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.145 -3.161 0.001 -3.264 0.001
## Flatness test 0.716 1.122 0.869 3.541 1.000
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 60% (21.8%-86.5%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: nopcurve(BadMeta)
## P-curve analysis
## -----------------------
## - Total number of provided studies: k = 33
## - Total number of p<0.05 studies included into the analysis: k = 8 (24.24%)
## - Total number of studies with p<0.025: k = 7 (21.21%)
##
## Results
## -----------------------
## pBinomial zFull pFull zHalf pHalf
## Right-skewness test 0.035 -4.855 0.000 -5.067 0
## Flatness test 0.932 2.575 0.995 4.709 1
## Note: p-values of 0 or 1 correspond to p<0.001 and p>0.999, respectively.
## Power Estimate: 82% (53.7%-94.9%)
##
## Evidential value
## -----------------------
## - Evidential value present: yes
## - Evidential value absent/inadequate: nosummary(trimfill(GoodMeta))## Warning: Fisher scoring algorithm may have gotten stuck at a local maximum.
## Setting tau^2 = 0. Check the profile likelihood plot with profile().
## Warning: Fisher scoring algorithm may have gotten stuck at a local maximum.
## Setting tau^2 = 0. Check the profile likelihood plot with profile().
## Warning: Fisher scoring algorithm may have gotten stuck at a local maximum.
## Setting tau^2 = 0. Check the profile likelihood plot with profile().
## Warning: Fisher scoring algorithm may have gotten stuck at a local maximum.
## Setting tau^2 = 0. Check the profile likelihood plot with profile().## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## IRR 95%-CI %W(random)
## Donohue, Aneja & Weber (2019a) 1.0200 [0.9230; 1.1272] 2.8
## Hamill et al. (2019a) 1.0000 [0.9228; 1.0837] 3.0
## Aneja, Donohue & Zhang (2014a) 1.0300 [0.9050; 1.1722] 2.6
## Kendall & Tamura (2010a) 1.0000 [0.9941; 1.0059] 3.4
## Hepburn et al. (2004) 1.0100 [0.9320; 1.0945] 3.0
## Donohue, Aneja & Weber (2019b) 1.0300 [0.9050; 1.1722] 2.6
## Hamill et al. (2019b) 1.0700 [0.9682; 1.1825] 2.8
## French & Heagerty (2008) 1.0600 [0.9975; 1.1264] 3.2
## Donohue, Aneja & Weber (2019c) 1.0200 [0.9561; 1.0882] 3.2
## Donohue, Aneja & Weber (2019d) 1.0900 [1.0257; 1.1583] 3.2
## Hamill et al. (2019c) 0.9900 [0.9708; 1.0096] 3.4
## Hamill et al. (2019d) 1.0000 [0.9710; 1.0298] 3.4
## Aneja, Donohue & Zhang (2014b) 1.1200 [0.9841; 1.2747] 2.6
## Kendall & Tamura (2010b) 1.0000 [0.9941; 1.0059] 3.4
## Hamill et al. (2019c) 0.9700 [0.9364; 1.0048] 3.3
## Aneja, Donohue & Zhang (2014c) 1.1500 [0.9603; 1.3772] 2.1
## Kendall & Tamura (2010c) 1.0000 [0.9980; 1.0020] 3.4
## Hamill et al. (2019d) 0.9900 [0.9708; 1.0096] 3.4
## Aneja, Donohue & Zhang (2014d) 1.0800 [0.9830; 1.1865] 2.9
## Kendall & Tamura (2010d) 1.0000 [0.9980; 1.0020] 3.4
## Crifasi, Pollack & Webster (2016a) 1.0200 [0.8031; 1.2955] 1.6
## Crifasi, Pollack & Webster (2016b) 0.9200 [0.7131; 1.1870] 1.5
## Crifasi, Pollack & Webster (2016c) 1.2700 [0.7585; 2.1265] 0.5
## Crifasi, Pollack & Webster (2016d) 0.7200 [0.5252; 0.9871] 1.1
## Crifasi, Pollack & Webster (2016e) 0.7400 [0.4669; 1.1729] 0.6
## Crifasi, Pollack & Webster (2016f) 0.7400 [0.4770; 1.1479] 0.7
## La Valle (2013a) 0.8700 [0.7826; 0.9671] 2.8
## La Valle (2013b) 0.8500 [0.7498; 0.9636] 2.6
## La Valle & Glover (2012a) 1.2300 [1.0131; 1.4934] 1.9
## Luca, Malhotra & Poliquin (2017a) 1.0600 [0.8730; 1.2870] 1.9
## La Valle & Glover (2012b) 1.3200 [1.0915; 1.5964] 2.0
## Luca, Malhotra & Poliquin (2017b) 1.0800 [0.8404; 1.3880] 1.5
## Luca, Malhotra & Poliquin (2017c) 1.0500 [0.9262; 1.1903] 2.6
## La Valle & Glover (2012c) 0.8100 [0.7287; 0.9004] 2.8
## Luca, Malhotra & Poliquin (2017d) 1.0600 [0.8628; 1.3022] 1.8
## La Valle & Glover (2012d) 0.7700 [0.6832; 0.8678] 2.7
## Luca, Malhotra & Poliquin (2017e) 1.0500 [0.8218; 1.3415] 1.5
## Luca, Malhotra & Poliquin (2017f) 1.1300 [0.9622; 1.3270] 2.2
## Filled: Aneja, Donohue & Zhang (2014c) 0.8694 [0.7259; 1.0412] 2.1
## Filled: La Valle & Glover (2012a) 0.8128 [0.6695; 0.9869] 1.9
## Filled: Crifasi, Pollack & Webster (2016c) 0.7872 [0.4702; 1.3182] 0.5
## Filled: La Valle & Glover (2012b) 0.7574 [0.6263; 0.9160] 2.0
##
## Number of studies combined: k = 42 (with 4 added studies)
##
## IRR 95%-CI t p-value
## Random effects model 0.9916 [0.9546; 1.0300] -0.45 0.6558
##
## Quantifying heterogeneity:
## tau^2 = 0.0126 [0.0038; 0.0200]; tau = 0.1122 [0.0618; 0.1415]
## I^2 = 64.2% [50.3%; 74.2%]; H = 1.67 [1.42; 1.97]
##
## Test of heterogeneity:
## Q d.f. p-value
## 114.50 41 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Hartung-Knapp adjustment for random effects model
## - Trim-and-fill method to adjust for funnel plot asymmetrysummary(trimfill(BadMeta))## Review: Effects of Concealed-Carry Laws on Violent Crime Rates
##
## IRR 95%-CI %W(random)
## Siegal et al. (2017a) 1.0600 [1.0233; 1.0981] 3.5
## Webster, Crifasi & Vernick (2014a) 1.0600 [0.9878; 1.1375] 3.4
## Grambsch (2008) 1.0100 [0.9846; 1.0361] 3.5
## Rosengart et al. (2005a) 1.0700 [0.9739; 1.1756] 3.3
## Martin & Legault (2005a) 0.9500 [0.8993; 1.0036] 3.4
## Kovandzic, Marvell & Vieraitis (2005a) 1.0000 [0.9374; 1.0668] 3.4
## Siegal et al. (2017b) 1.0900 [1.0481; 1.1336] 3.5
## Webster, Crifasi & Vernick (2014b) 1.0600 [0.9592; 1.1714] 3.2
## Crifasi et al. (2018a) 1.0400 [1.0198; 1.0606] 3.5
## Rosengart et al. (2005b) 1.1100 [0.9791; 1.2583] 3.1
## Siegal et al. (2017c) 1.0100 [0.9561; 1.0670] 3.4
## Webster, Crifasi & Vernick (2014c) 1.1000 [0.9857; 1.2276] 3.2
## Crifasi et al. (2018b) 1.0300 [1.0002; 1.0607] 3.5
## Martin & Legault (2005b) 0.9400 [0.9074; 0.9738] 3.5
## Kovandzic, Marvell & Vieraitis (2005b) 1.0000 [0.9559; 1.0461] 3.5
## Martin & Legault (2005c) 0.9800 [0.9368; 1.0252] 3.5
## Kovandzic, Marvell & Vieraitis (2005c) 1.0100 [0.9505; 1.0733] 3.4
## Martin & Legault (2005d) 0.9600 [0.9087; 1.0142] 3.4
## Kovandzic, Marvell & Vieraitis (2005d) 0.9800 [0.9423; 1.0192] 3.5
## Martin & Legault (2005e) 0.9300 [0.8890; 0.9729] 3.5
## DeSimone, Markowitz & Xu (2013a) 2.4900 [0.1983; 31.2680] 0.1
## DeSimone, Markowitz & Xu (2013b) 2.7200 [0.7713; 9.5916] 0.2
## Barati (2016a) 1.0200 [0.9655; 1.0775] 3.4
## Barati (2016b) 1.0500 [0.9557; 1.1536] 3.3
## Barati (2016c) 1.0500 [0.9501; 1.1604] 3.2
## Barati (2016d) 0.9400 [0.8456; 1.0449] 3.2
## Barati (2016e) 0.9300 [0.8633; 1.0019] 3.4
## Barati (2016f) 1.0400 [0.9503; 1.1381] 3.3
## Gius (2014) 1.1100 [1.0507; 1.1726] 3.4
## Roberts (2009a) 1.7100 [1.1176; 2.6164] 1.4
## Roberts (2009b) 1.1200 [0.8715; 1.4394] 2.3
## Roberts (2009c) 0.9600 [0.6189; 1.4892] 1.3
## Roberts (2009d) 0.8600 [0.5857; 1.2628] 1.5
## Filled: Roberts (2009a) 0.6030 [0.3941; 0.9226] 1.4
## Filled: DeSimone, Markowitz & Xu (2013a) 0.4141 [0.0330; 5.1998] 0.1
## Filled: DeSimone, Markowitz & Xu (2013b) 0.3791 [0.1075; 1.3367] 0.2
##
## Number of studies combined: k = 36 (with 3 added studies)
##
## IRR 95%-CI t p-value
## Random effects model 1.0163 [0.9729; 1.0616] 0.75 0.4578
##
## Quantifying heterogeneity:
## tau^2 = 0.0294 [0.0011; 0.0134]; tau = 0.1714 [0.0331; 0.1158]
## I^2 = 70.5% [58.6%; 78.9%]; H = 1.84 [1.55; 2.18]
##
## Test of heterogeneity:
## Q d.f. p-value
## 118.55 35 < 0.0001
##
## Details on meta-analytical method:
## - Inverse variance method
## - Sidik-Jonkman estimator for tau^2
## - Q-profile method for confidence interval of tau^2 and tau
## - Hartung-Knapp adjustment for random effects model
## - Trim-and-fill method to adjust for funnel plot asymmetryGoodLimit <- limitmeta(GoodMeta); funnel.limitmeta(GoodLimit, shrunken = T)
BadLimit <- limitmeta(BadMeta); funnel.limitmeta(BadLimit, shrunken = T)
Do the Conclusions of Studies of Violent Crime Rates in General Differ from Studies of Homicides and Police Assaults?
MetaVCR <- subgroup.analysis.mixed.effects(IRDMeta, data$VCR); MetaVCR #No = Homicides/Police Assaults, Yes = Violent Crime in General## Warning: Use argument 'fixed' instead of 'comb.fixed' (deprecated).## Warning: Use argument 'random' instead of 'comb.random' (deprecated).## Warning: Use argument 'subgroup' instead of 'byvar' (deprecated).## Warning: Use argument 'fixed' instead of 'comb.fixed' (deprecated).## Warning: Use argument 'subgroup' instead of 'byvar' (deprecated).## Subgroup Results:
## --------------
## k TE seTE IRR LLCI ULCI p Q I2
## No 47 0.017174630 0.01651466 1.017323 0.985 1.051 0.2983571 155.89389 0.70
## Yes 24 0.006617844 0.01343418 1.006640 0.980 1.033 0.6222864 55.32062 0.58
## I2.lower I2.upper
## No 0.60 0.78
## Yes 0.35 0.74
##
## Test for subgroup differences (mixed/fixed-effects (plural) model):
## --------------
## Q df p
## Between groups 0.2459025 1 0.6199751
##
## - Total number of studies included in subgroup analysis: 71
## - Tau estimator used for within-group pooling: SJforest(MetaVCR)
Conclusion
Right to carry laws do not appear to affect violent crime rates, homicide rates, or police assault rates, and differences between the studies RAND dubbed high versus low-quality, or categorized as violent crime studies versus homicide and police assault studies did not produce different results. The evidence for publication was minor because of the limited effect sizes, but it did trend in that direction, as can be seen in the funnel plots, which were asymmetric on the side of concealed carry increasing crime.
References
Smart, R. (2020). Effects of Concealed-Carry Laws on Violent Crime. RAND Corporation. https://www.rand.org/research/gun-policy/analysis/concealed-carry/violent-crime.html