── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.1.4 ✔ readr 2.1.5
✔ forcats 1.0.0 ✔ stringr 1.5.1
✔ ggplot2 3.5.0 ✔ tibble 3.2.1
✔ lubridate 1.9.3 ✔ tidyr 1.3.1
✔ purrr 1.0.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
gh_install_packages("konfound", ref ="newitcv_output", force =TRUE)
Warning in fread(download_url, sep = "\t", header = FALSE, stringsAsFactors =
FALSE, : Found and resolved improper quoting out-of-sample. First healed line
4848: <<Puriney honfleuR "Evening, honfleuR" by Seurat>>. If the fields are not
quoted (e.g. field separator does not appear within any field), try quote="" to
avoid this warning.
Suggestion:
- jrosen48/konfound R package for carrying out sensitivity analysis
The downloaded binary packages are in
/var/folders/gl/9nbm4d7s0zv_7r1wcdpj_kv80000gn/T//RtmpxGJHsJ/downloaded_packages
── R CMD build ─────────────────────────────────────────────────────────────────
* checking for file ‘/private/var/folders/gl/9nbm4d7s0zv_7r1wcdpj_kv80000gn/T/RtmpxGJHsJ/remotesf0a20659a49/konfound-project-konfound-ae1164c/DESCRIPTION’ ... OK
* preparing ‘konfound’:
* checking DESCRIPTION meta-information ... OK
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘konfound_0.4.0.tar.gz’
library(konfound)
Sensitivity analysis as described in Frank, Maroulis, Duong, and Kelcey (2013) and in Frank (2000).
For more information visit http://konfound-it.com.
library("Statamarkdown")
Stata found at /Applications/Stata/StataSE.app/Contents/MacOS/StataSE
The 'stata' engine is ready to use.
Konfound / Linear / ITCV
m <-lm(mpg ~ wt + hp, data = mtcars)konfound(m, wt, index="IT", to_return ="print")
Impact Threshold for a Confounding Variable:
The minimum (in absolute value) impact of an omitted variable to
invalidate an inference for a null hypothesis of 0 effect is based on
a correlation of -0.784 with the outcome and 0.784 with the predictor
of interest (conditioning on all observed covariates in the model;
signs are interchangeable) based on a threshold of -0.355 for
statistical significance (alpha = 0.05).
Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be
-0.784 X 0.784 = -0.614 to invalidate an inference for a null hypothesis of 0 effect.
For calculation of unconditional ITCV, request raw output. When using pkonfound(),
additionally include the R2, sdx, and sdy as input.
See Frank (2000) for a description of the method.
Citation:
Frank, K. (2000). Impact of a confounding variable on the inference of a
regression coefficient. Sociological Methods and Research, 29 (2), 147-194
Accuracy of results increases with the number of decimals reported.
For more detailed output, consider setting `to_return` to table
To consider other predictors of interest, consider setting `test_all` to TRUE.
Robustness of Inference to Replacement (RIR):
TO INVALIDATE:
RIR = 4020
The estimated effect is -0.1. To invalidate the inference of an effect using
the threshold of -0.02 for statistical significance with alpha = 0.05, 80.396% of
the (-0.1) estimate would have to be due to bias. This implies that to invalidate
the inference one would expect to have to replace 4020 (80.396%) observations
with cases for which the treatment effect is 0 (RIR = 4020).
See Frank et al. (2013) for a description of the method.
Citation: Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. (2013).
What would it take to change an inference?
Using Rubin's causal model to interpret the robustness of causal inferences.
Education, Evaluation and Policy Analysis, 35 437-460.
Accuracy of results increases with the number of decimals reported.
For other forms of output, run ?pkonfound and inspect the to_return argument
Robustness of Inference to Replacement (RIR):
TO SUSTAIN:
RIR = 4923
The estimated effect is 0.003, and specified threshold for inference is 0.001.
To reach that threshold, 98.47% of the (0.003) estimate would have to be due
to bias. This implies that to sustain an inference one would expect to have
to replace 4923 (98.47%) observations with effect of 0 with cases with effect
of 0.196 (RIR = 4923).
See Frank et al. (2013) for a description of the method.
Citation: Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. (2013).
What would it take to change an inference?
Using Rubin's causal model to interpret the robustness of causal inferences.
Education, Evaluation and Policy Analysis, 35 437-460.
Accuracy of results increases with the number of decimals reported.
For other forms of output, run ?pkonfound and inspect the to_return argument
Impact Threshold for a Confounding Variable:
The maximum impact (in absolute value) of an omitted variable to
sustain an inference for a null hypothesis of 0 effect is based on
a correlation of -0.163 with the outcome and 0.163 with the predictor
of interest (conditioning on all observed covariates in the model;
signs are interchangeable) based on a threshold of 0.196 for
statistical significance (alpha = 0.05).
Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be
-0.163 X 0.163 = -0.027 to sustain an inference for a null hypothesis of 0 effect.
For calculation of unconditional ITCV, request raw output. When using pkonfound(),
additionally include the R2, sdx, and sdy as input.
See Frank (2000) for a description of the method.
Citation:
Frank, K. (2000). Impact of a confounding variable on the inference of a
regression coefficient. Sociological Methods and Research, 29 (2), 147-194
Accuracy of results increases with the number of decimals reported.
For other forms of output, run ?pkonfound and inspect the to_return argument
Impact Threshold for a Confounding Variable:
The maximum impact of an omitted variable to sustain an inference
for a null hypothesis of 0 effect is based on a correlation of 0.163
with the outcome and 0.163 with the predictor of interest (conditioning
on all observed covariates in the model; signs are interchageable)
based on a threshold of -0.196 for statistical significance (alpha = 0.05).
Correspondingly the impact of an omitted variable (as defined in Frank 2000) must be
0.163 X 0.163 = 0.027 to sustain an inference for a null hypothesis of 0 effect.
For calculation of unconditional ITCV, request raw output. When using pkonfound(),
additionally include the R2, sdx, and sdy as input.
See Frank (2000) for a description of the method.
Citation:
Frank, K. (2000). Impact of a confounding variable on the inference of a
regression coefficient. Sociological Methods and Research, 29 (2), 147-194
Accuracy of results increases with the number of decimals reported.
For other forms of output, run ?pkonfound and inspect the to_return argument
RIR = 86
The table implied by the parameter estimates and sample sizes you entered:
Fail Success
Control 293 207
Treatment 207 293
The reported effect size = 0.700, SE = 0.100, p-value = 0.000.
The SE has been adjusted to 0.128 to generate real numbers in the
implied table for which the p-value would be 0.000. Numbers in
the table cells have been rounded to integers, which may slightly
alter the estimated effect from the value originally entered.
To invalidate the inference that the effect is different from 0
(alpha = 0.010) one would need to replace 86 (29.352%) treatment success
cases with cases for which the probability of failure in the control
group (58.600%) applies (RIR = 86). This is equivalent to transferring
50 cases from treatment success to treatment failure (Fragility = 50).
Note that RIR = Fragility/[1-P(success in the control group)]
The transfer of 50 cases yields the following table:
Fail Success
Control 293 207
Treatment 257 243
Effect size = 0.291, SE = 0.127, p-value = 0.011.
This is based on t = estimated effect/standard error
See Frank et al. (2013) for a description of the method.
Citation: Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. (2013).
What would it take to change an inference?
Using Rubin's causal model to interpret the robustness of causal inferences.
Education, Evaluation and Policy Analysis, 35 , 437-460.
Accuracy of results increases with the number of decimals reported.
For other forms of output, run ?pkonfound and inspect the to_return argument
RIR = 24
The table implied by the parameter estimates and sample sizes you entered:
Fail Success
Control 70 180
Treatment 68 182
The reported effect size = 0.030, and SE = 0.200, p-value = 0.841.
Values have been rounded to the nearest integer. This may cause
a little change to the estimated effect for the table.
To reach the threshold that would sustain an inference that the
effect is different from 0 (alpha = 0.050) one would need to replace 24
(35.294%) treatment failure cases with cases for which the probability of
failure in the control group (28.000%) applies (RIR = 24). This is equivalent
to transferring 17 cases from treatment failure to treatment success
(Fragility = 17).
Note that RIR = Fragility/[1-P(failure in the control group)]
The transfer of 17 cases yields the following table:
Fail Success
Control 70 180
Treatment 51 199
Effect size = 0.417, SE = 0.211, p-value = 0.049.
This is based on t = estimated effect/standard error
See Frank et al. (2013) for a description of the method.
Citation: Frank, K.A., Maroulis, S., Duong, M., and Kelcey, B. (2013).
What would it take to change an inference?
Using Rubin's causal model to interpret the robustness of causal inferences.
Education, Evaluation and Policy Analysis, 35 , 437-460.
Accuracy of results increases with the number of decimals reported.
For other forms of output, run ?pkonfound and inspect the to_return argument
For models fit in R, consider use of konfound().
pkonfound / 2 by 2 Table
# 2by2 table by tkonfound, Fisherpkonfound(a =35, b =17, c =10, d =5, test ="chisq", "to_return"="print")
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Background Information:
This function calculates the number of cases that would have to be replaced
with zero effect cases (RIR) to invalidate an inference made about the association
between the rows and columns in a 2x2 table.
One can also interpret this as switches from one cell to another, such as from
the treatment success cell to the treatment failure cell.
Conclusion:
To sustain an inference, one would need to replace 16 treatment failure
cases for which the probability of failure in the control group applies (RIR = 16).
This is equivalent to transferring 5 cases from treatment failure to treatment success.
For the User-entered Table, the Pearson's chi square is 0.002, with p-value of 0.963:
User-entered Table:
Fail Success
Control 35 17
Treatment 10 5
For the Transfer Table, the Pearson's chi square is 5.585, with p-value of 0.018:
Transfer Table:
Fail Success
Control 35 17
Treatment 5 10
RIR:
RIR = 16
For other forms of output, run ?pkonfound and inspect the to_return argument
For models fit in R, consider use of konfound().
# 2by2 table by tkonfound, chi-squarepkonfound(a =35, b =17, c =10, d =5, test ="chisq", "to_return"="print")
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Warning in chisq.test(table, correct = FALSE): Chi-squared approximation may be
incorrect
Background Information:
This function calculates the number of cases that would have to be replaced
with zero effect cases (RIR) to invalidate an inference made about the association
between the rows and columns in a 2x2 table.
One can also interpret this as switches from one cell to another, such as from
the treatment success cell to the treatment failure cell.
Conclusion:
To sustain an inference, one would need to replace 16 treatment failure
cases for which the probability of failure in the control group applies (RIR = 16).
This is equivalent to transferring 5 cases from treatment failure to treatment success.
For the User-entered Table, the Pearson's chi square is 0.002, with p-value of 0.963:
User-entered Table:
Fail Success
Control 35 17
Treatment 10 5
For the Transfer Table, the Pearson's chi square is 5.585, with p-value of 0.018:
Transfer Table:
Fail Success
Control 35 17
Treatment 5 10
RIR:
RIR = 16
For other forms of output, run ?pkonfound and inspect the to_return argument