Gabay
Literature
I reviewed the papers you sent. Here are the highlights relevant to the current analysis.
In page 34 @fluegge2017 write “The relation between each measurement and the system (S1 and S2) was evaluated with a nonparametric test as in Brunner et al.”. They are hiding behind a heavy methodological reference, without explicitly stating the methods of analysis!!! Even I don’t do such things…
The paper by [@mangano2016] is actually of worth, as they report their statistical analysis with detail.
@imburgia2017 rely on the same method as @mangano2016.
We use the VCA R package [@VCA].
$tooth.21
Result Variance Component Analysis:
-----------------------------------
Name DF SS MS VC %Total
1 total 38.173963 1.683879 100
2 device 1 2.911289 2.911289 0.079387 4.714536
3 model 15 26.309898 1.753993 0* 0*
4 model:replication_number 16 45.409325 2.838083 1.23359 73.258831
5 error 31 11.497961 0.370902 0.370902 22.026634
SD CV[%]
1 1.297644 85.169936
2 0.281757 18.492932
3 0* 0*
4 1.110671 72.898117
5 0.609017 39.972415
Mean: 1.523594 (N = 64)
Experimental Design: balanced | Method: ANOVA | * VC set to 0 | adapted MS used for total DF
$tooth.16
Result Variance Component Analysis:
-----------------------------------
Name DF SS MS VC %Total
1 total 35.288206 0.65227 100
2 device 1 0.49 0.49 0.008439 1.293807
3 model 15 23.480394 1.56536 0.248823 38.147338
4 model:replication_number 16 9.12105 0.570066 0.175059 26.838383
5 error 31 6.8184 0.219948 0.219948 33.720471
SD CV[%]
1 0.807632 86.988311
2 0.091865 9.894543
3 0.498822 53.727052
4 0.4184 45.064969
5 0.468987 50.51353
Mean: 0.928438 (N = 64)
Experimental Design: balanced | Method: ANOVAAttempt to use a linear model via the standard R package for linear modeling lme4 [@lme4].
Questions
In response to the following reviewer remark:
The evaluation is wrong. The planned position is defined as a reference. However, this is deviation from that due to implant placement error. The reference position should be real implant position, obtained by industrial or model scanner, and deviations on scans, obtained with regular scan body and FMA should be measured and compared.
You wrote (in
answers 29.04.23):Answer: we added another set of measurements, in which, laboratory 3D scanner was used for SBIO models scanning. These model scans were used as a reference for implants positions when compared to the MFA and SBIO intraoral scans.
Does this imply that the absolute errors you provided me with are (for a given device, tooth and model) of the form: \[|\text{Actual implant location} - \text{Measured implant location} |\]?
The reviewers seem insistent on this issue, as they all follow @mangano2016.
- Somewhat related to question 1 above, can we obtain data in the form of the actual value (not only the deviation?).
- Does it make sense to average out mesial and distal measurements? Or use them as additional replications?
- I do not understand why @mangano2016 reports obtaining estimates from ANOVA…
Trueness
According to @mangano2016, the correct estimation of mean and variance of trueness is conducted via ANOVA.
# A tibble: 512 × 7
model tooth replication_number device measurement value measure
<dbl> <dbl> <dbl> <chr> <chr> <dbl> <chr>
1 1 16 2 sb apex_vertical_mesial 0.12 apex
2 1 16 2 sb apex_vertical_distal 0.13 apex
3 1 16 2 sb apex_horizontal_mesial 0.19 apex
4 1 16 2 sb apex_horizontal_distal 0.2 apex
5 1 16 2 sb neck_vertical_mesial 0.1 neck
6 1 16 2 sb neck_vertical_distal 0.07 neck
7 1 16 2 sb neck_horizontal_mesial 0.11 neck
8 1 16 2 sb neck_horizontal_distal 0.07 neck
9 2 16 2 sb apex_vertical_mesial 0.19 apex
10 2 16 2 sb apex_vertical_distal 0.22 apex
# ℹ 502 more rowsPrecision
- Assume that mesial and distal can be regarded as samples from the same distribution.
For paper
Trueness
All four distances aggregated
We assume that the direction (horizontal/vertical) and location (apex-neck) all yield errors with equal variance and zero mean and can thus be aggregated.
https://search.r-project.org/CRAN/refmans/gmGeostats/html/accuracy.html
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ device + tooth + (1 | model)
Data: dat_distance
REML criterion at convergence: 366.1
Scaled residuals:
Min 1Q Median 3Q Max
-1.8270 -0.5972 -0.1939 0.3907 6.3349
Random effects:
Groups Name Variance Std.Dev.
model (Intercept) 0.008001 0.08945
Residual 0.079692 0.28230
Number of obs: 1024, groups: model, 16
Fixed effects:
Estimate Std. Error t value
(Intercept) -0.486729 0.070124 -6.941
devicesb 0.002754 0.017644 0.156
tooth 0.042207 0.003529 11.961
Correlation of Fixed Effects:
(Intr) devcsb
devicesb -0.126
tooth -0.931 0.000Analysis of Variance Table
npar Sum Sq Mean Sq F value
device 1 0.0019 0.0019 0.0244
tooth 1 11.4012 11.4012 143.0655ANOVA-like table for random-effects: Single term deletions
Model:
value ~ device + tooth + (1 | model)
npar logLik AIC LRT Df Pr(>Chisq)
<none> 5 -183.04 376.08
(1 | model) 4 -214.05 436.11 62.029 1 3.384e-15 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Model table
| term | estimate | std.error | statistic |
|---|---|---|---|
| (Intercept) | -0.4867 | 0.07012 | -6.941 |
| devicesb | 0.002754 | 0.01764 | 0.1561 |
| tooth | 0.04221 | 0.003529 | 11.96 |
| sd__(Intercept) | 0.08945 | NA | NA |
| sd__Observation | 0.2823 | NA | NA |
Confidence intervals (95%)
| 2.5 % | 97.5 % | |
|---|---|---|
| .sig01 | 0.05841 | 0.1331 |
| .sigma | 0.2701 | 0.2948 |
| (Intercept) | -0.6237 | -0.3497 |
| devicesb | -0.03183 | 0.03733 |
| tooth | 0.03529 | 0.04912 |
Only tooth part aggregated
We assume that the direction (horizontal/vertical) yields errors with equal variance and zero mean and can thus be aggregated. The tooth part is
https://search.r-project.org/CRAN/refmans/gmGeostats/html/accuracy.html
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ device + direction + tooth + (1 | model)
Data: dat_distance
REML criterion at convergence: 341.8
Scaled residuals:
Min 1Q Median 3Q Max
-2.0306 -0.5997 -0.1157 0.4508 6.2537
Random effects:
Groups Name Variance Std.Dev.
model (Intercept) 0.008037 0.08965
Residual 0.077382 0.27818
Number of obs: 1024, groups: model, 16
Fixed effects:
Estimate Std. Error t value
(Intercept) -0.535156 0.069765 -7.671
devicesb 0.002754 0.017386 0.158
directionvertical 0.096855 0.017386 5.571
tooth 0.042207 0.003477 12.138
Correlation of Fixed Effects:
(Intr) devcsb drctnv
devicesb -0.125
dirctnvrtcl -0.125 0.000
tooth -0.922 0.000 0.000Analysis of Variance Table
npar Sum Sq Mean Sq F value
device 1 0.0019 0.0019 0.0251
direction 1 2.4015 2.4015 31.0349
tooth 1 11.4012 11.4012 147.3368ANOVA-like table for random-effects: Single term deletions
Model:
value ~ device + direction + tooth + (1 | model)
npar logLik AIC LRT Df Pr(>Chisq)
<none> 6 -170.88 353.75
(1 | model) 5 -203.18 416.37 64.618 1 9.093e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Model table
| term | estimate | std.error | statistic |
|---|---|---|---|
| (Intercept) | -0.5352 | 0.06976 | -7.671 |
| devicesb | 0.002754 | 0.01739 | 0.1584 |
| directionvertical | 0.09686 | 0.01739 | 5.571 |
| tooth | 0.04221 | 0.003477 | 12.14 |
| sd__(Intercept) | 0.08965 | NA | NA |
| sd__Observation | 0.2782 | NA | NA |
Confidence intervals (95%)
| 2.5 % | 97.5 % | |
|---|---|---|
| .sig01 | 0.05873 | 0.1333 |
| .sigma | 0.2661 | 0.2903 |
| (Intercept) | -0.6714 | -0.3989 |
| devicesb | -0.0313 | 0.03681 |
| directionvertical | 0.0628 | 0.1309 |
| tooth | 0.0354 | 0.04902 |
Only direction aggregated
We assume that the direction (horizontal/vertical) yields errors with equal variance and zero mean and can thus be aggregated. The tooth part is
https://search.r-project.org/CRAN/refmans/gmGeostats/html/accuracy.html
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ device + part + tooth + (1 | model)
Data: dat_distance
REML criterion at convergence: 354.7
Scaled residuals:
Min 1Q Median 3Q Max
-1.8848 -0.5844 -0.1706 0.4183 6.2552
Random effects:
Groups Name Variance Std.Dev.
model (Intercept) 0.008022 0.08956
Residual 0.078385 0.27997
Number of obs: 1024, groups: model, 16
Fixed effects:
Estimate Std. Error t value
(Intercept) -0.449844 0.070162 -6.411
devicesb 0.002754 0.017498 0.157
partneck -0.073770 0.017498 -4.216
tooth 0.042207 0.003500 12.060
Correlation of Fixed Effects:
(Intr) devcsb prtnck
devicesb -0.125
partneck -0.125 0.000
tooth -0.923 0.000 0.000Analysis of Variance Table
npar Sum Sq Mean Sq F value
device 1 0.0019 0.0019 0.0248
part 1 1.3931 1.3931 17.7730
tooth 1 11.4012 11.4012 145.4508ANOVA-like table for random-effects: Single term deletions
Model:
value ~ device + part + tooth + (1 | model)
npar logLik AIC LRT Df Pr(>Chisq)
<none> 6 -177.35 366.70
(1 | model) 5 -209.09 428.17 63.473 1 1.626e-15 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Model table
| term | estimate | std.error | statistic |
|---|---|---|---|
| (Intercept) | -0.4498 | 0.07016 | -6.411 |
| devicesb | 0.002754 | 0.0175 | 0.1574 |
| partneck | -0.07377 | 0.0175 | -4.216 |
| tooth | 0.04221 | 0.0035 | 12.06 |
| sd__(Intercept) | 0.08956 | NA | NA |
| sd__Observation | 0.28 | NA | NA |
Confidence intervals (95%)
| 2.5 % | 97.5 % | |
|---|---|---|
| .sig01 | 0.0586 | 0.1332 |
| .sigma | 0.2678 | 0.2922 |
| (Intercept) | -0.5869 | -0.3128 |
| devicesb | -0.03152 | 0.03703 |
| partneck | -0.108 | -0.03949 |
| tooth | 0.03535 | 0.04906 |
Model report
We fitted a linear mixed model (estimated using REML and nloptwrap optimizer)
to predict value with device, part and tooth (formula: value ~ device + part +
tooth). The model included model as random effect (formula: ~1 | model). The
model's total explanatory power is moderate (conditional R2 = 0.21) and the
part related to the fixed effects alone (marginal R2) is of 0.13. The model's
intercept, corresponding to device = mfa, part = apex and tooth = 0, is at
-0.45 (95% CI [-0.59, -0.31], t(1018) = -6.41, p < .001). Within this model:
- The effect of device [sb] is statistically non-significant and positive (beta
= 2.75e-03, 95% CI [-0.03, 0.04], t(1018) = 0.16, p = 0.875; Std. beta =
8.79e-03, 95% CI [-0.10, 0.12])
- The effect of part [neck] is statistically significant and negative (beta =
-0.07, 95% CI [-0.11, -0.04], t(1018) = -4.22, p < .001; Std. beta = -0.24, 95%
CI [-0.34, -0.13])
- The effect of tooth is statistically significant and positive (beta = 0.04,
95% CI [0.04, 0.05], t(1018) = 12.06, p < .001; Std. beta = 0.34, 95% CI [0.28,
0.39])
Standardized parameters were obtained by fitting the model on a standardized
version of the dataset. 95% Confidence Intervals (CIs) and p-values were
computed using a Wald t-distribution approximation.Angular deviation
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ device + tooth + (1 | model)
Data: dat_distance
REML criterion at convergence: 357.4
Scaled residuals:
Min 1Q Median 3Q Max
-1.4540 -0.7174 -0.2311 0.4185 2.8023
Random effects:
Groups Name Variance Std.Dev.
model (Intercept) 0.2293 0.4789
Residual 0.7841 0.8855
Number of obs: 128, groups: model, 16
Fixed effects:
Estimate Std. Error t value
(Intercept) -0.91317 0.60169 -1.518
devicesb -0.12578 0.15653 -0.804
tooth 0.11903 0.03131 3.802
Correlation of Fixed Effects:
(Intr) devcsb
devicesb -0.130
tooth -0.963 0.000Analysis of Variance Table
npar Sum Sq Mean Sq F value
device 1 0.5063 0.5063 0.6457
tooth 1 11.3348 11.3348 14.4560ANOVA-like table for random-effects: Single term deletions
Model:
value ~ device + tooth + (1 | model)
npar logLik AIC LRT Df Pr(>Chisq)
<none> 5 -178.71 367.42
(1 | model) 4 -185.13 378.27 12.846 1 0.0003382 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Model table
| term | estimate | std.error | statistic |
|---|---|---|---|
| (Intercept) | -0.9132 | 0.6017 | -1.518 |
| devicesb | -0.1258 | 0.1565 | -0.8035 |
| tooth | 0.119 | 0.03131 | 3.802 |
| sd__(Intercept) | 0.4789 | NA | NA |
| sd__Observation | 0.8855 | NA | NA |
Confidence intervals (95%)
| 2.5 % | 97.5 % | |
|---|---|---|
| .sig01 | 0.2548 | 0.7588 |
| .sigma | 0.7741 | 1.006 |
| (Intercept) | -2.09 | 0.2633 |
| devicesb | -0.4325 | 0.1809 |
| tooth | 0.0577 | 0.1804 |
Direction and Orientation and Part - aggregated
Precision
[1] "95% CI for standard deviation of sb: [ 0.336591447061133 , 0.427711832796871 ]"[1] "95% CI for standard deviation of mfa: [ 0.348721555528647 , 0.443527541002128 ]"# A tibble: 4 × 3
...1 name value
<chr> <chr> <dbl>
1 sb ...2 0.437
2 sb ...3 0.467
3 mfa ...2 0.598
4 mfa ...3 0.652