Problem

Raytheon Company (NYSE: RTN) successfully test fired four Precision Extended Range Munition (PERM) guided projectiles. Three of the GPS-guided rounds flew to the required range and hit within the required distance of their targets. One round was fired to the minimum range requirement and impacted just a few meters from an off-axis target.

It means our estimate of probability of success for this test \(P=3/4\) or \(75\)%. Is it enough for GPS-guided round? Sure, it’s not! So how much is enough? We have some guess clue:

Raytheon and Israeli Military Industries jointly developed PERM. The two companies are expected to deliver 42 PERM rounds to the U.S. Marine Corps for additional testing later this year.

library(binom)
library(Bolstad)
library(pwr)
library(ggplot2)

Null hypothesis

We believe the true value for PERM hit probability must be 99%. So our null hypothesis \(H0|P=0.99\), while alternative is \(H1|P<0.99\). WE use binom.test to verify it.

binom.test(3,4,0.99,alternative = "less")
## 
##  Exact binomial test
## 
## data:  3 and 4
## number of successes = 3, number of trials = 4, p-value = 0.0394
## alternative hypothesis: true probability of success is less than 0.99
## 95 percent confidence interval:
##  0.0000000 0.9872585
## sample estimates:
## probability of success 
##                   0.75

Confidence interval

It’s a pity but after the three successful trials out of four we must reject our null hypothesis H0|P=0.99 in favor of alternative. Binomial test has provided confidence interval for our estimate of upper limit for \(P=0.987\). So it is very close to our goal but is not statistically significant to believe it.

Bayesian estimate

Let’s do the estimate for confidence interval using binom.bayes.

bs=binom.bayes(x = 3,n = 4,maxit = 1000)
print(bs)
##   method x n shape1 shape2 mean    lower     upper        sig
## 1  bayes 3 4    3.5    1.5  0.7 0.347072 0.9966562 0.04999999
binom.bayes.densityplot(bs,500)

Bayes is very generous giving for \(P\) confidence interval as \([0.347,0.996]\) with mean \(0.7\). So we got posterior only \(70\)% of success after 4 trials. But there will be 42 trials next year. Why 42 not 30 or 50?

Power of test

Here is the plot where is the answer.

ntrials=1:50
power=binom.power(p.alt=0.75,n=ntrials,p=0.99,alternative="less")
plot(ntrials,power,type="l",main="Power of binomial test",xlab="Number of trials",ylab="Power",col="red")
grid()

We must estimate the power of future experiment with 42 rounds. Let’s do it!

pwr.p.test(ES.h(3/4,0.99),n = 42,alternative = "less")
## 
##      proportion power calculation for binomial distribution (arcsine transformation) 
## 
##               h = -0.8468627
##               n = 42
##       sig.level = 0.05
##           power = 0.9999393
##     alternative = less

Fantastic power - \(99.99\)%. It means after 42 rounds been fired we’ll have only 0.01% chance to make error in assuming H0 by mistake. And what about pi-value for rejecting H0 by mistake? Well, it is only \(5\)%. Suppose we want to be \(1\)% confident for this error type.

pwr.p.test(ES.h(3/4,0.99),n = 42,alternative = "less",sig.level = 0.01)
## 
##      proportion power calculation for binomial distribution (arcsine transformation) 
## 
##               h = -0.8468627
##               n = 42
##       sig.level = 0.01
##           power = 0.9992164
##     alternative = less

The result for power of experiment will be the same - \(99.99\)%. But what if we assume \(80\)% power level and \(5\)% significance for test - how much would be enough?

pwr.p.test(ES.h(3/4,0.99),alternative = "less",sig.level = 0.05,power = 0.8)
## 
##      proportion power calculation for binomial distribution (arcsine transformation) 
## 
##               h = -0.8468627
##               n = 8.620693
##       sig.level = 0.05
##           power = 0.8
##     alternative = less

9 rounds only. Feel the difference. The answer is simple: GPS-rounds are cheaper then BMD (https://rpubs.com/alex-lev/42458) or ICBM missiles and PERM could be used by hundreds or even thousands anywhere without limitations.

PERM is designed to provide U.S. Marines with a lethal, extended range, high probability one-shot capability. The munition is a key capability of the Marine Corps Expeditionary Fire Support System. EFSS is a close support, all weather, quick response indirect fire system.

That is the case!