Fleet challenge Omega and Zeta Material probabilities

Author: Oberon
Guild: BaseDeltaZero
Posted: 14 May, 2018
Updated: 4 June, 2018

What’s the chance of winning a Zeta or two?

Everyone wants to be able to forecast their zeta material tokens in order to estimate the amount of time it will take to get those 20 zeta tokens. We can get them from irregular events (unpredicatable) and also buy them from the fleet arena store (quite predictable). That leaves us with the Fleet Challenges for ship ability materials which is quite regular but for unpredictable amounts.

A year ago I started to record my omega and zeta material wins from the Fleet Challenges and recorded each one with the exception of three separate fortnights of results. The following analysis is based on this data only.

Results so far

Cumlative sum and mean for Omega and Zeta Materials:

Summary data:

Table 1: Summary of Zeta and Omega wins to date
Statistic N Mean St. Dev. Min Max
Ability Material Omega 305 0.266 0.543 0 2
Ability Material Zeta 305 0.623 0.663 0 2

N = total number of attempts
Mean = the average number of wins per attempt

This gives an indication of the number of attempts it will take on average to get 20 zetas from Fleet Challenges alone – about 31 attempts or a little over 5 weeks.

Probability distribution

However we don’t know how much confidence we can have in these results since we don’t know how the random numbers were generated and what the probability distribution of those numbers are. What’s the likely worst case senario? The best case? Is it likely could you get 20 zetas in 3 weeks, or only 10 in ten weeks? To do that we need to find what probability distributuon was used to generate the zeta winnings.

My hypothesis is that the method used to generate the results would mimic two Bernoulli trials at a set probability - that is that the outcomes would be binomially distributed random variables with size = 2 and probability = mean / 2. Given the stability of the cumulative sum and mean results I am further assuming that Captial Games haven’t been playing silly buggers and changing the win probabilities over time.

Hypothesis testing

The hypothesis can be assessed:

Data is periodically summarised to provide comparisons at size = 2 (one trial) at size = 4 (two trials, ie one day), size = 12 (six trials, one week) and size = 24 (twelve trials, two weeks).

Goodness-of-fit

Empirical results for both Omega and Zeta Materials appear to have a good fit to the expected distributions:

Table 2: Kolmogorov - Smirnov test results
Material Estimated probability per trial No. of trials KS test p-value
Omega 0.133 1 0.186
Omega 0.133 2 0.244
Omega 0.133 6 0.846
Omega 0.133 12 0.776
Zeta 0.311 1 0.977
Zeta 0.311 2 0.982
Zeta 0.311 6 0.539
Zeta 0.311 12 0.543

Visual comparisons - Empirical data c/w theoretical binomial estimates

95% Confidence Intervals for the true mean

There are other statistical assessments and visual comparisons that can be used but these are sufficient for us to assume that the raw win data could be thought of as binomially distributed with size = 2 and probability ~0.125 (Omega) and ~0.321 (Zeta). Note that we haven’t proved that the this specific probability distribution was used to generate the Zeta and Omega results – but we can not rule out the possibility.

The first thing to do is determine the confidence we can have in the probabilities given above using an exact binomial test:

Table 3: Estimated binomial probabilities and confidence intervals
Material Estimated probability per trial Lower est. Upper est.
Omega 0.133 0.107 0.162
Zeta 0.311 0.275 0.350

These limits are not particularly tight, but the probability estimates are sufficiently accurate to use in estimating times to win.

95% Confidence Intervals for results

The 95% CIs above describe the confidence we have that the true probability of a win is as estimated. However we also want a predictive confidence interval to tell us something about the range in which future results might occur – another answer we get from the assumed probability distribution of wins:

Assuming the true binomial probabilities are close to the estimated ones, these plots show us 1. The 95% confidence limits of weeks taken for an arbitrary number of Zetas / Omegas wins, and 2. The 95% confidence limits of wins after an arbitrary number of weeks.

This answers the mean, worst and best case scenarios. In case you can’t follow the plots, here are a few examples:

Table 4: Number of weeks predicted for 20 Zetas or Omegas
Best case Average Worst case
Omega 7 12.3 20.3
Zeta 3.3 5.3 8.3
Table 5: Number of Zetas/Omegas predicted for four weeks
Best case Average Worst case
Omega 13 6.4 1
Zeta 23 15.0 7