Point estimation in computer science

Algorithms run repeatedly on many inputs. System noise affects each run.

You summarize performance with one value.
An estimate of average run time.

This is point estimation in computer science.

Population parameter and estimator

Let X represent run time from one execution.

The population mean run time is μ.

The sample mean estimates μ.

\[ \mu = E(X) \]

\[ \hat{\mu} = \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i \]

Example data, algorithm runtime

You measure run time in milliseconds.
You repeat the algorithm many times.

Runtime distribution

Point estimate of mean runtime

The sample mean estimates the true mean runtime.

## [1] 118.2637

Bias of the sample mean

Bias measures systematic error of an estimator.

\[ \text{Bias}(\hat{\mu}) = E(\hat{\mu}) - \mu \]

For the sample mean.

\[ E(\bar{X}) = \mu \]

Therefore, the sample mean is unbiased.

Effect of sample size

Larger samples give more stable estimates.

Mean runtime across input sizes

3D plotly view of point estimates

Key takeaways

Point estimation gives one number for an unknown parameter.
In computer science, you use it for benchmarking run time.
The sample mean estimates the true mean run time.
Larger samples reduce variability in the estimate.