2024-03-17

Statistical Distributions

In Statistics, there are distributions to help us find the likelihood of certain scenario occurring. These distributions are represented by equations called Probability Density Functions (PDFs). The following is a PDF for an exponential distribution:

\(f(x)=\lambda e^{-\lambda x}, x > 0\)


with the expected value of this distribution being:

\(E[X]=\frac{1}{\lambda}\)


The following slide is a graphical representation of a sample from that distribution with a sample size of 50 and \(\lambda = 1\) and how it compares with the pdf of an exponential distribution with \(\lambda = 1\)

Graph of Exponential Distribution \(\lambda = 1\) and Sample

Because the sample does not match the pdf exactly, statisticians must estimate the parameters to the best of their ability. One way to do that is the Method of Moments.

Intro to Method of Moments

When statisticians receive data samples, they have to find distributions that best model this data. This allows them to predict with a greater degree of certainty what future samples could look like.

However, after choosing a distribution to use in the model, a statistician still needs to determine the parameter in that PDF.

Going back to the original example, the parameter that needs to be determined in an exponential PDF is \(\lambda\).

This is done by using a given sample and solving for its first moment and then plugging that into the expected value of the PDF.

Solving for \(\lambda\)

We know that our sample mean is

\(\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i\).


We will set this equal to our expectation:

\(E[X]=\frac{1}{\lambda} = \bar{x}\)


When we solve for \(\lambda\), we get:

\(\lambda = \frac{1}{\bar{x}}\)

Example with real data:

The following is a plot (with a regression line solved for by loess) and a table of a sample of data points that we will use to solve for \(\lambda\):

## `geom_smooth()` using formula = 'y ~ x'

\(x_i = \{2.222, 1.616, 1.264, 0.96, 0.67, \\ 0.5, 0.376, 0.272, 0.202, 0.15\}\)

Continued

We can solve and find that

\(\bar{x}= \frac{1}{n} \sum_{i=1}^{n} x_i = 0.8232\)


Thus,

\(\lambda=\frac{1}{\bar{x}} = \frac{1}{0.8232}=1.214\)


We now can see that the sample of data most closely follows an exponential distribution with the parameter:

\(\lambda=1.214\)

When this parameter is used, the continuous pdf that emerges will match that of the red dotted line.

The Gamma Distribution

Another common distribution we can solve for parameters is the Gamma distribution. It has a pdf as follows:

\(f(x)=\frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}\)


Its expected value is

\(E[X]=\frac{\alpha}{\beta}\)


If we know that \(\alpha = 2\), we can solve for \(\beta\) using the Method of Moments

Continued

We can swap in the sample mean (\(\bar{x}\)) into the \(E[X]\) and solve for \(\beta\)

\(E[X] = \bar{x} = \frac{2}{\beta} => \beta = \frac{2}{\bar{x}}\)


Given the following sample:
## `geom_smooth()` using formula = 'y ~ x'

Continued

\(x_i = \{0, 0.079, 0.12, 0.13, 0.118, 0.104, \\ 0.1, 0.077, 0.0617, 0.0498, 0.04, \\ 0.0312, 0.025, 0.019, 0.011\}\)


We can determine the sample mean:

\(\bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i = 0.063\)

Thus:

\(\bar{x} = \frac{2}{\beta} => \beta = \frac{2}{\bar{x}} \approx 30\)

Conclusion

The Method of Moments serves as a useful, easy way to make a predictive guess as to the structure of a population by simply using a given sample.

This helps us better understand and predict the world in all its complexity, allowing for a little more peace of mind as we try to navigate our way through this complex life.