Probability Distribution

Assignment week 11

Cahaya Medina Semidang

Data Science Undergraduate at ITSB

1 About Probability Distribution

A probability distribution is a mathematical function that assigns the probabilities of different outcomes to the possible values of a random variable. It provides a way of modeling the likelihood of each outcome in a random experiment. While a Frequency Distribution shows how often outcomes occur in a sample or dataset, a probability distribution assigns probabilities to outcomes abstractly, theoretically, regardless of any specific dataset. These probabilities represent the likelihood of each outcome occurring.

Common types of probability distributions include:

Probability Distribution

source

Properties of a probability distribution include:

The probability of each outcome is greater than or equal to zero.
The sum of the probabilities of all possible outcomes equals 1.

This material will guide you through several key concept; Continous Random Variables, Sampling Distribution, The Central Limit Theorem, Sample proportion distribution.

2 Continuous Random Variables

In statistics and probability theory, a continuous random variable is a type of variable that can take any value within a given range. Unlike discrete random variables, which can only assume specific, separate values (like the number of students in a class), continuous random variables can assume any value within an interval, making them ideal for modelling quantities that vary smoothly without jumps. This makes them ideal for modelling a wide range of real-world phenomena, such as the height of individuals, the time taken to complete a task, or the amount of rainfall in a particular period. In this article, we will discuss the concept of “Continuous Random Variable” in detail including its examples and properties. We will also discuss how it is different from a discrete random variable.

2.1 Definition

Continuous random variable is a type of random variable that can take on an infinite number of possible values within a given range. These values are typically real numbers, and the range can be either bounded or unbounded. Unlike discrete random variables, which have countable outcomes, continuous random variables are associated with measurable and uncountable outcomes.

2.2 Examples

Continuous random variables can take any value within a given range and are commonly used in various fields to model and analyze real-world phenomena. Here are some examples:

Height of Individuals: The height of people within a population can vary continuously. Measurements can be as precise as the measurement tool allows, such as 172.3 cm, 172.33 cm, etc.
Weight of Objects: The weight of objects, such as fruits, animals, or packages, is another example. For instance, the weight of an apple can be 150.5 grams, 150.55 grams, and so on.
Temperature: Temperature can be measured to a high degree of precision, such as 23.1°C, 23.12°C, and so forth. It is a continuous variable because it can take any value within the thermometric scale.
Time: The time it takes to complete a task or event, like running a marathon, is a continuous random variable. For instance, a marathon might be completed in 3 hours, 2 minutes, and 47.5 seconds.
Distance:The distance between two points can vary continuously. For example, the distance someone runs can be 5.123 kilometers, 5.1234 kilometers, etc.

2.3 Properties of Continuous Random variable

Continuous random variables have several key properties that distinguish them from discrete random variables and are crucial for understanding their behavior and applications.

2.3.1 Probability Density Function (PDF)

A continuous random variable X is described by a probability density function $f(x)$ . The PDF $f(x)$ gives the relative likelihood of $x$ taking on a specific value $x$ .

The PDF must satisfy two conditions:

$f(x) ≥ 0$ for all $x$
$\int_{- \infty}^{\infty} f (x) d x = 1$

Called a density function, in the following way: the probability that $x$ assumes a value in the interval $[a, b]$ is equal to the area of the region that is bounded above by the graph of the equation $y =
f(x),$ bounded below by the x-axis, and bounded on the left and right by the vertical lines through $a$ and $b$

Probability Density Function (PDF)

$P(a < X < b) = area of shaded region$

source

Probability Given as Area of a Region under a Curve.

Note:The PDF can be understood as the limit of the relative frequency histogram as the number of data points becomes very large and measurement precision increases. When the histogram bars become very narrow, their peaks form a smooth curve called the PDF.

For any continuous random variable $x$ : $\begin{matrix} P (a \leq X \leq b) = P (a < X \leq b) = P (a \leq X < b) = P (a < X < b) \end{matrix}$

2.3.2 Cumulative Distribution Function (CDF)

The cumulative distribution function $F(x)$ of a continuous random variable $X$ represents the probability that $X$ takes a value less than or equal to $X$ : $F (x) = P (X \leq x) = \int_{- \infty}^{x} f (t) d t$

The CDF is non-decreasing and continuos, with: $lim_{x \to - \infty} F (x) = 0 and lim_{x \to \infty} F (x) = 1$

Some other important properties:

Probability of Intervals

The probability that $X$ lies within an interval $[a, b]$ is given by the integral of the PDF over that interval:

$P (a \leq X \leq b) = \int_{a}^{b} f (x) d x$

The probability of $X$ taking any specific value $X$ is zero, i.e., $P(X = x)
= 0$ .

Moments

Moments are quantitative measures related to the shape of the distribution. The nth moment about the origin is given by:

$E (X^{n}) = \int_{- \infty}^{\infty} x^{n} f (x) d x$

The nth central moment (about the mean) is given by:

$E [{(X - μ)}^{n}] = \int_{- \infty}^{\infty} {(x - μ)}^{n} f (x) d x$

Skewness and Kurtosis

Skewness measures the asymmetry of the probability distribution of a real-valued random variable about its mean:

$Skewness = \frac{E [{(X - μ)}^{3}]}{σ^{3}}$

Kurtosis measures the “tailedness” of the probability distribution:

$Kurtosis = \frac{E [{(X - μ)}^{4}]}{σ^{4}}$

2.4 Common Continuous Random Variables

Some of the common distributions where continuous random variable is used are:

Uniform Distribution
Normal Distribution
Exponential Distribution

2.4.1 Uniform Distribution

A continous random variable $X$ is uniformly distributed between a and b if its PDF is:

$f (x) = {\begin{matrix} \frac{1}{b - a} & for a \leq x \leq b \\ 0 & otherwise \end{matrix}$

Properties

Each value in the interval $[a, b]$ is equally likely.
The mean is $\frac{a\ +b}{2}$
The variance is $\frac{{(b-a)}^2}{12}$

2.4.2 Normal Distribution

A continuous random variable X is said to follow a normal distribution with mean μ and variance σ2 if its PDF is given by:

$f (x) = \frac{1}{σ \sqrt{2 π}} \exp (- \frac{(x - μ) ²}{2 σ^{2}})$

Properties

Symmetrical about the mean $\mu$
The mean, median, and mode are all equal
The distribution is described by the parameters $\mu$ (mean) and $𝜎$ (standard deviation).

2.4.3 Exponential Distribution

A continuous random variable X follows an exponential distribution with rate parameter $λ$ if its PDF is:

$f (x) = {\begin{matrix} λ e^{- λ x} & for x \geq 0 \\ 0 & for x < 0 \end{matrix}$

Properties

Describes the time between events in a Poisson process.
The mean is $1/λ$.
The variance is $1/λ^2$.

Note: Other then these there are some distributions where continuous random variable is useful are:

Gamma Distribution
Beta Distribution
Chi-Square Distribution
Student’s t-Distribution

Differences between continuous and discrete random variables

The key differences between continuous and discrete random variables are listed in the following table:

Aspect	Continuous	Discrete
Definition	Can take any value in a range	Can take only specific values
Examples	Height, weight, temperature	Number of children, dice outcomes
Distribution	PDF	PMF
Probability	Over intervals	At specific values
P(X = x)	Always 0	Can be > 0

3 Sampling Disributions

Sampling distribution is essential in various aspects of real life, essential in inferential statistics. A sampling distribution represents the probability distribution of a statistic (such as the mean or standard deviation) that is calculated from multiple samples of a population. It helps us to understand how a statistic varies across different samples and is crucial for making inferences about the population.

Before we talk about sampling distributions, we need to know the difference between a sample distribution and a sample distribution

Sample Distribution

Involves taking a singular sample from a population, and interpreting the data.

Sampling Distribution

A dstribution of a statistic made from multiple simple random samples drawn from a specific population

Definition: Sampling distribution is the probability distribution of a statistic based on random samples of a given population. It is also know as finite distribution.

Important Terminologies in Sampling Distribution

Some important terminologies related to Sampling Disribution are given below:

Statistic: Summary value from a sample (ex: mean, median).
Parameter: Summary value from a population.
sample: A subset of a population.
Population: The entore group being studied.
Sampling Distribution: Distribution of a statistic across many samples.
Central Limit Theorem: Sample means follow a nornal distribution as the sample size increases.
Standard Error: Standard deviation of the sampling distribution.
Bias: Systematic error causing deviation from the true value.
Confidence Interval: Range likely to contain the population parameter.
Sampling Method: How samples are chosen (random, stratified, etc.).
Inferential Statistics: Concluding a population from samples.
Hypothesis Testing: making decision about population parameters using sample data.

3.1 Sampling Distribution of Mean

The sampling distribution of the mean refers to the probability distribution of sample means that you get by repeatedly taking samples (of the same size) from a population and calculating the mean of each sample. Suppose we wish to estimate the mean of a population. In actual practice we would typically take just one sample. Imagine however that we take sample after sample, all of the same size $n$ , and compute the sample mean $\bar{x}$ each time. The sample mean $x$ is a random variable: it varies from sample to sample in a way that cannot be predicted with certainty. We will write $\bar{X}$ when the sample mean is thought of as a random variable, and write $x$ for the values that it takes. The random variable $\bar{X}$ has a mean, denoted $μ_{\bar{X}}$ , and a standard deviation, denoted $σ_{\bar{X}}$ .

Mean

or center of the sampling distribution of $\bar{x}$ , is equal to the population mean, $\mu$. $μ_{\bar{X}} = μ$

Standard Deviation

$σ_{\bar{X}} = \frac{σ}{\sqrt{n}}$

Where:

$σ_{\bar{X}}$ : Standard Deviation of Sampling Deviation
$\sigma$ : Population Standard Deviation
$n$ : Sample Size

4 Central Limit Theorem

4.1 Definition

Definition: Central Limit Theorem predicts the shape of a sampling distribution based on the sample size. Specifically the central limit theorem states that if the sample size n is large enough, then the sampling distribution of the sample mean will be approximately normal. The Central Limit Theorem (CLT) says that when you take many random samples from a population, the average of those sample means will get closer to the population mean as the sample size grows.

KEY TAKEAWAYS

The CLT shows that with a large enough sample size, the distribution of sample means can accurately reflect the characteristics of a population.
Sample sizes equal to or greater than 30 are often considered sufficient under the central limit theorem.
Investors may use CLT to study a random sample of stocks to estimate returns for a portfolio.

If the sample is sufficiently large (usually n > 30), then the sample means’ distribution will be normally distributed regardless of the underlying population distribution, whether it is normal, skewed, or otherwise.

4.2 Formula

Let us assume we have a random variable X. Let $σ$ be its standard deviation, and $μ$ be the mean of the random variable.

$Z = \frac{\overline{X} - μ}{\frac{σ}{\sqrt{n}}}$

where:

Sample mean = population mean = $μ$
sample standard deviation = $\frac{standard\ deviation}{n}$

4.2.1 Mean of the Sample Mean

according to the Central Limit Theorem: if you have a population with mean $μ$, the mean of the sample means (also called the expected value of the sample mean) will be equal to the population mean:

E (\overline{X}) = μ

4.2.2 Standard Deviation of the Sample Mean

The standard deviation of the sample mean (often called the standard error) describes how much the sample mean is expected to vary from the true population mean. It is calculated using the population standard deviation σ and the sample size n:

Standard deviation from distribution of the sample mean $σ_{\overline{X}} = \frac{σ}{\sqrt{n}}$
Standard deviation from distribution of the proportion sample $σ_{\hat{p}} = \sqrt{\frac{p (1 - p)}{n}}$

5 Sampling Distribution of the Sample Proportion

In inferential statistics, we often rely on sample data to draw conclusions about a larger population. However, every sample we take does not necessarily produce the same result. This natural variation is exactly why we need to understand how a statistic—such as a mean or a proportion—behaves when calculated from many different samples. Previously, we explored the sampling distribution of the sample mean and saw how the means from multiple samples form a predictable distribution. The same idea applies to other statistics, including the sample proportion, which is especially important when working with categorical or success–failure data.

5.1 Basic Concept of Proportion

In Statistics a proportion describes the fraction of favorable outcomes in relation to the whole. Favorable outcome is just any variable you were trying to study, example someones’s height, someone’s weight, someone’s eye color or the score they got on a test. These are all measurable variables that we can record from a population or sample.

Formula:

$Proportion = \frac{number\ of\ favourable\ outcomes(X)}{total\ number\ of\ outcomes (N)}$

Notation used:

$X$ : Number of desired outcomes
$N$ : Total population size

Example:

If $N=5,000$ and $X=900$ (green-eyed individuals), then $\mathbf{p}= \frac{900}{5,000} = 0.18$

5.2 Sampling Distribution of the Sample Proportion ($\hat{p}$)

sampling distribution involves repeatedly taking a sample from a population calculating a statistic for each individual such as $\bar{x}$ and then combining that information on a graph to create a distribution.

Example

If $n=10$ and $X=2$ (green-eyed individuals), then $\mathbf{\hat {p}}=\frac{2}{10} = 0.2$.

Notation

$X$: Number of desired outcomes
$n$: Total sample size

5.2.1 Key Characteristics of the Sampling Distribution of $\hat {p}$

When the sampling distribution of approximates a normal distribution (according to the Central Limit Theorem), it has the following characteristics:

Characteristics of the Sampling Distribution for Proportion
Characteristic	Notation	Formula	Description
Mean	$\mu_{\hat{p}}$	$\mu_{\hat{p}} = p$	The mean of all $\hat{p}$ values equals the population proportion ($p$).
Standard Deviation	$\sigma_{\hat{p}}$	$\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}$	Known as the Standard Error of Proportion. (With $q = 1 - p$.)

5.3 Condition for Applying CLT

Unlike the sampling distribution of the mean (which typically requires $n\geq 30$), the CLT for proportions requires two conditions to be met:

Sufficient number of successes: $n\cdot p\geq 10$
Sufficient number of failures: $n\cdot (1-p)\geq 10$

If both conditions are satisfied, the sampling distribution of $\hat {p}$ is considered approximately normal, and we can use the Z-score formula.

5.3.1 Standardization Formula (Z-Score)

When CLT conditions are met, you can standardize the sample proportion ($\hat {p}$) to find probabilities (area under the normal curve) using the Z-score formula:

$Z = \frac{\hat{p} - p}{σ_{\hat{p}}} or Z = \frac{\hat{p} - p}{\sqrt{\frac{p (1 - p)}{n}}}$

$Z$: Standardized value (Z-score)
$\mathbf{\hat {p}}$: Observed sample proportion
$\mathbf{p}$: Population proportion
$\mathbf{n}$: Sample size

6 Review Sampling Distribution

This video is a structured review of three major topics in introductory statistics; Basic probability, The binomial distribution, The sampling distribution of the sample proportion

6.1 Probability

Sample Space

The set of all possible outcomes of an experiment. Example: rolling a die -> {1,2,3,4,5,6}

Simple Probability

for equally likely outcomes: \[P(A)=\frac{\mathrm{number\ of\ favorable\ outcomes}}{\mathrm{total\ outcomes}}\]

Basic Probability Rules

Probabilities range from 0 to 1
Complement rule: \[P(A^c)=1-P(A)\]
Addition rule for mutuallyy exclusive events: \[P(A\cup B)=P(A)+P(B)\]

6.2 Binomial Distribution

Condition for a Binomial Experiment

Fixed number of trials $n$
Two possible outcomes (success/failure)
Constant probability of success $p$
Independent trials

Binomial Probability Formula

\[P(X=k)={n \choose k}p^k(1-p)^{n-k}\]

where:

$X$ = number of successes
${n \choose k}$ = number of ways to choose $x$ successes

6.3 Sampling Distribution of the Sample Proportion

Sampling distribution is the probability of a statistic (like$\hat {p}$) based on all possible sample of size $n$

\[Sample Proportion(\hat {p})=\frac{x}{n}\]

6.3.1 Properties of the Sampling Distributin of $\hat {p}$

mean: \[\mu _{\hat {p}}=p\] Standard Deviation (standard error): \[\sigma _{\hat {p}}=\sqrt{\frac{p(1-p)}{n}}\]

6.3.2 Normal Approximation Conditions

The distribution of $\hat {p}$ is approximately normal if:

\[np\geq 10\quad \mathrm{and}\quad n(1-p)\geq 10\]

7 References

[1] Shafer, D. S., & Zhang, Z. (2012). Introductory statistics. Saylor Foundation.

[2] geeksforgeeks.(2025, December 5). Probability Distribution

[3] geeksforgeeks.(2025, July 23). Continous Random Variable

[4] geeksforgeeks.(2025, August 01). Sampling Distribution

[5] Ganti, Akhilesh.(2025, october 09). Central Limit Theorem

Probability Distribution

Assignment week 11