Tugas Week 11 ~ Probability Distribution
1 . Introduction
This introduction serves as the definitive gateway to one of the most foundational and transformative concepts in statistical analysis: Probability Distributions.
Statistics is fundamentally the science of making confident decisions amid uncertainty. When we encounter variable outcomes such as the number of “Heads” we get after tossing a coin \(N\) times we cannot rely on a single, isolated value. This is where the Probability Distribution steps in as the primary analytical tool. A Probability Distribution is the “master map” that precisely describes the likelihood for every possible outcome. The Coin Toss Example: If you toss a coin 10 times, the possible outcomes (Random Variable \(X\) = Number of Heads) range from 0 to 10. The Probability Distribution tells you:
• What is the chance of getting exactly 5 Heads?
• What is the chance of getting at least 8 Heads? (\(P(X \ge 8)\))
Its function goes far beyond simple prediction. Crucially, it forms the core analytical basis for virtually all inferential statistical methods we use to:
Quantify Risk and Uncertainty: It measures the expected variation or spread of results around the most likely value (e.g., the expected average of 5 Heads).
Drive Inference: It allows us to draw valid, responsible conclusions from sample data such as determining if a deviation (like getting 9 Heads) is unusual enough to conclude the coin is biased. The central concept is the Random Variable (\(X\)), which numerically represents the experimental outcome. The Distribution then maps the probability onto each possible value.
The Critical Insight: Understanding Distribution Properties To make accurate and robust predictions, it is absolutely mandatory that we grasp the characteristics of the distribution we are dealing with:
• Shape: Is the distribution symmetric (like a fair coin’s outcome), skewed, or uniform?
• Spread (Variability): How far are the values likely to scatter from the mean?
• Center (Location): What is the most expected or average value? Without this deep understanding of a distribution’s properties, our probability calculations will be flawed, and our predictive models will be fundamentally unreliable. In essence, Probability Distributions are the universal language that translates raw data into quantifiable, actionable probabilities.
The Four Pillars of Statistical Inference Having established the indispensable role of Probability Distributions, this report will guide you through four essential topics that constitute the building blocks of inferential statistics the field that allows us to move from simply describing data to drawing powerful, general conclusions:
- Continuous Random Variables
We shift our focus to outcomes that can take any value within a given range (e.g., the time it takes to get a Head, weight, or temperature). This necessitates moving from calculating the probability of a specific point to calculating the probability as the area under a curve, with special emphasis on the paramount Normal Distribution.
- Sampling Distributions
This is the hinge concept, explaining how sample statistics behave (e.g., the average proportion of Heads \(\hat{p}\) across many different sets of 10 coin tosses). This distribution is the critical link that connects the small sample we observe to the large population we seek to understand.
- The Central Limit Theorem (CLT)
The CLT is often called the “Gold Standard” or the most crucial result in statistics. Its magical power states that, regardless of the original population distribution’s shape, the distribution of sample averages (the Sampling Distribution) will tend toward a Normal Distribution provided the sample size (\(n\)) is large enough. This allows us to leverage the power of Normal probability for almost any real-world analysis.
- Sample Proportion Distributions
This topic specifically addresses categorical or binary outcomes (Heads/Tails, Yes/No, Success/Failure). We will study the probability distribution of the proportion of successes (\(\hat{p}\)) found in a sample, a vital skill for survey data analysis and percentage-based hypothesis testing. The Introductory Takeaway
Mastering these four concepts from continuous variables to the revolutionary power of the CLT will equip you with the critical capability to analyze data deeply, construct sound statistical models, and ultimately draw robust, solid conclusions grounded in strong probabilistic principles. This foundation will transform you from a simple data consumer into a sophisticated, data-driven decision-maker.
2 . Countinuous Random
2.1 . Summary of the video:
2.1.1 . Discrete Variables: Data Obtained by Counting
A discrete variable represents data whose values are countable and limited to a finite number of possibilities. The key characteristic of this type of data is that it is generated through the process of counting. Consequently, discrete variables can only take on certain, isolated values, leaving no possibility for any value in between the defined steps.
For example, when considering the number of children in a family, the outcomes are restricted to integers like 0, 1, 2, or 3. The video emphasizes that asking for a fraction of a child is nonsensical in this context. Similarly, the score on a test is discrete because the possible outcomes are limited to a finite set of numbers. Even in cases involving currency, where decimals (cents) are present, the variable remains discrete because the precision is limited to a countable number of decimal places, making the set of values finite.
In terms of visualization, discrete data is represented by a Bar Chart, where the gaps between the bars clearly signify the separation between the countable values.
2.1.2 . Continuous Variables: Data Obtained by Measuring
In contrast, a continuous variable is data that can take on any numerical value within a given range. The fundamental source of continuous data is measurement (measuring), not counting.
This reliance on measurement means the number of possible outcomes within any range is infinite and uncountable. The defining trait of continuous data is that you can always achieve greater precision by adding more decimal places.
The video uses age as a prime example. While one might state an age of 23 years, precise measurement can extend this value to 23 years, 6 months, 2 days, 3 seconds, and down to milliseconds or nanoseconds, theoretically without end. The same concept applies to weight. Since measurement can be refined infinitely, all values within the range are theoretically possible.
Continuous variables are typically visualized using a Histogram or a Density Curve. The absence of gaps between the bars in a histogram illustrates the continuity of the data, reflecting the infinite possibilities of measurement within the observed range.
2.1.3 . Probability Formulas and Concepts
The difference between counting and measuring fundamentally changes how we calculate probability:
For Discrete Variables Focus: Finding the probability of a single, exact value (\(P(X = x)\)).
Calculation: Probability is found by Summation (adding up) the probabilities of all relevant outcomes.
For Continuous Variables Focus: Probability is calculated for a range of values (\(P(a < X < b)\)), not a single point.Calculation: Probability is equal to the Area Under the Density Curve (which involves integration, or calculus).
Key Fact: The probability of a single, exact value in a continuous distribution (e.g., \(P(X = 170.000...)\)) is always zero.
Major Example: The Normal Distribution is the most common example of a density curve used for continuous variables.
2.1.4 . Visualization
A. Discrete Data Visualization (Bar Chart with Gaps)
This code demonstrates a discrete variable (e.g., the probability of
getting a certain number of heads from four coin flips), emphasizing the
gaps between bars.
B. Continuous Data Visualization (Density Curve/Histogram)
This code demonstrates a continuous variable (like height, following a Normal Distribution), showing the smooth curve and connected histogram bars.
C. Conceptual Formula in R (Area Under the Curve)
The norm() function in R calculates the area under the Normal Distribution curve (cumulative probability), which is the method used for continuous variables.Scenario:
If height (\(X\)) is normally distributed with a mean (\(\mu\)) of \(170\) cm and a standard deviation (\(\sigma\)) of \(10\) cm, what is the probability that a person’s height is less than \(180\) cm? (\(P(X < 180)\))
The relationship between the Continuous Random Variable (RV), the Probability Density Function (PDF), and the Cumulative Distribution Function (CDF) is the fundamental bridge connecting the field of statistics with the powerful tools of integral and differential calculus.
- The Continuous Random VariableA random variable (\(X\)) is classified as continuous if it can assume any value within a given interval on the real number line, such as \([a, b]\) or \((-\infty, \infty)\). Examples include measured quantities like height, time, temperature, and velocity.
Zero Point Probability: Since there is an infinite number of possible values within any interval, the probability of the variable landing exactly on any single point is always zero:\[P(X = x) = 0\]
Interval-Based Probability: Probabilities are meaningful only when measured over an interval:\[P(a \le X \le b) = \int_{a}^{b} f(x) dx\]
- The Probability Density Function (PDF), \(f(x)\)The PDF, denoted \(f(x)\), is the descriptive function that illustrates the density of probability around each value \(x\). The PDF is the operational heart of any continuous distribution.
Interpretation Caveat: A crucial distinction is that the value of \(f(x)\) itself is not a probability.
Density Indicator: Larger values of \(f(x)\) merely indicate a higher probability density around that specific value.
Validation Requirements: A function \(f(x)\) is a valid PDF only if it satisfies two essential rules:
Non-negativity: \(f(x) \ge 0\) for all \(x\).Total Area Equals 1:
The total area under the PDF curve must integrate to 1:\[\int_{-\infty}^{\infty} f(x) dx = 1\]
Probability Calculation: The probability for an interval (\(P(a \le X \le b)\)) is computed as the area under the PDF curve between \(a\) and \(b\):\[P(a \le X \le b) = \int_{a}^{b} f(x) dx\]Example PDF: \(f(x) = 3x^2\) on \([0, 1]\).
- The Cumulative Distribution Function (CDF), \(F(x)\)The CDF, denoted \(F(x)\), is defined as the probability that the Random Variable \(X\) takes a value less than or equal to \(x\) (\(P(X \le x)\)). The CDF is an inherently cumulative measure.
Integral Relationship: The CDF is the integral of the PDF from the lower bound of the distribution up to the point \(x\):\[F(x) = P(X \le x) = \int_{low}^{x} f(t) dt\]Example CDF: For the PDF \(f(x) = 3x^2\) on \([0, 1]\), the CDF is calculated as \(F(x) = \int_{0}^{x} 3t^2 dt = x^3\).
The Reciprocal Relationship: PDF vs. CDF
The relationship between \(f(x)\) and \(F(x)\) is the backbone of the Fundamental Theorem of Calculus. They are an integral/derivative pair:\[\text{PDF } f(x) = \frac{d}{dx} F(x) = F'(x)\]This means the PDF is simply the rate of change of the CDF. You can obtain the PDF by differentiating the CDF, and you can obtain the CDF by integrating the PDF.
3 . Sampling Distribuition
This summary, based on the video “Sampling Distributions (7.2)”, unlocks one of the most vital concepts in inferential statistics: understanding not just single data points, but the behavior of averages drawn repeatedly from a population.
3.1 . Summary of the video:
The video starts by resolving a major source of confusion: differentiating between three types of statistical distributions:
The Population Distribution: this is the complete data set from every individual in the entire population, characterized by its mean (\(\mu\)) and standard deviation (\(\sigma\)).
The Sample Distribution: this is merely the data collected from one single random sample. It’s highly variable and often fails to perfectly mirror the population.
The Sampling Distribution: this is our central focus. It is a distribution built not from individual scores, but from the statistic (like the mean, \(\bar{x}\)) calculated from hundreds or thousands of random samples taken from the same population. It’s essentially a “stack of sample means” that tells us the probability of observing a specific sample average.
The power of the Sampling Distribution lies in its predictability, which allows us to generalize results from a small group to a large population:
Stable Mean: the mean of the Sampling Distribution (\(\mu_{\bar{x}}\)) will always equal the population mean (\(\mu\)).Reduced Variability (The Standard Error): Crucially, sample means are less variable than individual observations. Therefore, the spread of the Sampling Distribution is much narrower and taller than the original population curve. This measure of spread is called the Standard Error (\(\sigma_{\bar{x}}\)).A Move Toward Normality: Even if the original Population Distribution is strangely shaped, the Sampling Distribution will tend toward a Normal (bell-shaped) distribution as the sample size increases. This principle is the cornerstone of the Central Limit Theorem.
The relationship between the population and the sampling distribution is governed by these two fundamental formulas:
Mean of the Sampling Distribution:\[\mu_{\bar{x}} = \mu\]Standard Error (Standard Deviation of the Sampling Distribution): This formula shows that as the sample size (\(n\)) increases, the Standard Error decreases, making the distribution tighter.\[\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}\]These are used to standardize a sample mean (\(\bar{x}\)) into a Z-score:\[Z = \frac{\bar{x} - \mu}{\sigma/\sqrt{n}}\]For example, using these formulas, the video calculates that the probability that the average height (\(\bar{x}\)) of a random sample of 10 Canadians is less than 157 cm (given \(\mu = 160 \text{ cm}, \sigma = 7 \text{ cm}\)) is only 8.69%.
4 . Central Limit Theorem
The video “The Central Limit Theorem (7.3)” dives into what is arguably the most fundamental and revolutionary principle in all of modern statistics. The CLT isn’t just a formula; it’s the mathematical magic that allows data scientists to trust their inferential conclusions.
4.1 . Summary of the video:
Before the CLT, statisticians struggled with population data that came in all shapes and sizes skewed, flat, or wildly irregular. The CLT offered a stunning solution, essentially providing a predictive guarantee:
The CLT Statement: If you take simple random samples (\(n\)) that are large enough (typically \(n \ge 30\)) from any population (regardless of its original distribution shape)
Guaranteed Normality: The Sampling Distribution of the sample means (\(\bar{x}\)) will tend toward a Normal Distribution (the bell curve).
Centralized Mean: The mean of the Sampling Distribution (\(\mu_{\bar{x}}\)) will be equal to the true population mean (\(\mu\)).
Standard Error: The spread of this new, normal distribution (the Standard Error, \(\sigma_{\bar{x}}\)) is perfectly predictable by the formula \(\sigma / \sqrt{n}\).
The Simple Power: The CLT essentially normalizes the chaos of the real world. It guarantees that we can use the simple, powerful tools of the Normal Distribution (like Z-scores) to calculate probabilities and test hypotheses on our sample means—a feat impossible on the messy original population data.
If the Population is Normal: If the original population is already Normal, the Sampling Distribution is always Normal, even if the sample size (\(n\)) is small.If the Population is NOT Normal: We must rely on the large sample size condition. The general consensus is that \(n\) must be at least 30 (\(n \ge 30\)) to assume the Sampling Distribution is Normal.
Using the CLT guarantee of normality, we rely on the Z-score formula to find the probability of observing a specific sample mean (\(\bar{x}\)).
5 . Sample Proportion
These two videos—“The Central Limit Theorem (7.3)” and “Sampling Distribution of the Sample Proportion (7.4)”—represent the ultimate transition from descriptive analysis to actionable inferential statistics. Their essential message is that, under the right conditions, the chaotic results of sampling can be reliably transformed into a predictable Normal Distribution.
5.1 . Summary of the video:
5.1.1 . The Central Limit Theorem (CLT) for the Mean (\(\bar{x}\))
The CLT is often called the “magic of statistics” because it provides an incredible guarantee it ensures that the sampling distribution of the mean will be normal, regardless of the original population’s bizarre shape.
5.1.2 . The CLT Mechani
When we take multiple samples, the average of those sample means (\(\mu_{\bar{x}}\)) will precisely equal the true population mean (\(\mu\)). More importantly, the spread of this new distribution known as the Standard Error (\(\sigma_{\bar{x}}\)) is drastically reduced compared to the original population’s standard deviation (\(\sigma\)). This relationship is defined as \(\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}\).
5.1.3 . The Normal Condition (The Rule of 30)
For the CLT to kick in and ensure normality when the population is not normal, the sample size (\(n\)) must be large enough, with the common threshold being \(n \ge 30\).
5.1.4 . The Sampling Distribution of the Proportion (\(\hat{p}\))
When dealing with categorical data (like “yes/no,” “success/failure”), we use the proportion (\(\hat{p}\)) instead of the mean (\(\bar{x}\)). Just like the mean, if we repeatedly sample and plot the resulting proportions, they also form a Sampling Distribution.
5.1.5 . The Proportion Rules
The mean of the sample proportions (\(\mu_{\hat{p}}\)) will equal the true population proportion (\(p\)). The Standard Error for the proportion is calculated using the formula \(\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}\).
5.1.6 . The Normal Condition (The Rule of Successes and Failures)
For the sampling distribution of the proportion to be considered normal, the CLT applies differently. Instead of relying only on \(n \ge 30\), we must verify that there are enough expected successes and enough expected failures: both \(n \cdot p\) and \(n \cdot (1-p)\) must be greater than or equal to 10. Meeting these two simple conditions guarantees that the distribution is sufficiently symmetric to use the Z-score table.
6 . Review Sampling Distribuition
These three videos covering the Central Limit Theorem (CLT), the Sampling Distribution of the Proportion, and Binomial Approximation represent the ultimate transition from analyzing messy sample data to making confident predictions about a large population. Their essential teaching is how we leverage the laws of probability to turn chaotic sample results into a reliable Normal Distribution.
6.1 . Summary of the video:
6.1.1 . The Central Promise
The core idea shifts our focus.Instead of worrying about a single sample’s outcome, we consider the distribution of a statistic (like the mean \(\bar{x}\) or the proportion \(\hat{p}\)) if we were to draw thousands of random samples from the same population. This distribution is called the Sampling Distribution.This shift is crucial because while individual data points can be highly erratic, the averages and proportions of multiple samples follow highly predictable rules dictated by the CLT.
6.1.2 . The Central Limit Theorem (CLT) for the Sample Mean (\(\bar{x}\))
The CLT is statistics’ “magic spell,” guaranteeing that the sampling distribution of the mean will take on the familiar Normal (bell-shaped) Distribution, regardless of how weirdly shaped the original population data is.
Key Principles for \(\bar{x}\):
Stable Center: The average of all sample means (\(\mu_{\bar{x}}\)) will equal the true population mean (\(\mu\)).
Reduced Spread: The variability of this new distribution is drastically smaller than the population’s. This controlled spread is the Standard Error (\(\sigma_{\bar{x}}\)), calculated as \(\sigma / \sqrt{n}\). The bigger the sample size (\(n\)), the smaller and taller the resulting curve is.
The Rule of 30: To ensure this normality takes effect when the population is non-normal, the sample size (\(n\)) must be at least 30 (\(n \ge 30\)).
6.1.3 . The Sampling Distribution of the Sample Proportion (\(\hat{p}\))
When analyzing categorical or binary data (e.g., success/failure, yes/no), we use the sample proportion (\(\hat{p}\)). Similar to the mean, if we repeatedly calculate \(\hat{p}\), its distribution will also approach normality.
Key Principles for \(\hat{p}\):
Stable Center: The mean of the sample proportions (\(\mu_{\hat{p}}\)) equals the true population proportion (\(p\)).
Standard Error: The variability is calculated as \(\sigma_{\hat{p}} = \sqrt{\frac{p(1-p)}{n}}\).
The Rule of Successes and Failures: the condition for normality here is stricter: we must ensure that the number of expected successes (\(n \cdot p\)) and expected failures (\(n \cdot (1-p)\)) are both greater than or equal to 10. Meeting these two conditions guarantees the distribution is symmetric enough for normal approximation.
6.1.4 . Why CLT is Essential (The Binomial Bridge)
One of the most practical applications of the CLT for proportions is solving complicated problems involving the Binomial Distribution.Calculating the exact probability of, say, at least 35 successes out of 100 trials using the traditional Binomial formula is tedious, requiring the calculation and summing of many probabilities. Thanks to the CLT, once the \(n p \ge 10\) and \(n (1-p) \ge 10\) conditions are met, we can approximate that complex probability easily using a single Z-score calculation, saving immense time and effort.
7 . Referensi
“The Basic Practice of Statistics” (Edisi Pertama: 1995)
“Introductory Statistics with R” (Edisi Pertama: 2008)
”Foundations of Statistical Inference and Data Science”
Elementary Statistics: Picturing the World by Ron Larson and Betsy Farber, 7th Edition, published in 2018.
Statistics by James T. McClave, P. George Benson, and Terry Sincich, 13th Edition, published in 2018.