Assignment Week 13 ~ Confidence Interval
Rafael Yogi Septiadi P.
21 December 2025
- 1 Introduction
- 2 Statistical Theory and Computation Framework
- 3 Z-Score and t-Score Reference Tables
- 4 Case Study 1: Confidence Interval for Mean (\(\sigma\) Known)
- 5 Case Study 2: Confidence Interval for Mean (\(\sigma\) Unknown)
- 6 Case Study 3: Confidence Interval for a Proportion (A/B Testing)
- 7 Case Study 4: Precision Comparison (Z-Test vs t-Test)
- 8 Case Study 5: One-Sided Confidence Interval (SaaS Feature)
- References
Rafael Yogi S.P.
Data Science Major
1 Introduction
In data science and analytics, confidence intervals (CIs) are fundamental tools for statistical inference. Unlike point estimates, which provide a single best guess of a population parameter, confidence intervals quantify the uncertainty arising from sampling variability. This is particularly important in business analytics, UX research, and product experimentation, where decisions must be made under uncertainty Ripley (2015).
From a data scientist’s perspective, confidence intervals help answer questions such as:
- How precise is our estimate?
- How reliable is the observed effect?
- Does the data statistically support a business or product decision?
2 Statistical Theory and Computation Framework
Before solving each case study, we summarize the statistical theory and formulas used throughout the analysis.
2.1 Confidence Levels and Significance
A confidence level represents the proportion of intervals that would contain the true population parameter if the sampling process were repeated many times. It is defined as:\[ 1 - \alpha \]
where \(\alpha\) is the significance level. Higher confidence levels require larger critical values and therefore produce wider confidence intervals DSCI Labs (2025).
2.2 Confidence Interval for the Mean (Population Standard Deviation Known)
When the population standard deviation \(\sigma\) is known and the sample size is large (or the population is normally distributed), the sampling distribution of the sample mean follows a normal distribution.
The confidence interval for the population mean \(\mu\) is:
\[ \bar{x} \pm z_{\alpha/2} \frac{\sigma}{\sqrt{n}} \]
where:
- \(\bar{x}\) : sample mean
- \(\sigma\) : population standard
deviation
- \(n\) : sample size
- \(z_{\alpha/2}\) : critical value
from the standard normal distribution
This method is referred to as a Z-confidence interval.
2.3 Confidence Interval for the Mean (Population Standard Deviation Unknown)
When \(\sigma\) is unknown and the sample size is small, it is estimated using the sample standard deviation \(s\). In this case, the sampling distribution follows a t-distribution with \(n-1\) degrees of freedom.The confidence interval becomes:
\[ \bar{x} \pm t_{\alpha/2,\,n-1} \frac{s}{\sqrt{n}} \]
This interval is wider than the Z-interval due to additional uncertainty from estimating \(\sigma\).
2.4 Confidence Interval for a Population Proportion
For binary outcomes (e.g., click vs no-click), the parameter of interest is the population proportion \(p\). The sample proportion is:\[ \hat{p} = \frac{x}{n} \]
For sufficiently large samples, the confidence interval for \(p\) is approximated by:
\[ \hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}} \]
This approach is widely used in A/B testing and product analytics.
2.5 One-Sided Confidence Intervals
In some business contexts, interest lies only in whether a parameter exceeds or falls below a certain threshold. In such cases, a one-sided confidence interval is appropriate.A lower one-sided confidence interval for a proportion is given by:
\[ \hat{p} - z_{\alpha} \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}} \]
This formulation ensures that the true parameter is at least as large as the lower bound with a specified confidence level.
3 Z-Score and t-Score Reference Tables
3.1 Z-Score Table
The Z-score table is a reference table that converts Z-scores into standard deviation units. It is commonly used in statistical analysis for converting Z-scores into meaningful values.
The Z-score table is a graphical representation of a standard normal distribution. Each Z-score represents a specific distance from the mean on the y-axis, and the corresponding standard deviation on the x-axis.
3.2 T-Score Table
The T-score table is a reference table that converts T-scores into degrees of freedom. It is commonly used in statistical analysis for converting T-scores into meaningful values when estimating population parameters.
The T-score table is a graphical representation of a t-distribution. Each T-score represents a specific distance from the mean on the y-axis, and the corresponding degrees of freedom on the x-axis.
4 Case Study 1: Confidence Interval for Mean (\(\sigma\) Known)
4.1 Problem Description
An e-commerce platform wants to estimate the average number of daily transactions per user after launching a new feature. Based on large-scale historical data, the population standard deviation is known.
4.1.1 Given Data:
Population standard deviation (\(\sigma\)): \(3.2\)
Sample size (\(n\)): \(100\)
Sample mean (\(\bar{x}\)): \(12.6\)
4.2 Task 1: Identify and Justify the Statistical Test
The appropriate statistical test to use is the Z-Test (Standard Normal Distribution).
Justification:
Known \(\sigma\): The population standard deviation is explicitly provided (\(\sigma = 3.2\)).
Sample Size: The sample size (\(n = 100\)) is sufficiently large (\(n \ge 30\)), satisfying the Central Limit Theorem (CLT) requirements, which ensures that the sampling distribution of the mean is approximately normal.
4.3 Task 2: Compute Confidence Intervals (90%, 95%, 99%)
The following table shows the critical Z-scores and the resulting intervals for the three requested confidence levels.
4.4 Task 3: Comparison Visualization
The plot below visualizes how the interval width expands as the confidence level increases.
4.5 Task 4: Business Interpretation
From a business analytics perspective, these results provide several insights:1. Reliability of the New Feature: We are 95% confident that the true average number of daily transactions per user falls between 11.973 and 13.227. This narrow range suggests high precision in our estimate.
2. Risk vs. Precision: If the management requires absolute certainty (99%), the interval becomes wider (11.776 to 13.424). This trade-off means that higher confidence requires accepting a less specific estimate.
3. Strategic Planning: If the business goal was to achieve an average of at least 12 transactions per user, the lower bound of even the 99% interval (11.776) is above 11.7, which gives strong evidence that the feature is performing close to or above expectations.
5 Case Study 2: Confidence Interval for Mean (\(\sigma\) Unknown)
5.1 Problem Description
A UX Research team analyzes task completion time (in minutes) for a new mobile application. The data are collected from 12 users:
5.1.1 Data:
8.4, 7.9, 9.1, 8.7, 8.2, 9.0, 7.8, 8.5, 8.9, 8.1, 8.6, 8.3
5.2 Task 1: Identify and Explain the Statistical Test
The appropriate test is the t-Test (Student’s t-distribution).
Explanation: 1. Unknown \(\sigma\): The population standard deviation is unknown; we must estimate it using the sample standard deviation \(s\).
2. Small Sample Size: The sample size (\(n = 12\)) is small (\(n < 30\)). The t-distribution is designed to account for the extra uncertainty in small samples.
5.3 Task 2: Compute Confidence Intervals
5.4 Task 3: Visualize the Three Intervals
5.5 Task 4: Factors Influencing Interval Width
1. Confidence Level: As the confidence level increases (e.g., from 90% to 99%), the interval width increases. This is because we need a wider range to be more “certain” that it contains the true mean.
2. Sample Size: The interval width is inversely proportional to \(\sqrt{n}\). A smaller sample size (like \(n=12\)) results in a wider interval and a higher critical t-value, reflecting greater uncertainty compared to larger samples.
6 Case Study 3: Confidence Interval for a Proportion (A/B Testing)
6.1 Problem Description
A data science team runs an A/B test on a new Call-To-Action (CTA) button design.
6.1.1 Data:
\(n = 400\) (total users)
\(x = 156\) (users who clicked the CTA)
6.2 Task 1: Compute the Sample Proportion (\(\hat{p}\))
The sample proportion is computed as:
\[\hat{p} = \frac{x}{n} = \frac{156}{400} = 0.39\]
6.3 Task 2: Compute Confidence Intervals (90%, 95%, 99%)
6.4 Task 3: Visualize and Compare Intervals
6.5 Task 4: Confidence Level and Decision-Making
In product experiments:
1. Risk Mitigation: Higher confidence levels (99%) reduce the “False Positive” risk (Type I error), which is crucial for major design overhauls.
2. Agility: Lower confidence levels (90%) produce narrower intervals, allowing teams to make faster decisions if the conversion lift is significant, though with a higher risk of error.
3. Experimental Baseline: If the interval contains the baseline conversion rate, the new design is not statistically “better,” regardless of the point estimate \(\hat{p}\).
7 Case Study 4: Precision Comparison (Z-Test vs t-Test)
7.1 Problem Description
Two teams measure API latency (\(ms\)).
Team A: \(n=36, \bar{x}=210, \sigma=24\) (Known)
Team B: \(n=36, \bar{x}=210, s=24\) (Sample)
7.2 Task 1: Identify Statistical Tests
Team A uses the Z-Test because the population standard deviation (\(\sigma\)) is known.
Team B uses the t-Test because the population standard deviation is unknown and estimated via sample \(s\).
7.3 Task 2: Compute Confidence Intervals
7.4 Task 3: Visualization Comparison
7.5 Task 4: Why do the widths differ?
The interval widths differ because the t-distribution has heavier tails than the Z-distribution.
1. Uncertainty Penalty: Team B does not know \(\sigma\). Using \(s\) introduces extra sampling error. To maintain the same confidence level, the t-distribution must use a larger critical value.
2. Critical Values: At 95%, \(Z \approx 1.96\) while \(t_{35} \approx 2.03\). This \(\approx 3.6\%\) difference in the multiplier directly results in a wider interval for Team B.
8 Case Study 5: One-Sided Confidence Interval (SaaS Feature)
8.1 Problem Description
A SaaS company wants to ensure at least 70% of weekly active users utilize a premium feature.
\(n = 250\) (total users)
\(x = 185\) (active premium users)
Goal: Lower bound \(\ge 0.70\)
8.2 Task 1: Identify the Statistical Test
The appropriate test is a One-Sided Z-Test for Proportions. Since \(n\hat{p} = 185\) and \(n(1-\hat{p}) = 65\), both are greater than 10, satisfying the normal approximation criteria.
8.3 Task 2: Compute One-Sided Lower Bounds
8.4 Task 3: Visualize Lower Bounds
8.5 Task 4: Statistical Satisfaction of Target
The management’s target is 70% (0.7000).
At 90% confidence, the lower bound is 0.7044.
At 95% confidence, the lower bound is 0.6944.
At 99% confidence, the lower bound is 0.6755.
Since all computed lower bounds are greater than 0.70, we can conclude that the 70% target is statistically satisfied at all three confidence levels.