STUDY CASES – CONFIDENCE INTERVAL
Basic Statistics – Data Science – Assignment Week 13
INSTITUT TEKNOLOGI SAINS BANDUNG
Name : Hirose Kawarin Sirait
Student ID : 52250012
Major : Data Science
Lecturer : Mr. Bakti Siregar, M.Sc., CDS.
1 CASE STUDY 1
In this analysis, the concept of confidence intervals is used to estimate the average number of daily transactions per user after the implementation of a new feature on the e-commerce platform. With the population standard deviation known and a sufficiently large sample size, the normal distribution approach is used to obtain a reliable estimate.
Next, confidence intervals will be calculated at several confidence levels, accompanied by visualizations and interpretations in the context of business analysis.
1.1 Question
An e-commerce platform wants to estimate the average number of daily transactions per user after launching a new feature. Based on large-scale historical data, the population standard deviation is known.
Given:
Population standard deviation (\(\sigma\)) = 3.2
Sample size (\(n\)) = 100
Sample mean (\(\bar{x}\)) = 12.6
Tasks:
Identify the appropriate statistical test and justify your choice.
Compute the confidence intervals for:
90%
95%
99%
Create a comparison visualization of the three confidence intervals.
Interpret the results in a business analytics context.
1.2 Answer
The Z distribution is used when performing statistical inference on the population mean and population standard deviation (\(\sigma\)) known. This distribution is also used when the sample size is large (\(n \geq 30\)), because based on the Central Limit Theorem, the distribution of the sample mean will approach a normal distribution even if the population data is not normal. If the population is normally distributed, the Z distribution can still be used even for small sample sizes. The use of the Z distribution is considered appropriate because the value of \(\sigma\) is known for certain, resulting in stable critical values and more accurate confidence interval estimates compared to the t distribution.
why in this case we can use the z distribution
reason:
All conditions for the z distribution are met
Population standard deviation \(\sigma\) is known
Sample size is large (\(n \geq 30\))
2. Compute the confidence intervals for:
90%
95%
99%
❖
❖ 90%
Given:
Population standard deviation (\(\sigma\)) = 3.2
Sample size (\(n\)) = 100
Sample mean (\(\bar{x}\)) = 12.6
Confidence level (CL) = 90%
For a 90% confidence level: \[ \alpha = 1 - 0.90 = 0.10 \Rightarrow \frac{\alpha}{2} = 0.05 \]
The critical value from the standard normal distribution is: \[ Z_{\alpha/2} = Z_{0.05} = 1.645 \]
The margin of error (E) is calculated as: \[ E = Z_{\alpha/2}\left(\frac{\sigma}{\sqrt{n}}\right) = 1.645\left(\frac{3.2}{\sqrt{100}}\right) \] \[ = 1.645(0.32) = 0.5264 \approx 0.53 \]
Therefore, the 90% confidence interval for the population mean is: \[ \bar{x} - E < \mu < \bar{x} + E \] \[ 12.6 - 0.53 < \mu < 12.6 + 0.53 \]
\[ \boxed{12.07 < \mu < 13.13} \]
❖ 95%
Given:
Population standard deviation (\(\sigma\)) = 3.2
Sample size (\(n\)) = 100
Sample mean (\(\bar{x}\)) = 12.6
Confidence level (CL) = 95%
For a 95% confidence level: \[ \alpha = 1 - 0.95 = 0.05 \Rightarrow \frac{\alpha}{2} = 0.025 \]
The critical value from the standard normal distribution is: \[ Z_{\alpha/2} = Z_{0.025} = 1.96 \]
The margin of error (E) is calculated as: \[ E = Z_{\alpha/2}\left(\frac{\sigma}{\sqrt{n}}\right) = 1.96\left(\frac{3.2}{\sqrt{100}}\right) \] \[ = 1.96(0.32) = 0.6272 \approx 0.63 \]
Therefore, the 95% confidence interval for the population mean is: \[ \bar{x} - E < \mu < \bar{x} + E \] \[ 12.6 - 0.63 < \mu < 12.6 + 0.63 \]
\[ \boxed{11.97 < \mu < 13.23} \]
❖ 99%
Given:
Population standard deviation (\(\sigma\)) = 3.2
Sample size (\(n\)) = 100
Sample mean (\(\bar{x}\)) = 12.6
Confidence level (CL) = 99%
For a 99% confidence level: \[ \alpha = 1 - 0.99 = 0.01 \Rightarrow \frac{\alpha}{2} = 0.005 \]
The critical value from the standard normal distribution is: \[ Z_{\alpha/2} = Z_{0.005} = 2.576 \]
The margin of error (E) is calculated as: \[ E = Z_{\alpha/2}\left(\frac{\sigma}{\sqrt{n}}\right) = 2.576\left(\frac{3.2}{\sqrt{100}}\right) \] \[ = 2.576(0.32) = 0.824 \]
Therefore, the 99% confidence interval for the population mean is: \[ \bar{x} - E < \mu < \bar{x} + E \] \[ 12.6 - 0.824 < \mu < 12.6 + 0.824 \]
\[ \boxed{11.78 < \mu < 13.42} \]
The confidence interval results indicate that as the confidence level increases, the width of the confidence interval also becomes wider. At the 90% confidence level, the interval is relatively narrower, providing a more precise estimate of the average number of daily transactions per user, but with a lower level of certainty. In contrast, the 99% confidence level produces the widest interval, reflecting a higher degree of confidence that the interval contains the true population mean. This demonstrates a clear trade-off between confidence and precision in statistical estimation. In a business analytics context, the choice of confidence level should be aligned with decision-making needs; higher confidence levels offer stronger assurance to management but result in greater uncertainty in the estimated range of average daily transactions per user. In conclusion, the 95% confidence interval is often considered the most balanced option, as it provides a sufficiently high level of confidence while maintaining a reasonably narrow interval to support effective business decision-making.
2 CASE STUDY 2
In this analysis, the concept of confidence intervals is applied to estimate the average task completion time for a new mobile application based on data collected from users. Since the population standard deviation is unknown and the sample size is relatively small, the t-distribution approach is used to provide an appropriate estimation of the population mean.
Furthermore, confidence intervals will be constructed at several confidence levels, namely 90%, 95%, and 99%. The results will be visualized and interpreted to illustrate how different confidence levels influence the width of the confidence interval in the context of user experience analysis.
2.1 Question
Problem Statement
Confidence Interval for Mean, \(\sigma\) Unknown: A UX Research team analyzes task completion time (in minutes) for a new mobile application.The dara are collected from 12 users:
8.4, 7.9, 9.1, 8.7, 8.2, 9.0, 7.8, 8.5, 8.9, 8.1, 8.6, 8.3
Tasks
Identify the appropriate statistical test and explain why.
Compute the confidence intervals for:
90%
95%
99%
Visualize the three confidence intervals on a single plot.
Explain how sample size and confidence level influence the interval width.
2.2 Answer
The t distribution is used when performing statistical inference on the population mean and the population standard deviation (σ) is unknown. This distribution is particularly appropriate for small sample sizes because it accounts for additional uncertainty introduced by estimating the population standard deviation from the sample data. When the population standard deviation is unknown and the sample size is limited, the t distribution provides more reliable confidence interval estimates compared to the Z distribution.
In this case, the use of the t distribution is considered appropriate because the necessary conditions for applying the Z distribution are not fully satisfied. Instead, the t distribution is more suitable as it adjusts for the increased variability associated with small samples and unknown population parameters.
reason:
The population standard deviation \(\sigma\) is unknown
The sample size is small (\(n = 12 < 30\)).
The confidence interval is constructed using the sample standard deviation (\(s\)).
2. Compute the confidence intervals for:
90%
95%
99%
❖
| Description | Value |
|---|---|
| Task Completion Time Data | 8.4, 7.9, 9.1, 8.7, 8.2, 9, 7.8, 8.5, 8.9, 8.1, 8.6, 8.3 |
| Sample Mean | 8.458 |
| Sample Standard Deviation | 0.421 |
The confidence interval for the mean when the population standard deviation is unknown is calculated using the formula: \[ CI = \bar{x} \pm t_{\alpha/2, df} \times \frac{s}{\sqrt{n}} \]
Explanation of the symbols:
\(\bar{x}\) = sample mean
\(s\) = sample standard deviation
\(n\) = sample size
\(df = n - 1\) = degrees of freedom
\(t_{\alpha/2, df}\) = critical t-value
❖ 90%
given
Sample data: 8.4, 7.9, 9.1, 8.7, 8.2, 9.0, 7.8, 8.5, 8.9, 8.1, 8.6, 8.3
Sample size: \(n = 12\)
Sample mean: \(\bar{x} = 8.458 \approx\) (from data)
Sample standard deviation: \(s = 0.421 \approx\) (from data)
Degrees of freedom: \(df = n-1 = 11\)
Confidence level: 90%
\[ \alpha = 1 - 0.90 = 0.10 \quad \Rightarrow \quad \frac{\alpha}{2} = 0.05 \]
\[ t_{0.05, 11} \approx 1.796 \]
\[ E = t_{\alpha/2, df} \times \frac{s}{\sqrt{n}} = 1.796 \times \frac{0.421}{\sqrt{12}} \] \[ = 1.796 \times 0.1216 \approx 0.218 \]
Therefore, the 90% confidence interval for the population mean is: \[
\bar{x} - E < \mu < \bar{x} + E
\] \[
8.458 - 0.218 < \mu < 8.458 + 0.218
\]
\[ \boxed{8.24 < \mu < 8.68} \]
❖ 95%
given
Sample data: 8.4, 7.9, 9.1, 8.7, 8.2, 9.0, 7.8, 8.5, 8.9, 8.1, 8.6, 8.3
Sample size: \(n = 12\)
Sample mean: \(\bar{x} = 8.458 \approx\) (from data)
Sample standard deviation: \(s = 0.421 \approx\) (from data)
Degrees of freedom: \(df = n-1 = 11\)
Confidence level: 95%
\[ \alpha = 1 - 0.95 = 0.05 \quad \Rightarrow \quad \frac{\alpha}{2} = 0.025 \]
\[ t_{0.025, 11} \approx 2.201 \]
\[ E = t_{\alpha/2, df} \times \frac{s}{\sqrt{n}} = 2.201 \times \frac{0.421}{\sqrt{12}} \] \[ = 2.201 \times 0.1215 \approx 0.267 \]
Therefore, the 95% confidence interval for the population mean is: \[ \bar{x} - E < \mu < \bar{x} + E \] \[ 8.458 - 0.267 < \mu < 8.458 + 0.267 \]
\[ \boxed{8.19 < \mu < 8.73} \]
❖ 99%
given
Sample data: 8.4, 7.9, 9.1, 8.7, 8.2, 9.0, 7.8, 8.5, 8.9, 8.1, 8.6, 8.3
Sample size: \(n = 12\)
Sample mean: \(\bar{x} = 8.458 \approx\) (from data)
Sample standard deviation: \(s = 0.421 \approx\) (from data)
Degrees of freedom: \(df = n-1 = 11\)
Confidence level: 99%
\[ \alpha = 1 - 0.99 = 0.01 \quad \Rightarrow \quad \frac{\alpha}{2} = 0.005 \]
\[ t_{0.005, 11} \approx 3.106 \]
\[ E = t_{\alpha/2, df} \times \frac{s}{\sqrt{n}} = 3.106 \times \frac{0.421}{\sqrt{12}} \] \[ = 3.106 \times 0.122 \approx 0.38 \]
Therefore, the 99% confidence interval for the population mean is: \[ \bar{x} - E < \mu < \bar{x} + E \] \[ 8.458 - 0.38 < \mu < 8.458 + 0.38 \]
\[ \boxed{8.08 < \mu < 8.84} \]
The width of a confidence interval is mainly influenced by sample size and confidence level. As the sample size increases, the standard error \(s/\sqrt{n}\) becomes smaller, resulting in a narrower confidence interval and a more precise estimate of the population mean. Conversely, a smaller sample size leads to a wider interval due to greater uncertainty.
In addition, a higher confidence level requires a larger critical value \(t_{\alpha/2}\), which increases the margin of error defined as \(t_{\alpha/2,\,n-1}(s/\sqrt{n})\). As a result, the confidence interval becomes wider. This can be observed in the results, where the 99% confidence interval is wider than the 95% interval, and the 95% interval is wider than the 90% interval.
Conclusion
Higher confidence levels provide greater certainty that the interval contains the true population mean, but this comes at the cost of a wider interval. Increasing the sample size is an effective way to obtain a narrower and more precise confidence interval.
3 CASE STUDY 3
A/B testing is a common experimental method used in data-driven product development to compare user responses to different design alternatives. In this study, an A/B test is conducted to evaluate a new Call-To-Action (CTA) button design.
The objective of this analysis is to estimate the proportion of users who clicked the CTA button and to construct confidence intervals at multiple confidence levels. These intervals are used to assess the reliability of the estimated click-through rate and to understand how uncertainty affects experimental conclusions.
3.1 Question
Problem Description
Confidence Interval for a Proportion, A/B Testing: A data science team runs an A/B test on a new Call-To-Action (CTA) button design. The experiment yields:
Total number of users:
\[ n = 400 \]
Number of users who clicked the CTA button:
\[ x = 156 \]
Tasks
Compute the sample proportion of users who clicked the CTA button.
Compute confidence intervals for the population proportion at the following confidence levels:
90%
95%
99%
Visualize and compare the three confidence intervals to observe their differences.
Explain how the choice of confidence level affects decision-making in product experiments, particularly in the context of A/B testing.
3.2 Answer
The sample proportion represents the fraction of users who clicked the Call-To-Action (CTA) button among all users in the experiment. It is defined as:
\[ \hat{p} = \frac{x}{n} \]
where:
\(x\) is the number of users who clicked the CTA button, and
\(n\) is the total number of users in the experiment.
Substituting the given values:
\[ \hat{p} = \frac{156}{400} = 0.39 \]
Therefore, the sample proportion of users who clicked the CTA button is 0.39, which means that 39% of users in the sample clicked the CTA.
2. Compute the confidence intervals for:
90%
95%
99%
❖
To estimate the population proportion of users who clicked the Call-To-Action (CTA) button, confidence intervals for a population proportion are constructed. The general form of the confidence interval is given by:
\[ \hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \]
where:
\(\hat{p}\) is the sample proportion,
\(z_{\alpha/2}\) is the critical value from the standard normal distribution, and
\(n\) is the sample size.
Standard Error
The standard error of the sample proportion is calculated as:
\[ SE = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \]
Substituting the values:
\[ SE = \sqrt{\frac{0.39(1-0.39)}{400}} = \sqrt{0.00059475} \approx 0.0244 \]
❖ 90%
For a 90% confidence level, the critical value is \(z_{0.05} = 1.645\).
\[ CI_{90\%} = 0.39 \pm 1.645(0.0244) \]
\[ CI_{90\%} = (0.3498,\; 0.4302) \]
❖ 95%
For a 95% confidence level, the critical value is \(z_{0.025} = 1.96\).
\[ CI_{95\%} = 0.39 \pm 1.96(0.0244) \]
\[ CI_{95\%} = (0.3422,\; 0.4378) \]
❖ 99%
For a 99% confidence level, the critical value is \(z_{0.005} = 2.576\).
\[ CI_{99\%} = 0.39 \pm 2.576(0.0244) \]
\[ CI_{99\%} = (0.3271,\; 0.4529) \]
❖
In product experiments such as A/B testing, the confidence level directly affects how conservative the decision-making process is. A higher confidence level produces a wider confidence interval, which increases certainty but reduces precision. This means the team is more cautious when declaring an improvement in performance.
Conversely, a lower confidence level yields a narrower interval, allowing faster and more decisive conclusions, but with a higher risk of error. Therefore, product teams often use a 95% confidence level as a balance between reliability and actionable insight. As the confidence level increases, certainty increases, but the ability to detect small improvements decreases.
Conclusion
Based on the analysis, the estimated CTA click-through rate is 39%. As the confidence level increases from 90% to 99%, the confidence interval becomes wider, reflecting greater certainty but less precision. In A/B testing, choosing an appropriate confidence level is crucial to balance decision speed and reliability.
4 CASE STUDY 4
In performance analysis, estimating the average API latency is crucial for evaluating system reliability and user experience. Different statistical approaches can be used to construct confidence intervals depending on whether the population standard deviation is known or unknown.
In this problem, two data teams analyze API latency measurements under similar conditions. Team A applies a Z-based confidence interval because the population standard deviation is known, while Team B uses a t-based confidence interval since the standard deviation must be estimated from the sample.
This analysis aims to compare the confidence intervals obtained from both methods at different confidence levels (90%, 95%, and 99%) and to explain the differences in interval precision.
4.1 Question
Problem Statement
Precision Comparison (Z-Test vs t-Test): Two data teams measure API latency (in milliseconds) under different conditions.
Team A
Sample size: \(n = 36\)
Sample mean: \(\bar{x} = 210\)
Population standard deviation: \(\sigma = 24\) (known)
Team B
Sample size: \(n = 36\)
Sample mean: \(\bar{x} = 210\)
Sample standard deviation: \(s = 24\) (unknown population standard deviation / sample standard deviation)
Tasks
Identify the statistical test used by each team
Compute the confidence intervals for the mean at the 90%, 95%, and 99% confidence levels.
Create a visualization comparing all confidence intervals.
Explain why the interval widths differ, even with similar data
4.2 Answer
Team A uses a Z-test (Z-based confidence interval) because the population standard deviation (\(\sigma = 24\)) is known and the sample size is sufficiently large (\(n = 36\)).
Team B uses a t-test (t-based confidence interval) because the population standard deviation is unknown and must be estimated using the sample standard deviation (\(s = 24\)).
2. Compute the confidence intervals for:
90%
95%
99%
❖
The confidence interval for the population mean is calculated using the following general formulas.
Team A (Z-based Confidence Interval)
Because the population standard deviation is known, the confidence interval is given by
\[ \bar{x} \pm z_{\alpha/2} \left( \frac{\sigma}{\sqrt{n}} \right) \]
where
\(\bar{x}\) is the sample mean,
\(\sigma\) is the population standard deviation,
\(n\) is the sample size, and
\(z_{\alpha/2}\) is the critical value from the standard normal distribution.
Substituting the given values:
\[ 210 \pm z_{\alpha/2} \left( \frac{24}{\sqrt{36}} \right) = 210 \pm z_{\alpha/2} (4) \]
Using the corresponding critical values:
❖ 90%
\[ 210 \pm 1.645(4) = (203.42,\ 216.58) \]
❖ 95%
\[ 210 \pm 1.96(4) = (202.16,\ 217.84) \]
❖ 99%
\[ 210 \pm 2.576(4) = (199.70,\ 220.30) \]
❖
Team B (t-based Confidence Interval)
Because the population standard deviation is unknown, the confidence interval is computed using the t-distribution:
\[ \bar{x} \pm t_{\alpha/2,\,df} \left( \frac{s}{\sqrt{n}} \right) \]
where
\(s\) is the sample standard deviation and
\(df = n - 1\) is the degrees of freedom.
Substituting the given values:
\[ 210 \pm t_{\alpha/2,35} \left( \frac{24}{\sqrt{36}} \right) = 210 \pm t_{\alpha/2,35} (4) \]
Using the corresponding critical values:
❖ 90%
\[ 210 \pm 1.69(4) = (203.24,\ 216.76) \]
❖ 95%
\[ 210 \pm 2.03(4) = (201.88,\ 218.12) \]
❖ 99%
\[ 210 \pm 2.724(4) = (199.10,\ 220.90) \]
Even though both teams have the same sample mean, sample size, and variability, the confidence intervals differ because:
Team A (Z-test) assumes the population standard deviation is known, leading to smaller critical values.
Team B (t-test) must account for additional uncertainty in estimating the standard deviation from the sample.
As a result, t critical values are larger than z critical values, especially at higher confidence levels.
Therefore, t-based confidence intervals are always wider than z-based intervals when σ is unknown.
Conclusion
Although both teams observe identical data, the confidence intervals from Team B (t-test) are wider due to uncertainty in estimating the population standard deviation. As the confidence level increases, this difference becomes more pronounced, reflecting the conservative nature of the t-distribution.
5 CASE STUDY 5
In many digital-based businesses, particularly Software as a Service (SaaS) companies, user engagement is a critical performance indicator. One important metric is the proportion of users who actively utilize premium features, as this reflects both product value and customer adoption. To evaluate whether a company has met its minimum engagement target, statistical inference can be applied using confidence intervals. In this analysis, a one-sided confidence interval for a population proportion is constructed to estimate the minimum percentage of weekly active users who use a premium feature. The focus is placed on the lower bound of the confidence interval, as management is interested in ensuring that the true proportion meets or exceeds a predefined threshold.
The following sections present the identification of the appropriate statistical method, the calculation of one-sided confidence intervals at different confidence levels, a visualization of the results, and a final conclusion regarding whether the company’s target has been statistically satisfied.
5.1 Question
Problem Statement
One-Sided Confidence Interval: A Software as a Service (SaaS) company wants to ensure that at least 70% of weekly active users utilize a premium feature.
From an experiment, the following data were obtained:
Total number of users observed: \(n = 250\)
Number of users utilizing the premium feature: \(x = 185\)
Management is only interested in the lower bound of the estimate.
Tasks
Identify the type of confidence interval and the appropriate statistical test.
Compute the one-sided lower confidence interval at:
90% confidence level
95% confidence level
99% confidence level
Visualize the lower bounds for all confidence levels.
Determine whether the 70% target is statistically satisfied.
5.2 Answer
The parameter of interest in this problem is the population proportion \(p\) of weekly active users who utilize the premium feature.
Since the objective is to estimate the minimum value of the population proportion, a one-sided (lower) confidence interval for a proportion is appropriate.
Given the large sample size and the satisfaction of the normal approximation conditions, the confidence interval is constructed using the normal (Z) approximation method for a population proportion.
The sample proportion is defined as:
\[ \hat{p} = \frac{x}{n} \]
❖
Parameter: population proportion \(p\)
Confidence Interval: one-sided (lower) confidence interval for a population proportion
Method: normal approximation (Z-interval)
The normal approximation method is appropriate because the following conditions are satisfied:
\(n\hat{p} = 185 > 10\)
\(n(1 - \hat{p}) = 65 > 10\)
Lower One-Sided Confidence Interval Formula
\[ \text{Lower bound} = \hat{p} - z_{\alpha} \sqrt{ \frac{\hat{p}(1 - \hat{p})}{n} } \]- Compute the one-sided lower confidence interval at:
90% confidence level
95% confidence level
99% confidence level**
The sample proportion is first calculated as:
\[ \hat{p} = \frac{x}{n} = \frac{185}{250} = 0.74 \]
The standard error of the sample proportion is given by:
\[ SE = \sqrt{\frac{\hat{p}(1 - \hat{p})}{n}} = \sqrt{\frac{0.74(1 - 0.74)}{250}} = \sqrt{0.0007696} \approx 0.0277 \]
❖ 90%
For a 90% one-sided confidence interval, the critical value is
\(z_{\alpha} = 1.282\).
\[ \text{Lower bound}_{90\%} = \hat{p} - z_{\alpha} \times SE = 0.74 - 1.282(0.0277) = 0.704 \]
❖ 95%
For a 95% one-sided confidence interval, the critical value is
\(z_{\alpha} = 1.645\).
\[ \text{Lower bound}_{95\%} = 0.74 - 1.645(0.0277) = 0.694 \]
❖ 99%
For a 99% one-sided confidence interval, the critical value is
\(z_{\alpha} = 2.33\).
\[ \text{Lower bound}_{99\%} = 0.74 - 2.33(0.0277) = 0.675 \]
At the 90% confidence level, the lower bound of the one-sided confidence interval is 0.704, which is greater than the target value of 0.70. Therefore, the 70% target is statistically satisfied at the 90% confidence level.
However, at the 95% and 99% confidence levels, the lower bounds are 0.694 and 0.675, respectively, both of which are below 0.70. Hence, the 70% target is not statistically satisfied at these higher confidence levels.
❖ 90% CI: lower bound = 0.704 > 0.70 → Satisfied
❖ 95% CI: lower bound = 0.694 < 0.70 → not satisfied
❖ 99% CI: lower bound = 0.675 < 0.70 → not satisfied