Study Cases

Confidence Interval~ Week 13

Logo

1 Case Study 1

Confidence Interval for Mean, \(\sigma\) Known: An e-commerce platform wants to estimate the average number of daily transactions per user after launching a new feature. Based on large-scale historical data, the population standard deviation is known.

\[ \begin{eqnarray*} \sigma &=& 3.2 \quad \text{(population standard deviation)} \\ n &=& 100 \quad \text{(sample size)} \\ \bar{x} &=& 12.6 \quad \text{(sample mean)} \end{eqnarray*} \]

Tasks

  1. Identify the appropriate statistical test and justify your choice.
  2. Compute the Confidence Intervals for:
    • \(90\%\)
    • \(95\%\)
    • \(99\%\)
  3. Create a comparison visualization of the three confidence intervals.
  4. Interpret the results in a business analytics context.

2 Case Study 2

Confidence Interval for Mean, \(\sigma\) Unknown: A UX Research team analyzes task completion time (in minutes) for a new mobile application. The data are collected from 12 users:

\[ 8.4,\; 7.9,\; 9.1,\; 8.7,\; 8.2,\; 9.0,\; 7.8,\; 8.5,\; 8.9,\; 8.1,\; 8.6,\; 8.3 \]

Tasks:

  1. Identify the appropriate statistical test and explain why.
  2. Compute the Confidence Intervals for:
    • \(90\%\)
    • \(95\%\)
    • \(99\%\)
  3. Visualize the three intervals on a single plot.
  4. Explain how sample size and confidence level influence the interval width.

3 Case Study 3

Confidence Interval for a Proportion, A/B Testing: A data science team runs an A/B test on a new Call-To-Action (CTA) button design. The experiment yields:

\[ \begin{eqnarray*} n &=& 400 \quad \text{(total users)} \\ x &=& 156 \quad \text{(users who clicked the CTA)} \end{eqnarray*} \]

Tasks:

  1. Compute the sample proportion \(\hat{p}\).
  2. Compute Confidence Intervals for the proportion at:
    • \(90\%\)
    • \(95\%\)
    • \(99\%\)
  3. Visualize and compare the three intervals.
  4. Explain how confidence level affects decision-making in product experiments.

4 Case Study 4

Precision Comparison (Z-Test vs t-Test): Two data teams measure API latency (in milliseconds) under different conditions.

\[\begin{eqnarray*} \text{Team A:} \\ n &=& 36 \quad \text{(sample size)} \\ \bar{x} &=& 210 \quad \text{(sample mean)} \\ \sigma &=& 24 \quad \text{(known population standard deviation)} \\[6pt] \text{Team B:} \\ n &=& 36 \quad \text{(sample size)} \\ \bar{x} &=& 210 \quad \text{(sample mean)} \\ s &=& 24 \quad \text{(sample standard deviation)} \end{eqnarray*}\]

Tasks

  1. Identify the statistical test used by each team.
  2. Compute Confidence Intervals for 90%, 95%, and 99%.
  3. Create a visualization comparing all intervals.
  4. Explain why the interval widths differ, even with similar data.

5 Case Study 5

One-Sided Confidence Interval: A Software as a Service (SaaS) company wants to ensure that at least 70% of weekly active users utilize a premium feature.

From the experiment:

\[ \begin{eqnarray*} n &=& 250 \quad \text{(total users)} \\ x &=& 185 \quad \text{(active premium users)} \end{eqnarray*} \]

Management is only interested in the lower bound of the estimate.

Tasks:

  1. Identify the type of Confidence Interval and the appropriate test.
  2. Compute the one-sided lower Confidence Interval at:
    • \(90\%\)
    • \(95\%\)
    • \(99\%\)
  3. Visualize the lower bounds for all confidence levels.
  4. Determine whether the 70% target is statistically satisfied.
LS0tCnRpdGxlOiAiU3R1ZHkgQ2FzZXMiICAgICAgICAgICAgIyBNYWluIHRpdGxlIG9mIHRoZSBkb2N1bWVudApzdWJ0aXRsZTogIkNvbmZpZGVuY2UgSW50ZXJ2YWx+IFdlZWsgMTMiICAjIFN1YnRpdGxlIG9yIHRvcGljIGZvciB3ZWVrIDIKYXV0aG9yOiAiQmFrdGkgU2lyZWdhciwgTS5TYy4sIENEUy4iICAgICAgIyBSZXBsYWNlIHdpdGggeW91ciBmdWxsIG5hbWUKZGF0ZTogICJgciBmb3JtYXQoU3lzLkRhdGUoKSwgJyVCICVkLCAlWScpYCIgIyBBdXRvIGRpc3BsYXlzIHRoZSBjdXJyZW50IGRhdGUKb3V0cHV0OiAgICAgICAgICAgICAgICAgICAgICAgICAjIE91dHB1dCBzZWN0aW9uIGRlZmluZXMgdGhlIGZvcm1hdCBhbmQgbGF5b3V0IAogIHJtZGZvcm1hdHM6OnJlYWR0aGVkb3duOiAgICAgICMgaHR0cHM6Ly9naXRodWIuY29tL2p1YmEvcm1kZm9ybWF0cwogICAgc2VsZl9jb250YWluZWQ6IHRydWUgICAgICAgICMgRW1iZWRzIGFsbCByZXNvdXJjZXMgKENTUywgSlMsIGltYWdlcykgCiAgICB0aHVtYm5haWxzOiB0cnVlICAgICAgICAgICAgIyBEaXNwbGF5cyBpbWFnZSB0aHVtYm5haWxzIGluIHRoZSBkb2MKICAgIGxpZ2h0Ym94OiB0cnVlICAgICAgICAgICAgICAjIEVuYWJsZXMgY2xpY2sgdG8gZW5sYXJnZSBpbWFnZXMKICAgIGdhbGxlcnk6IHRydWUgICAgICAgICAgICAgICAjIEdyb3VwcyBpbWFnZXMgaW50byBhbiBpbnRlcmFjdGl2ZSBnYWxsZXJ5CiAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUgICAgICAgIyBBdXRvbWF0aWNhbGx5IG51bWJlcnMgYWxsIHNlY3Rpb25zCiAgICBsaWJfZGlyOiBsaWJzICAgICAgICAgICAgICAgIyBEaXJlY3Rvcnkgd2hlcmUgSmF2YVNjcmlwdC9DU1MgbGlicmFyaWVzCiAgICBkZl9wcmludDogInBhZ2VkIiAgICAgICAgICAgIyBEaXNwbGF5cyBkYXRhIGZyYW1lcyBhcyBpbnRlcmFjdGl2ZSBwYWdlZCAKICAgIGNvZGVfZm9sZGluZzogInNob3ciICAgICAgICAjIEFsbG93cyBmb2xkaW5nL3VuZm9sZGluZyBSIGNvZGUgYmxvY2tzIAogICAgY29kZV9kb3dubG9hZDogeWVzICAgICAgICAgICMgQWRkcyBhIGJ1dHRvbiB0byBkb3dubG9hZCBhbGwgUiBjb2RlCi0tLQoKCjxpbWcgaWQ9IkZvdG8iIHNyYz0iaHR0cHM6Ly9naXRodWIuY29tL2RzY2llbmNlbGFicy9pbWFnZXMvYmxvYi9tYXN0ZXIvTG9nb19Ec2NpZW5jZWxhYnNfdjEucG5nP3Jhdz10cnVlIiBhbHQ9IkxvZ28iIHN0eWxlPSJ3aWR0aDoyMDBweDsgZGlzcGxheTogYmxvY2s7IG1hcmdpbjogYXV0bzsiPgoKLS0tCgojIyBDYXNlIFN0dWR5IDEKCioqQ29uZmlkZW5jZSBJbnRlcnZhbCBmb3IgTWVhbiwgJFxzaWdtYSQgS25vd246KiogQW4gKiplLWNvbW1lcmNlIHBsYXRmb3JtKiogd2FudHMgdG8gZXN0aW1hdGUgdGhlICoqYXZlcmFnZSBudW1iZXIgb2YgZGFpbHkgdHJhbnNhY3Rpb25zIHBlciB1c2VyKiogYWZ0ZXIgbGF1bmNoaW5nIGEgbmV3IGZlYXR1cmUuIEJhc2VkIG9uIGxhcmdlLXNjYWxlIGhpc3RvcmljYWwgZGF0YSwgdGhlICoqcG9wdWxhdGlvbiBzdGFuZGFyZCBkZXZpYXRpb24qKiBpcyBrbm93bi4KCiQkClxiZWdpbntlcW5hcnJheSp9ClxzaWdtYSAmPSYgMy4yIFxxdWFkIFx0ZXh0eyhwb3B1bGF0aW9uIHN0YW5kYXJkIGRldmlhdGlvbil9IFxcCm4gJj0mIDEwMCBccXVhZCBcdGV4dHsoc2FtcGxlIHNpemUpfSBcXApcYmFye3h9ICY9JiAxMi42IFxxdWFkIFx0ZXh0eyhzYW1wbGUgbWVhbil9ClxlbmR7ZXFuYXJyYXkqfQokJAoKKipUYXNrcyoqCgoxLiBJZGVudGlmeSB0aGUgKiphcHByb3ByaWF0ZSBzdGF0aXN0aWNhbCB0ZXN0KiogYW5kIGp1c3RpZnkgeW91ciBjaG9pY2UuCjIuIENvbXB1dGUgdGhlIENvbmZpZGVuY2UgSW50ZXJ2YWxzIGZvcjoKICAgLSAkOTBcJSQKICAgLSAkOTVcJSQKICAgLSAkOTlcJSQKMy4gQ3JlYXRlIGEgKipjb21wYXJpc29uIHZpc3VhbGl6YXRpb24qKiBvZiB0aGUgdGhyZWUgY29uZmlkZW5jZSBpbnRlcnZhbHMuCjQuIEludGVycHJldCB0aGUgcmVzdWx0cyBpbiBhIGJ1c2luZXNzIGFuYWx5dGljcyBjb250ZXh0LgoKLS0tCgojIyBDYXNlIFN0dWR5IDIgCgoqKkNvbmZpZGVuY2UgSW50ZXJ2YWwgZm9yIE1lYW4sICRcc2lnbWEkIFVua25vd246KiogQSAqKlVYIFJlc2VhcmNoIHRlYW0qKiBhbmFseXplcyAqKnRhc2sgY29tcGxldGlvbiB0aW1lIChpbiBtaW51dGVzKSoqIGZvciBhIG5ldyBtb2JpbGUgYXBwbGljYXRpb24uIFRoZSBkYXRhIGFyZSBjb2xsZWN0ZWQgZnJvbSAqKjEyIHVzZXJzKio6CgokJAo4LjQsXDsgNy45LFw7IDkuMSxcOyA4LjcsXDsgOC4yLFw7IDkuMCxcOwo3LjgsXDsgOC41LFw7IDguOSxcOyA4LjEsXDsgOC42LFw7IDguMwokJAoKKipUYXNrczoqKgoKMS4gSWRlbnRpZnkgdGhlICoqYXBwcm9wcmlhdGUgc3RhdGlzdGljYWwgdGVzdCoqIGFuZCBleHBsYWluIHdoeS4KMi4gQ29tcHV0ZSB0aGUgQ29uZmlkZW5jZSBJbnRlcnZhbHMgZm9yOgogICAtICQ5MFwlJAogICAtICQ5NVwlJAogICAtICQ5OVwlJAozLiBWaXN1YWxpemUgdGhlIHRocmVlIGludGVydmFscyBvbiBhIHNpbmdsZSBwbG90Lgo0LiBFeHBsYWluIGhvdyAqKnNhbXBsZSBzaXplIGFuZCBjb25maWRlbmNlIGxldmVsKiogaW5mbHVlbmNlIHRoZSBpbnRlcnZhbCB3aWR0aC4KCi0tLQoKIyMgQ2FzZSBTdHVkeSAzIAoKKipDb25maWRlbmNlIEludGVydmFsIGZvciBhIFByb3BvcnRpb24sIEEvQiBUZXN0aW5nOioqIEEgZGF0YSBzY2llbmNlIHRlYW0gcnVucyBhbiAqKkEvQiB0ZXN0Kiogb24gYSBuZXcgKkNhbGwtVG8tQWN0aW9uIChDVEEpKiBidXR0b24gZGVzaWduLiBUaGUgZXhwZXJpbWVudCB5aWVsZHM6CgokJApcYmVnaW57ZXFuYXJyYXkqfQpuICY9JiA0MDAgXHF1YWQgXHRleHR7KHRvdGFsIHVzZXJzKX0gXFwKeCAmPSYgMTU2IFxxdWFkIFx0ZXh0eyh1c2VycyB3aG8gY2xpY2tlZCB0aGUgQ1RBKX0KXGVuZHtlcW5hcnJheSp9CiQkCgoqKlRhc2tzOioqCgoxLiBDb21wdXRlIHRoZSAqKnNhbXBsZSBwcm9wb3J0aW9uKiogJFxoYXR7cH0kLgoyLiBDb21wdXRlIENvbmZpZGVuY2UgSW50ZXJ2YWxzIGZvciB0aGUgcHJvcG9ydGlvbiBhdDoKICAgLSAkOTBcJSQKICAgLSAkOTVcJSQKICAgLSAkOTlcJSQKMy4gVmlzdWFsaXplIGFuZCBjb21wYXJlIHRoZSB0aHJlZSBpbnRlcnZhbHMuCjQuIEV4cGxhaW4gaG93IGNvbmZpZGVuY2UgbGV2ZWwgYWZmZWN0cyBkZWNpc2lvbi1tYWtpbmcgaW4gcHJvZHVjdCBleHBlcmltZW50cy4KCiMjIENhc2UgU3R1ZHkgNCAKCioqUHJlY2lzaW9uIENvbXBhcmlzb24gKFotVGVzdCB2cyB0LVRlc3QpOioqIFR3byBkYXRhIHRlYW1zIG1lYXN1cmUgKipBUEkgbGF0ZW5jeSAoaW4gbWlsbGlzZWNvbmRzKSoqIHVuZGVyIGRpZmZlcmVudCBjb25kaXRpb25zLgoKXGJlZ2lue2VxbmFycmF5Kn0KXHRleHR7VGVhbSBBOn0gXFwKbiAmPSYgMzYgXHF1YWQgXHRleHR7KHNhbXBsZSBzaXplKX0gXFwKXGJhcnt4fSAmPSYgMjEwIFxxdWFkIFx0ZXh0eyhzYW1wbGUgbWVhbil9IFxcClxzaWdtYSAmPSYgMjQgXHF1YWQgXHRleHR7KGtub3duIHBvcHVsYXRpb24gc3RhbmRhcmQgZGV2aWF0aW9uKX0gXFxbNnB0XQoKXHRleHR7VGVhbSBCOn0gXFwKbiAmPSYgMzYgXHF1YWQgXHRleHR7KHNhbXBsZSBzaXplKX0gXFwKXGJhcnt4fSAmPSYgMjEwIFxxdWFkIFx0ZXh0eyhzYW1wbGUgbWVhbil9IFxcCnMgJj0mIDI0IFxxdWFkIFx0ZXh0eyhzYW1wbGUgc3RhbmRhcmQgZGV2aWF0aW9uKX0KXGVuZHtlcW5hcnJheSp9CgoqKlRhc2tzKioKCjEuIElkZW50aWZ5IHRoZSBzdGF0aXN0aWNhbCB0ZXN0IHVzZWQgYnkgZWFjaCB0ZWFtLgoyLiBDb21wdXRlIENvbmZpZGVuY2UgSW50ZXJ2YWxzIGZvciAqKjkwJSwgOTUlLCBhbmQgOTklKiouCjMuIENyZWF0ZSBhIHZpc3VhbGl6YXRpb24gY29tcGFyaW5nIGFsbCBpbnRlcnZhbHMuCjQuIEV4cGxhaW4gd2h5IHRoZSAqKmludGVydmFsIHdpZHRocyBkaWZmZXIqKiwgZXZlbiB3aXRoIHNpbWlsYXIgZGF0YS4KCgojIyBDYXNlIFN0dWR5IDUKCioqT25lLVNpZGVkIENvbmZpZGVuY2UgSW50ZXJ2YWw6KiogQSAqKlNvZnR3YXJlIGFzIGEgU2VydmljZSAoU2FhUykqKiBjb21wYW55IHdhbnRzIHRvIGVuc3VyZSB0aGF0ICoqYXQgbGVhc3QgNzAlIG9mIHdlZWtseSBhY3RpdmUgdXNlcnMqKiB1dGlsaXplIGEgcHJlbWl1bSBmZWF0dXJlLgoKRnJvbSB0aGUgZXhwZXJpbWVudDoKCiQkClxiZWdpbntlcW5hcnJheSp9Cm4gJj0mIDI1MCBccXVhZCBcdGV4dHsodG90YWwgdXNlcnMpfSBcXAp4ICY9JiAxODUgXHF1YWQgXHRleHR7KGFjdGl2ZSBwcmVtaXVtIHVzZXJzKX0KXGVuZHtlcW5hcnJheSp9CiQkCgpNYW5hZ2VtZW50IGlzIG9ubHkgaW50ZXJlc3RlZCBpbiB0aGUgKipsb3dlciBib3VuZCoqIG9mIHRoZSBlc3RpbWF0ZS4KCioqVGFza3M6KioKCjEuIElkZW50aWZ5IHRoZSAqKnR5cGUgb2YgQ29uZmlkZW5jZSBJbnRlcnZhbCoqIGFuZCB0aGUgYXBwcm9wcmlhdGUgdGVzdC4KMi4gQ29tcHV0ZSB0aGUgKipvbmUtc2lkZWQgbG93ZXIgQ29uZmlkZW5jZSBJbnRlcnZhbCoqIGF0OgogICAtICQ5MFwlJAogICAtICQ5NVwlJAogICAtICQ5OVwlJAozLiBWaXN1YWxpemUgdGhlIGxvd2VyIGJvdW5kcyBmb3IgYWxsIGNvbmZpZGVuY2UgbGV2ZWxzLgo0LiBEZXRlcm1pbmUgd2hldGhlciB0aGUgKio3MCUgdGFyZ2V0KiogaXMgc3RhdGlzdGljYWxseSBzYXRpc2ZpZWQuCgoK