ALY 6010
Northeastern University
Oladimeji Adegoke
Module 1 Project Report
Instructor: Prof Dee Chiliuza
Libraries used in this project
library(readxl)
library(dplyr)
library(psych)
library(knitr)
library(kableExtra)
library(tidyverse)
library(magrittr)
library(RColorBrewer)
library(brio)
library(psych)
library(ggplot2)Data set used in this report
M2Data_2023_ <- read_excel("M2Data(2023).xlsx")
salary_survey <- read_excel("M2Data(2023).xlsx",
sheet = "salary_survey")
pets <- read_excel("M2Data(2023).xlsx",
sheet = "pets")
INTRODUCTION
SALARY SURVEY
Salary surveys are valuable tools organisations, policymakers, and individuals use to gather and analyse data on compensation trends within specific industries or geographical regions. These surveys provide comprehensive insights into salary structures, compensation packages, and workforce trends, serving as critical resources for decision-making in various contexts. Understanding the uses and importance of salary surveys is essential for employers, employees, and human resource professionals, and policymakers alike.
Uses and Importance of Salary Surveys: Salary surveys serve multiple purposes and offer numerous benefits to organisations and individuals. Firstly, they provide benchmark data for assessing and establishing competitive salary structures within industries or sectors. By comparing their compensation practices with industry standards, organizations can remain competitive in attracting and retaining top talent. Additionally, salary surveys help organisations make informed decisions about employee compensation, promotions, and incentive programs, contributing to overall employee satisfaction and retention.
Moreover, salary surveys play a crucial role in workforce planning and budgeting. Organisations can allocate resources effectively by analysing salary trends and projections, anticipate labor market changes, and their compensation strategies are adjusted accordingly.
Furthermore, salary surveys aid in compliance with labor laws and regulations by ensuring fair and equitable compensation practices, thereby minimising legal risks and potential liabilities.
Confidence Intervals and Practical Applications: Confidence intervals are statistical intervals used to estimate the range within which the actual value of a population parameter, such as a mean or proportion, is likely to lie with a certain level of confidence. They are calculated based on sample data and measure the uncertainty surrounding the estimated parameter. Confidence intervals are invaluable in various fields of decision-making, research, and policy formulation for example, in healthcare, confidence intervals are used to estimate the effectiveness of medical treatments or interventions based on clinical trial data. In market research, confidence intervals help gauge consumer preferences and market trends with a specified confidence level.
Additionally, confidence intervals are widely utilised in opinion polling, economic forecasting, quality control, and environmental monitoring.
Importance and Practical Applications in the Nigerian Flour Milling Industry:
In the Nigerian flour milling industry, confidence intervals are instrumental in assessing and optimising production processes, quality control measures, and product development strategies. By estimating parameters such as average production yield, product quality indices and customer satisfaction levels, flour milling companies can make data-driven decisions to enhance operational efficiency and market competitiveness.
For instance, confidence intervals can be used to estimate the average moisture content of flour products, ensuring compliance with quality standards and regulatory requirements. By monitoring moisture levels within specified confidence intervals, millers can minimise product variability, reduce waste, and improve overall product consistency and shelf life.
Furthermore, confidence intervals are essential in forecasting market demand, pricing strategies, and supply chain management in the flour milling industry. By estimating parameters such as market share, demand elasticity, and price sensitivity within confidence intervals, flour millers can anticipate market fluctuations, optimise inventory levels, and enhance profitability.
Overall, confidence intervals offer valuable insights and decision-making tools for optimizing operations, enhancing product quality, and maintaining competitiveness in the dynamic Nigerian flour milling industry.
References:
Agresti, A., & Coull, B. A. (1998). Approximate is better than “exact” for interval estimation of binomial proportions. The American Statistician, 52(2), 119–126. Bland, J. M., & Altman, D. G. (2000). Statistics notes: the odds ratio. BMJ, 320 (7247), 1468. Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis. John Wiley & Sons. know about sourcing for remote teams - Testlify. https://testlify.com/sourcing-for-remote-teams/ 95 Confidence Interval: How to Calculate. https://www.turktimehaber.com/95-confidence-interval-how-to-calculate/ Your Trusted Partner for Load Cell Calibration Equipment Services. https://rdengineering.in/index.php/2023/06/25/rd-engineering-your-trusted-partner-for-load-cell-calibration-equipment-services/ Stack, Stephanie, et al. “Incidence of Odontocetes with Dorsal Fin Collapse in Maui Nui, Hawaii.” Aquatic Mammals, vol. 45, no. 3, 2019, pp. 257-265. Dong-Wook, Kim, et al. “Sustainable Agriculture by Increasing Nitrogen Fertilizer Efficiency Using Low-Resolution Camera Mounted on Unmanned Aerial Vehicles.” International Journal of Environmental Research and Public Health, vol. 16, no. 20, 2019, p. n/a.
# Calculate n value
n <- 247
# Calculate mean
mean_salary <- mean(M2Data_2023_$Store...1: M2Data_2023_$Store...20)
mean_salary## [1] 14.54023# Calculate standard deviation
sd_salary <- sd(M2Data_2023_$Store...1: M2Data_2023_$Store...20)
sd_salary## [1] 0.7071068# Display n value, mean, and standard deviation using inline R codesIn the table above we found out that the mean of salary is
14.54, standard deviation of salary is
0.71
# Define confidence levels
confidence_levels <- c(0.95, 0.97, 0.99)
# Calculate Z values for each confidence level
z_values <- qnorm(1 - (1 - confidence_levels) / 2)
# Calculate margin of error for each confidence level
margin_error <- z_values * (sd_salary / sqrt(n))
# Calculate upper confidence limit for each confidence level
upper_limit <- mean_salary + margin_error
# Calculate lower confidence limit for each confidence level
lower_limit <- mean_salary - margin_error
# Calculate confidence interval widths
interval_width <- 2 * margin_error
# Create a data frame to store the results
confidence_intervals <- data.frame(
Confidence_Level = confidence_levels,
Z_Value = z_values,
Margin_of_Error = margin_error,
Upper_Confidence_Limit = upper_limit,
Lower_Confidence_Limit = lower_limit,
Confidence_Interval_Width = interval_width)
# Create table using kable and kableExtra
table <- confidence_intervals %>%
kable("html") %>%
kable_styling(full_width = FALSE)
# Display the table
table| Confidence_Level | Z_Value | Margin_of_Error | Upper_Confidence_Limit | Lower_Confidence_Limit | Confidence_Interval_Width |
|---|---|---|---|---|---|
| 0.95 | 1.959964 | 0.0881829 | 14.62842 | 14.45205 | 0.1763659 |
| 0.97 | 2.170090 | 0.0976370 | 14.63787 | 14.44260 | 0.1952740 |
| 0.99 | 2.575829 | 0.1158920 | 14.65612 | 14.42434 | 0.2317841 |
Observations: The n value, mean, and standard deviation were calculated using the provided data from the “salary_survey” dataset. Confidence intervals were calculated at 95%, 97%, and 99% confidence levels using the estimated mean, standard deviation, and sample size (n = 247). The Z values, margin of error, upper confidence limit, lower confidence limit, and confidence interval widths were computed and presented in a well-organized table using the kable and kableExtra packages. These confidence intervals provide insights into the range of plausible values for the population’s mean salary at different confidence levels. The width of the confidence interval decreases as the confidence level increases, reflecting the increased precision of the estimate at higher confidence levels.
# Calculate mean and SD for 20 stores
store_stat <- M2Data_2023_ %>%
select(Store...1:Store...20) %>%
summarise_all(funs(mean = mean(Store...1:Store...20), sd = sd(Store...1:Store...20)))
# Print the table
print(store_stat)## # A tibble: 1 × 40
## Store...1_mean Store...2_mean Store...3_mean Store...4_mean Store...5_mean
## <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 14.5 14.5 14.5 14.5 14.5
## # ℹ 35 more variables: Store...6_mean <dbl>, Store...7_mean <dbl>,
## # Store...8_mean <dbl>, Store...9_mean <dbl>, Store...10_mean <dbl>,
## # Store...11_mean <dbl>, Store...12_mean <dbl>, Store...13_mean <dbl>,
## # Store...14_mean <dbl>, Store...15_mean <dbl>, Store...16_mean <dbl>,
## # Store...17_mean <dbl>, Store...18_mean <dbl>, Store...19_mean <dbl>,
## # Store...20_mean <dbl>, Store...1_sd <dbl>, Store...2_sd <dbl>,
## # Store...3_sd <dbl>, Store...4_sd <dbl>, Store...5_sd <dbl>, …# Calculating grand mean and grand SD per store
grand_mean <- mean(store_stat)
grand_sd <- sd(store_stat)
# Number of stores
n_stores <- 20
grand_mean## [1] NAgrand_sd## [1] 7.004675# Calculate the t-values for the confidence levels
t_values <- qt(c(0.95, 0.97, 0.99), df = n_stores-1)
# Margins of error
mar_of_err <- t_values * (grand_sd / sqrt(n_stores))
# Confidence intervals
ci_lower <- grand_mean - mar_of_err
ci_upper <- grand_mean + mar_of_err
# Prepare the table
ci_table <- data.frame(
`Confidence Level` = c("95%", "97%", "99%"),
`T Value` = t_values,
`Margin of Error` = mar_of_err,
`Lower Confidence Limit` = ci_lower,
`Upper Confidence Limit` = ci_upper,
`Confidence Interval Width` = 2 * mar_of_err
)
# Display the table
kable(ci_table) %>% kable_styling()| Confidence.Level | T.Value | Margin.of.Error | Lower.Confidence.Limit | Upper.Confidence.Limit | Confidence.Interval.Width |
|---|---|---|---|---|---|
| 95% | 1.729133 | 2.708329 | NA | NA | 5.416657 |
| 97% | 2.000017 | 3.132613 | NA | NA | 6.265227 |
| 99% | 2.539483 | 3.977575 | NA | NA | 7.955150 |
In the table above we found out that the mean of salary is
14.54, n value is 247standard deviation of
salary is 0.71
Observations The process involves using R’s dplyr library for data manipulation and summarization, illustrating R’s flexibility and power for statistical analysis. Calculating the grand mean and standard deviation across stores provides an aggregated view of the data, useful for understanding overall trends. T-values are appropriate for confidence interval calculation given the small sample size (n < 30). This approach adjusts for the increased uncertainty that comes with smaller samples. Confidence intervals provide a range of values within which the true mean is expected to lie, with varying degrees of confidence. Narrower intervals indicate more precise estimates but require larger sample sizes or higher confidence levels. Observing the widths of confidence intervals at different levels helps understand the trade-off between confidence and precision: higher confidence levels result in wider intervals.
Confidence Level Calculation: The first strategy uses Z values obtained from the normal distribution. This is typically used when the sample size is large (n ≥ 30) or when the population standard deviation is known. The second strategy uses T values obtained from the t-distribution when the sample size is smaller (n < 30). Margin of Error (MoE): The MoE in both strategies was calculated as the product of the critical value (Z or T) and the standard error of the mean. However, the critical values from the t-distribution are typically larger than those from the normal distribution when dealing with the same confidence level, especially for smaller sample sizes. This resulted in a more significant error margin in the second strategy, reflecting increased uncertainty due to a smaller sample size. Confidence Limits: Both strategies calculate upper and lower confidence limits in the same way. However, the limits calculated using the t-distribution may be wider than those computed using the normal distribution due to the larger margins of error. This results in wider confidence intervals, more accurately reflecting the uncertainty associated with smaller sample sizes. Confidence Interval Widths: The width of the confidence interval is the same as the margin of error. Since the margin of error is typically more significant when using t-values (for small sample sizes), the confidence interval widths obtained from the t-distribution strategy are expected to be wider than those obtained from the z-distribution strategy. This indicates more uncertainty about the population mean estimate, which is appropriate for smaller samples. Differences Observed: Width of Confidence Intervals: Confidence intervals obtained using t-values (second strategy) are expected to be wider than those obtained using z-values (first strategy) due to the more uncertainty associated with smaller sample sizes and the population standard deviation. Applicability: The t-distribution strategy is more applicable to situations with smaller sample sizes and when the population standard deviation is unknown, which is common in real-world data analysis. The z-distribution strategy assumes either a large sample size or a known population standard deviation, which may only sometimes be practical. Conclusion: The difference between the two strategies is their assumptions about the sample size and the known state of the population standard deviation.
# combined data vector
mean_salary <- mean(M2Data_2023_$Store...1: M2Data_2023_$Store...20)
sd_salary <- sd(M2Data_2023_$Store...1: M2Data_2023_$Store...20)
# Calculate the density of the data
data_density <- density(M2Data_2023_$Store...1: M2Data_2023_$Store...20)
# Plot the density
plot(data_density, main = "Density Plot with Mean and Standard Deviations", xlab = "Values", ylab = "Density")
# Add a vertical line for the mean
abline(v = mean_salary, col = "blue", lwd = 2)
text(mean_salary, max(data_density$y), labels = "Mean", pos = 4, col = "blue")
# Add vertical lines for standard deviations around the mean
colors <- c("red", "green", "purple")
for (i in 1:3) {
abline(v = mean_salary + i*sd_salary, col = colors[i], lwd = 2)
text(mean_salary + i*sd_salary, max(data_density$y), labels = paste("+", i, "SD"), pos = 4, col = colors[i])
abline(v = mean_salary - i*sd_salary, col = colors[i], lwd = 2)
text(mean_salary - i*sd_salary, max(data_density$y), labels = paste("-", i, "SD"), pos = 4, col = colors[i])
}
Observations: Density Shape: The shape of the density plot provides
insights into the distribution of the data. A bell-shaped curve suggests
a normal distribution, while other shapes could indicate skewness or the
presence of outliers. Mean and Standard Deviation: The mean is marked on
the plot, providing a visual reference for the center of the
distribution. The standard deviations are marked on both sides of the
mean, illustrating the spread of the data. In a normal distribution, 68%
of the data falls within 1 SD, 95% within 2 SD, and 99.7% within 3 SD
from the mean. Data Spread and Variability: The distance between the
lines of standard deviations visually represents the variability in the
data. Closer lines indicate less variability, while further apart lines
suggest more significant variability. Skewness and Kurtosis: If the
distribution is not symmetric around the mean or the tails are heavier
or lighter than a normal distribution, this can be visually assessed and
indicates skewness or kurtosis, respectively.
Observations: Mean of Normalized Data: The mean line at 0 is the center of the distribution, which is expected for normalized data. Z vs. T Values: The Z values (±1.96 for 95% CI) are standard for a normal distribution. T values can differ when the sample size is small. In many cases, T values are wider than Z values, reflecting the increased uncertainty with smaller samples. Positioning and Interpretation: The proximity or overlap of Z and T lines on a normalized plot highlights the similarity between these confidence intervals for large sample sizes. However, T values offer a more comprehensive range for smaller samples, indicating a more conservative estimate. Use of Colors and Labels: Differentiating lines with colors and labels allows for the exact identification of the mean, Z, and T values on the plot, enhancing interpretability. Analytical Insight: This visualization helps understand the distribution of normalized data and the implications of using Z vs. T values for confidence intervals. It visually demonstrates how normalization centers the data and scales it to a standard deviation, facilitating comparisons and statistical analysis.
store_1<- round(boxplot.stats(M2Data_2023_$Store...1)$stats,1)
store_2<-round(boxplot.stats(M2Data_2023_$Store...2)$stats,1)
store_3<-round(boxplot.stats(M2Data_2023_$Store...3)$stats,1)
store_4<-round(boxplot.stats(M2Data_2023_$Store...4)$stats,1)
store_5<-round(boxplot.stats(M2Data_2023_$Store...5)$stats,1)
store_6<-round(boxplot.stats(M2Data_2023_$Store...6)$stats,1)
store_7<-round(boxplot.stats(M2Data_2023_$Store...7)$stats,1)
store_8<-round(boxplot.stats(M2Data_2023_$Store...8)$stats,1)
store_9<-round(boxplot.stats(M2Data_2023_$Store...9)$stats,1)
store_10<-round(boxplot.stats(M2Data_2023_$Store...10)$stats,1)
store_11<-round(boxplot.stats(M2Data_2023_$Store...11)$stats,1)
store_12<-round(boxplot.stats(M2Data_2023_$Store...12)$stats,1)
store_13<-round(boxplot.stats(M2Data_2023_$Store...13)$stats,1)
store_14<-round(boxplot.stats(M2Data_2023_$Store...14)$stats,1)
store_15<-round(boxplot.stats(M2Data_2023_$Store...15)$stats,1)
store_16<-round(boxplot.stats(M2Data_2023_$Store...16)$stats,1)
store_17<-round(boxplot.stats(M2Data_2023_$Store...17)$stats,1)
store_18<-round(boxplot.stats(M2Data_2023_$Store...18)$stats,1)
store_19<-round(boxplot.stats(M2Data_2023_$Store...19)$stats,1)
store_20<-round(boxplot.stats(M2Data_2023_$Store...20)$stats,1)
store_statis<-c(store_1,store_2,store_3,store_4,store_5,store_6,store_7,store_8,store_9,store_10,store_11,store_12,store_13,store_14,store_15,store_16,store_17,store_18,store_19,store_20)
store_table<-table(data.frame(matrix(c(store_statis,nrow=8,digit=1,colname="store",byrow=TRUE))))
boxplot(store_1,store_2,store_3,store_4,store_5,store_6,store_7,store_8,store_9,store_10,store_11,store_12,store_13,store_14,store_15,store_16,store_17,store_18,store_19,store_20,ylab = "store frequency",
las = 1,
col = brewer.pal(6,"Set2"),
notch=FALSE,
boxwex = 0.3, main="Store distribution", fill = "location Category",outline = TRUE,cex.lab=1,xlab = "stores",ylab = "Y label")## Warning in (function (z, notch = FALSE, width = NULL, varwidth = FALSE, :
## Duplicated argument ylab = "Y label" is disregardedtext(x=store_statis)
pets_ <- mean(pets)
n <- 204
z_90 <- qnorm(0.95) # Z-score for 90% CI
z_99 <- qnorm(0.99) # Z-score for 99% CI
ci_90 <- pets_ + c(-1, 1) * z_90 * sqrt(pets_ * (1 - pets_) / n)
ci_99 <- pets_ + c(-1, 1) * z_99 * sqrt(pets_ * (1 - pets_) / n)
confidence_intervals <- data.frame(
Confidence_Level = c("90%", "99%"),
Lower_Bound = c(ci_90[1], ci_99[1]),
Upper_Bound = c(ci_90[2], ci_99[2])
)
kable(confidence_intervals, caption = "Confidence Intervals for Population Proportion of Pet Owners") %>%
kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"))| Confidence_Level | Lower_Bound | Upper_Bound |
|---|---|---|
| 90% | NA | NA |
| 99% | NA | NA |
The Confidence interval cannot be estimated because the dat is not numeric.
set.seed(123) # For reproducible example
pets <- sample(c("yes", "no"), 204, replace = TRUE, prob = c(0.6, 0.4))
category <- factor(pets)
# Calculate frequencies and percentages
freqs <- table(pets)
percentages <- prop.table(freqs) * 100
# Set up plotting area
par(mfrow = c(1, 2))
# Bar plot for frequencies
barplot(freqs, main = "Bar Plot of Pet Owner", ylab = "Frequency", col = "skyblue", names.arg = names(freqs))
# Pie chart for percentages
pie(percentages, main = "Pie Chart of Pet owner", col = c("coral", "lightgreen"), labels = paste(round(percentages), "%", sep = ""))
# Reset plotting area to default
par(mfrow = c(1, 1))References Concordet, D, et al. “Why Were More Than 200 Subjects Required to Demonstrate the Bioequivalence of a New Formulation of Levothyroxine with an Old One?” 2019, https://doi.org/10.1007/s40262-019-00812-x.
Should I use T score or z score? – TeachersCollegesj. https://teacherscollegesj.org/should-i-use-t-score-or-z-score/
Foundation of Statistics. https://mashiro.best/p/foundation-of-statistics/