library(readr)
library(dplyr)
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(ggplot2)
mydata <- read_delim("C:\\Users\\abhir\\OneDrive\\Documents\\luddy\\H510-Stats,R\\adult.csv", delim = ",")
## Rows: 32561 Columns: 15
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (9): Work-Class, Education, Marital Status, Occupation, Relationship, Ra...
## dbl (6): Age, Fnl_Wgt, Education_Level, Capital_Gain, Capital_Loss, hrs_week
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
mydata
## # A tibble: 32,561 × 15
##      Age `Work-Class`        Fnl_Wgt Education  Education_Level `Marital Status`
##    <dbl> <chr>                 <dbl> <chr>                <dbl> <chr>           
##  1    39 " State-gov"          77516 " Bachelo…              13 " Never-married"
##  2    50 " Self-emp-not-inc"   83311 " Bachelo…              13 " Married-civ-s…
##  3    38 " Private"           215646 " HS-grad"               9 " Divorced"     
##  4    53 " Private"           234721 " 11th"                  7 " Married-civ-s…
##  5    28 " Private"           338409 " Bachelo…              13 " Married-civ-s…
##  6    37 " Private"           284582 " Masters"              14 " Married-civ-s…
##  7    49 " Private"           160187 " 9th"                   5 " Married-spous…
##  8    52 " Self-emp-not-inc"  209642 " HS-grad"               9 " Married-civ-s…
##  9    31 " Private"            45781 " Masters"              14 " Never-married"
## 10    42 " Private"           159449 " Bachelo…              13 " Married-civ-s…
## # ℹ 32,551 more rows
## # ℹ 9 more variables: Occupation <chr>, Relationship <chr>, Race <chr>,
## #   Sex <chr>, Capital_Gain <dbl>, Capital_Loss <dbl>, hrs_week <dbl>,
## #   Country <chr>, Annual_pay <chr>

#1)Response Variable I am considering hrs_week as the response variable as other factors are dependent on it.

#2)Selection of Categorical column. I have selected Occupation as my categorical column, As it influences hrs_week .

ANOVA Test and Results: Null Hypothesis (H0): There is no significant difference in hours per week among the different Occupations.

Alternative Hypothesis (H1): There is a significant difference in hours per week among the different Occupations.

model <- aov(hrs_week ~ Occupation, data = mydata)
anova_result <- anova(model)
print(anova_result)
## Analysis of Variance Table
## 
## Response: hrs_week
##               Df  Sum Sq Mean Sq F value    Pr(>F)    
## Occupation    14  481500   34393  249.71 < 2.2e-16 ***
## Residuals  32546 4482565     138                      
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

##Response: This indicates the dependent variable being analyzed, which in this case is “hrs_week” (hours per week).

Occupation: This is the independent variable or factor being studied. It has 14 degrees of freedom (Df), indicating that there are 15 levels or categories within the variable (including the reference category).

Sum Sq (Sum of Squares): This represents the total variability in the dependent variable, “hrs_week”, that is attributed to the independent variable, “Occupation”. It is partitioned into two parts: the variability explained by the model (Occupation) and the unexplained variability (Residuals).

Mean Sq (Mean Square): This is the average variability within each group. It is calculated by dividing the sum of squares by the degrees of freedom.

F value: This is the test statistic for the ANOVA test, which assesses whether there are statistically significant differences in means between the groups. A higher F value indicates greater differences between groups.

Pr(>F): This is the p-value associated with the F statistic. It indicates the probability of observing the F value (or a more extreme value) if the null hypothesis (i.e., no difference between groups) were true. Here, the p-value is extremely small (< 2.2e-16), suggesting strong evidence against the null hypothesis.

In summary, this ANOVA table suggests that there are statistically significant differences in “hrs_week” across different occupations, as evidenced by the very low p-value (< 0.001) and the large F value (249.71).

#3)a)Selection of Continuos column

I have selected Age as my continuous Variable and built the regression model with the response variable .

model <- lm(hrs_week ~ Age, data = mydata)

#3.b) Hypothesis Testing. H0 (Null Hypothesis): There is no significant relationship between hrs_week and Age. H1 (Alternative Hypothesis): There is a significant relationship between hrs_week and Age.

summary(lm(hrs_week ~ Age, data = mydata))
## 
## Call:
## lm(formula = hrs_week ~ Age, data = mydata)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -42.077  -1.957  -0.215   4.474  59.781 
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 38.036203   0.204804  185.72   <2e-16 ***
## Age          0.062238   0.005005   12.44   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.32 on 32559 degrees of freedom
## Multiple R-squared:  0.004727,   Adjusted R-squared:  0.004697 
## F-statistic: 154.6 on 1 and 32559 DF,  p-value: < 2.2e-16

#Residual standard error: Represents the standard deviation of the residuals, a measure of the variability of the observed values around the fitted regression line.

Multiple R-squared: Indicates the proportion of variance in the dependent variable explained by the independent variable(s). In this case, it is very low (0.004727), suggesting that only about 0.47% of the variability in “hrs_week” can be explained by “Age”.

Adjusted R-squared: Similar to R(2) , but adjusted for the number of predictors in the model. It penalizes for adding unnecessary predictors.

F-statistic: The test statistic for the overall significance of the regression model. It assesses whether the regression model as a whole explains a significant amount of variability in the dependent variable.

p-value: The probability associated with the F-statistic. Here, it is extremely small (< 2.2e-16), suggesting strong evidence against the null hypothesis that the regression model has no predictive power.

Overall, this output indicates that there is a statistically significant relationship between “Age” and “hrs_week”, as evidenced by the low p-value and the significant coefficients. However, the model’s explanatory power is very weak, as indicated by the low R(2) value.

par(mfrow = c(2, 2))
plot(lm(hrs_week ~ Age, data = mydata))

model <- lm(hrs_week ~ Age * Sex, data = mydata)

summary(model)
## 
## Call:
## lm(formula = hrs_week ~ Age * Sex, data = mydata)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -42.559  -2.879  -1.920   4.539  63.470 
## 
## Coefficients:
##               Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  33.886051   0.325406 104.135  < 2e-16 ***
## Age           0.068487   0.008252   8.299  < 2e-16 ***
## Sex Male      7.322283   0.412309  17.759  < 2e-16 ***
## Age:Sex Male -0.037555   0.010251  -3.664 0.000249 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12 on 32557 degrees of freedom
## Multiple R-squared:  0.05533,    Adjusted R-squared:  0.05524 
## F-statistic: 635.7 on 3 and 32557 DF,  p-value: < 2.2e-16

This is the output of a linear regression model with “hrs_week” (hours per week) as the dependent variable and both “Age” and “Sex” (gender) as independent variables, including their interaction term. Here’s the breakdown of the components:

Call: Indicates the call to the linear regression function, specifying the formula and the dataset used.

Residuals: Provides summary statistics for the residuals (the differences between the observed and predicted values of the dependent variable).

Min: Minimum residual value. 1Q: First quartile of residual values. Median: Median (second quartile) of residual values. 3Q: Third quartile of residual values. Max: Maximum residual value. Coefficients: Displays the estimated coefficients of the linear regression model.

Estimate: The estimated intercept and slope coefficients of the regression equation. Std. Error: Standard errors of the estimated coefficients. t value: The t-statistic associated with each coefficient, indicating the significance of the coefficient. Pr(>|t|): The p-value associated with each coefficient, indicating the significance of the coefficient. Residual standard error: Represents the standard deviation of the residuals, a measure of the variability of the observed values around the fitted regression line.

Multiple R-squared: Indicates the proportion of variance in the dependent variable explained by the independent variable(s), including their interaction term. In this case, it is 0.05533, suggesting that about 5.53% of the variability in “hrs_week” can be explained by “Age”, “Sex”, and their interaction.

Adjusted R-squared: Similar to R(2) , but adjusted for the number of predictors in the model. It penalizes for adding unnecessary predictors.

F-statistic: The test statistic for the overall significance of the regression model. It assesses whether the regression model as a whole explains a significant amount of variability in the dependent variable.

p-value: The probability associated with the F-statistic. Here, it is extremely small (< 2.2e-16), suggesting strong evidence against the null hypothesis that the regression model has no predictive power.

Overall, this output indicates that there is a statistically significant relationship between “hrs_week” and both “Age” and “Sex”, as evidenced by the low p-value and the significant coefficients. Additionally, the interaction term “Age:Sex” is also significant, suggesting that the effect of age on hours per week may differ between males and females. However, the model’s explanatory power is still relatively low, as indicated by the low R(2) value.