Lab Test 2
Dario DeBlasis
2022-03-21
## Loading required package: splines## Loading required package: RcmdrMisc## Loading required package: car## Loading required package: carData## Loading required package: sandwich## Warning in register(): Can't find generic `scale_type` in package ggplot2 to
## register S3 method.## Loading required package: effects## lattice theme set by effectsTheme()
## See ?effectsTheme for details.## The Commander GUI is launched only in interactive sessions##
## Attaching package: 'Rcmdr'## The following object is masked from 'package:base':
##
## errorCondition> marketing <- read.table("C:/Users/Dario/Downloads/DirectMarketing(1).csv",
+ header=TRUE, stringsAsFactors=TRUE, sep=",", na.strings="NA", dec=".",
+ strip.white=TRUE)> with(marketing, Hist(AmountSpent, scale="frequency", breaks="Sturges",
+ col="darkgray"))
The distribution for Amount Spent is a right tailed distribution as indicated by the low-end of the tail to the right of the above histogram.
> Boxplot( ~ AmountSpent, data=marketing, id=list(method="y"))
[1] "181" "256" "386" "7" "133" "278" "240" "291" "202" "140"The top whisker is longer than the bottom whisker and the box is not split evenly in the middle. This indicates that Amount Spent is indeed right skewed.
> with(marketing, qqPlot(AmountSpent, dist="norm", id=list(method="y", n=2,
+ labels=rownames(marketing))))
[1] 181 256Since Amount Spent deviates up from the 45 degree line in the QQ Plot, this is another indication that it is not normally distributed and it is right skewed.
> summary(marketing) Age Gender OwnHome Married
Min. :1.000 Min. :0.000 Min. :0.000 Min. :0.000
1st Qu.:1.000 1st Qu.:0.000 1st Qu.:0.000 1st Qu.:0.000
Median :2.000 Median :1.000 Median :1.000 Median :1.000
Mean :1.918 Mean :0.506 Mean :0.516 Mean :0.502
3rd Qu.:2.000 3rd Qu.:1.000 3rd Qu.:1.000 3rd Qu.:1.000
Max. :3.000 Max. :1.000 Max. :1.000 Max. :1.000
Salary Children Catalogs AmountSpent
Min. : 10100 Min. :0.000 Min. : 6.00 Min. : 38.0
1st Qu.: 29975 1st Qu.:0.000 1st Qu.: 6.00 1st Qu.: 488.2
Median : 53700 Median :1.000 Median :12.00 Median : 962.0
Mean : 56104 Mean :0.934 Mean :14.68 Mean :1216.8
3rd Qu.: 77025 3rd Qu.:2.000 3rd Qu.:18.00 3rd Qu.:1688.5
Max. :168800 Max. :3.000 Max. :24.00 Max. :6217.0
X X.1 X.2 X.3 X.4
Mode:logical Mode:logical Mode:logical Mode:logical :998
NA's:1000 NA's:1000 NA's:1000 NA's:1000 Female -1: 1
Male -0 : 1
Mean is greater than median for Amount Spent, also meaning it is right skewed. Lets examine Salary next which also has a mean > median, but not as major.
> with(marketing, Hist(Salary, scale="frequency", breaks="Sturges",
+ col="darkgray"))
Salary is a right skewed by looking at the histogram but it is not as right skewed as Amount Spent.
> with(marketing, qqPlot(Salary, dist="norm", id=list(method="y", n=2,
+ labels=rownames(marketing))))
[1] 177 262The QQ Plot for Salary shows how many points stay within the 45-degree line but some points deviate from it, giving Salary it’s right skewness.
> normalityTest(~Salary, test="shapiro.test", data=marketing)
Shapiro-Wilk normality test
data: Salary
W = 0.96338, p-value = 3.763e-15> library(abind, pos=17)> with(marketing, (t.test(Salary, alternative='two.sided', mu=0.0,
+ conf.level=.95)))
One Sample t-test
data: Salary
t = 57.948, df = 999, p-value < 2.2e-16
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
54204.01 58003.79
sample estimates:
mean of x
56103.9 > with(marketing, cor.test(AmountSpent, Salary, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: AmountSpent and Salary
t = 30.93, df = 998, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.6665092 0.7299316
sample estimates:
cor
0.6995957 Using Pearson’s product-moment correlation test, we can see that the sample estimate of correlation between Salary and Amount Spent is 0.6995 which is a strong correlation. This means that Salary is a good estimator of Amount Spent.
> with(marketing, cor.test(AmountSpent, Catalogs, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: AmountSpent and Catalogs
t = 16.944, df = 998, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.4230527 0.5194233
sample estimates:
cor
0.4726499 When using Pearson’s product-moment correlation test for Catalogs and Amount Spent, it has a correlation of 0.473 which is weaker than the Salary sample estimate. It is safe to say Catalogs is a worse estimator of Amount Spent than Salary. Let’s explore the correlation with more variables.
> with(marketing, cor.test(Age, AmountSpent, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: Age and AmountSpent
t = 11.736, df = 998, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.2925738 0.4015739
sample estimates:
cor
0.3482505 > with(marketing, cor.test(AmountSpent, Married, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: AmountSpent and Married
t = 17.093, df = 998, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.4264682 0.5224598
sample estimates:
cor
0.47588 > with(marketing, cor.test(AmountSpent, Gender, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: AmountSpent and Gender
t = -6.5053, df = 998, p-value = 1.224e-10
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
-0.2604271 -0.1414659
sample estimates:
cor
-0.2016902 > with(marketing, cor.test(AmountSpent, OwnHome, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: AmountSpent and OwnHome
t = 11.835, df = 998, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.2952356 0.4040147
sample estimates:
cor
0.350808 > with(marketing, cor.test(AmountSpent, Married, alternative="two.sided",
+ method="pearson"))
Pearson's product-moment correlation
data: AmountSpent and Married
t = 17.093, df = 998, p-value < 2.2e-16
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
0.4264682 0.5224598
sample estimates:
cor
0.47588 After examining the other variables using Pearson’s product-moment correlation, it seems that all of the other variables have a weaker correlation than Salary. This means that in this dataset, Salary is the best predictor of Amount Spent.
> library(lattice, pos=18)> scatterplot(AmountSpent~Salary, regLine=FALSE, smooth=FALSE, boxplots=FALSE,
+ data=marketing)
When looking at a scatterplot of Salary (as the x variable) and Amount Spent (as the y variable) it is visible that there is a decently strong positive correlation between the two variables.
> scatterplot(AmountSpent~Catalogs, regLine=FALSE, smooth=FALSE,
+ boxplots=FALSE, data=marketing)
While the number of Catalogs increases with the amount spent, it is not consistent for all Amounts Spent. From this scatterplot, we can tell that Catalogs does not have a strong correlation with Amount Spent.
> RegModel.1 <- lm(AmountSpent~Age+Catalogs+Children+OwnHome+Salary,
+ data=marketing)
> summary(RegModel.1)
Call:
lm(formula = AmountSpent ~ Age + Catalogs + Children + OwnHome +
Salary, data = marketing)
Residuals:
Min 1Q Median 3Q Max
-1762.0 -349.2 -38.9 260.6 3214.4
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -4.293e+02 7.113e+01 -6.035 2.24e-09 ***
Age -1.142e+01 3.062e+01 -0.373 0.709
Catalogs 4.771e+01 2.758e+00 17.303 < 2e-16 ***
Children -1.999e+02 1.800e+01 -11.105 < 2e-16 ***
OwnHome 3.829e+01 4.220e+01 0.907 0.364
Salary 2.022e-02 6.902e-04 29.294 < 2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 562.9 on 994 degrees of freedom
Multiple R-squared: 0.6587, Adjusted R-squared: 0.657
F-statistic: 383.7 on 5 and 994 DF, p-value: < 2.2e-16When conducting the Linear Regression model for Amount Spent with the other variables in the dataset we notice that the ones with *** are statistically significant. This means that there is some relationship amongst those variables with amount spent. When looking at the Estimate column, Salary is 0.02 which is the closest to 0 making it the best estimator amongst all the variables. This confirms my earlier hypothesis, and while OwnHome and Catalogs are not negative they still are not as good of estimators as Salary. The multiple R squared value is equal to 0.6587 which means that the selected variables are relatively good predictors of the proportion of variance in the response variable which was Amount Spent.
> library(nnet, pos=19)> Anova(RegModel.1, type="II")Anova Table (Type II tests)
Response: AmountSpent
Sum Sq Df F value Pr(>F)
Age 44065 1 0.1391 0.7093
Catalogs 94846256 1 299.3805 <2e-16 ***
Children 39071454 1 123.3283 <2e-16 ***
OwnHome 260783 1 0.8232 0.3645
Salary 271869033 1 858.1497 <2e-16 ***
Residuals 314907547 994
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1> library(Matrix, pos=20)> library(lme4, pos=20)The Anova (type 2 regression model) gives us a similar output to the first Linear regression model, however we get a sum of the variances instead of the estimate and the standard deviation. For this reason, I reccomend that the marketing agent use a Linear Regression model to find the best estimator of Amount Spent. Since the linear regression model tells us how much variability in the data is explained by the model (using R-squared, estimate, etc) then it is the best model to use for this type of analysis.