Many college courses conclude by giving students the opportunity to evaluate the course and the instructor anonymously. However, the use of these student evaluations as an indicator of course quality and teaching effectiveness is often criticized because these measures may reflect the influence of non-teaching related characteristics, such as the physical appearance of the instructor. The article titled, “Beauty in the classroom: instructors’ pulchritude and putative pedagogical productivity” (Hamermesh and Parker, 2005) found that instructors who are viewed to be better looking receive higher instructional ratings. (Daniel S. Hamermesh, Amy Parker, Beauty in the classroom: instructors pulchritude and putative pedagogical productivity, Economics of Education Review, Volume 24, Issue 4, August 2005, Pages 369-376, ISSN 0272-7757, 10.1016/j.econedurev.2004.07.013. http://www.sciencedirect.com/science/article/pii/S0272775704001165.)
In this lab we will analyze the data from this study in order to learn what goes into a positive professor evaluation.
download.file("http://www.openintro.org/stat/data/evals.RData", destfile = "evals.RData")
load("evals.RData")
variable | description |
---|---|
score |
average professor evaluation score: (1) very unsatisfactory - (5) excellent. |
rank |
rank of professor: teaching, tenure track, tenured. |
ethnicity |
ethnicity of professor: not minority, minority. |
gender |
gender of professor: female, male. |
language |
language of school where professor received education: english or non-english. |
age |
age of professor. |
cls_perc_eval |
percent of students in class who completed evaluation. |
cls_did_eval |
number of students in class who completed evaluation. |
cls_students |
total number of students in class. |
cls_level |
class level: lower, upper. |
cls_profs |
number of professors teaching sections in course in sample: single, multiple. |
cls_credits |
number of credits of class: one credit (lab, PE, etc.), multi credit. |
bty_f1lower |
beauty rating of professor from lower level female: (1) lowest - (10) highest. |
bty_f1upper |
beauty rating of professor from upper level female: (1) lowest - (10) highest. |
bty_f2upper |
beauty rating of professor from second upper level female: (1) lowest - (10) highest. |
bty_m1lower |
beauty rating of professor from lower level male: (1) lowest - (10) highest. |
bty_m1upper |
beauty rating of professor from upper level male: (1) lowest - (10) highest. |
bty_m2upper |
beauty rating of professor from second upper level male: (1) lowest - (10) highest. |
bty_avg |
average beauty rating of professor. |
pic_outfit |
outfit of professor in picture: not formal, formal. |
pic_color |
color of professor’s picture: color, black & white. |
Answer:
This is an observational study. Eventhough the data were gathered from end of semester student evaluations for a large sample of professors, only 6 students rated the professors’ physical appearance. Basically we are observing sample data collected from the University of Texas at Austin.
score
. Is the distribution skewed? What does that tell you about how students rate courses? Is this what you expected to see? Why, or why not?hist(evals$score)
summary(evals$score)
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 2.300 3.800 4.300 4.175 4.600 5.000
Answer
scores have a left skewed distribution with a mean of 4.175 and a median of 4.3. Most of the students rated the scores from 4-5 ratings. In general I feel like not many students want to give bad reviews on the professors. Even the minimum rating is 2.3 is just little below the half wich is not considered as really bad. I would expect to see a more normal distribution of scores, unless the sample was drawn from unusually good classes at UT Austin.
score
, select two other variables and describe their relationship using an appropriate visualization (scatterplot, side-by-side boxplots, or mosaic plot).Answer:
I chose gender and beauty rating variable for the comparison. I have two side-by-side boxpolts for easy visualization of the relatinships of those two variables. It shows that female professors have higher average beauti rating compared to male professors.
library(ggplot2)
library(ggpubr)
## Loading required package: magrittr
male_beauty_level <- ggplot(evals, aes(gender, bty_m1lower))
figure_A <- male_beauty_level + geom_boxplot()
female_beauty_level <- ggplot(evals, aes(gender, bty_f1lower))
figure_B <- female_beauty_level + geom_boxplot()
figure <- ggarrange(figure_A, figure_B,
labels = c("A", "B"),
ncol = 2, nrow = 2)
figure
The fundamental phenomenon suggested by the study is that better looking teachers are evaluated more favorably. Let’s create a scatterplot to see if this appears to be the case:
plot(evals$score ~ evals$bty_avg)
Before we draw conclusions about the trend, compare the number of observations in the data frame with the approximate number of points on the scatterplot. Is anything awry?
jitter()
on the \(y\)- or the \(x\)-coordinate. (Use ?jitter
to learn more.) What was misleading about the initial scatterplot?plot(jitter(evals$score, factor = 1, amount=NULL) ~ evals$bty_avg)
m_bty
to predict average professor score by average beauty rating and add the line to your plot using abline(m_bty)
. Write out the equation for the linear model and interpret the slope. Is average beauty score a statistically significant predictor? Does it appear to be a practically significant predictor?Answer:
m_bty <- lm(score ~ bty_avg, data = evals)
plot(jitter(evals$score) ~ jitter(evals$bty_avg), ylab="Evaluation Score", xlab="Average Beauty Rating",
main="Evaluation Scores and Average Beauty Rating")
abline(m_bty)
m_bty$coefficients
## (Intercept) bty_avg
## 3.88033795 0.06663704
\(\hat{score} = 3.88+0.07*bty\underline{}avg\)
With a slopw of 0.067 we would expect to see an increase of 0.067 in teh professor’s evaluation score for each increase of 1 point on the beauty rating scale.
Answer:
par(mfrow=c(1,3))
plot(m_bty$residuals ~ evals$bty_avg)
abline(h = 0)
qqnorm(m_bty$residuals)
qqline(m_bty$residuals)
hist(m_bty$residuals)
Linearity
There does not seem to be an obvious pattern to the way the dots are scattered. This supports a linear relationship.
Constant variability
There appears to be more points below the horizontal than above. This suggests that variability is not constant.
Nearly normal residuals
When we plot the residuals, we see that they are not normal – they are left-skewed. That means there is a bias in the data. This condition is not met.
The data set contains several variables on the beauty score of the professor: individual ratings from each of the six students who were asked to score the physical appearance of the professors and the average of these six scores. Let’s take a look at the relationship between one of these scores and the average beauty score.
plot(evals$bty_avg ~ evals$bty_f1lower)
cor(evals$bty_avg, evals$bty_f1lower)
As expected the relationship is quite strong - after all, the average score is calculated using the individual scores. We can actually take a look at the relationships between all beauty variables (columns 13 through 19) using the following command:
plot(evals[,13:19])
These variables are collinear (correlated), and adding more than one of these variables to the model would not add much value to the model. In this application and with these highly-correlated predictors, it is reasonable to use the average beauty score as the single representative of these variables.
In order to see if beauty is still a significant predictor of professor score after we’ve accounted for the gender of the professor, we can add the gender term into the model.
m_bty_gen <- lm(score ~ bty_avg + gender, data = evals)
summary(m_bty_gen)
##
## Call:
## lm(formula = score ~ bty_avg + gender, data = evals)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.8305 -0.3625 0.1055 0.4213 0.9314
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.74734 0.08466 44.266 < 2e-16 ***
## bty_avg 0.07416 0.01625 4.563 6.48e-06 ***
## gendermale 0.17239 0.05022 3.433 0.000652 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.5287 on 460 degrees of freedom
## Multiple R-squared: 0.05912, Adjusted R-squared: 0.05503
## F-statistic: 14.45 on 2 and 460 DF, p-value: 8.177e-07
Answer:
plot(abs(m_bty_gen$residuals) ~ m_bty_gen$fitted.values)
plot(m_bty_gen$residuals ~ c(1:nrow(evals)))
plot(evals$score ~ evals$gender)
plot(evals$score ~ evals$bty_avg)
par(mfrow = c(1,2))
plot(evals$score ~ evals$gender)
plot(evals$score ~ evals$bty_avg)
bty_avg
still a significant predictor of score
? Has the addition of gender
to the model changed the parameter estimate for bty_avg
?Answer:
summary(m_bty_gen)
##
## Call:
## lm(formula = score ~ bty_avg + gender, data = evals)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.8305 -0.3625 0.1055 0.4213 0.9314
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.74734 0.08466 44.266 < 2e-16 ***
## bty_avg 0.07416 0.01625 4.563 6.48e-06 ***
## gendermale 0.17239 0.05022 3.433 0.000652 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.5287 on 460 degrees of freedom
## Multiple R-squared: 0.05912, Adjusted R-squared: 0.05503
## F-statistic: 14.45 on 2 and 460 DF, p-value: 8.177e-07
It does look like that the bty_avg is a significant predictor of scores. And gender made bty_avg more significant as the p-value computed is even smaller when compared to the model where we computed bty_avg as the sole variable.
Note that the estimate for gender
is now called gendermale
. You’ll see this name change whenever you introduce a categorical variable. The reason is that R recodes gender
from having the values of female
and male
to being an indicator variable called gendermale
that takes a value of \(0\) for females and a value of \(1\) for males. (Such variables are often referred to as “dummy” variables.)
As a result, for females, the parameter estimate is multiplied by zero, leaving the intercept and slope form familiar from simple regression.
\[ \begin{aligned} \widehat{score} &= \hat{\beta}_0 + \hat{\beta}_1 \times bty\_avg + \hat{\beta}_2 \times (0) \\ &= \hat{\beta}_0 + \hat{\beta}_1 \times bty\_avg\end{aligned} \]
We can plot this line and the line corresponding to males with the following custom function.
multiLines(m_bty_gen)
Answer:
For males, the equation of the line is:
\(\hat{score} = 3.91973 + 0.07416 × bty-avg\)
This is just the genderemale parameter plus the intercept, since if we’re only looking at males this will always be 1.
For two professors who received the same beauty rating, male professor tends to have the higher course evaluation score.
The decision to call the indicator variable gendermale
instead ofgenderfemale
has no deeper meaning. R simply codes the category that comes first alphabetically as a \(0\). (You can change the reference level of a categorical variable, which is the level that is coded as a 0, using therelevel
function. Use ?relevel
to learn more.)
m_bty_rank
with gender
removed and rank
added in. How does R appear to handle categorical variables that have more than two levels? Note that the rank variable has three levels: teaching
, tenure track
, tenured
.Answer:
m_bty_rank <- lm(score ~ bty_avg + rank, data = evals)
summary(m_bty_rank)
##
## Call:
## lm(formula = score ~ bty_avg + rank, data = evals)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.8713 -0.3642 0.1489 0.4103 0.9525
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.98155 0.09078 43.860 < 2e-16 ***
## bty_avg 0.06783 0.01655 4.098 4.92e-05 ***
## ranktenure track -0.16070 0.07395 -2.173 0.0303 *
## ranktenured -0.12623 0.06266 -2.014 0.0445 *
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.5328 on 459 degrees of freedom
## Multiple R-squared: 0.04652, Adjusted R-squared: 0.04029
## F-statistic: 7.465 on 3 and 459 DF, p-value: 6.88e-05
R automatically Creates dummy variables for categorical variable which have more than two levels.
The interpretation of the coefficients in multiple regression is slightly different from that of simple regression. The estimate for bty_avg
reflects how much higher a group of professors is expected to score if they have a beauty rating that is one point higher while holding all other variables constant. In this case, that translates into considering only professors of the same rank with bty_avg
scores that are one point apart.
We will start with a full model that predicts professor score based on rank, ethnicity, gender, language of the university where they got their degree, age, proportion of students that filled out evaluations, class size, course level, number of professors, number of credits, average beauty rating, outfit, and picture color.
Answer:
“number of professors” cls_profs as the variable to have the least assoication with the professor’s evaluation score.
Let’s run the model…
m_full <- lm(score ~ rank + ethnicity + gender + language + age + cls_perc_eval
+ cls_students + cls_level + cls_profs + cls_credits + bty_avg
+ pic_outfit + pic_color, data = evals)
summary(m_full)
##
## Call:
## lm(formula = score ~ rank + ethnicity + gender + language + age +
## cls_perc_eval + cls_students + cls_level + cls_profs + cls_credits +
## bty_avg + pic_outfit + pic_color, data = evals)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.77397 -0.32432 0.09067 0.35183 0.95036
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.0952141 0.2905277 14.096 < 2e-16 ***
## ranktenure track -0.1475932 0.0820671 -1.798 0.07278 .
## ranktenured -0.0973378 0.0663296 -1.467 0.14295
## ethnicitynot minority 0.1234929 0.0786273 1.571 0.11698
## gendermale 0.2109481 0.0518230 4.071 5.54e-05 ***
## languagenon-english -0.2298112 0.1113754 -2.063 0.03965 *
## age -0.0090072 0.0031359 -2.872 0.00427 **
## cls_perc_eval 0.0053272 0.0015393 3.461 0.00059 ***
## cls_students 0.0004546 0.0003774 1.205 0.22896
## cls_levelupper 0.0605140 0.0575617 1.051 0.29369
## cls_profssingle -0.0146619 0.0519885 -0.282 0.77806
## cls_creditsone credit 0.5020432 0.1159388 4.330 1.84e-05 ***
## bty_avg 0.0400333 0.0175064 2.287 0.02267 *
## pic_outfitnot formal -0.1126817 0.0738800 -1.525 0.12792
## pic_colorcolor -0.2172630 0.0715021 -3.039 0.00252 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.498 on 448 degrees of freedom
## Multiple R-squared: 0.1871, Adjusted R-squared: 0.1617
## F-statistic: 7.366 on 14 and 448 DF, p-value: 6.552e-14
Answer:
The highest p value has the cls_profs with value of 0.77806.
Answer:
As mentioned above, ethnicity does not appear to be significant in predicting the scores of professors. The p-value here is 0.12, which is above our 5% threshold.
Answer:
m_full_nosingle <- lm(score ~ rank + ethnicity + gender + language + age +
cls_perc_eval + cls_students + cls_level +
cls_credits + bty_avg + pic_outfit + pic_color,
data = evals)
summary(m_full_nosingle)
##
## Call:
## lm(formula = score ~ rank + ethnicity + gender + language + age +
## cls_perc_eval + cls_students + cls_level + cls_credits +
## bty_avg + pic_outfit + pic_color, data = evals)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.7836 -0.3257 0.0859 0.3513 0.9551
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 4.0872523 0.2888562 14.150 < 2e-16 ***
## ranktenure track -0.1476746 0.0819824 -1.801 0.072327 .
## ranktenured -0.0973829 0.0662614 -1.470 0.142349
## ethnicitynot minority 0.1274458 0.0772887 1.649 0.099856 .
## gendermale 0.2101231 0.0516873 4.065 5.66e-05 ***
## languagenon-english -0.2282894 0.1111305 -2.054 0.040530 *
## age -0.0089992 0.0031326 -2.873 0.004262 **
## cls_perc_eval 0.0052888 0.0015317 3.453 0.000607 ***
## cls_students 0.0004687 0.0003737 1.254 0.210384
## cls_levelupper 0.0606374 0.0575010 1.055 0.292200
## cls_creditsone credit 0.5061196 0.1149163 4.404 1.33e-05 ***
## bty_avg 0.0398629 0.0174780 2.281 0.023032 *
## pic_outfitnot formal -0.1083227 0.0721711 -1.501 0.134080
## pic_colorcolor -0.2190527 0.0711469 -3.079 0.002205 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4974 on 449 degrees of freedom
## Multiple R-squared: 0.187, Adjusted R-squared: 0.1634
## F-statistic: 7.943 on 13 and 449 DF, p-value: 2.336e-14
Yes. There was a slight change in the coefficients and significance of the other explanatory variables when cls_profs was removed. All the values are now slightly lower - meaning they are more significant now to the level than before
Answer:
bmodel <- step(m_full, direction = "backward")
## Start: AIC=-630.9
## score ~ rank + ethnicity + gender + language + age + cls_perc_eval +
## cls_students + cls_level + cls_profs + cls_credits + bty_avg +
## pic_outfit + pic_color
##
## Df Sum of Sq RSS AIC
## - cls_profs 1 0.0197 111.11 -632.82
## - cls_level 1 0.2740 111.36 -631.76
## - cls_students 1 0.3599 111.44 -631.40
## - rank 2 0.8930 111.98 -631.19
## <none> 111.08 -630.90
## - pic_outfit 1 0.5768 111.66 -630.50
## - ethnicity 1 0.6117 111.70 -630.36
## - language 1 1.0557 112.14 -628.52
## - bty_avg 1 1.2967 112.38 -627.53
## - age 1 2.0456 113.13 -624.45
## - pic_color 1 2.2893 113.37 -623.46
## - cls_perc_eval 1 2.9698 114.06 -620.69
## - gender 1 4.1085 115.19 -616.09
## - cls_credits 1 4.6495 115.73 -613.92
##
## Step: AIC=-632.82
## score ~ rank + ethnicity + gender + language + age + cls_perc_eval +
## cls_students + cls_level + cls_credits + bty_avg + pic_outfit +
## pic_color
##
## Df Sum of Sq RSS AIC
## - cls_level 1 0.2752 111.38 -633.67
## - cls_students 1 0.3893 111.49 -633.20
## - rank 2 0.8939 112.00 -633.11
## <none> 111.11 -632.82
## - pic_outfit 1 0.5574 111.66 -632.50
## - ethnicity 1 0.6728 111.78 -632.02
## - language 1 1.0442 112.15 -630.49
## - bty_avg 1 1.2872 112.39 -629.49
## - age 1 2.0422 113.15 -626.39
## - pic_color 1 2.3457 113.45 -625.15
## - cls_perc_eval 1 2.9502 114.06 -622.69
## - gender 1 4.0895 115.19 -618.08
## - cls_credits 1 4.7999 115.90 -615.24
##
## Step: AIC=-633.67
## score ~ rank + ethnicity + gender + language + age + cls_perc_eval +
## cls_students + cls_credits + bty_avg + pic_outfit + pic_color
##
## Df Sum of Sq RSS AIC
## - cls_students 1 0.2459 111.63 -634.65
## - rank 2 0.8140 112.19 -634.30
## <none> 111.38 -633.67
## - pic_outfit 1 0.6618 112.04 -632.93
## - ethnicity 1 0.8698 112.25 -632.07
## - language 1 0.9015 112.28 -631.94
## - bty_avg 1 1.3694 112.75 -630.02
## - age 1 1.9342 113.31 -627.70
## - pic_color 1 2.0777 113.46 -627.12
## - cls_perc_eval 1 3.0290 114.41 -623.25
## - gender 1 3.8989 115.28 -619.74
## - cls_credits 1 4.5296 115.91 -617.22
##
## Step: AIC=-634.65
## score ~ rank + ethnicity + gender + language + age + cls_perc_eval +
## cls_credits + bty_avg + pic_outfit + pic_color
##
## Df Sum of Sq RSS AIC
## - rank 2 0.7892 112.42 -635.39
## <none> 111.63 -634.65
## - ethnicity 1 0.8832 112.51 -633.00
## - pic_outfit 1 0.9700 112.60 -632.65
## - language 1 1.0338 112.66 -632.38
## - bty_avg 1 1.5783 113.20 -630.15
## - pic_color 1 1.9477 113.57 -628.64
## - age 1 2.1163 113.74 -627.96
## - cls_perc_eval 1 2.7922 114.42 -625.21
## - gender 1 4.0945 115.72 -619.97
## - cls_credits 1 4.5163 116.14 -618.29
##
## Step: AIC=-635.39
## score ~ ethnicity + gender + language + age + cls_perc_eval +
## cls_credits + bty_avg + pic_outfit + pic_color
##
## Df Sum of Sq RSS AIC
## <none> 112.42 -635.39
## - pic_outfit 1 0.7141 113.13 -634.46
## - ethnicity 1 1.1790 113.59 -632.56
## - language 1 1.3403 113.75 -631.90
## - age 1 1.6847 114.10 -630.50
## - pic_color 1 1.7841 114.20 -630.10
## - bty_avg 1 1.8553 114.27 -629.81
## - cls_perc_eval 1 2.9147 115.33 -625.54
## - gender 1 4.0577 116.47 -620.97
## - cls_credits 1 6.1208 118.54 -612.84
summary(bmodel)
##
## Call:
## lm(formula = score ~ ethnicity + gender + language + age + cls_perc_eval +
## cls_credits + bty_avg + pic_outfit + pic_color, data = evals)
##
## Residuals:
## Min 1Q Median 3Q Max
## -1.8455 -0.3221 0.1013 0.3745 0.9051
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.907030 0.244889 15.954 < 2e-16 ***
## ethnicitynot minority 0.163818 0.075158 2.180 0.029798 *
## gendermale 0.202597 0.050102 4.044 6.18e-05 ***
## languagenon-english -0.246683 0.106146 -2.324 0.020567 *
## age -0.006925 0.002658 -2.606 0.009475 **
## cls_perc_eval 0.004942 0.001442 3.427 0.000666 ***
## cls_creditsone credit 0.517205 0.104141 4.966 9.68e-07 ***
## bty_avg 0.046732 0.017091 2.734 0.006497 **
## pic_outfitnot formal -0.113939 0.067168 -1.696 0.090510 .
## pic_colorcolor -0.180870 0.067456 -2.681 0.007601 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.4982 on 453 degrees of freedom
## Multiple R-squared: 0.1774, Adjusted R-squared: 0.161
## F-statistic: 10.85 on 9 and 453 DF, p-value: 2.441e-15
The final model: score = 3.907030 + 0.163818 * not_ethnicity + 0.202597 * gendermale -0.246683 language-english -0.006925 age + 0.004942 * cls_perc_eval + 0.517205 * cls_creditsone_credit + 0.046732bty_avg $-0.180870 pic_colorcolor
Answer:
par(mfrow = c(2,2))
plot(bmodel)
Answer:
The conditions for the model is not satisfied we can see from the residuals vs fitted plot theirs is v shaped data pattern and from the normal q-q plot we can see that the residuals are not normally distributed.
Class courses are independent of each other so evaluation scores from one course is indpendent of the other even if the course is being taught by the same professor.
Answer:
Based on the final model, the qualities being associated with a high evaluation score are being of teaching rank, non-minority, male, speaking english, younger, and teaching one credit classes.
Answer:
The sample size of 6 is too small. Also, some of the predictor variables are subjective and may vary with culture. Beauty, for one, is in the eye of the beholder.Picture preferences may also be culturally biased.