Midterm Exam Coding part
Instructions
This is the R portion of your midterm exam. You will analyze the Salary dataset, which contains information salary for Assistant Professors, Associate Professors and Professors in a college in the U.S in 2008-2009. For each of the variables, please check the code book here:
- rank: a factor with levels: AssocProf, AsstProf, Prof.
- discipline: a factor with levels A (“theoretical” departments) or B
(“applied” departments).
- yrs.since.phd: years since PhD.
- yrs.service: years of service.
- sex: a factor with levels Female, Male.
- salary: nine-month salary, in thousand dollars.
I’ve reviewed this dataset, and confirmed that there is no missing values.
Please follow the instructions carefully and write your R code in the provided chunks. You will be graded on the correctness of your code, the quality of your analysis, and your interpretation of the results.
Total points: 10
Good luck!
1. Data Import and Exploration (2 points)
- Import the Salary dataset provided on Canvas, named it as
Salary, and display the first few rows. (1 points)
Salary <- read.csv("salaries.csv")
head(Salary, n = 5)## rank discipline yrs.since.phd yrs.service sex salary
## 1 Prof B 19 18 Male 139.75
## 2 Prof B 20 16 Male 173.20
## 3 AsstProf B 4 3 Male 79.75
## 4 Prof B 45 39 Male 115.00
## 5 Prof B 40 41 Male 141.50- Use appropriate R functions to display the structure of the dataset and report how many observations and variables are in the dataset? Among these variables, how many of them are numeric? (1 points)
str(Salary)## 'data.frame': 397 obs. of 6 variables:
## $ rank : chr "Prof" "Prof" "AsstProf" "Prof" ...
## $ discipline : chr "B" "B" "B" "B" ...
## $ yrs.since.phd: int 19 20 4 45 40 6 30 45 21 18 ...
## $ yrs.service : int 18 16 3 39 41 6 23 45 20 18 ...
## $ sex : chr "Male" "Male" "Male" "Male" ...
## $ salary : num 139.8 173.2 79.8 115 141.5 ...There are 397 observations and 6 variables. Among these, 3 are numeric and 3 are categorical.
2. Data Preprocessing and Visualization (3 points)
- Rename the variables to snake case (like “snake_case”) (1 points)
snake_case <- Salary$yrs.since.phd
snake_case1 <- Salary$yrs.service- Please create frequency tables for variable
rankanddiscipline. (How many AsstProf, AssocProf, and Prof; and how many of them are in theoretical departments and how many in applied departments). (1)
table(Salary$rank)##
## AssocProf AsstProf Prof
## 64 67 266table(Salary$discipline)##
## A B
## 181 216Assistant Prof: 67. Associate Prof: 64. Prof: 266 Theoretical: 181. Applied: 216
- Create a box plot of ‘salary’ vs ‘rank’ (you can use
plot()orggplot()). Add a title and proper axis labels. You don’t need to interpret the result here but you should know how. (1 points)
library(ggplot2)
ggplot(data = Salary, aes(y = salary, x = rank)) +
geom_boxplot() +
labs(title = "Salary vs Rank", x = "Rank", y = "Salary (1000s)")
3. Linear Regression Analysis (5 points)
- Split the data into training data (80%) and testing data (20%), and
name them as
Salary_trainandSalary_test. A part of the code was given, please finish it. (1 points)
training_index <- sample(1:nrow(Salary), round(0.8 * nrow(Salary)))
Salary_train <- Salary[training_index, ]
Salary_train## rank discipline yrs.since.phd yrs.service sex salary
## 274 AsstProf A 8 4 Male 74.000
## 375 Prof A 27 19 Male 103.275
## 263 Prof A 31 26 Male 121.200
## 285 AssocProf A 8 6 Male 88.650
## 181 Prof B 11 11 Male 142.467
## 30 Prof B 12 8 Male 118.223
## 160 Prof B 15 16 Male 137.167
## 367 Prof A 15 10 Male 115.435
## 166 Prof B 21 8 Male 105.890
## 44 Prof B 38 38 Male 231.545
## 213 Prof B 15 7 Male 128.400
## 266 Prof A 36 30 Male 134.800
## 363 Prof A 30 30 Male 138.771
## 139 AssocProf A 10 7 Male 73.877
## 278 Prof A 31 27 Male 163.200
## 298 Prof A 17 11 Male 148.800
## 12 AsstProf B 7 2 Male 79.800
## 279 Prof A 24 18 Male 107.100
## 62 AsstProf B 3 2 Male 75.243
## 224 Prof B 34 20 Male 129.600
## 342 Prof B 17 17 Female 124.312
## 206 Prof B 21 2 Male 96.545
## 317 AssocProf B 12 9 Female 71.065
## 135 Prof A 35 25 Male 168.635
## 267 Prof A 43 43 Male 143.940
## 171 AsstProf B 5 5 Male 91.227
## 281 Prof A 39 38 Male 136.500
## 185 Prof B 23 23 Male 101.000
## 91 AsstProf B 10 5 Female 97.032
## 326 AsstProf B 8 4 Male 84.500
## 140 Prof A 21 18 Male 152.664
## 23 Prof A 34 30 Male 93.904
## 35 AsstProf B 4 2 Female 80.225
## 3 AsstProf B 4 3 Male 79.750
## 151 Prof B 14 12 Male 128.148
## 124 AssocProf A 25 22 Female 62.884
## 309 AsstProf A 5 0 Male 74.000
## 4 Prof B 45 39 Male 115.000
## 116 Prof A 21 9 Male 120.806
## 319 Prof B 16 16 Male 134.550
## 234 Prof A 36 19 Female 117.555
## 146 Prof B 28 28 Male 119.015
## 75 Prof B 28 23 Male 113.398
## 16 Prof B 12 3 Male 117.150
## 147 AsstProf B 4 4 Male 92.000
## 270 Prof A 13 7 Male 103.700
## 144 AsstProf B 3 3 Male 89.942
## 92 AssocProf B 10 7 Male 105.128
## 361 Prof A 14 11 Male 121.946
## 379 Prof A 38 38 Male 150.680
## 118 Prof A 39 36 Male 117.515
## 47 Prof B 40 28 Male 98.193
## 384 Prof A 44 44 Male 105.000
## 322 AssocProf B 9 9 Male 95.642
## 99 Prof B 30 14 Male 102.235
## 330 Prof B 23 23 Male 134.778
## 216 Prof B 16 11 Male 145.350
## 104 Prof B 20 14 Female 127.512
## 138 Prof A 17 14 Male 105.668
## 105 AssocProf A 18 10 Male 83.850
## 391 Prof A 40 19 Male 166.605
## 252 Prof A 20 8 Male 102.000
## 250 Prof A 29 7 Male 204.000
## 255 Prof A 28 7 Female 116.450
## 130 AsstProf A 4 2 Male 73.000
## 193 Prof B 19 18 Male 122.100
## 122 Prof A 32 32 Male 124.309
## 108 AssocProf A 10 8 Male 82.600
## 5 Prof B 40 41 Male 141.500
## 345 Prof B 32 35 Male 150.376
## 71 Prof B 17 2 Male 126.320
## 65 AsstProf B 4 3 Male 68.404
## 158 AsstProf B 1 0 Male 88.000
## 238 AsstProf A 7 6 Female 63.100
## 325 Prof B 30 31 Male 162.221
## 41 Prof B 23 2 Male 146.500
## 202 Prof B 40 40 Male 119.700
## 136 Prof A 20 18 Male 136.000
## 304 Prof A 14 4 Male 105.260
## 305 Prof A 46 44 Male 144.050
## 265 Prof A 37 35 Male 99.000
## 6 AssocProf B 6 6 Male 97.000
## 169 AssocProf B 8 6 Male 101.210
## 194 AssocProf B 19 19 Male 86.250
## 184 Prof B 26 22 Male 150.000
## 178 AssocProf B 13 9 Male 100.944
## 36 AsstProf B 5 0 Female 77.000
## 89 Prof B 25 25 Male 172.272
## 273 AsstProf A 4 1 Male 73.000
## 28 AsstProf B 5 3 Male 82.379
## 26 Prof A 21 8 Male 106.294
## 31 Prof B 20 4 Male 132.261
## 349 AsstProf B 4 3 Male 80.139
## 1 Prof B 19 18 Male 139.750
## 226 Prof A 20 20 Male 122.400
## 8 Prof B 45 45 Male 147.765
## 106 Prof A 31 28 Male 113.543
## 295 Prof A 19 7 Male 107.300
## 176 Prof B 28 25 Male 111.751
## 262 Prof A 45 45 Male 107.550
## 154 AssocProf B 12 10 Female 103.994
## 221 Prof B 21 21 Male 170.000
## 308 Prof A 31 28 Male 122.500
## 240 Prof A 19 6 Male 96.200
## 132 Prof A 56 57 Male 76.840
## 207 Prof B 35 33 Male 162.200
## 371 AssocProf A 13 8 Male 78.182
## 128 AsstProf A 2 0 Female 72.500
## 275 AsstProf A 8 3 Female 78.500
## 360 AsstProf A 11 4 Male 78.785
## 172 Prof B 19 19 Male 151.575
## 211 AsstProf B 4 3 Male 91.000
## 79 AsstProf B 3 1 Male 86.100
## 180 AsstProf B 3 3 Female 92.000
## 52 Prof B 12 11 Male 108.875
## 58 AssocProf B 9 8 Male 90.215
## 323 AssocProf B 13 11 Male 126.431
## 259 AsstProf A 9 3 Male 73.800
## 219 AssocProf B 14 7 Female 109.650
## 314 Prof A 35 35 Male 100.351
## 161 AsstProf B 2 2 Male 89.516
## 327 Prof B 23 15 Male 124.714
## 374 Prof A 30 26 Male 136.660
## 350 Prof B 27 28 Male 144.309
## 51 Prof B 28 28 Male 126.621
## 157 AssocProf B 12 18 Male 113.341
## 125 Prof A 24 22 Male 96.614
## 246 Prof A 17 11 Female 90.450
## 15 Prof B 20 18 Male 104.800
## 352 Prof B 38 38 Male 93.519
## 126 Prof A 54 49 Male 78.162
## 291 Prof A 33 7 Male 174.500
## 155 AsstProf B 4 0 Male 92.000
## 66 AssocProf B 9 8 Male 100.522
## 189 AssocProf B 28 28 Male 106.300
## 318 Prof B 46 45 Male 67.559
## 241 AsstProf A 5 3 Male 69.200
## 94 Prof B 38 38 Male 166.024
## 148 Prof B 27 27 Male 156.938
## 343 Prof B 38 38 Male 114.596
## 395 Prof A 42 25 Male 101.738
## 235 AsstProf A 8 3 Male 69.700
## 396 Prof A 25 15 Male 95.329
## 170 Prof B 25 18 Male 181.257
## 324 Prof B 24 15 Female 161.101
## 306 Prof A 33 31 Male 111.350
## 348 Prof B 39 33 Male 128.250
## 177 AssocProf B 10 7 Male 95.436
## 59 AssocProf B 10 9 Male 100.135
## 133 AssocProf A 10 8 Female 77.500
## 54 Prof B 16 9 Male 106.639
## 131 AssocProf A 11 9 Male 83.001
## 347 Prof B 41 27 Male 142.023
## 29 AsstProf B 11 0 Male 77.000
## 48 Prof B 23 19 Female 151.768
## 386 Prof A 15 9 Male 114.330
## 301 Prof A 39 36 Male 88.600
## 340 Prof B 37 15 Male 137.317
## 156 Prof B 21 21 Male 118.971
## 50 AsstProf B 1 1 Male 70.768
## 42 AssocProf B 23 23 Male 93.418
## 260 Prof A 32 30 Male 92.550
## 107 AssocProf A 11 8 Male 82.099
## 353 Prof B 26 27 Male 142.500
## 229 Prof A 16 11 Male 88.175
## 145 Prof B 27 27 Male 112.696
## 331 Prof B 49 60 Male 192.253
## 129 Prof A 32 30 Male 113.278
## 76 AsstProf B 8 3 Male 73.266
## 80 AsstProf B 6 2 Male 84.240
## 209 AsstProf B 7 2 Male 91.300
## 153 Prof B 21 9 Male 111.168
## 212 Prof B 39 39 Male 111.350
## 233 Prof A 38 19 Male 148.750
## 328 Prof B 37 37 Male 151.650
## 40 AssocProf B 9 9 Male 100.938
## 199 Prof B 34 33 Male 189.409
## 187 AssocProf B 13 10 Female 103.750
## 114 Prof A 37 37 Male 104.279
## 261 AssocProf A 41 33 Male 88.600
## 225 Prof A 38 35 Male 87.800
## 208 Prof B 18 18 Male 120.000
## 198 AsstProf B 4 4 Male 92.000
## 210 Prof B 20 20 Male 163.200
## 257 Prof A 22 22 Male 140.300
## 366 Prof A 43 40 Male 101.036
## 82 Prof B 17 16 Male 135.585
## 276 Prof A 12 6 Male 93.000
## 393 Prof A 33 30 Male 103.106
## 27 Prof A 35 23 Male 134.885
## 332 Prof B 20 9 Male 116.518
## 297 Prof A 18 18 Male 126.300
## 394 Prof A 31 19 Male 150.564
## 365 Prof A 43 43 Male 205.500
## 53 AsstProf B 11 3 Female 74.692
## 355 AsstProf B 8 1 Male 83.600
## 228 AssocProf A 9 7 Male 70.000
## 87 Prof B 37 37 Male 152.708
## 164 AsstProf B 3 3 Male 89.942
## 39 Prof B 41 31 Male 125.196
## 149 Prof B 36 26 Female 144.651
## 73 Prof B 29 19 Male 100.131
## 287 Prof A 28 27 Male 115.800
## 141 AssocProf A 14 8 Male 100.102
## 143 Prof A 19 11 Male 106.608
## 175 AssocProf B 17 6 Male 105.000
## 117 Prof A 30 29 Male 148.500
## 10 Prof B 18 18 Female 129.000
## 364 AssocProf A 20 17 Male 81.285
## 286 AssocProf A 49 49 Male 81.800
## 74 Prof B 35 34 Male 92.391
## 152 AsstProf B 4 4 Male 92.000
## 19 Prof A 37 23 Male 124.750
## 64 AssocProf B 11 11 Female 103.613
## 242 Prof A 31 30 Male 122.875
## 227 AsstProf A 3 1 Male 63.900
## 335 AssocProf B 19 6 Female 104.542
## 271 Prof A 42 40 Male 143.250
## 43 Prof B 40 27 Male 101.299
## 72 Prof B 45 45 Male 146.856
## 336 Prof B 36 38 Male 151.445
## 236 Prof A 28 17 Male 81.700
## 55 AssocProf B 12 11 Male 103.760
## 142 AssocProf A 15 10 Male 81.500
## 103 Prof B 16 5 Male 153.303
## 32 AsstProf B 7 2 Male 79.916
## 165 AsstProf B 1 0 Male 88.795
## 389 Prof A 38 36 Male 119.450
## 222 Prof B 23 10 Male 145.200
## 338 Prof B 13 12 Male 145.000
## 354 Prof B 22 20 Male 138.000
## 230 Prof A 39 38 Male 133.900
## 321 Prof B 24 23 Male 104.428
## 383 AssocProf A 8 5 Male 86.895
## 20 Prof A 39 36 Female 137.000
## 56 AssocProf B 14 5 Male 83.900
## 115 Prof A 12 0 Female 105.000
## 120 AsstProf A 5 3 Female 73.500
## 377 AsstProf A 4 1 Male 74.856
## 86 Prof B 15 14 Male 132.825
## 17 Prof B 19 20 Male 101.000
## 382 Prof A 27 23 Male 172.505
## 127 Prof A 28 26 Male 155.500
## 93 AssocProf B 10 7 Male 105.631
## 333 Prof B 18 10 Female 105.450
## 200 Prof B 38 22 Male 114.500
## 293 Prof A 39 9 Male 183.800
## 329 AssocProf B 10 10 Male 99.247
## 256 AssocProf A 12 8 Male 83.000
## 110 Prof A 40 31 Male 131.205
## 362 Prof A 23 15 Female 109.646
## 369 Prof A 35 30 Male 131.950
## 356 Prof B 25 21 Male 145.028
## 380 AssocProf A 11 8 Male 104.121
## 97 AssocProf B 17 12 Male 95.611
## 163 AssocProf B 22 7 Male 98.510
## 296 Prof A 40 36 Male 97.150
## 196 AssocProf B 9 7 Male 113.600
## 192 Prof B 43 22 Male 133.700
## 372 Prof A 23 20 Male 110.515
## 96 AsstProf B 4 0 Male 84.000
## 2 Prof B 20 16 Male 173.200
## 37 Prof B 22 21 Male 155.750
## 302 Prof A 27 16 Male 127.100
## 385 Prof A 27 21 Male 125.192
## 249 Prof A 28 23 Male 128.800
## 111 Prof A 20 16 Male 112.429
## 378 AsstProf A 6 3 Male 77.081
## 373 Prof A 12 7 Male 109.707
## 159 AssocProf B 6 6 Male 95.408
## 191 Prof B 22 9 Male 180.000
## 268 Prof A 14 10 Male 104.350
## 294 AssocProf A 11 1 Male 104.800
## 232 AssocProf A 26 24 Female 73.300
## 150 AsstProf B 4 3 Male 95.079
## 334 Prof B 33 19 Male 145.098
## 21 Prof A 31 26 Male 89.565
## 339 Prof B 32 25 Male 128.464
## 69 Prof B 17 17 Female 111.512
## 70 Prof B 28 36 Male 91.412
## 300 AssocProf A 45 39 Male 70.700
## 283 Prof A 51 51 Male 57.800
## 84 AsstProf B 6 2 Male 88.825
## 34 AsstProf B 4 2 Male 80.225
## 312 Prof A 14 9 Male 108.100
## 11 AssocProf B 12 8 Male 119.800
## 85 Prof B 17 18 Female 122.960
## 33 Prof B 13 9 Male 117.256
## 186 Prof B 33 30 Male 134.000
## 190 Prof B 25 19 Male 153.750
## 95 Prof B 21 20 Male 123.683
## 78 Prof B 26 19 Male 193.000
## 284 Prof A 45 43 Male 155.865
## 248 Prof A 21 18 Male 101.100
## 392 Prof A 30 19 Male 151.292
## 368 AssocProf A 10 1 Male 108.413
## 214 Prof B 26 19 Male 126.200
## 289 Prof A 29 27 Male 150.500
## 316 AsstProf B 6 3 Male 84.716
## 90 AssocProf B 9 7 Male 107.008
## 357 Prof A 49 40 Male 88.709
## 201 AsstProf B 4 4 Male 92.700
## 311 Prof A 20 7 Male 92.050
## 60 AsstProf B 8 3 Male 75.044
## 113 AsstProf A 3 1 Male 72.500
## 14 AsstProf B 2 0 Male 78.000
## 134 AsstProf A 3 1 Female 72.500
## 182 Prof B 18 5 Male 141.136
## 390 Prof A 33 18 Male 186.023
## 102 Prof B 28 23 Male 126.933
## 119 AsstProf A 4 1 Male 72.500
## 13 AsstProf B 1 1 Male 77.700
## 83 Prof B 22 20 Male 144.640
## 203 Prof B 28 17 Male 160.400
## 237 Prof A 25 25 Male 114.000
## 258 AssocProf A 30 23 Male 74.000
## 272 Prof A 42 18 Male 194.800
## 174 Prof B 20 20 Male 134.185Salary_test <- Salary[-training_index, ]
Salary_test## rank discipline yrs.since.phd yrs.service sex salary
## 7 Prof B 30 23 Male 175.000
## 9 Prof B 21 20 Male 119.250
## 18 Prof A 38 34 Male 103.450
## 22 Prof A 36 31 Male 102.580
## 24 Prof A 24 19 Male 113.068
## 25 AssocProf A 13 8 Female 74.830
## 38 AsstProf B 7 4 Male 86.373
## 45 Prof B 19 19 Male 94.384
## 46 Prof B 25 15 Male 114.778
## 49 Prof B 25 25 Female 140.096
## 57 Prof B 23 21 Male 117.704
## 61 AssocProf B 9 8 Male 90.304
## 63 Prof B 33 31 Male 109.785
## 67 Prof B 22 12 Male 101.000
## 68 Prof B 35 31 Male 99.418
## 77 Prof B 17 3 Male 150.480
## 81 Prof B 43 28 Male 150.743
## 88 AsstProf B 2 2 Male 88.400
## 98 Prof B 13 7 Male 129.676
## 100 Prof B 41 26 Male 106.689
## 101 Prof B 42 25 Male 133.217
## 109 AssocProf A 15 8 Male 81.500
## 112 AssocProf A 19 16 Male 82.100
## 121 Prof A 14 14 Male 115.313
## 123 Prof A 24 22 Male 97.262
## 137 Prof A 16 14 Male 108.262
## 162 Prof B 26 19 Male 176.500
## 167 Prof B 16 16 Male 167.284
## 168 Prof B 18 19 Male 130.664
## 173 Prof B 37 24 Male 93.164
## 179 Prof B 27 14 Male 147.349
## 183 AssocProf B 8 8 Male 100.000
## 188 Prof B 18 10 Male 107.500
## 195 AssocProf B 48 53 Male 90.000
## 197 AsstProf B 4 4 Male 92.700
## 204 Prof B 17 17 Male 152.500
## 205 Prof B 19 5 Male 165.000
## 215 AssocProf B 11 1 Male 118.700
## 217 Prof B 15 11 Male 146.000
## 218 AssocProf B 29 22 Male 105.350
## 220 Prof B 13 11 Male 119.500
## 223 AssocProf B 13 6 Male 107.150
## 231 Prof A 29 27 Female 91.000
## 239 Prof A 46 40 Male 77.202
## 243 Prof A 38 37 Male 102.600
## 244 Prof A 23 23 Male 108.200
## 245 Prof A 19 23 Male 84.273
## 247 Prof A 30 23 Male 91.100
## 251 Prof A 39 39 Male 109.000
## 253 Prof A 31 12 Male 132.000
## 254 AsstProf A 4 2 Female 77.500
## 264 Prof A 31 31 Male 126.000
## 269 Prof A 47 44 Male 89.650
## 277 Prof A 52 48 Male 107.200
## 280 Prof A 46 46 Male 100.600
## 282 Prof A 37 27 Male 103.600
## 288 AsstProf A 2 0 Male 85.000
## 290 AsstProf A 8 5 Male 74.000
## 292 Prof A 32 28 Male 168.500
## 299 Prof A 49 43 Male 72.300
## 303 Prof A 28 13 Male 170.500
## 307 AsstProf A 7 4 Male 74.500
## 310 Prof A 22 15 Male 166.800
## 313 Prof A 29 19 Male 94.350
## 315 Prof A 22 6 Male 146.800
## 320 Prof B 16 15 Male 135.027
## 337 Prof B 35 23 Male 98.053
## 341 Prof B 13 11 Male 106.231
## 344 Prof B 31 31 Male 162.150
## 346 Prof B 15 10 Male 107.986
## 351 Prof B 56 49 Male 186.960
## 358 Prof A 39 35 Male 107.309
## 359 Prof A 28 14 Female 109.954
## 370 Prof A 33 31 Male 134.690
## 376 Prof A 28 26 Male 103.649
## 381 AsstProf A 8 3 Male 75.996
## 387 Prof A 29 27 Male 139.219
## 388 Prof A 29 15 Male 109.305
## 397 AsstProf A 8 4 Male 81.035- Using ‘salary’ as the response variable, and based on the
training data, fit a full (linear) model (using all the
other variables as predictors), and name it as
model_full; fit a null (linear) model (no predictor, only an intercept), and name it asmodel_null. Display the summary of both the models. (1 points)
model_full <- lm(salary ~ ., data = Salary_train)
summary(model_full)##
## Call:
## lm(formula = salary ~ ., data = Salary_train)
##
## Residuals:
## Min 1Q Median 3Q Max
## -68.809 -13.091 -1.886 9.832 96.874
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 78.0937 5.3963 14.472 < 2e-16 ***
## rankAsstProf -12.7503 4.4293 -2.879 0.00427 **
## rankProf 32.0951 3.8978 8.234 5.06e-15 ***
## disciplineB 13.9132 2.6141 5.322 1.96e-07 ***
## yrs.since.phd 0.5790 0.2697 2.147 0.03256 *
## yrs.service -0.4192 0.2329 -1.800 0.07278 .
## sexMale 4.4979 4.1136 1.093 0.27506
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 22.32 on 311 degrees of freedom
## Multiple R-squared: 0.4817, Adjusted R-squared: 0.4717
## F-statistic: 48.17 on 6 and 311 DF, p-value: < 2.2e-16model_null <- lm(salary ~ 1, data = Salary_train)
summary(model_null)##
## Call:
## lm(formula = salary ~ 1, data = Salary_train)
##
## Residuals:
## Min 1Q Median 3Q Max
## -55.804 -23.331 -6.404 20.854 117.941
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 113.604 1.722 65.98 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 30.7 on 317 degrees of freedom- Interpret the coefficient of
discipline, and the coefficient ofyrs.service. What do they tell us about the relationship between these predictors andsalary? (1 points)
Discipline: for every 1 unit increase in discipline, the salary will go up by 13.92 thousand (since the numbers are in thousands) Yrs.Service: for every 1 year of service added on, the salary will decrease by .5283 thousand dollars
- Conduct a stepwise variable selection using BIC, and name the
selected model as
model_step_BIC. Which variables are selected in the final model? (1 points)
model_step_BIC <- step(model_null, scope = list(lower = model_null, upper = model_full),
direction = "both", trace = F, k = log(nrow(Salary_train)))
model_step_BIC##
## Call:
## lm(formula = salary ~ rank + discipline, data = Salary_train)
##
## Coefficients:
## (Intercept) rankAsstProf rankProf disciplineB
## 86.39 -14.53 35.35 12.70The variables used in the final model are salary (response variable), then rank and discipline as predictor variables.
- Calculate the out-of-sample MSE with
model_fullandmodel_step_BIC. Based on this results, which model performs better in prediction? (1 points)
out_MSE <- predict(model_full, newdata = Salary_test)
out_MSE## 7 9 18 22 24 25 38 45
## 136.32745 132.37421 122.43468 122.53437 120.61717 82.26676 86.13046 131.63546
## 46 49 57 61 63 67 68 77
## 136.78628 128.09620 133.11297 98.36189 134.71063 136.30698 135.86861 137.18504
## 81 88 98 100 101 109 112 121
## 141.75819 84.07396 133.19219 141.43866 142.43687 87.92260 86.88477 116.92339
## 123 137 162 167 168 173 179 183
## 119.35950 118.08137 135.68838 131.15616 131.05647 139.96115 138.36348 97.78290
## 188 195 197 204 205 215 217 218
## 134.82947 102.07744 84.39350 131.31592 137.50458 102.45443 132.67328 104.07255
## 220 223 231 239 243 244 245 247
## 131.51530 101.51629 115.66048 124.55125 121.17701 118.36129 116.04533 122.41421
## 251 253 254 264 269 277 280 282
## 120.91755 127.60466 66.82086 119.63942 123.45335 124.67141 122.03592 124.79025
## 288 290 292 299 303 307 310 313
## 70.99918 72.37700 121.47608 125.03055 125.44847 72.21723 121.13608 123.51212
## 315 320 337 341 344 346 351 358
## 124.90909 131.57538 139.22239 131.51530 133.55265 133.09250 140.48137 122.59444
## 359 370 376 381 387 388 397
## 120.53139 120.79740 119.99857 73.21545 120.15833 125.18901 72.79622mean((Salary$salary - out_MSE)^2)## Warning in Salary$salary - out_MSE: longer object length is not a multiple of
## shorter object length## [1] 1286.308out_MSE_BIC <- predict(model_step_BIC, newdata = Salary_test)
out_MSE_BIC## 7 9 18 22 24 25 38 45
## 134.44890 134.44890 121.74422 121.74422 121.74422 86.39288 84.57192 134.44890
## 46 49 57 61 63 67 68 77
## 134.44890 134.44890 134.44890 99.09756 134.44890 134.44890 134.44890 134.44890
## 81 88 98 100 101 109 112 121
## 134.44890 84.57192 134.44890 134.44890 134.44890 86.39288 86.39288 121.74422
## 123 137 162 167 168 173 179 183
## 121.74422 121.74422 134.44890 134.44890 134.44890 134.44890 134.44890 99.09756
## 188 195 197 204 205 215 217 218
## 134.44890 99.09756 84.57192 134.44890 134.44890 99.09756 134.44890 99.09756
## 220 223 231 239 243 244 245 247
## 134.44890 99.09756 121.74422 121.74422 121.74422 121.74422 121.74422 121.74422
## 251 253 254 264 269 277 280 282
## 121.74422 121.74422 71.86724 121.74422 121.74422 121.74422 121.74422 121.74422
## 288 290 292 299 303 307 310 313
## 71.86724 71.86724 121.74422 121.74422 121.74422 71.86724 121.74422 121.74422
## 315 320 337 341 344 346 351 358
## 121.74422 134.44890 134.44890 134.44890 134.44890 134.44890 134.44890 121.74422
## 359 370 376 381 387 388 397
## 121.74422 121.74422 121.74422 71.86724 121.74422 121.74422 71.86724mean((Salary$salary - out_MSE_BIC)^2)## Warning in Salary$salary - out_MSE_BIC: longer object length is not a multiple
## of shorter object length## [1] 1260.251model_full out-of-sample: 1343.771 model_step_BIC out-of-sample: 1326.16
The model_step_BIC performs better in prediction than the model_full
End of Exam. Please submit this RMD file along with a knitted HTML report.