Assignment-1

0.1 Introduction

In this assignment, we are tasked to explore, analyze and model a major league baseball dataset which contains around 2000 records where each record presents a baseball team from 1871 to 2006. Each observation provides the perforamce of the team for that particular year with all the statistics for the performance of 162 game season. The problem statement for the main objective is that “Can we predict the number of wins for the team with the given attributes of each record?”. In order to provide a solution for the problem, our goal is to build a linear regression model on the training data that creates this prediction.

0.1.1 About the Data

The data set are provided in csv format as moneyball-evaluation-data and moneyball-training-data where we will explore, preperate and create our model with the training data and further test the model with the evaluation data. Below is short description of the variables within the datasets.

**INDEX: Identification Variable(Do not use)

**TARGET_WINS: Number of wins

**TEAM_BATTING_H : Base Hits by batters (1B,2B,3B,HR)

**TEAM_BATTING_2B: Doubles by batters (2B)

**TEAM_BATTING_3B: Triples by batters (3B)

**TEAM_BATTING_HR: Homeruns by batters (4B)

**TEAM_BATTING_BB: Walks by batters

**TEAM_BATTING_HBP: Batters hit by pitch (get a free base)

**TEAM_BATTING_SO: Strikeouts by batters

**TEAM_BASERUN_SB: Stolen bases

**TEAM_BASERUN_CS: Caught stealing

**TEAM_FIELDING_E: Errors

**TEAM_FIELDING_DP: Double Plays

**TEAM_PITCHING_BB: Walks allowed

**TEAM_PITCHING_H: Hits allowed

**TEAM_PITCHING_HR: Homeruns allowed

**TEAM_PITCHING_SO: Strikeouts by pitchers

0.2 Data Exploration

0.2.1 Descriptive Statistics

# load libraries
library(ggplot2)
library(ggcorrplot)
library(psych)

## 
## Attaching package: 'psych'

## The following objects are masked from 'package:ggplot2':
## 
##     %+%, alpha

library(statsr)

## Warning: package 'statsr' was built under R version 3.6.2

## Loading required package: BayesFactor

## Warning: package 'BayesFactor' was built under R version 3.6.2

## Loading required package: coda

## Loading required package: Matrix

## ************
## Welcome to BayesFactor 0.9.12-4.2. If you have questions, please contact Richard Morey (richarddmorey@gmail.com).
## 
## Type BFManual() to open the manual.
## ************

library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(PerformanceAnalytics)

## Warning: package 'PerformanceAnalytics' was built under R version 3.6.2

## Loading required package: xts

## Warning: package 'xts' was built under R version 3.6.2

## Loading required package: zoo

## 
## Attaching package: 'zoo'

## The following objects are masked from 'package:base':
## 
##     as.Date, as.Date.numeric

## 
## Attaching package: 'xts'

## The following objects are masked from 'package:dplyr':
## 
##     first, last

## 
## Attaching package: 'PerformanceAnalytics'

## The following object is masked from 'package:graphics':
## 
##     legend

library(tidyr)

## 
## Attaching package: 'tidyr'

## The following objects are masked from 'package:Matrix':
## 
##     expand, pack, unpack

library(reshape2)

## 
## Attaching package: 'reshape2'

## The following object is masked from 'package:tidyr':
## 
##     smiths

library(rcompanion)

## Warning: package 'rcompanion' was built under R version 3.6.2

## 
## Attaching package: 'rcompanion'

## The following object is masked from 'package:psych':
## 
##     phi

library(caret)

## Loading required package: lattice

library(MASS)

## 
## Attaching package: 'MASS'

## The following object is masked from 'package:dplyr':
## 
##     select

library(imputeTS)

## Warning: package 'imputeTS' was built under R version 3.6.2

## Registered S3 method overwritten by 'quantmod':
##   method            from
##   as.zoo.data.frame zoo

## 
## Attaching package: 'imputeTS'

## The following object is masked from 'package:zoo':
## 
##     na.locf

library(rsample)

## Warning: package 'rsample' was built under R version 3.6.2

library(huxtable)

## Warning: package 'huxtable' was built under R version 3.6.2

## 
## Attaching package: 'huxtable'

## The following object is masked from 'package:dplyr':
## 
##     add_rownames

## The following object is masked from 'package:ggplot2':
## 
##     theme_grey

library(glmnet)

## Loaded glmnet 3.0-1

## 
## Attaching package: 'glmnet'

## The following object is masked from 'package:imputeTS':
## 
##     na.replace

library(sjPlot)

## Warning: package 'sjPlot' was built under R version 3.6.2

## 
## Attaching package: 'sjPlot'

## The following object is masked from 'package:huxtable':
## 
##     font_size

library(modelr)

## 
## Attaching package: 'modelr'

## The following object is masked from 'package:psych':
## 
##     heights

# Load data sets

baseball_eva <- read.csv("https://raw.githubusercontent.com/anilak1978/data621/master/moneyball-evaluation-data.csv")
baseball_train <- read.csv("https://raw.githubusercontent.com/anilak1978/data621/master/moneyball-training-data.csv")

We can start exploring our training data set by looking at basic descriptive statistics.

# look at training dataset structure
str(baseball_train)

## 'data.frame':    2276 obs. of  17 variables:
##  $ INDEX           : int  1 2 3 4 5 6 7 8 11 12 ...
##  $ TARGET_WINS     : int  39 70 86 70 82 75 80 85 86 76 ...
##  $ TEAM_BATTING_H  : int  1445 1339 1377 1387 1297 1279 1244 1273 1391 1271 ...
##  $ TEAM_BATTING_2B : int  194 219 232 209 186 200 179 171 197 213 ...
##  $ TEAM_BATTING_3B : int  39 22 35 38 27 36 54 37 40 18 ...
##  $ TEAM_BATTING_HR : int  13 190 137 96 102 92 122 115 114 96 ...
##  $ TEAM_BATTING_BB : int  143 685 602 451 472 443 525 456 447 441 ...
##  $ TEAM_BATTING_SO : int  842 1075 917 922 920 973 1062 1027 922 827 ...
##  $ TEAM_BASERUN_SB : int  NA 37 46 43 49 107 80 40 69 72 ...
##  $ TEAM_BASERUN_CS : int  NA 28 27 30 39 59 54 36 27 34 ...
##  $ TEAM_BATTING_HBP: int  NA NA NA NA NA NA NA NA NA NA ...
##  $ TEAM_PITCHING_H : int  9364 1347 1377 1396 1297 1279 1244 1281 1391 1271 ...
##  $ TEAM_PITCHING_HR: int  84 191 137 97 102 92 122 116 114 96 ...
##  $ TEAM_PITCHING_BB: int  927 689 602 454 472 443 525 459 447 441 ...
##  $ TEAM_PITCHING_SO: int  5456 1082 917 928 920 973 1062 1033 922 827 ...
##  $ TEAM_FIELDING_E : int  1011 193 175 164 138 123 136 112 127 131 ...
##  $ TEAM_FIELDING_DP: int  NA 155 153 156 168 149 186 136 169 159 ...

We have 2276 observations and 17 variables. All of our variables are integer type as expected.

# look at descriptive statistics
metastats <- data.frame(describe(baseball_train))
metastats <- tibble::rownames_to_column(metastats, "STATS")
metastats["pct_missing"] <- round(metastats["n"]/2276, 3)
head(metastats)

STATS	vars	n	mean	sd	median	trimmed	mad	min	max	range	skew	kurtosis	se	pct_missing
INDEX	1	2.28e+03	1.27e+03	736	1.27e+03	1.27e+03	953	1	2.54e+03	2.53e+03	0.00421	-1.22	15.4	1
TARGET_WINS	2	2.28e+03	80.8	15.8	82	81.3	14.8	0	146	146	-0.399	1.03	0.33	1
TEAM_BATTING_H	3	2.28e+03	1.47e+03	145	1.45e+03	1.46e+03	114	891	2.55e+03	1.66e+03	1.57	7.28	3.03	1
TEAM_BATTING_2B	4	2.28e+03	241	46.8	238	240	47.4	69	458	389	0.215	0.00616	0.981	1
TEAM_BATTING_3B	5	2.28e+03	55.2	27.9	47	52.2	23.7	0	223	223	1.11	1.5	0.586	1
TEAM_BATTING_HR	6	2.28e+03	99.6	60.5	102	97.4	78.6	0	264	264	0.186	-0.963	1.27	1

With the descriptive statistics, we are able to see mean, standard deviation, median, min, max values and percentage of each missing value of each variable. For example, when we look at TEAM_BATTING_H, we see that average 1469 Base hits by batters, with standard deviation of 144, median of 1454 with maximum base hits of 2554.

# Look for missing values
colSums(is.na(baseball_train))

##            INDEX      TARGET_WINS   TEAM_BATTING_H  TEAM_BATTING_2B 
##                0                0                0                0 
##  TEAM_BATTING_3B  TEAM_BATTING_HR  TEAM_BATTING_BB  TEAM_BATTING_SO 
##                0                0                0              102 
##  TEAM_BASERUN_SB  TEAM_BASERUN_CS TEAM_BATTING_HBP  TEAM_PITCHING_H 
##              131              772             2085                0 
## TEAM_PITCHING_HR TEAM_PITCHING_BB TEAM_PITCHING_SO  TEAM_FIELDING_E 
##                0                0              102                0 
## TEAM_FIELDING_DP 
##              286

# Percentage of missing values
missing_values <- metastats %>%
  filter(pct_missing < 1) %>%
  dplyr::select(STATS, pct_missing) %>%
  arrange(pct_missing)
missing_values

STATS	pct_missing
TEAM_BATTING_HBP	0.084
TEAM_BASERUN_CS	0.661
TEAM_FIELDING_DP	0.874
TEAM_BASERUN_SB	0.942
TEAM_BATTING_SO	0.955
TEAM_PITCHING_SO	0.955

When we look at the missing values within the training data set, we see that proportionaly against the total observations, TEAM_BATTING_HBP and TEAM_BESARUN_CS variables have the most missing values. We will be handling these missing values in our Data Preperation section.

0.2.2 Correlation and Distribution

# Look at correlation between variables

corr <- round(cor(baseball_train), 1)

ggcorrplot(corr,
           type="lower",
           lab=TRUE,
           lab_size=3,
           method="circle",
           colors=c("tomato2", "white", "springgreen3"),
           title="Correlation of variables in Training Data Set",
           ggtheme=theme_bw)

Team_Batting_H and Team_Batting_2B have the strongest positive correlation with Target_Wins. We also see that, there is a strong correlation between Team_Batting_H and Team_Batting_2B, Team_Pitching_B and TEAM_FIELDING_E. We will consider these findings on model creation as collinearity might complicate model estimation and we want to have explanotry variables to be independent from each other. We will try to avoid adding explanotry variables that are correlated to each other.

Let’s look at the correlations and distribution of the variables in more detail.

# Look at correlation from batting, baserunning, pitching and fielding perspective
Batting_df <- baseball_train[c(2:7, 10)] 
BaseRunning_df <- baseball_train[c(8:9)] 
Pitching_df <- baseball_train[c(11:14)] 
Fielding_df <- baseball_train[c(15:16)]

0.2.2.1 Batting

# Batting Correlations
chart.Correlation(Batting_df, histogram=TRUE, pch=19)

We can see that our response variable TARGET_WINS, TEAM_BATTING_H, TEAM_BATTING_2B, TEAM_BATTING_BB and TEAM_BASERUN_CS are normaly distributed. TEAM_BATTING_HR on the other hand is bimodal.

0.2.2.2 Baserunning

# baserunning Correlation

chart.Correlation(BaseRunning_df, histogram=TRUE, pch=19)

TEAM_BASERUN_SB is right skewed and TEAM_BATTING_SO is bimodal.

0.2.2.3 Pitching

#pitching correlations
chart.Correlation(Pitching_df, histogram=TRUE, pch=19)

TEAM_BATTING_HBP seems to be normally distributed however we shouldnt forget that we have a lot of missing values in this variable.

# fielding correlations
chart.Correlation(Fielding_df, histogram=TRUE, pch=19)

Let’s also look at the outliers and skewness for each varibale.

0.2.3 Outliers and Skewness

par(mfrow=c(3,3))
datasub_1 <- melt(baseball_train)

## No id variables; using all as measure variables

suppressWarnings(ggplot(datasub_1, aes(x= "value", y=value)) + 
                   geom_boxplot(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

## Warning: Removed 3478 rows containing non-finite values (stat_boxplot).

Based on the boxplot we created, TEAM_FIELDING_DP, TEAM_PITCHING_HR, TEAM_BATTING_HR and TEAM_BATTING_SO seem to have the least amount of outliers.

par(mfrow = c(3, 3))
datasub = melt(baseball_train) 

## No id variables; using all as measure variables

suppressWarnings(ggplot(datasub, aes(x= value)) + 
                   geom_density(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

## Warning: Removed 3478 rows containing non-finite values (stat_density).

metastats %>%
  filter(skew > 1) %>%
  dplyr::select(STATS, skew) %>%
  arrange(desc(skew))

STATS	skew
TEAM_PITCHING_SO	22.2
TEAM_PITCHING_H	10.3
TEAM_PITCHING_BB	6.74
TEAM_FIELDING_E	2.99
TEAM_BASERUN_CS	1.98
TEAM_BASERUN_SB	1.97
TEAM_BATTING_H	1.57
TEAM_BATTING_3B	1.11

We can see that the most skewed variable is TEAM_PITCHING_SO. We will correct the skewed variables in our data preperation section.

When we are creating a linear regression model, we are looking for the fitting line with the least sum of squares, that has the small residuals with minimized squared residuals. From our correlation analysis, we can see that the explatory variable that has the strongest correlation with TARGET_WINS is TEAM_BATTING_H. Let’s look at a simple model example to further expand our explaroty analysis.

0.2.4 Simple Model Example

# line that follows the best assocation between two variables

plot_ss(x = TEAM_BATTING_H, y = TARGET_WINS, data=baseball_train, showSquares = TRUE, leastSquares = TRUE)

## 
                                
## Call:
## lm(formula = y ~ x, data = data)
## 
## Coefficients:
## (Intercept)            x  
##    18.56233      0.04235  
## 
## Sum of Squares:  479178.4

When we are exploring to build a linear regression, one of the first thing we do is to create a scatter plot of the response and explanatory variable.

# scatter plot between TEAM_BATTING_H and TARGET_WINS

ggplot(baseball_train, aes(x=TEAM_BATTING_H, y=TARGET_WINS))+
  geom_point()

One of the conditions for least square lines or linear regression are Linearity. From the scatter plot between TEAM_BATTING_H and TARGET_WINS, we can see this condition is met. We can also create a scatterplot that shows the data points between TARGET_WINS and each variable.

baseball_train %>%
  gather(var, val, -TARGET_WINS) %>%
  ggplot(., aes(val, TARGET_WINS))+
  geom_point()+
  facet_wrap(~var, scales="free", ncol=4)

## Warning: Removed 3478 rows containing missing values (geom_point).

As we displayed earlier, hits walks and home runs have the strongest correlations with TARGET_WINS and also meets the linearity condition.

# create a simple example model
lm_sm <- lm(baseball_train$TARGET_WINS ~ baseball_train$TEAM_BATTING_H)
summary(lm_sm)

## 
## Call:
## lm(formula = baseball_train$TARGET_WINS ~ baseball_train$TEAM_BATTING_H)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -71.768  -8.757   0.856   9.762  46.016 
## 
## Coefficients:
##                                Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                   18.562326   3.107523   5.973 2.69e-09 ***
## baseball_train$TEAM_BATTING_H  0.042353   0.002105  20.122  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 14.52 on 2274 degrees of freedom
## Multiple R-squared:  0.1511, Adjusted R-squared:  0.1508 
## F-statistic: 404.9 on 1 and 2274 DF,  p-value: < 2.2e-16

TARET_BATTING_H has the strongest correlation with TARGET_WINS response variable, however when we create a simple model just using TARGET_BATTING_H, we can only explain 15% of the variablity. (Adjusted R-squared: 0.1508). The remainder of the varibility can be explained with selected other variables within the training dataset.

#histogram of residuals for the simple model
hist(lm_sm$residuals)

# check for constant variability (honoscedasticity)

plot(lm_sm$residuals ~ baseball_train$TEAM_BATTING_H)

We do see that the residuals are distributed normally and variability around the regression line is roughly constant.

Based on our explatory analysis, we were able to see the correlation level between the possible explanatory variables and repsonse variable TARGET_WINS. Some of the variables such as TARGET_BATTING_H has somewhat strong positive correlation, however some of the variables such as TEAM_PITCHING_BB has weak positive relationship with TARGET_WINS. We also found out, hit by the pitcher(TEAM_BATTING_HBP) and caught stealing (TEAM_BASERUN_CS) variables are missing majority of the values. Skewness and distribution analysis gave us the insights that we have some variables that are right-tailed. Considering all of these insights, we will handle missing values, correct skewness and outliers and select our explaratory variables based on correlation in order to create our regression model.

0.3 Data Preparation

0.3.1 Objective

In this section, we will prepare the dataset for linear regression modeling. We accomplish this by handling missing values and outliers and by tranforming the data into more normal distributions. This section covers:

Identify and Handle Missing Data Correct Outliers *Adjust Skewed value - Box Cox Transformation

First, we will start by copying the dataset into a new variable, baseball_train_01, and we will remove the Index variable from the new dataset as well. We will now have 16 variables.

baseball_train_01 <- baseball_train

baseball_train_01 <-subset(baseball_train_01, select = -c(INDEX))

0.3.2 Identify and Handle Missing Data

0.3.2.1 Removal of Sparsely Populated Variables - MCAR

In the Data Exploration section, we identified these variables as having missing data values.The table below lists the variables with missing data. The variable, TEAM_BATTING_HBP, is sparsely populated. Since this data is Missing Completely at Random (MCAR) and is not related to any other variable, it is safe to completely remove the variable from the dataset.

missing_values

STATS	pct_missing
TEAM_BATTING_HBP	0.084
TEAM_BASERUN_CS	0.661
TEAM_FIELDING_DP	0.874
TEAM_BASERUN_SB	0.942
TEAM_BATTING_SO	0.955
TEAM_PITCHING_SO	0.955

baseball_train_01 <-subset(baseball_train_01, select = -c(TEAM_BATTING_HBP))

There are now 15 variables.

dim(baseball_train_01)

## [1] 2276   15

0.3.2.2 Imputation of Missing Values

For the remaining variables with missing values, we will impute the mean of the variable. The function, “na_mean” updates all missing values with the mean of the variable.

baseball_train_01 <- na_mean(baseball_train_01, option = "mean")  

Re-running the metastats dataframe on the new baseball_train_01 dataset shows that there are no missing values.

# look at descriptive statistics
metastats <- data.frame(describe(baseball_train_01))
metastats <- tibble::rownames_to_column(metastats, "STATS")
metastats["pct_missing"] <- round(metastats["n"]/2276, 3)

# Percentage of missing values
missing_values2 <- metastats %>%
  filter(pct_missing < 1) %>%
  dplyr::select(STATS, pct_missing) %>%
  arrange(pct_missing)
missing_values2

## Warning in max(nchar(as.character(col), type = "width")): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(col), type = "width")): no non-missing
## arguments to max; returning -Inf

STATS

pct_missing

0.3.3 Correct Outliers

In this section, we created two functions that can identify outliers. The funcion, Identify_Outlier, uses the Turkey method, where outliers are identified by being below Q1-1.5IQR and above Q3+1.5IQR. The second function, tag_outlier, returns a binary list of values, “Acceptable” or “Outlier” that will be added to the dataframe.

Identify_Outlier <- function(value){

    interquartile_range = IQR(sort(value),na.rm = TRUE)
    q1 = matrix(c(quantile(sort(value),na.rm = TRUE)))[2]
    q3 = matrix(c(quantile(sort(value),na.rm = TRUE)))[4]
    lower = q1-(1.5*interquartile_range)
    upper = q3+(1.5*interquartile_range)
    
    bound <- c(lower, upper)
    
    return (bound)
}

tag_outlier <- function(value) {
    
   boundaries <- Identify_Outlier(value)
   tags <- c()
   counter = 1
    for (i in as.numeric(value))
    {

        if (i >= boundaries[1] & i <= boundaries[2]){
          tags[counter] <- "Acceptable"
        } else{
          tags[counter] <- "Outlier"
        }
      
      counter = counter +1
    }
   
   return (tags)
}

As seen in the box plots from the previous section, “TEAM_BASERUN_SB”, “TEAM_BASERUN_CS”, “TEAM_PITCHING_H”, “TEAM_PITCHING_BB”, “TEAM_PITCHING_SO”, and “TEAM_FIELDING_E” all have a high number of outliers. We will use the two functions above to tag those rows with extreme outliers.

tags<- tag_outlier(baseball_train_01$TEAM_BASERUN_SB)
baseball_train_01$TEAM_BASERUN_SB_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_BASERUN_CS)
baseball_train_01$TEAM_BASERUN_CS_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_PITCHING_H)
baseball_train_01$TEAM_PITCHING_H_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_PITCHING_BB)
baseball_train_01$TEAM_PITCHING_BB_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_PITCHING_SO)
baseball_train_01$TEAM_PITCHING_SO_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_FIELDING_E)
baseball_train_01$TEAM_FIELDING_E_Outlier <- tags

Below, we filtered out all of the outliers and created a new dataframe, baseball_train_02

baseball_train_02 <- baseball_train_01 %>%
                filter(
                        TEAM_BASERUN_SB_Outlier != "Outlier" &
                        TEAM_BASERUN_CS_Outlier != "Outlier" &
                        TEAM_PITCHING_H_Outlier != "Outlier" &
                        TEAM_PITCHING_BB_Outlier != "Outlier" &
                        TEAM_PITCHING_SO_Outlier != "Outlier" &
                        TEAM_FIELDING_E_Outlier != "Outlier"
                )

Re-running the boxplots show data that has a better normal distribution except for the variable, TEAM_FIELDING_E which is still skewed. We will handle this next.

par(mfrow=c(3,3))
datasub_1 <- melt(baseball_train_02)

## Using TEAM_BASERUN_SB_Outlier, TEAM_BASERUN_CS_Outlier, TEAM_PITCHING_H_Outlier, TEAM_PITCHING_BB_Outlier, TEAM_PITCHING_SO_Outlier, TEAM_FIELDING_E_Outlier as id variables

suppressWarnings(ggplot(datasub_1, aes(x= "value", y=value)) + 
                   geom_boxplot(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

0.3.4 Adjust Skewed values

0.3.4.1 Box Cox Transformation

Removing the outliers tranformed each variable to a closer to a normal distribution and checking the skewness of the variables confirm this with the exception of TEAM_FIELDING_E. This variable is still skewed and not normal. In this section, we will use the Box Cox tranformation from the MASS library to normalize this variable.

metastats_02 <- data.frame(describe(baseball_train_02))

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

metastats_02 <- tibble::rownames_to_column(metastats_02, "STATS") 
 
metastats_02 %>%
 filter(skew > 1 | skew < -1) %>%
  dplyr::select(STATS, skew) %>%
  arrange(desc(skew))

STATS	skew
TEAM_FIELDING_E	1.45

Looking at the histogram and QQ plots we can confirm that the variable, TEAM_FIELDING_E, is not normally distributed. It is skewed to the right.

plotNormalHistogram(baseball_train_02$TEAM_FIELDING_E)

qqnorm(baseball_train_02$TEAM_FIELDING_E,
       ylab="Sample Quantiles for TEAM_FIELDING_E")    
         qqline(baseball_train_02$TEAM_FIELDING_E,
           col="blue")

The following Box Cox transformation section is based on the tutorial at the link below:

[://rcompanion.org/handbook/I_12.html][Summary and Analysis of Extension Program Evaluation in R]

The Box Cox procedure uses a log-likelihood to find the lambda to use to transform a variable to a normal distribution.

TEAM_FIELDING_E <- as.numeric(dplyr::pull(baseball_train_02, TEAM_FIELDING_E))

#Transforms TEAM_FIELDING_E as a single vector 
Box = boxcox(TEAM_FIELDING_E ~ 1, lambda = seq(-6,6,0.1))

#Creates a dataframe with results
Cox = data.frame(Box$x, Box$y)

# Order the new data frame by decreasing y to find the best lambda.Displays the lambda with the greatest log likelihood.
Cox2 = Cox[with(Cox, order(-Cox$Box.y)),]
Cox2[1,] 

Box.x	Box.y
-0.8	-3.95e+03

#Extract that lambda and Transform the data
lambda = Cox2[1, "Box.x"]
T_box = (TEAM_FIELDING_E ^ lambda - 1)/lambda

We can now see that TEAM_FIELDING_E has a normal distribution.

plotNormalHistogram(T_box)

qqnorm(T_box, ylab="Sample Quantiles for TEAM_FIELDING_E")
qqline(T_box,
        col="blue")

baseball_train_02$TEAM_FIELDING_E <- T_box

The density plots below show that all of the variables for the dataset baseball_train_02 are now normally distributed. In the next section, we will use this dataset to build the models and discuss the coefficients of the models.

par(mfrow = c(3, 3))
datasub = melt(baseball_train_02) 
suppressWarnings(ggplot(datasub, aes(x= value)) + 
                   geom_density(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

Viewing the dataframe shows that the dataset contains characters resulting from the transfromation of the outliers. These non numeric characters will impact our models especially if we build the intial baseline model with all the variables. We will need one more step to have our data ready for the models.

str(baseball_train_02)

## 'data.frame':    1521 obs. of  21 variables:
##  $ TARGET_WINS             : int  70 82 75 80 85 76 78 87 88 66 ...
##  $ TEAM_BATTING_H          : int  1387 1297 1279 1244 1273 1271 1305 1417 1563 1460 ...
##  $ TEAM_BATTING_2B         : int  209 186 200 179 171 213 179 226 242 239 ...
##  $ TEAM_BATTING_3B         : int  38 27 36 54 37 18 27 28 43 32 ...
##  $ TEAM_BATTING_HR         : int  96 102 92 122 115 96 82 108 164 107 ...
##  $ TEAM_BATTING_BB         : int  451 472 443 525 456 441 374 539 589 546 ...
##  $ TEAM_BATTING_SO         : num  922 920 973 1062 1027 ...
##  $ TEAM_BASERUN_SB         : num  43 49 107 80 40 72 60 86 100 92 ...
##  $ TEAM_BASERUN_CS         : num  30 39 59 54 36 34 39 69 53 64 ...
##  $ TEAM_PITCHING_H         : int  1396 1297 1279 1244 1281 1271 1364 1417 1563 1478 ...
##  $ TEAM_PITCHING_HR        : int  97 102 92 122 116 96 86 108 164 108 ...
##  $ TEAM_PITCHING_BB        : int  454 472 443 525 459 441 391 539 589 553 ...
##  $ TEAM_PITCHING_SO        : num  928 920 973 1062 1033 ...
##  $ TEAM_FIELDING_E         : num  1.23 1.23 1.22 1.23 1.22 ...
##  $ TEAM_FIELDING_DP        : num  156 168 149 186 136 159 141 136 172 146 ...
##  $ TEAM_BASERUN_SB_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_BASERUN_CS_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_PITCHING_H_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_PITCHING_BB_Outlier: chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_PITCHING_SO_Outlier: chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_FIELDING_E_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...

Subsetting - The code below will subset the data to have only numeric or integer values that will be used for our models. This will create baseball_train_03 dataframe.

baseball_train_03 <- baseball_train_02[c(1:15) ]
str(baseball_train_03)

## 'data.frame':    1521 obs. of  15 variables:
##  $ TARGET_WINS     : int  70 82 75 80 85 76 78 87 88 66 ...
##  $ TEAM_BATTING_H  : int  1387 1297 1279 1244 1273 1271 1305 1417 1563 1460 ...
##  $ TEAM_BATTING_2B : int  209 186 200 179 171 213 179 226 242 239 ...
##  $ TEAM_BATTING_3B : int  38 27 36 54 37 18 27 28 43 32 ...
##  $ TEAM_BATTING_HR : int  96 102 92 122 115 96 82 108 164 107 ...
##  $ TEAM_BATTING_BB : int  451 472 443 525 456 441 374 539 589 546 ...
##  $ TEAM_BATTING_SO : num  922 920 973 1062 1027 ...
##  $ TEAM_BASERUN_SB : num  43 49 107 80 40 72 60 86 100 92 ...
##  $ TEAM_BASERUN_CS : num  30 39 59 54 36 34 39 69 53 64 ...
##  $ TEAM_PITCHING_H : int  1396 1297 1279 1244 1281 1271 1364 1417 1563 1478 ...
##  $ TEAM_PITCHING_HR: int  97 102 92 122 116 96 86 108 164 108 ...
##  $ TEAM_PITCHING_BB: int  454 472 443 525 459 441 391 539 589 553 ...
##  $ TEAM_PITCHING_SO: num  928 920 973 1062 1033 ...
##  $ TEAM_FIELDING_E : num  1.23 1.23 1.22 1.23 1.22 ...
##  $ TEAM_FIELDING_DP: num  156 168 149 186 136 159 141 136 172 146 ...

0.4 Build Models

The first Model is using stepwise in Backward direction to eliminate variables, this is an automated process which is different from the manual variable selction process. We will not pay much attention to this process as the focus of the project is to manually identify and select those significant variables that will predict TARGET WINS.

Model <- step(lm(TARGET_WINS ~ ., data=baseball_train_03), direction = "backward")

## Start:  AIC=7313.35
## TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_3B + 
##     TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + TEAM_BASERUN_SB + 
##     TEAM_BASERUN_CS + TEAM_PITCHING_H + TEAM_PITCHING_HR + TEAM_PITCHING_BB + 
##     TEAM_PITCHING_SO + TEAM_FIELDING_E + TEAM_FIELDING_DP
## 
##                    Df Sum of Sq    RSS    AIC
## - TEAM_PITCHING_H   1       0.7 182710 7311.4
## - TEAM_PITCHING_HR  1     162.2 182872 7312.7
## - TEAM_BATTING_H    1     216.2 182926 7313.2
## <none>                          182709 7313.4
## - TEAM_BASERUN_CS   1     330.3 183040 7314.1
## - TEAM_BATTING_HR   1     338.0 183047 7314.2
## - TEAM_PITCHING_BB  1     363.7 183073 7314.4
## - TEAM_BATTING_BB   1     629.6 183339 7316.6
## - TEAM_PITCHING_SO  1    1242.9 183952 7321.7
## - TEAM_BATTING_SO   1    1857.6 184567 7326.7
## - TEAM_BATTING_2B   1    1864.9 184574 7326.8
## - TEAM_FIELDING_DP  1    6690.2 189400 7366.1
## - TEAM_BATTING_3B   1    7536.4 190246 7372.8
## - TEAM_BASERUN_SB   1    8080.4 190790 7377.2
## - TEAM_FIELDING_E   1   18743.7 201453 7459.9
## 
## Step:  AIC=7311.36
## TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_3B + 
##     TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + TEAM_BASERUN_SB + 
##     TEAM_BASERUN_CS + TEAM_PITCHING_HR + TEAM_PITCHING_BB + TEAM_PITCHING_SO + 
##     TEAM_FIELDING_E + TEAM_FIELDING_DP
## 
##                    Df Sum of Sq    RSS    AIC
## - TEAM_PITCHING_HR  1     173.3 182883 7310.8
## <none>                          182710 7311.4
## - TEAM_BASERUN_CS   1     331.1 183041 7312.1
## - TEAM_BATTING_HR   1     358.9 183069 7312.3
## - TEAM_PITCHING_SO  1    1259.1 183969 7319.8
## - TEAM_PITCHING_BB  1    1509.7 184220 7321.9
## - TEAM_BATTING_2B   1    1876.6 184587 7324.9
## - TEAM_BATTING_SO   1    1880.0 184590 7324.9
## - TEAM_BATTING_BB   1    2658.3 185368 7331.3
## - TEAM_BATTING_H    1    4833.0 187543 7349.1
## - TEAM_FIELDING_DP  1    6705.4 189416 7364.2
## - TEAM_BATTING_3B   1    7548.6 190259 7370.9
## - TEAM_BASERUN_SB   1    8142.6 190853 7375.7
## - TEAM_FIELDING_E   1   18841.1 201551 7458.6
## 
## Step:  AIC=7310.8
## TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_3B + 
##     TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + TEAM_BASERUN_SB + 
##     TEAM_BASERUN_CS + TEAM_PITCHING_BB + TEAM_PITCHING_SO + TEAM_FIELDING_E + 
##     TEAM_FIELDING_DP
## 
##                    Df Sum of Sq    RSS    AIC
## <none>                          182883 7310.8
## - TEAM_BASERUN_CS   1     418.7 183302 7312.3
## - TEAM_PITCHING_SO  1    1202.0 184085 7318.8
## - TEAM_PITCHING_BB  1    1537.3 184421 7321.5
## - TEAM_BATTING_2B   1    1928.3 184812 7324.8
## - TEAM_BATTING_SO   1    1977.9 184861 7325.2
## - TEAM_BATTING_BB   1    2678.0 185561 7330.9
## - TEAM_BATTING_H    1    4860.2 187744 7348.7
## - TEAM_BATTING_HR   1    5541.1 188424 7354.2
## - TEAM_BATTING_3B   1    7420.8 190304 7369.3
## - TEAM_FIELDING_DP  1    7423.8 190307 7369.3
## - TEAM_BASERUN_SB   1    9570.9 192454 7386.4
## - TEAM_FIELDING_E   1   18687.6 201571 7456.8

summary(Model)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + 
##     TEAM_BATTING_3B + TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + 
##     TEAM_BASERUN_SB + TEAM_BASERUN_CS + TEAM_PITCHING_BB + TEAM_PITCHING_SO + 
##     TEAM_FIELDING_E + TEAM_FIELDING_DP, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -45.468  -6.985  -0.128   7.454  34.637 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       1.368e+03  1.084e+02  12.624  < 2e-16 ***
## TEAM_BATTING_H    3.169e-02  5.006e-03   6.331 3.22e-10 ***
## TEAM_BATTING_2B  -4.198e-02  1.053e-02  -3.987 7.00e-05 ***
## TEAM_BATTING_3B   1.756e-01  2.244e-02   7.822 9.68e-15 ***
## TEAM_BATTING_HR   7.519e-02  1.112e-02   6.759 1.97e-11 ***
## TEAM_BATTING_BB   1.564e-01  3.328e-02   4.699 2.85e-06 ***
## TEAM_BATTING_SO  -8.768e-02  2.171e-02  -4.038 5.65e-05 ***
## TEAM_BASERUN_SB   6.625e-02  7.458e-03   8.884  < 2e-16 ***
## TEAM_BASERUN_CS  -6.510e-02  3.503e-02  -1.858 0.063350 .  
## TEAM_PITCHING_BB -1.130e-01  3.175e-02  -3.560 0.000382 ***
## TEAM_PITCHING_SO  6.484e-02  2.059e-02   3.148 0.001675 ** 
## TEAM_FIELDING_E  -1.085e+03  8.739e+01 -12.413  < 2e-16 ***
## TEAM_FIELDING_DP -1.121e-01  1.433e-02  -7.824 9.56e-15 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.01 on 1508 degrees of freedom
## Multiple R-squared:  0.3725, Adjusted R-squared:  0.3675 
## F-statistic: 74.59 on 12 and 1508 DF,  p-value: < 2.2e-16

The step backward variable selection process identified eleven significant variables with an R-squared of 37%, Residual Error of 11.01 and F-Statistic of 74.59. Notice that some of the coefficients are negative which means these Team will most likely result in negative wins. We will explore these coefficient a little further in this analysis.

0.4.1 OLS- MODEL 1

Using all the 15 Variables

Model1 <-lm(TARGET_WINS ~ ., data=baseball_train_03)
summary(Model1)

## 
## Call:
## lm(formula = TARGET_WINS ~ ., data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -45.067  -7.014  -0.101   7.499  34.361 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       1.376e+03  1.089e+02  12.634  < 2e-16 ***
## TEAM_BATTING_H    2.992e-02  2.242e-02   1.335   0.1821    
## TEAM_BATTING_2B  -4.140e-02  1.056e-02  -3.921 9.23e-05 ***
## TEAM_BATTING_3B   1.775e-01  2.252e-02   7.882 6.15e-15 ***
## TEAM_BATTING_HR   2.424e-01  1.452e-01   1.669   0.0953 .  
## TEAM_BATTING_BB   1.606e-01  7.051e-02   2.278   0.0229 *  
## TEAM_BATTING_SO  -1.072e-01  2.741e-02  -3.913 9.52e-05 ***
## TEAM_BASERUN_SB   6.361e-02  7.794e-03   8.161 6.94e-16 ***
## TEAM_BASERUN_CS  -5.853e-02  3.547e-02  -1.650   0.0992 .  
## TEAM_PITCHING_H   1.587e-03  2.058e-02   0.077   0.9386    
## TEAM_PITCHING_HR -1.617e-01  1.398e-01  -1.156   0.2478    
## TEAM_PITCHING_BB -1.166e-01  6.735e-02  -1.732   0.0836 .  
## TEAM_PITCHING_SO  8.366e-02  2.614e-02   3.201   0.0014 ** 
## TEAM_FIELDING_E  -1.092e+03  8.785e+01 -12.430  < 2e-16 ***
## TEAM_FIELDING_DP -1.086e-01  1.463e-02  -7.426 1.87e-13 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.01 on 1506 degrees of freedom
## Multiple R-squared:  0.3731, Adjusted R-squared:  0.3672 
## F-statistic: 64.01 on 14 and 1506 DF,  p-value: < 2.2e-16

This Model identified seven significant variables at = 0.05 with an R-squared of 37%, Residual Error of 11.01 and F-Statistic of 64.01. Although the F-Statistic reduced, this model does not improve significantly from the previous model.

Metrics1 <- data.frame(
  R2 = rsquare(Model1, data = baseball_train_03),
  RMSE = rmse(Model1, data = baseball_train_03),
  MAE = mae(Model1, data = baseball_train_03)
)
print(Metrics1)

##          R2     RMSE      MAE
## 1 0.3730717 10.96013 8.741105

0.4.2 OLS- MODEL 2

Using all the seven (7) significant variables from Model 1

Model2 <- lm(TARGET_WINS~TEAM_FIELDING_E + TEAM_BASERUN_SB + TEAM_BATTING_3B + TEAM_FIELDING_DP + TEAM_PITCHING_SO + TEAM_BATTING_SO + TEAM_BATTING_2B,data=baseball_train_03)
summary(Model2)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_FIELDING_E + TEAM_BASERUN_SB + 
##     TEAM_BATTING_3B + TEAM_FIELDING_DP + TEAM_PITCHING_SO + TEAM_BATTING_SO + 
##     TEAM_BATTING_2B, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -48.799  -8.299  -0.053   8.472  39.785 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       1.808e+03  1.158e+02  15.607  < 2e-16 ***
## TEAM_FIELDING_E  -1.410e+03  9.313e+01 -15.141  < 2e-16 ***
## TEAM_BASERUN_SB   5.429e-02  7.497e-03   7.242 7.02e-13 ***
## TEAM_BATTING_3B   1.788e-01  2.289e-02   7.808 1.08e-14 ***
## TEAM_FIELDING_DP -5.319e-02  1.525e-02  -3.488 0.000501 ***
## TEAM_PITCHING_SO -8.587e-03  9.176e-03  -0.936 0.349497    
## TEAM_BATTING_SO  -8.186e-03  9.308e-03  -0.880 0.379267    
## TEAM_BATTING_2B   4.475e-02  7.971e-03   5.614 2.35e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.19 on 1513 degrees of freedom
## Multiple R-squared:  0.2288, Adjusted R-squared:  0.2253 
## F-statistic: 64.14 on 7 and 1513 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 22%, Residual Error of 12.19 and F-Statistic of 64.14. The R-Squared decreased and the Error increased slightly.

Metrics2 <- data.frame(
  R2 = rsquare(Model2, data = baseball_train_03),
  RMSE = rmse(Model2, data = baseball_train_03),
  MAE = mae(Model2, data = baseball_train_03)
)
print(Metrics2)

##          R2     RMSE      MAE
## 1 0.2288348 12.15572 9.731629

0.4.3 OLS- MODEL 3

All offensive categories which include hitting and base running

Model3 <-lm(TARGET_WINS~TEAM_BATTING_H + TEAM_BATTING_BB + TEAM_BATTING_HR + TEAM_BATTING_2B + TEAM_BATTING_SO + TEAM_BASERUN_CS + TEAM_BATTING_3B + TEAM_BASERUN_SB,data=baseball_train_03)
summary(Model3)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_BB + 
##     TEAM_BATTING_HR + TEAM_BATTING_2B + TEAM_BATTING_SO + TEAM_BASERUN_CS + 
##     TEAM_BATTING_3B + TEAM_BASERUN_SB, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -49.812  -7.822   0.247   8.166  35.877 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     15.763131   7.024077   2.244   0.0250 *  
## TEAM_BATTING_H   0.024765   0.005285   4.686 3.03e-06 ***
## TEAM_BATTING_BB  0.037681   0.003994   9.435  < 2e-16 ***
## TEAM_BATTING_HR  0.099319   0.011448   8.676  < 2e-16 ***
## TEAM_BATTING_2B -0.013435   0.010919  -1.230   0.2187    
## TEAM_BATTING_SO -0.010801   0.002767  -3.904 9.88e-05 ***
## TEAM_BASERUN_CS -0.068614   0.037166  -1.846   0.0651 .  
## TEAM_BATTING_3B  0.115379   0.022950   5.027 5.57e-07 ***
## TEAM_BASERUN_SB  0.076701   0.007431  10.321  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.73 on 1512 degrees of freedom
## Multiple R-squared:  0.2857, Adjusted R-squared:  0.2819 
## F-statistic: 75.58 on 8 and 1512 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 28%, Residual Error of 11.73 and F-Statistic of 75.58. Although the R-squared is not that great, the standard errors are more reasonable. We will hold onto this Model as performing better than the previous models for now.

Metrics3 <- data.frame(
  R2 = rsquare(Model3, data = baseball_train_03),
  RMSE = rmse(Model3, data = baseball_train_03),
  MAE = mae(Model3, data = baseball_train_03)
)
print(Metrics3)

##          R2     RMSE      MAE
## 1 0.2856527 11.69934 9.330048

0.4.4 OLS- MODEL 4

All defensive categories which include fielding and pitching

Model4 <- lm(TARGET_WINS~TEAM_PITCHING_H + TEAM_PITCHING_BB + TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E,data=baseball_train_03)
summary(Model4)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_PITCHING_H + TEAM_PITCHING_BB + 
##     TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -48.818  -8.397   0.393   8.617  42.600 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       7.915e+02  1.079e+02   7.335 3.61e-13 ***
## TEAM_PITCHING_H   2.420e-02  2.705e-03   8.947  < 2e-16 ***
## TEAM_PITCHING_BB  2.542e-02  3.943e-03   6.448 1.52e-10 ***
## TEAM_PITCHING_HR  1.503e-02  9.799e-03   1.533 0.125415    
## TEAM_PITCHING_SO -9.205e-03  2.369e-03  -3.886 0.000106 ***
## TEAM_FIELDING_E  -6.153e+02  8.770e+01  -7.016 3.44e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.46 on 1515 degrees of freedom
## Multiple R-squared:  0.1932, Adjusted R-squared:  0.1905 
## F-statistic: 72.56 on 5 and 1515 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 19%, Residual Error of 12.46 and F-Statistic of 75.56.There is no significant improvement with this model.

Metrics4 <- data.frame(
  R2 = rsquare(Model4, data = baseball_train_03),
  RMSE = rmse(Model4, data = baseball_train_03),
  MAE = mae(Model4, data = baseball_train_03)
)
print(Metrics4)

##          R2     RMSE      MAE
## 1 0.1932003 12.43339 9.945165

0.4.5 OLS- MODEL 5

Using only the significant variables from Model 3

Model5 <- lm(TARGET_WINS~TEAM_PITCHING_H + TEAM_PITCHING_BB + TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_BATTING_3B + TEAM_BASERUN_SB,data=baseball_train_03)
summary(Model5)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_PITCHING_H + TEAM_PITCHING_BB + 
##     TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_BATTING_3B + TEAM_BASERUN_SB, 
##     data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -51.002  -7.725   0.407   8.171  36.647 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      44.113607   4.898047   9.006  < 2e-16 ***
## TEAM_PITCHING_H   0.002544   0.002951   0.862    0.389    
## TEAM_PITCHING_BB  0.029880   0.003774   7.917 4.66e-15 ***
## TEAM_PITCHING_HR  0.134928   0.009503  14.198  < 2e-16 ***
## TEAM_PITCHING_SO -0.016789   0.002522  -6.657 3.91e-11 ***
## TEAM_BATTING_3B   0.141192   0.023104   6.111 1.26e-09 ***
## TEAM_BASERUN_SB   0.077656   0.007190  10.800  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.93 on 1514 degrees of freedom
## Multiple R-squared:  0.2606, Adjusted R-squared:  0.2576 
## F-statistic: 88.92 on 6 and 1514 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 26%, Residual Error of 11.93 and F-Statistic of 88.92. Although the R-squared is not better than than Model3, the F-statistic improved with smaller Standard Error.

Metrics5 <- data.frame(
  R2 = rsquare(Model5, data = baseball_train_03),
  RMSE = rmse(Model5, data = baseball_train_03),
  MAE = mae(Model5, data = baseball_train_03)
)
print(Metrics5)

##          R2     RMSE      MAE
## 1 0.2605772 11.90291 9.506094

0.4.6 Compare OLS Model Quality

anova(Model, Model1, Model2, Model3, Model4, Model5)

Res.Df	RSS	Df	Sum of Sq	F	Pr(>F)
1.51e+03	1.83e+05
1.51e+03	1.83e+05	2	174	0.717	0.488
1.51e+03	2.25e+05	-7	-4.2e+04	49.5	1.39e-63
1.51e+03	2.08e+05	1	1.66e+04	136	3.01e-30
1.52e+03	2.35e+05	-3	-2.69e+04	74	1.15e-44
1.51e+03	2.15e+05	1	1.96e+04	162	2.72e-35

tab_model(Model, Model1, Model2, Model3, Model4, Model5)

	TARGET WINS			TARGET WINS			TARGET WINS			TARGET WINS			TARGET WINS			TARGET WINS
Predictors	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p
(Intercept)	1368.26	1155.66 – 1580.87	<0.001	1376.10	1162.45 – 1589.76	<0.001	1807.57	1580.39 – 2034.75	<0.001	15.76	1.99 – 29.54	0.025	791.49	579.82 – 1003.16	<0.001	44.11	34.51 – 53.72	<0.001
TEAM_BATTING_H	0.03	0.02 – 0.04	<0.001	0.03	-0.01 – 0.07	0.182				0.02	0.01 – 0.04	<0.001
TEAM_BATTING_2B	-0.04	-0.06 – -0.02	<0.001	-0.04	-0.06 – -0.02	<0.001	0.04	0.03 – 0.06	<0.001	-0.01	-0.03 – 0.01	0.219
TEAM_BATTING_3B	0.18	0.13 – 0.22	<0.001	0.18	0.13 – 0.22	<0.001	0.18	0.13 – 0.22	<0.001	0.12	0.07 – 0.16	<0.001				0.14	0.10 – 0.19	<0.001
TEAM_BATTING_HR	0.08	0.05 – 0.10	<0.001	0.24	-0.04 – 0.53	0.095				0.10	0.08 – 0.12	<0.001
TEAM_BATTING_BB	0.16	0.09 – 0.22	<0.001	0.16	0.02 – 0.30	0.023				0.04	0.03 – 0.05	<0.001
TEAM_BATTING_SO	-0.09	-0.13 – -0.05	<0.001	-0.11	-0.16 – -0.05	<0.001	-0.01	-0.03 – 0.01	0.379	-0.01	-0.02 – -0.01	<0.001
TEAM_BASERUN_SB	0.07	0.05 – 0.08	<0.001	0.06	0.05 – 0.08	<0.001	0.05	0.04 – 0.07	<0.001	0.08	0.06 – 0.09	<0.001				0.08	0.06 – 0.09	<0.001
TEAM_BASERUN_CS	-0.07	-0.13 – 0.00	0.063	-0.06	-0.13 – 0.01	0.099				-0.07	-0.14 – 0.00	0.065
TEAM_PITCHING_BB	-0.11	-0.18 – -0.05	<0.001	-0.12	-0.25 – 0.02	0.084							0.03	0.02 – 0.03	<0.001	0.03	0.02 – 0.04	<0.001
TEAM_PITCHING_SO	0.06	0.02 – 0.11	0.002	0.08	0.03 – 0.13	0.001	-0.01	-0.03 – 0.01	0.349				-0.01	-0.01 – -0.00	<0.001	-0.02	-0.02 – -0.01	<0.001
TEAM_FIELDING_E	-1084.76	-1256.17 – -913.35	<0.001	-1091.94	-1264.26 – -919.62	<0.001	-1410.16	-1592.85 – -1227.48	<0.001				-615.31	-787.34 – -443.27	<0.001
TEAM_FIELDING_DP	-0.11	-0.14 – -0.08	<0.001	-0.11	-0.14 – -0.08	<0.001	-0.05	-0.08 – -0.02	0.001
TEAM_PITCHING_H				0.00	-0.04 – 0.04	0.939							0.02	0.02 – 0.03	<0.001	0.00	-0.00 – 0.01	0.389
TEAM_PITCHING_HR				-0.16	-0.44 – 0.11	0.248							0.02	-0.00 – 0.03	0.125	0.13	0.12 – 0.15	<0.001
Observations	1521			1521			1521			1521			1521			1521
R2 / R2 adjusted	0.372 / 0.367			0.373 / 0.367			0.229 / 0.225			0.286 / 0.282			0.193 / 0.191			0.261 / 0.258

0.4.7 RIDGE Regression- MODEL 6

The Ridge regression is an extension of linear regression where the loss function is modified to minimize the complexity of the model. This modification is done by adding a penalty parameter that is equivalent to the square of the magnitude of the coefficients.

Before implementing the RIDGE model, we will split the training dataset into 2 parts that is - training set within the training set and a test set that can be used for evaluation. By enforcing stratified sampling both our training and testing sets have approximately equal response “TARGET_WINS” distributions.

Transforming the variables into the form of a matrix will enable us to penalize the model using the ‘glmnet’ method in glmnet package.

#Split the data into Training and Test Set
baseball_train_set<- initial_split(baseball_train_03, prop = 0.7, strata = "TARGET_WINS")
train_baseball  <- training(baseball_train_set)
test_baseball   <- testing(baseball_train_set)

train_Ind<- as.matrix(train_baseball)
train_Dep<- as.matrix(train_baseball$TARGET_WINS)

test_Ind<- as.matrix(test_baseball)
test_Dep<- as.matrix(test_baseball$TARGET_WINS)

For the avoidance of multicollinearity, avoiding overfitting and predicting better, implementing RIDGE regression will become useful.

lambdas <- 10^seq(2, -3, by = -.1)
Model6 <- glmnet(train_Ind,train_Dep, nlambda = 25, alpha = 0, family = 'gaussian', lambda = lambdas)
summary(Model6)

##           Length Class     Mode   
## a0         51    -none-    numeric
## beta      765    dgCMatrix S4     
## df         51    -none-    numeric
## dim         2    -none-    numeric
## lambda     51    -none-    numeric
## dev.ratio  51    -none-    numeric
## nulldev     1    -none-    numeric
## npasses     1    -none-    numeric
## jerr        1    -none-    numeric
## offset      1    -none-    logical
## call        7    -none-    call   
## nobs        1    -none-    numeric

print(Model6, digits = max(3, getOption("digits") - 3),
           signif.stars = getOption("show.signif.stars"))

## 
## Call:  glmnet(x = train_Ind, y = train_Dep, family = "gaussian", alpha = 0,      nlambda = 25, lambda = lambdas) 
## 
##    Df   %Dev  Lambda
## 1  15 0.3002 100.000
## 2  15 0.3498  79.430
## 3  15 0.4032  63.100
## 4  15 0.4597  50.120
## 5  15 0.5181  39.810
## 6  15 0.5771  31.620
## 7  15 0.6354  25.120
## 8  15 0.6916  19.950
## 9  15 0.7444  15.850
## 10 15 0.7924  12.590
## 11 15 0.8350  10.000
## 12 15 0.8716   7.943
## 13 15 0.9021   6.310
## 14 15 0.9268   5.012
## 15 15 0.9463   3.981
## 16 15 0.9613   3.162
## 17 15 0.9725   2.512
## 18 15 0.9808   1.995
## 19 15 0.9868   1.585
## 20 15 0.9910   1.259
## 21 15 0.9939   1.000
## 22 15 0.9959   0.794
## 23 15 0.9973   0.631
## 24 15 0.9982   0.501
## 25 15 0.9988   0.398
## 26 15 0.9993   0.316
## 27 15 0.9995   0.251
## 28 15 0.9997   0.200
## 29 15 0.9998   0.158
## 30 15 0.9999   0.126
## 31 15 0.9999   0.100
## 32 15 0.9999   0.079
## 33 15 1.0000   0.063
## 34 15 1.0000   0.050
## 35 15 1.0000   0.040
## 36 15 1.0000   0.032
## 37 15 1.0000   0.025
## 38 15 1.0000   0.020
## 39 15 1.0000   0.016
## 40 15 1.0000   0.013
## 41 15 1.0000   0.010
## 42 15 1.0000   0.008
## 43 15 1.0000   0.006
## 44 15 1.0000   0.005
## 45 15 1.0000   0.004
## 46 15 1.0000   0.003
## 47 15 1.0000   0.003
## 48 15 1.0000   0.002
## 49 15 1.0000   0.002
## 50 15 1.0000   0.001
## 51 15 1.0000   0.001

The significant difference between the OLS and the Ridge Regresion is the hyperparameter tuning using lambda. The Ridge regression does not perform Feature Selection, but it predicts better and solve overfitting. Cross Validating the Ridge Regression will help us to identify the optimal lambda to penalize the model and enhance the predictability.

CrossVal_ridge <- cv.glmnet(train_Ind,train_Dep, alpha = 0, lambda = lambdas)
optimal_lambda <- CrossVal_ridge$lambda.min
optimal_lambda #The optimal lambda is 0.001 which we will use to penelize the Ridge Regression model.

## [1] 0.001

coef(CrossVal_ridge) # Shows the coefficients

## 16 x 1 sparse Matrix of class "dgCMatrix"
##                              1
## (Intercept)       2.811677e-01
## TARGET_WINS       9.998729e-01
## TEAM_BATTING_H   -8.349498e-06
## TEAM_BATTING_2B  -7.913110e-06
## TEAM_BATTING_3B   1.639477e-05
## TEAM_BATTING_HR  -4.928352e-04
## TEAM_BATTING_BB   9.259542e-05
## TEAM_BATTING_SO   1.605365e-05
## TEAM_BASERUN_SB   1.673979e-05
## TEAM_BASERUN_CS  -2.885973e-05
## TEAM_PITCHING_H   1.218556e-05
## TEAM_PITCHING_HR  4.823411e-04
## TEAM_PITCHING_BB -8.538437e-05
## TEAM_PITCHING_SO -1.864112e-05
## TEAM_FIELDING_E  -2.221726e-01
## TEAM_FIELDING_DP -2.291662e-05

plot(CrossVal_ridge)

The plot shows that the errors increases as the magnitude of lambda increases, previously, we identified that the optimal lambda is 0.001 which is very obvious from the plot above. The coefficients are restricted to be small but not quite zero as Ridge Regression does not force the coefficient to zero. This indicates that the model is performing well so far. But let’s make it better using the optimal labmda.

eval_results <- function(true, predicted, df){
  SSE <- sum((predicted - true)^2)
  SST <- sum((true - mean(true))^2)
  R_square <- 1 - SSE / SST
  RMSE = sqrt(SSE/nrow(df))
data.frame(   
  RMSE = RMSE,
  Rsquare = R_square
)
  
}
# Prediction and evaluation on train data
predictions_train <- predict(Model6, s = optimal_lambda, newx = train_Ind)
eval_results(train_Dep, predictions_train, train_baseball)

RMSE	Rsquare
0.00177	1

We should be a little concern about the 100% R-squared performance for this Model. Although the Ridge Regression forces the coefficients towards zero to improve the Model performance and enhance the predictability, the very high peformance may require further investigation. Lets improve the model using a more reason lambda because optimal might not always be the best.

0.4.8 The Improved Ridge Regression

Model6_Improved <- glmnet(train_Ind,train_Dep, nlambda = 25, alpha = 0, family = 'gaussian', lambda = 6.310)
summary(Model6_Improved)

##           Length Class     Mode   
## a0         1     -none-    numeric
## beta      15     dgCMatrix S4     
## df         1     -none-    numeric
## dim        2     -none-    numeric
## lambda     1     -none-    numeric
## dev.ratio  1     -none-    numeric
## nulldev    1     -none-    numeric
## npasses    1     -none-    numeric
## jerr       1     -none-    numeric
## offset     1     -none-    logical
## call       7     -none-    call   
## nobs       1     -none-    numeric

coef(Model6_Improved)

## 16 x 1 sparse Matrix of class "dgCMatrix"
##                             s0
## (Intercept)       2.076819e+02
## TARGET_WINS       6.246886e-01
## TEAM_BATTING_H    6.964070e-03
## TEAM_BATTING_2B   4.671956e-04
## TEAM_BATTING_3B   2.621307e-02
## TEAM_BATTING_HR   8.582284e-03
## TEAM_BATTING_BB   6.871063e-03
## TEAM_BATTING_SO  -9.172081e-04
## TEAM_BASERUN_SB   1.345116e-02
## TEAM_BASERUN_CS  -7.305997e-03
## TEAM_PITCHING_H   1.819725e-03
## TEAM_PITCHING_HR  7.785479e-03
## TEAM_PITCHING_BB  4.440364e-03
## TEAM_PITCHING_SO -1.543466e-03
## TEAM_FIELDING_E  -1.589633e+02
## TEAM_FIELDING_DP -2.307774e-02

Let’s compute the Model’s Performance Metric to see how this model is doing.

eval_results <- function(true, predicted, df){
  SSE <- sum((predicted - true)^2)
  SST <- sum((true - mean(true))^2)
  R_square <- 1 - SSE / SST
  RMSE = sqrt(SSE/nrow(df))
data.frame(   
  RMSE = RMSE,
  Rsquare = R_square
)
  
}

# Prediction and evaluation on train data
predictions_train <- predict(Model6_Improved, s = lambda, newx = train_Ind)
eval_results(train_Dep, predictions_train, train_baseball)

RMSE	Rsquare
4.32	0.902

# Prediction and evaluation on test data
predictions_test <- predict(Model6_Improved, s = lambda, newx = test_Ind)
eval_results(test_Dep, predictions_test, test_baseball)

RMSE	Rsquare
4.38	0.902

The improved Model6 output shows that the RMSE and R-squared values for the Ridge Regression model on the training and test data are significantly improved. The Loss Function (RMSE) are severely reduced compared to the OLS models which indicates that the Ridge Regression is not overfitting. These performance is significantly improved compared to the OLS Models 1 to 5.

0.4.9 Model Performance Comparison

ModelName <- c("Model", "Model1","Model2","Model3","Model4","Model5","Model6")
Model_RSquared <- c("37%", "37%", "22%", "28%", "19%", "26% ", "90%")
Model_RMSE <- c("11.01", "10.96", "12.15", "11.69", "12.43", "11.93 ", "4.33")
Model_FStatistic <- c("74.59", "64.01", "64.14", "75.58", "72.56", "88.92 ", "NA")
Model_Performance <- data.frame(ModelName,Model_RSquared,Model_RMSE,Model_FStatistic)
Model_Performance

ModelName	Model_RSquared	Model_RMSE	Model_FStatistic
Model	37%	11.01	74.59
Model1	37%	10.96	64.01
Model2	22%	12.15	64.14
Model3	28%	11.69	75.58
Model4	19%	12.43	72.56
Model5	26%	11.93	88.92
Model6	90%	4.33	NA

0.4.10 Model Prediction

Based on the Model metrics above, we’re ready to make prediction and we will select our acceptable OLS Model3 and Model5 which has better F-Statistic, smaller standard errors and less negative coefficient as our best OLS models. We will also compare the prediction accuracy of these models to that of the improved Ridge Regression Model which is our champion Model for this exercise based on the very small RMSE and the highest R-squared of over 90%.

predicted <- predict(Model3, newx = test_baseball)# predict on test data
predicted_values <- cbind (actual=test_baseball$TARGET_WINS, predicted)  # combine

## Warning in cbind(actual = test_baseball$TARGET_WINS, predicted): number of
## rows of result is not a multiple of vector length (arg 1)

predicted_values

##      actual predicted
## 1        70  69.49875
## 2        80  67.56125
## 3        78  68.38378
## 4        88  73.25628
## 5        81  67.37030
## 6        90  66.86382
## 7        75  63.36361
## 8        87  76.58169
## 9        75  89.59170
## 10       85  76.54657
## 11       92  86.70081
## 12       98  77.31870
## 13       51  78.60647
## 14       53  84.73255
## 15      100  89.15530
## 16       70  86.01142
## 17       74  76.15138
## 18       81  76.18203
## 19       75  79.02243
## 20       90  72.03653
## 21       79  89.30377
## 22       68  84.16328
## 23       84  79.30964
## 24       88  76.00667
## 25       81  93.36918
## 26       74  79.72780
## 27       61  83.96472
## 28       67  81.83322
## 29       69  86.75976
## 30       65  83.85432
## 31      101  75.24767
## 32      106  96.12431
## 33       88  85.18416
## 34       96  87.23777
## 35       90  86.66397
## 36       69  80.90673
## 37       48  71.73633
## 38       83  81.22686
## 39       71  90.64517
## 40       77  94.18601
## 41       87  95.53100
## 42       69  80.08285
## 43       57  75.86419
## 44       81  66.74352
## 45       95  64.57578
## 46       91  60.91243
## 47       92  72.28834
## 48       91  68.48019
## 49       84  65.67250
## 50       73  72.48471
## 51       67  86.44011
## 52       87  80.65902
## 53       76  82.06254
## 54       67  78.70595
## 55       89  77.58369
## 56       80  72.97126
## 57       88  78.18561
## 58       78  72.24895
## 59       74  76.25716
## 60       67  77.02116
## 61      107  75.30225
## 62       52  76.46272
## 63       64  78.36297
## 64       94  68.74638
## 65       96  70.51741
## 66       50  66.00240
## 67       49  74.32965
## 68       54  68.02212
## 69       55  71.72855
## 70       45  69.41157
## 71       96  69.61566
## 72       81  70.79319
## 73       80  69.48860
## 74       89  65.80599
## 75       98  70.70940
## 76       89  82.73716
## 77       78  77.23526
## 78       83  81.04205
## 79       73  86.68577
## 80       80  81.16218
## 81       95  88.42305
## 82       98  85.99275
## 83      104  73.58515
## 84       82  82.22049
## 85      105  85.06236
## 86       87  86.09993
## 87       96  76.29960
## 88      104  79.91231
## 89       92  77.53938
## 90      105  84.34369
## 91       98  74.79848
## 92       88  70.02759
## 93      103  82.39423
## 94       87  79.28918
## 95       73  91.97048
## 96       81  78.96951
## 97       63  74.19733
## 98       72  72.42719
## 99       88  77.14336
## 100      81  76.15731
## 101      79  77.37089
## 102      77  72.05640
## 103      93  76.89373
## 104      82  75.92414
## 105      67  85.81217
## 106      92  75.59281
## 107      98  80.60366
## 108      83  85.62160
## 109      56  82.00590
## 110      79  84.74713
## 111      87  84.88357
## 112      81  86.74637
## 113      78  89.85103
## 114      99  88.78040
## 115      99  78.46174
## 116      86  84.86044
## 117      98  83.42672
## 118      79  82.79613
## 119      75  82.72617
## 120      64  66.91725
## 121      71  69.51332
## 122      87  78.66910
## 123      71  72.33942
## 124      95  67.50771
## 125      81  64.77165
## 126      98  69.79720
## 127      93  71.15595
## 128      84  77.34661
## 129      63  77.95811
## 130      64  86.54917
## 131     111  73.89007
## 132      83  79.45996
## 133      63  76.41158
## 134      62  81.88671
## 135      56  84.23277
## 136      72  81.62631
## 137      59  78.66292
## 138     102  78.01240
## 139      93  82.19463
## 140      81  84.27986
## 141      94  73.82318
## 142      65  75.50558
## 143      73  72.20240
## 144      79  71.73169
## 145      98  78.70550
## 146      98  82.60550
## 147      86  79.89101
## 148      77  80.98410
## 149      79  73.62353
## 150     108  74.78823
## 151     102  70.09769
## 152      86  69.79510
## 153      76  73.21664
## 154      80  76.42604
## 155      64  70.00951
## 156      91  79.24643
## 157      81  76.55529
## 158      93  77.45710
## 159      71  77.29383
## 160      93  74.83120
## 161      65  80.57564
## 162      93  75.27052
## 163      92  72.96984
## 164      92  88.01300
## 165     101  89.33125
## 166      97  86.25568
## 167      98  69.87604
## 168      94  89.64940
## 169      79  80.48480
## 170      87  81.38282
## 171      62  78.12840
## 172      60  80.46360
## 173      70  77.90473
## 174      61  86.93266
## 175      57  88.60549
## 176      76  86.22276
## 177      87  85.13130
## 178      97  80.44713
## 179      68  88.03475
## 180      80  80.82213
## 181      88  83.47961
## 182      83  72.80089
## 183      67  80.37391
## 184      77  79.72336
## 185      84  78.69135
## 186      77  85.57211
## 187      70  83.20429
## 188      64  89.48194
## 189      85  96.68228
## 190     106  87.75298
## 191     100  91.87313
## 192      85  86.26651
## 193      93  77.82142
## 194      97  76.04487
## 195      82  84.88891
## 196     101  81.55068
## 197      88  83.70493
## 198      79  80.38773
## 199      85 101.92399
## 200      72  81.10336
## 201      84  73.46680
## 202     104  93.36972
## 203      85  86.05010
## 204      66  85.59259
## 205      43  73.88114
## 206      64  74.83380
## 207      76  64.08681
## 208      80  65.84094
## 209      64  71.98560
## 210      83  79.13560
## 211      78  85.15069
## 212      65  84.82851
## 213      66  88.64454
## 214      66  87.36139
## 215      69  73.46671
## 216      79  73.32881
## 217      79  76.75376
## 218      89  80.18206
## 219      82  75.03975
## 220      76  75.66433
## 221      85  70.51948
## 222      87  74.54246
## 223      89  72.27715
## 224      72  70.35686
## 225      91  72.18585
## 226      72  79.91554
## 227      77  88.74324
## 228      65  87.01201
## 229      58  85.96054
## 230      80  89.84882
## 231      93  93.28619
## 232      88  91.12115
## 233     102  94.67752
## 234      97  73.79792
## 235      88  80.75316
## 236      75  76.99427
## 237      91  87.39336
## 238     100  77.79257
## 239      88  75.87232
## 240      89  87.87867
## 241      79  79.02455
## 242      73  79.53593
## 243      78  77.39581
## 244      81  71.37449
## 245      88  85.39949
## 246      68  78.33072
## 247      93  86.04030
## 248      96  79.55776
## 249      95  82.75494
## 250      71  86.19625
## 251      78  87.67022
## 252      74  85.00793
## 253      56  80.45297
## 254      68  87.60385
## 255      81  82.44495
## 256      59  77.50487
## 257      91  79.20244
## 258      89  69.31674
## 259      79  74.80593
## 260      97  83.14847
## 261      77  91.63208
## 262      69  93.48635
## 263      53  84.53632
## 264      65  86.35405
## 265      56  86.94776
## 266      62  82.27458
## 267      64  80.34078
## 268      60  97.84021
## 269      85  90.77786
## 270      80  86.29480
## 271      78  89.59173
## 272      68  81.45834
## 273      70  79.98754
## 274      90  76.10745
## 275      83  78.12297
## 276      51  76.91669
## 277      61  74.90087
## 278      73  87.24509
## 279      92  95.14666
## 280     101  90.93669
## 281      59  90.70110
## 282      88  81.29394
## 283      96  72.64682
## 284      82  82.98819
## 285      83  75.16897
## 286      87  78.10723
## 287      99  82.14880
## 288      53  84.91555
## 289      79  97.32176
## 290     114 101.10846
## 291     103  84.72247
## 292      87  75.83547
## 293      92  76.87742
## 294     100  84.10901
## 295     108  89.08411
## 296      97  81.50902
## 297      71  81.45233
## 298      83  78.28904
## 299      84  74.25636
## 300      79  75.53547
## 301      90  78.88184
## 302      89  77.90593
## 303      89  81.68807
## 304      98  76.22767
## 305      96  71.70319
## 306     101  74.06627
## 307      88  74.02593
## 308      90  79.72855
## 309      87  78.32084
## 310      98  75.75259
## 311     101  83.80749
## 312     110  79.14016
## 313      56  77.77473
## 314      85  77.15735
## 315      63  76.99388
## 316      58  68.85761
## 317      52  74.32405
## 318      76  76.06391
## 319      56  76.38482
## 320      74  77.40995
## 321      59  77.07957
## 322      76  81.59777
## 323      99  73.66385
## 324      78  79.18950
## 325     102  77.63615
## 326      91  79.20131
## 327      85  79.58513
## 328     100  78.45513
## 329      88  85.18691
## 330      96  87.36482
## 331      66  81.06456
## 332      78  86.55401
## 333      94  76.85508
## 334      96  80.08373
## 335      72  80.76698
## 336      46  78.96028
## 337      69  81.18968
## 338      73  84.35796
## 339      85  78.91002
## 340      92  75.96436
## 341      62  85.42552
## 342      63  80.55205
## 343      71  84.56468
## 344      80  82.79470
## 345      90  76.50017
## 346      91  78.80238
## 347      81  81.74452
## 348      67  81.81584
## 349      97  83.28464
## 350      76  75.40413
## 351      67  74.86536
## 352      74  75.32920
## 353     102  70.05289
## 354     103  72.91093
## 355     101  73.69728
## 356      88  84.47503
## 357      93  72.99975
## 358      73  83.43029
## 359      95  87.95245
## 360      79  78.31703
## 361      80  78.64492
## 362      90  81.71404
## 363      97  80.02600
## 364      98  77.59988
## 365      57  79.43621
## 366      95  79.28112
## 367      98  83.90819
## 368      79  93.54258
## 369      78  86.40102
## 370      72  82.58538
## 371      75  85.89380
## 372      61  79.81702
## 373      71  76.95876
## 374      69  70.17323
## 375      92  74.13669
## 376      83  72.30131
## 377      89  75.25157
## 378      84  76.54545
## 379      91  78.41872
## 380      76  81.53011
## 381      74  85.93305
## 382      79  83.95041
## 383      66  88.37161
## 384      64  86.31066
## 385      74  82.44512
## 386      93  83.91708
## 387      93  86.67909
## 388      69  87.44296
## 389      78  82.21262
## 390      61  76.52642
## 391      88  75.16841
## 392      98  64.24604
## 393      92  72.58621
## 394      90  77.46501
## 395      91  72.80068
## 396      58  78.05726
## 397      83  80.25288
## 398      85  76.11026
## 399      87  90.32384
## 400      90  74.10965
## 401      88  79.40059
## 402      88  70.55224
## 403      74  86.59573
## 404      87  87.70315
## 405      77  83.05623
## 406      88  78.62248
## 407      86  75.43148
## 408      92  83.51626
## 409      50  74.53313
## 410      42  75.06487
## 411      79  86.07172
## 412      67  87.84534
## 413      82  94.14527
## 414      67  96.04200
## 415      86  90.90022
## 416      87  86.34220
## 417      83  94.79885
## 418      97  86.76487
## 419     106  87.56303
## 420      76  85.09908
## 421      87  88.22749
## 422     112  93.78125
## 423     110  90.14901
## 424     102  97.39963
## 425      83  88.67552
## 426      80  87.20702
## 427      78  80.27788
## 428      87  85.93653
## 429      72  74.39622
## 430      86  74.75851
## 431      85  72.61288
## 432      62  90.56274
## 433      55  89.47333
## 434      63  78.69775
## 435      67  77.35048
## 436      76  79.53762
## 437      64  83.39287
## 438      57  83.62948
## 439      76  76.57712
## 440      70  77.10248
## 441      83  83.34904
## 442      77  78.56847
## 443      89  69.57909
## 444      53  73.96331
## 445      89  63.06532
## 446      87  69.99296
## 447      80  76.64262
## 448      87  79.98629
## 449      75  73.50146
## 450      82  77.92409
## 451      78  82.13105
## 452      68  77.90857
## 453      68  72.95778
## 454      83  69.73774
## 455      67  69.46233
## 456      70  67.52634
## 457      80  72.19872
## 458      78  78.59388
## 459      88  73.35113
## 460      81  71.86038
## 461      90  75.50214
## 462      75  70.66007
## 463      87  73.52882
## 464      75  88.14247
## 465      85  88.25647
## 466      92  81.56281
## 467      98  85.11315
## 468      51  80.24222
## 469      53  81.24244
## 470     100  82.69428
## 471      70  73.51443
## 472      74  75.48643
## 473      81  88.59721
## 474      75  74.75932
## 475      90  69.56123
## 476      79  88.86014
## 477      68  71.28452
## 478      84  86.22500
## 479      88  88.95228
## 480      81  92.69140
## 481      74 101.10893
## 482      61  93.85807
## 483      67  84.70288
## 484      69  82.88195
## 485      65  84.57159
## 486     101  73.01808
## 487     106  78.50309
## 488      88  87.77309
## 489      96  90.41206
## 490      90  81.01700
## 491      69  81.18505
## 492      48  78.76486
## 493      83  79.22418
## 494      71  89.33282
## 495      77  81.16533
## 496      87  79.74078
## 497      69  92.39093
## 498      57  89.28796
## 499      81  77.12440
## 500      95  79.77105
## 501      91  75.68524
## 502      92  87.43997
## 503      91  76.52938
## 504      84  84.78700
## 505      73  80.50877
## 506      67  77.35972
## 507      87  72.72118
## 508      76  80.34685
## 509      67  72.55936
## 510      89  70.06595
## 511      80  83.25037
## 512      88  80.59516
## 513      78  81.71215
## 514      74  78.13880
## 515      67  82.04003
## 516     107  88.64843
## 517      52  84.59548
## 518      64  90.67361
## 519      94  84.94869
## 520      96  82.43145
## 521      50  77.98885
## 522      49  72.89823
## 523      54  83.43799
## 524      55  86.92602
## 525      45  87.18417
## 526      96  77.09404
## 527      81  89.40353
## 528      80  93.34790
## 529      89  86.96778
## 530      98  82.14507
## 531      89  83.43017
## 532      78  75.10548
## 533      83  74.03722
## 534      73  74.16809
## 535      80  94.69567
## 536      95  85.56016
## 537      98  93.39981
## 538     104  85.25712
## 539      82  87.29422
## 540     105  87.46496
## 541      87  83.61357
## 542      96  79.45141
## 543     104  79.45387
## 544      92  78.43478
## 545     105  73.38522
## 546      98  76.03883
## 547      88  72.47713
## 548     103  66.23869
## 549      87  67.67506
## 550      73  74.29160
## 551      81  76.15938
## 552      63  68.53339
## 553      72  64.18492
## 554      88  76.55550
## 555      81  72.31920
## 556      79  75.70448
## 557      77  86.25488
## 558      93  87.13625
## 559      82  85.66963
## 560      67  75.01403
## 561      92  72.48383
## 562      98  76.99319
## 563      83  67.65273
## 564      56  78.88975
## 565      79  85.81330
## 566      87  99.38599
## 567      81 102.30433
## 568      78  93.87881
## 569      99 100.37487
## 570      99  98.32826
## 571      86  89.11290
## 572      98  78.88624
## 573      79  73.97117
## 574      75  86.87866
## 575      64  82.38174
## 576      71  82.41774
## 577      87  70.81168
## 578      71  94.27943
## 579      95  96.56763
## 580      81  96.17381
## 581      98  83.56929
## 582      93  91.26643
## 583      84  93.49057
## 584      63  95.36925
## 585      64  80.85944
## 586     111  84.39534
## 587      83  82.97421
## 588      63  77.45613
## 589      62  82.72740
## 590      56  77.65384
## 591      72  67.80243
## 592      59  66.62810
## 593     102  74.11153
## 594      93  75.35735
## 595      81  91.41911
## 596      94  89.37015
## 597      65  82.88452
## 598      73  82.28763
## 599      79  84.44431
## 600      98  77.11652
## 601      98  89.87563
## 602      86  87.62389
## 603      77  93.76432
## 604      79  87.78726
## 605     108  93.25463
## 606     102  97.35486
## 607      86  92.79773
## 608      76  98.47135
## 609      80  94.72234
## 610      64  93.80173
## 611      91  96.01043
## 612      81  82.86228
## 613      93  76.13348
## 614      71  77.17624
## 615      93  77.23491
## 616      65  84.72118
## 617      93  84.54481
## 618      92  91.45792
## 619      92  81.48643
## 620     101  76.05063
## 621      97  75.66432
## 622      98  77.26623
## 623      94  77.11791
## 624      79  85.23634
## 625      87  94.58567
## 626      62  82.27350
## 627      60  76.21672
## 628      70  80.88749
## 629      61  77.09902
## 630      57  79.78842
## 631      76  80.55472
## 632      87  78.32830
## 633      97  74.66971
## 634      68  70.44247
## 635      80  75.76856
## 636      88  81.25690
## 637      83  85.21570
## 638      67  87.45149
## 639      77  83.24715
## 640      84  85.12337
## 641      77  90.08131
## 642      70  85.69963
## 643      64  91.72563
## 644      85  95.57921
## 645     106  80.82969
## 646     100  76.76201
## 647      85  84.40290
## 648      93  88.69413
## 649      97  83.20373
## 650      82  92.68599
## 651     101  82.30209
## 652      88  79.07586
## 653      79  86.73771
## 654      85  81.29509
## 655      72  66.71945
## 656      84  73.26650
## 657     104  86.38450
## 658      85  80.74925
## 659      66  72.12318
## 660      43  86.47275
## 661      64  79.04706
## 662      76  82.85552
## 663      80  74.31326
## 664      64  75.84208
## 665      83  84.08627
## 666      78  79.05523
## 667      65  79.31311
## 668      66  80.23662
## 669      66  75.09644
## 670      69  61.44278
## 671      79  60.04693
## 672      79  73.69484
## 673      89  75.61348
## 674      82  75.09419
## 675      76  60.58337
## 676      85  79.31909
## 677      87  85.07393
## 678      89  71.05731
## 679      72  82.34659
## 680      91  75.70126
## 681      72  76.36214
## 682      77  79.97192
## 683      65  79.96235
## 684      58  77.60617
## 685      80  75.63627
## 686      93  72.69408
## 687      88  78.08013
## 688     102  77.26828
## 689      97  81.21731
## 690      88  75.48051
## 691      75  71.40040
## 692      91  74.64094
## 693     100  78.67222
## 694      88  88.47661
## 695      89  82.65871
## 696      79  89.85068
## 697      73  97.41705
## 698      78  84.06305
## 699      81  83.32637
## 700      88  79.06699
## 701      68  81.32051
## 702      93  81.98676
## 703      96  72.57010
## 704      95  73.44789
## 705      71  74.81811
## 706      78  74.98640
## 707      74  84.21973
## 708      56  84.56636
## 709      68  82.75465
## 710      81  77.97886
## 711      59  84.76507
## 712      91  79.18286
## 713      89  82.33910
## 714      79  78.44898
## 715      97  81.50181
## 716      77  84.18849
## 717      69  80.75256
## 718      53  87.77617
## 719      65  79.01648
## 720      56  84.53152
## 721      62  73.76380
## 722      64  74.64648
## 723      60  97.05403
## 724      85  95.97254
## 725      80  77.70426
## 726      78  91.18758
## 727      68  75.73231
## 728      70  73.62204
## 729      90  65.45411
## 730      83  78.44786
## 731      51  84.83279
## 732      61  85.78446
## 733      73  82.58456
## 734      92  80.45664
## 735     101  74.78727
## 736      59  70.95567
## 737      88  73.82233
## 738      96  84.23530
## 739      82  79.76443
## 740      83  75.55942
## 741      87  83.90318
## 742      99  78.55669
## 743      53  80.46496
## 744      79  76.10448
## 745     114  77.89772
## 746     103  71.32829
## 747      87  73.19595
## 748      92  84.23038
## 749     100  80.71645
## 750     108  82.88104
## 751      97  86.58766
## 752      71  77.59884
## 753      83  76.13671
## 754      84  87.43058
## 755      79  86.48412
## 756      90  90.45556
## 757      89  86.15989
## 758      89  97.07526
## 759      98  95.61839
## 760      96  91.30071
## 761     101 102.70882
## 762      88  94.41229
## 763      90  97.02811
## 764      87  89.96791
## 765      98  82.18003
## 766     101  83.80303
## 767     110  85.00570
## 768      56  80.53210
## 769      85  87.92249
## 770      63  94.71954
## 771      58  73.29979
## 772      52  71.60731
## 773      76  67.21755
## 774      56  61.79597
## 775      74  75.69141
## 776      59  84.94305
## 777      76  76.45704
## 778      99  76.87543
## 779      78  79.08576
## 780     102  82.82991
## 781      91  77.53815
## 782      85  88.68762
## 783     100  86.93289
## 784      88  86.40456
## 785      96  85.95538
## 786      66  71.33692
## 787      78  81.90683
## 788      94  74.62795
## 789      96  72.43715
## 790      72  68.46631
## 791      46  80.61536
## 792      69  77.72868
## 793      73  78.76628
## 794      85  79.50862
## 795      92  83.21869
## 796      62  78.19842
## 797      63  89.64355
## 798      71  89.13728
## 799      80  82.46795
## 800      90  77.71627
## 801      91  68.72122
## 802      81  73.81261
## 803      67  90.70552
## 804      97  88.53050
## 805      76  93.36524
## 806      67  81.87482
## 807      74  69.53519
## 808     102  65.83305
## 809     103  80.91972
## 810     101  75.66819
## 811      88  69.78134
## 812      93  74.65439
## 813      73  86.20933
## 814      95  90.52296
## 815      79  89.88804
## 816      80  91.22499
## 817      90  85.30952
## 818      97  76.50755
## 819      98  77.81370
## 820      57  76.53768
## 821      95  80.85762
## 822      98  83.55077
## 823      79  86.28180
## 824      78  90.36860
## 825      72  76.09565
## 826      75  75.87291
## 827      61  87.02377
## 828      71  78.48445
## 829      69  76.31062
## 830      92  73.67242
## 831      83  82.00670
## 832      89  76.92192
## 833      84  77.12030
## 834      91  75.08059
## 835      76  87.08930
## 836      74  84.10366
## 837      79  69.87058
## 838      66  67.95321
## 839      64  76.27043
## 840      74  77.82358
## 841      93  61.48364
## 842      93  73.93711
## 843      69  80.09102
## 844      78  80.39881
## 845      61  77.94942
## 846      88  79.27261
## 847      98  80.39794
## 848      92  79.57612
## 849      90  88.63689
## 850      91  85.94038
## 851      58  83.89021
## 852      83  84.32137
## 853      85  88.01176
## 854      87  89.30777
## 855      90  84.21572
## 856      88  89.72324
## 857      88  88.68351
## 858      74  93.65513
## 859      87  86.63118
## 860      77  80.90752
## 861      88  83.25118
## 862      86  82.34711
## 863      92  83.24225
## 864      50  77.00634
## 865      42  79.45941
## 866      79  70.73381
## 867      67  76.64514
## 868      82  80.04985
## 869      67  77.07574
## 870      86  81.67504
## 871      87  74.98745
## 872      83  84.02452
## 873      97  72.21179
## 874     106  77.78425
## 875      76  81.11352
## 876      87  82.50825
## 877     112  76.61177
## 878     110  74.25842
## 879     102  69.83767
## 880      83  78.11892
## 881      80  75.86576
## 882      78  70.01321
## 883      87  82.94723
## 884      72  80.03326
## 885      86  82.57406
## 886      85  87.68125
## 887      62  74.67282
## 888      55  67.15518
## 889      63  76.66567
## 890      67  77.28677
## 891      76  73.35303
## 892      64  85.23277
## 893      57  82.52631
## 894      76  80.73811
## 895      70  74.93185
## 896      83  84.73855
## 897      77  75.75555
## 898      89  81.92132
## 899      53  87.57509
## 900      89  81.64124
## 901      87  78.23981
## 902      80  79.85345
## 903      87  83.07927
## 904      75  77.58223
## 905      82  83.27506
## 906      78  84.80112
## 907      68  77.35284
## 908      68  83.92642
## 909      83  80.88043
## 910      67  79.19680
## 911      70  61.31942
## 912      80  58.38833
## 913      78  66.35593
## 914      88  60.64071
## 915      81  59.60618
## 916      90  71.52196
## 917      75  82.97232
## 918      87  69.80963
## 919      75  73.80638
## 920      85  71.90841
## 921      92  76.54061
## 922      98  81.68224
## 923      51  85.81771
## 924      53  92.23216
## 925     100  84.05760
## 926      70  79.86707
## 927      74  83.63096
## 928      81  72.09129
## 929      75  76.10990
## 930      90  76.51728
## 931      79  79.08606
## 932      68  75.26965
## 933      84  92.94321
## 934      88  85.87283
## 935      81  73.31166
## 936      74  77.15060
## 937      61  70.75125
## 938      67  83.97144
## 939      69  95.72627
## 940      65  77.69374
## 941     101  79.19794
## 942     106  76.35351
## 943      88  78.32832
## 944      96  75.91017
## 945      90  70.76003
## 946      69  86.96236
## 947      48  78.23717
## 948      83  81.18502
## 949      71  75.78016
## 950      77  75.92134
## 951      87  77.63943
## 952      69  97.27400
## 953      57  85.27491
## 954      81  88.53154
## 955      95  93.95050
## 956      91 104.40963
## 957      92  93.08013
## 958      91  94.25311
## 959      84  96.39318
## 960      73 102.13149
## 961      67  93.15151
## 962      87  93.26443
## 963      76  88.38960
## 964      67  83.12927
## 965      89  87.16569
## 966      80  86.20118
## 967      88  84.48924
## 968      78  93.72888
## 969      74 100.22576
## 970      67  90.75092
## 971     107  85.92259
## 972      52  90.08918
## 973      64  88.66595
## 974      94  94.18213
## 975      96  85.07843
## 976      50  87.16259
## 977      49  76.98871
## 978      54  66.83786
## 979      55  71.05327
## 980      45  80.22158
## 981      96  77.96724
## 982      81  74.09699
## 983      80  75.16087
## 984      89  75.85082
## 985      98  78.57326
## 986      89  86.20376
## 987      78  90.85388
## 988      83  82.07620
## 989      73  92.37084
## 990      80  84.76236
## 991      95  86.21000
## 992      98  82.30628
## 993     104  93.02867
## 994      82  92.67147
## 995     105  87.26267
## 996      87  85.61029
## 997      96  81.43227
## 998     104  78.74050
## 999      92  85.55504
## 1000    105  85.67697
## 1001     98  89.49447
## 1002     88  89.33847
## 1003    103  97.19323
## 1004     87  94.41711
## 1005     73  90.90383
## 1006     81  89.17507
## 1007     63  90.21799
## 1008     72  93.50255
## 1009     88  93.87559
## 1010     81  95.81350
## 1011     79  90.93701
## 1012     77  92.44312
## 1013     93  78.34708
## 1014     82  72.71619
## 1015     67  76.46792
## 1016     92  74.59291
## 1017     98  72.01169
## 1018     83  63.68687
## 1019     56  80.15956
## 1020     79  87.11331
## 1021     87  92.71264
## 1022     81  72.71623
## 1023     78  73.56595
## 1024     99  69.61434
## 1025     99  73.36825
## 1026     86  79.86398
## 1027     98  79.38112
## 1028     79  76.32710
## 1029     75  87.36658
## 1030     64  87.46441
## 1031     71  94.51520
## 1032     87  93.38030
## 1033     71  91.96627
## 1034     95  89.23678
## 1035     81  82.81230
## 1036     98  80.88571
## 1037     93  87.73838
## 1038     84  86.40998
## 1039     63  81.97909
## 1040     64  82.52483
## 1041    111  68.20531
## 1042     83  66.22217
## 1043     63  70.54708
## 1044     62  67.31787
## 1045     56  70.66388
## 1046     72  84.08191
## 1047     59  85.42203
## 1048    102  72.46110
## 1049     93  76.83792
## 1050     81  75.06820
## 1051     94  75.85712
## 1052     65  76.59533
## 1053     73  70.17289
## 1054     79  68.75619
## 1055     98  73.16580
## 1056     98  82.69221
## 1057     86  78.45319
## 1058     77  74.46707
## 1059     79  84.53133
## 1060    108  72.54941
## 1061    102  85.11678
## 1062     86  83.35557
## 1063     76  82.59932
## 1064     80  83.53339
## 1065     64  86.85838
## 1066     91  84.46278
## 1067     81  88.67685
## 1068     93  83.37239
## 1069     71  88.58651
## 1070     93  89.59870
## 1071     65  84.64656
## 1072     93  81.64766
## 1073     92  92.61935
## 1074     92  86.27243
## 1075    101  82.85143
## 1076     97  85.66203
## 1077     98  95.84800
## 1078     94  90.95834
## 1079     79  95.81290
## 1080     87  98.69009
## 1081     62  85.71811
## 1082     60  69.76796
## 1083     70  75.60889
## 1084     61  71.34770
## 1085     57  72.41132
## 1086     76  72.15901
## 1087     87  71.27513
## 1088     97  84.49067
## 1089     68  82.25327
## 1090     80  81.79879
## 1091     88  85.53387
## 1092     83  80.98372
## 1093     67  73.90174
## 1094     77  73.92732
## 1095     84  71.53401
## 1096     77  74.23172
## 1097     70  81.61689
## 1098     64  82.79164
## 1099     85  77.45100
## 1100    106  76.58674
## 1101    100  78.73276
## 1102     85  96.77330
## 1103     93  87.60659
## 1104     97  76.93790
## 1105     82  87.38424
## 1106    101  72.88084
## 1107     88  80.21365
## 1108     79  79.76677
## 1109     85  66.28857
## 1110     72  69.89638
## 1111     84  67.14770
## 1112    104  70.31318
## 1113     85  63.48091
## 1114     66  71.00934
## 1115     43  70.69567
## 1116     64  70.69689
## 1117     76  73.50918
## 1118     80  78.85426
## 1119     64  83.60694
## 1120     83  83.10800
## 1121     78  80.02899
## 1122     65  71.49187
## 1123     66  66.74572
## 1124     66  73.57565
## 1125     69  80.18532
## 1126     79  72.35786
## 1127     79  72.86273
## 1128     89  71.97395
## 1129     82  65.32023
## 1130     76  73.37091
## 1131     85  70.95516
## 1132     87  67.70542
## 1133     89  71.73850
## 1134     72  76.98511
## 1135     91  84.88781
## 1136     72  94.03477
## 1137     77  83.22815
## 1138     65  84.15055
## 1139     58  89.26872
## 1140     80  79.14093
## 1141     93  84.17269
## 1142     88  84.81615
## 1143    102  72.06287
## 1144     97  77.75736
## 1145     88  78.77255
## 1146     75  72.63020
## 1147     91  78.03885
## 1148    100  90.17251
## 1149     88  78.48975
## 1150     89  79.38583
## 1151     79  77.11963
## 1152     73  90.17322
## 1153     78  81.64387
## 1154     81  83.99443
## 1155     88  85.64188
## 1156     68  82.44665
## 1157     93  95.51429
## 1158     96  77.02829
## 1159     95  89.46990
## 1160     71  94.55289
## 1161     78  95.36864
## 1162     74  91.57210
## 1163     56  80.87037
## 1164     68  82.79777
## 1165     81  75.87495
## 1166     59  80.97836
## 1167     91  84.59811
## 1168     89  81.97436
## 1169     79  94.32579
## 1170     97  85.66276
## 1171     77  74.16725
## 1172     69  77.35892
## 1173     53  71.56327
## 1174     65  89.51002
## 1175     56  83.62805
## 1176     62  96.01226
## 1177     64  99.63880
## 1178     60  86.66543
## 1179     85  85.08412
## 1180     80  88.86202
## 1181     78  96.16164
## 1182     68  94.35191
## 1183     70  77.78130
## 1184     90  79.97524
## 1185     83  76.64023
## 1186     51  81.35226
## 1187     61  82.58319
## 1188     73  83.45274
## 1189     92  80.12935
## 1190    101  82.74097
## 1191     59  78.38666
## 1192     88  85.76137
## 1193     96  80.03589
## 1194     82  75.26520
## 1195     83  79.94641
## 1196     87  86.44159
## 1197     99  84.42915
## 1198     53  76.77597
## 1199     79  85.49227
## 1200    114  85.43649
## 1201    103  82.01907
## 1202     87  85.49127
## 1203     92  76.48413
## 1204    100  72.47560
## 1205    108  74.71764
## 1206     97  79.17754
## 1207     71  75.89540
## 1208     83  72.52410
## 1209     84  72.99217
## 1210     79  73.95031
## 1211     90  80.28342
## 1212     89  76.84542
## 1213     89  72.66814
## 1214     98  79.43934
## 1215     96  84.21485
## 1216    101  76.13103
## 1217     88  75.85239
## 1218     90  80.34523
## 1219     87  81.40989
## 1220     98  89.32086
## 1221    101  71.65314
## 1222    110  83.47910
## 1223     56  79.31333
## 1224     85  83.99140
## 1225     63  86.04019
## 1226     58  80.58368
## 1227     52  78.55305
## 1228     76  81.19871
## 1229     56  80.75316
## 1230     74  83.34638
## 1231     59  72.71053
## 1232     76  83.05962
## 1233     99  75.30919
## 1234     78  61.84184
## 1235    102  75.49493
## 1236     91  66.04304
## 1237     85  67.75688
## 1238    100  69.12003
## 1239     88  71.72482
## 1240     96  69.96031
## 1241     66  70.90493
## 1242     78  82.19781
## 1243     94  78.83265
## 1244     96  79.97341
## 1245     72  73.89507
## 1246     46  74.58934
## 1247     69  77.85825
## 1248     73  80.11585
## 1249     85  72.59577
## 1250     92  73.24053
## 1251     62  74.94112
## 1252     63  80.93204
## 1253     71  82.70670
## 1254     80  83.81551
## 1255     90  81.01255
## 1256     91  83.63764
## 1257     81  83.91016
## 1258     67  75.41719
## 1259     97  77.28019
## 1260     76  85.05006
## 1261     67  76.05888
## 1262     74  78.48064
## 1263    102  86.08551
## 1264    103  75.94236
## 1265    101  79.40133
## 1266     88  84.21428
## 1267     93  76.93622
## 1268     73  76.13965
## 1269     95  80.08179
## 1270     79  80.51164
## 1271     80  78.57486
## 1272     90  82.35286
## 1273     97  91.28096
## 1274     98  96.60549
## 1275     57  94.87582
## 1276     95  92.31089
## 1277     98  94.05118
## 1278     79  94.20321
## 1279     78  96.22611
## 1280     72  87.91910
## 1281     75  83.99419
## 1282     61  78.77420
## 1283     71  75.09218
## 1284     69  82.55089
## 1285     92  71.98056
## 1286     83  69.20131
## 1287     89  77.96775
## 1288     84  87.78023
## 1289     91  86.35932
## 1290     76  82.21422
## 1291     74  79.65022
## 1292     79  75.35074
## 1293     66  81.20123
## 1294     64  90.70037
## 1295     74  84.24705
## 1296     93  78.67993
## 1297     93  87.97745
## 1298     69  87.69785
## 1299     78  92.90616
## 1300     61  96.57243
## 1301     88  85.69885
## 1302     98  78.76782
## 1303     92  73.07410
## 1304     90  77.54960
## 1305     91  83.26963
## 1306     58  78.98654
## 1307     83  79.07793
## 1308     85  80.37722
## 1309     87  79.71387
## 1310     90  78.50491
## 1311     88  76.75080
## 1312     88  80.27969
## 1313     74  73.46866
## 1314     87  79.25616
## 1315     77  82.00955
## 1316     88  79.89839
## 1317     86  91.02800
## 1318     92  88.08337
## 1319     50  88.85369
## 1320     42  94.45356
## 1321     79  84.35462
## 1322     67  78.05740
## 1323     82  83.95127
## 1324     67  86.33623
## 1325     86  82.90238
## 1326     87  81.67246
## 1327     83  85.51611
## 1328     97  80.18255
## 1329    106  79.54560
## 1330     76  75.54402
## 1331     87  73.65885
## 1332    112  71.01422
## 1333    110  81.45969
## 1334    102  82.86082
## 1335     83  80.62995
## 1336     80  88.25696
## 1337     78  76.46766
## 1338     87  80.34835
## 1339     72  72.91620
## 1340     86  76.90302
## 1341     85  83.18826
## 1342     62  70.94189
## 1343     55  76.87526
## 1344     63  80.45797
## 1345     67  69.39293
## 1346     76  84.77448
## 1347     64  85.62956
## 1348     57  83.98628
## 1349     76  80.12358
## 1350     70  86.59676
## 1351     83  87.14948
## 1352     77  89.34894
## 1353     89  96.16870
## 1354     53  89.87599
## 1355     89  82.54057
## 1356     87  72.35155
## 1357     80  75.48538
## 1358     87  89.39629
## 1359     75  90.79489
## 1360     82  72.17537
## 1361     78  98.82305
## 1362     68  91.07166
## 1363     68  74.70489
## 1364     83  74.08408
## 1365     67  75.45591
## 1366     70  75.76548
## 1367     80  67.74708
## 1368     78  63.73702
## 1369     88  63.09795
## 1370     81  79.36250
## 1371     90  85.89266
## 1372     75  88.93737
## 1373     87  86.42517
## 1374     75  79.24316
## 1375     85  80.67956
## 1376     92  74.32748
## 1377     98  77.51726
## 1378     51  74.68399
## 1379     53  90.58364
## 1380    100  84.36898
## 1381     70  89.61974
## 1382     74  87.27046
## 1383     81  91.72583
## 1384     75  90.46183
## 1385     90  92.84914
## 1386     79  83.81388
## 1387     68  78.44059
## 1388     84  78.52910
## 1389     88  84.60661
## 1390     81  80.35381
## 1391     74  81.01220
## 1392     61  81.58834
## 1393     67  83.27561
## 1394     69  83.42052
## 1395     65  88.09058
## 1396    101  80.71739
## 1397    106  82.57078
## 1398     88  79.27823
## 1399     96  79.45690
## 1400     90  87.45363
## 1401     69  82.67362
## 1402     48  83.26842
## 1403     83  79.96190
## 1404     71  83.17930
## 1405     77  81.28235
## 1406     87  88.91157
## 1407     69  79.89535
## 1408     57  82.59148
## 1409     81  83.88969
## 1410     95  79.11848
## 1411     91  80.02279
## 1412     92  79.38053
## 1413     91  79.62617
## 1414     84  81.86965
## 1415     73  81.52487
## 1416     67  75.59362
## 1417     87  76.78916
## 1418     76  74.30279
## 1419     67  76.30224
## 1420     89  69.65634
## 1421     80  74.26352
## 1422     88  83.82491
## 1423     78  84.75257
## 1424     74  73.73032
## 1425     67  75.06767
## 1426    107  84.63083
## 1427     52  78.97639
## 1428     64  76.45999
## 1429     94  71.29329
## 1430     96  81.13600
## 1431     50  80.70558
## 1432     49  78.14821
## 1433     54  71.04719
## 1434     55  80.36170
## 1435     45  81.90376
## 1436     96  93.33645
## 1437     81  88.19413
## 1438     80  90.25974
## 1439     89  84.66770
## 1440     98  82.96837
## 1441     89  89.12204
## 1442     78  87.95702
## 1443     83  82.57955
## 1444     73  82.70475
## 1445     80  79.25230
## 1446     95  80.20210
## 1447     98  71.72168
## 1448    104  74.42225
## 1449     82  79.63366
## 1450    105  81.94346
## 1451     87  82.75808
## 1452     96  80.78277
## 1453    104  76.62972
## 1454     92  77.85689
## 1455    105  68.36665
## 1456     98  69.11303
## 1457     88  66.24626
## 1458    103  81.84655
## 1459     87  78.35787
## 1460     73  69.94233
## 1461     81  75.76441
## 1462     63  81.02432
## 1463     72  74.52049
## 1464     88  81.01784
## 1465     81  71.46302
## 1466     79  75.61764
## 1467     77  78.34152
## 1468     93  77.83881
## 1469     82  77.22750
## 1470     67  86.79195
## 1471     92  77.71251
## 1472     98  82.62507
## 1473     83  81.35461
## 1474     56  89.93796
## 1475     79  96.51608
## 1476     87  86.16285
## 1477     81  91.33961
## 1478     78  86.93554
## 1479     99  84.63384
## 1480     99  72.70103
## 1481     86  71.08248
## 1482     98  91.35231
## 1483     79  86.00672
## 1484     75  91.94352
## 1485     64  85.43193
## 1486     71  83.53370
## 1487     87  85.50160
## 1488     71  80.42214
## 1489     95  87.39467
## 1490     81  93.21013
## 1491     98  83.54572
## 1492     93  77.31713
## 1493     84  89.01612
## 1494     63  89.92811
## 1495     64  76.09057
## 1496    111  77.41702
## 1497     83  85.67225
## 1498     63  87.78315
## 1499     62  71.42951
## 1500     56  78.50359
## 1501     72  70.81870
## 1502     59  80.91072
## 1503    102  75.32227
## 1504     93  68.16678
## 1505     81  79.64992
## 1506     94  71.17387
## 1507     65  79.90700
## 1508     73  84.00078
## 1509     79  82.16401
## 1510     98  77.16445
## 1511     98  76.00898
## 1512     86  78.13104
## 1513     77  72.61470
## 1514     79  77.78654
## 1515    108  77.61568
## 1516    102  72.10171
## 1517     86  81.84125
## 1518     76  77.74045
## 1519     80  78.32464
## 1520     64  67.84336
## 1521     91  80.92785

mean (apply(predicted_values, 1, min)/apply(predicted_values, 1, max)) # calculate accuracy

## [1] 0.8553814

The prediction accuracy here is at 85.85%

predicted <- predict(Model5, newx = test_baseball)# predict on test data
predicted_values <- cbind (actual=test_baseball$TARGET_WINS, predicted)  # combine

## Warning in cbind(actual = test_baseball$TARGET_WINS, predicted): number of
## rows of result is not a multiple of vector length (arg 1)

predicted_values

##      actual predicted
## 1        70  67.44342
## 2        80  67.45113
## 3        78  70.07451
## 4        88  75.43397
## 5        81  67.72684
## 6        90  67.72587
## 7        75  63.76240
## 8        87  77.57821
## 9        75  87.50168
## 10       85  75.33778
## 11       92  86.57140
## 12       98  76.10532
## 13       51  79.30210
## 14       53  85.05514
## 15      100  89.33395
## 16       70  86.35917
## 17       74  75.87327
## 18       81  75.27661
## 19       75  76.92580
## 20       90  72.38257
## 21       79  91.24888
## 22       68  81.21417
## 23       84  76.85482
## 24       88  75.73465
## 25       81  92.95906
## 26       74  79.74796
## 27       61  82.59460
## 28       67  82.93132
## 29       69  84.01025
## 30       65  83.67314
## 31      101  74.98318
## 32      106  94.40116
## 33       88  85.35736
## 34       96  86.31272
## 35       90  85.20792
## 36       69  79.86968
## 37       48  71.39335
## 38       83  79.65920
## 39       71  88.48478
## 40       77  92.89590
## 41       87  95.70531
## 42       69  80.15908
## 43       57  77.64883
## 44       81  67.18263
## 45       95  64.94270
## 46       91  62.58915
## 47       92  71.88851
## 48       91  67.42428
## 49       84  66.89260
## 50       73  74.11101
## 51       67  84.19595
## 52       87  77.19389
## 53       76  81.04087
## 54       67  78.77630
## 55       89  78.14143
## 56       80  76.04199
## 57       88  79.46042
## 58       78  73.69400
## 59       74  74.54247
## 60       67  75.74929
## 61      107  74.97222
## 62       52  75.44410
## 63       64  79.02010
## 64       94  69.75831
## 65       96  70.64981
## 66       50  69.12150
## 67       49  74.71038
## 68       54  69.62466
## 69       55  70.17964
## 70       45  70.25523
## 71       96  69.23863
## 72       81  71.09084
## 73       80  72.67250
## 74       89  67.57287
## 75       98  73.96325
## 76       89  82.29090
## 77       78  76.84995
## 78       83  80.96384
## 79       73  86.00466
## 80       80  80.99199
## 81       95  90.30120
## 82       98  86.68144
## 83      104  76.35323
## 84       82  83.38313
## 85      105  86.70403
## 86       87  88.15920
## 87       96  77.29176
## 88      104  78.74435
## 89       92  78.89828
## 90      105  84.86922
## 91       98  76.52274
## 92       88  68.53117
## 93      103  80.63921
## 94       87  79.04574
## 95       73  91.06636
## 96       81  78.71918
## 97       63  73.94550
## 98       72  72.47404
## 99       88  76.97092
## 100      81  78.30645
## 101      79  77.84933
## 102      77  73.92133
## 103      93  87.30513
## 104      82  76.38672
## 105      67  87.67499
## 106      92  77.46469
## 107      98  82.05258
## 108      83  85.24247
## 109      56  87.16002
## 110      79  83.45556
## 111      87  83.35626
## 112      81  86.73727
## 113      78  91.21914
## 114      99  88.04261
## 115      99  78.51105
## 116      86  81.95540
## 117      98  82.76242
## 118      79  81.70924
## 119      75  85.27418
## 120      64  69.20964
## 121      71  69.37010
## 122      87  78.11881
## 123      71  69.72151
## 124      95  66.82142
## 125      81  65.28600
## 126      98  71.84567
## 127      93  71.77897
## 128      84  76.57889
## 129      63  79.01450
## 130      64  85.26364
## 131     111  75.40463
## 132      83  82.54799
## 133      63  78.50081
## 134      62  82.48798
## 135      56  84.16834
## 136      72  81.26085
## 137      59  80.88645
## 138     102  75.53196
## 139      93  80.44320
## 140      81  84.92377
## 141      94  75.02694
## 142      65  75.21066
## 143      73  71.99137
## 144      79  71.21597
## 145      98  81.47611
## 146      98  80.90290
## 147      86  80.66912
## 148      77  82.13911
## 149      79  74.68290
## 150     108  73.79684
## 151     102  69.14518
## 152      86  69.07586
## 153      76  74.64032
## 154      80  75.73128
## 155      64  71.89865
## 156      91  78.09027
## 157      81  76.71291
## 158      93  77.55373
## 159      71  77.24322
## 160      93  75.43189
## 161      65  81.07324
## 162      93  75.80984
## 163      92  75.01335
## 164      92  87.32782
## 165     101  87.38260
## 166      97  85.36363
## 167      98  72.33313
## 168      94  88.77234
## 169      79  80.72687
## 170      87  81.51611
## 171      62  79.82869
## 172      60  79.48619
## 173      70  79.26894
## 174      61  88.37324
## 175      57  86.43100
## 176      76  85.22346
## 177      87  83.65356
## 178      97  80.09002
## 179      68  88.30086
## 180      80  79.69794
## 181      88  84.31400
## 182      83  74.81675
## 183      67  80.47708
## 184      77  79.70448
## 185      84  78.06843
## 186      77  85.14572
## 187      70  82.05263
## 188      64  94.71854
## 189      85  96.65872
## 190     106  87.64550
## 191     100  90.21192
## 192      85  87.23107
## 193      93  78.75851
## 194      97  79.00789
## 195      82  81.25164
## 196     101  81.88150
## 197      88  84.03268
## 198      79  82.40478
## 199      85  95.78563
## 200      72  80.30579
## 201      84  75.01089
## 202     104  95.34271
## 203      85  87.50287
## 204      66  87.19443
## 205      43  76.17791
## 206      64  76.06067
## 207      76  64.91472
## 208      80  65.72480
## 209      64  72.39191
## 210      83  77.65363
## 211      78  84.15721
## 212      65  82.04471
## 213      66  86.08161
## 214      66  87.63339
## 215      69  75.05263
## 216      79  74.98438
## 217      79  83.25936
## 218      89  82.71286
## 219      82  75.53272
## 220      76  76.88389
## 221      85  71.86619
## 222      87  76.06333
## 223      89  74.51141
## 224      72  70.23775
## 225      91  73.52272
## 226      72  79.16581
## 227      77  86.89115
## 228      65  87.13819
## 229      58  85.46662
## 230      80  89.75357
## 231      93  91.54361
## 232      88  90.59831
## 233     102  95.02468
## 234      97  75.40648
## 235      88  79.71284
## 236      75  75.96297
## 237      91  85.91518
## 238     100  78.85678
## 239      88  76.84806
## 240      89  86.14862
## 241      79  78.05120
## 242      73  77.47512
## 243      78  78.23876
## 244      81  73.50749
## 245      88  86.78095
## 246      68  79.08171
## 247      93  84.72774
## 248      96  78.34657
## 249      95  82.76116
## 250      71  86.34586
## 251      78  87.92397
## 252      74  84.11055
## 253      56  77.99811
## 254      68  85.45186
## 255      81  77.80467
## 256      59  75.25132
## 257      91  76.95882
## 258      89  68.38314
## 259      79  73.50657
## 260      97  94.02116
## 261      77  93.71058
## 262      69  89.96523
## 263      53  80.98936
## 264      65  84.84977
## 265      56  85.19842
## 266      62  79.47680
## 267      64  79.10396
## 268      60  95.64288
## 269      85  86.71087
## 270      80  85.77351
## 271      78  88.98923
## 272      68  80.79937
## 273      70  77.98964
## 274      90  75.28088
## 275      83  77.35333
## 276      51  76.26755
## 277      61  75.33738
## 278      73  88.17904
## 279      92  96.27179
## 280     101  89.92458
## 281      59  92.67221
## 282      88  84.05349
## 283      96  74.83770
## 284      82  85.95691
## 285      83  78.72426
## 286      87  78.71273
## 287      99  81.26346
## 288      53  85.39316
## 289      79  94.85380
## 290     114  98.58835
## 291     103  82.07467
## 292      87  75.18221
## 293      92  76.37619
## 294     100  82.90378
## 295     108  88.14256
## 296      97  81.52590
## 297      71  82.87604
## 298      83  78.59107
## 299      84  75.43887
## 300      79  75.87870
## 301      90  77.95901
## 302      89  77.42188
## 303      89  79.91021
## 304      98  75.70473
## 305      96  72.30603
## 306     101  74.72546
## 307      88  77.04007
## 308      90  84.57445
## 309      87  79.03250
## 310      98  75.61020
## 311     101  85.12809
## 312     110  83.01153
## 313      56  81.21832
## 314      85  77.58116
## 315      63  76.16731
## 316      58  71.52026
## 317      52  75.22951
## 318      76  75.48431
## 319      56  76.87937
## 320      74  75.93558
## 321      59  75.95209
## 322      76  82.46585
## 323      99  74.48463
## 324      78  77.49127
## 325     102  75.66515
## 326      91  77.13311
## 327      85  79.35974
## 328     100  78.58116
## 329      88  85.79704
## 330      96  87.45633
## 331      66  82.37835
## 332      78  86.72390
## 333      94  75.70592
## 334      96  80.05905
## 335      72  80.56484
## 336      46  81.56194
## 337      69  79.82098
## 338      73  87.10452
## 339      85  79.65431
## 340      92  76.07465
## 341      62  83.48986
## 342      63  80.22453
## 343      71  82.98359
## 344      80  82.75474
## 345      90  75.27901
## 346      91  79.49034
## 347      81  82.43166
## 348      67  82.29389
## 349      97  84.23164
## 350      76  76.59806
## 351      67  74.96025
## 352      74  76.25467
## 353     102  68.62133
## 354     103  73.26311
## 355     101  75.04697
## 356      88  81.57823
## 357      93  76.97103
## 358      73  84.95317
## 359      95  88.65480
## 360      79  81.29242
## 361      80  80.11348
## 362      90  81.05779
## 363      97  78.21721
## 364      98  78.12708
## 365      57  77.36967
## 366      95  77.29128
## 367      98  81.79287
## 368      79  87.54212
## 369      78  85.17605
## 370      72  81.07061
## 371      75  85.04887
## 372      61  78.68070
## 373      71  77.38031
## 374      69  70.92452
## 375      92  74.03955
## 376      83  71.75593
## 377      89  75.89851
## 378      84  76.24042
## 379      91  78.59571
## 380      76  82.86759
## 381      74  86.95993
## 382      79  84.03568
## 383      66  87.55968
## 384      64  85.98648
## 385      74  82.14632
## 386      93  85.69631
## 387      93  87.07823
## 388      69  86.90617
## 389      78  81.14569
## 390      61  74.98691
## 391      88  74.22010
## 392      98  66.23871
## 393      92  70.80897
## 394      90  79.02572
## 395      91  74.46659
## 396      58  76.17838
## 397      83  78.85746
## 398      85  75.16575
## 399      87  90.13430
## 400      90  76.43702
## 401      88  80.42516
## 402      88  71.76322
## 403      74  85.62887
## 404      87  88.54009
## 405      77  85.41391
## 406      88  80.87090
## 407      86  76.33751
## 408      92  85.13585
## 409      50  76.30060
## 410      42  72.82338
## 411      79  85.53497
## 412      67  88.16092
## 413      82  94.67713
## 414      67  92.65560
## 415      86  90.99936
## 416      87  85.43587
## 417      83  92.80923
## 418      97  89.13903
## 419     106  87.66494
## 420      76  86.26550
## 421      87  87.30838
## 422     112  95.64263
## 423     110  89.77173
## 424     102  96.58498
## 425      83  89.08766
## 426      80  84.87436
## 427      78  79.09352
## 428      87  83.13129
## 429      72  74.78158
## 430      86  75.01630
## 431      85  75.37997
## 432      62  91.42830
## 433      55  89.53917
## 434      63  80.57424
## 435      67  77.44170
## 436      76  80.56396
## 437      64  86.13667
## 438      57  86.17377
## 439      76  76.58950
## 440      70  76.80901
## 441      83  83.22035
## 442      77  78.71702
## 443      89  68.39029
## 444      53  74.51458
## 445      89  65.97702
## 446      87  69.94125
## 447      80  78.07337
## 448      87  79.56762
## 449      75  75.47085
## 450      82  78.95959
## 451      78  82.04844
## 452      68  78.93698
## 453      68  75.55269
## 454      83  73.11614
## 455      67  71.16156
## 456      70  68.87201
## 457      80  74.10248
## 458      78  80.17785
## 459      88  77.30115
## 460      81  72.49663
## 461      90  77.63991
## 462      75  72.55387
## 463      87  75.26255
## 464      75  89.56768
## 465      85  89.34414
## 466      92  81.88434
## 467      98  84.87743
## 468      51  81.65693
## 469      53  82.50675
## 470     100  81.01440
## 471      70  75.36897
## 472      74  74.83014
## 473      81  86.95051
## 474      75  75.64894
## 475      90  71.91194
## 476      79  88.80437
## 477      68  72.45152
## 478      84  85.64352
## 479      88  87.18982
## 480      81  88.23807
## 481      74  97.61437
## 482      61  94.15000
## 483      67  85.46822
## 484      69  81.29913
## 485      65  83.24641
## 486     101  73.97669
## 487     106  78.98370
## 488      88  87.86617
## 489      96  90.77691
## 490      90  82.77272
## 491      69  81.84398
## 492      48  78.91267
## 493      83  78.56951
## 494      71  90.81276
## 495      77  82.15841
## 496      87  77.31243
## 497      69  91.95794
## 498      57  87.79896
## 499      81  76.99782
## 500      95  79.89077
## 501      91  74.67528
## 502      92  86.66090
## 503      91  76.67202
## 504      84  84.36278
## 505      73  82.37365
## 506      67  77.49743
## 507      87  72.37151
## 508      76  75.22012
## 509      67  73.18528
## 510      89  71.40753
## 511      80  78.79972
## 512      88  77.39860
## 513      78  81.06421
## 514      74  78.92919
## 515      67  83.29334
## 516     107  92.13062
## 517      52  86.88210
## 518      64  86.09691
## 519      94  83.65993
## 520      96  80.00122
## 521      50  75.79357
## 522      49  71.29807
## 523      54  80.14395
## 524      55  85.04878
## 525      45  86.42506
## 526      96  77.88073
## 527      81  89.99265
## 528      80  93.12574
## 529      89  86.35583
## 530      98  82.66900
## 531      89  86.55700
## 532      78  76.00762
## 533      83  75.10227
## 534      73  76.24862
## 535      80  93.95561
## 536      95  86.80163
## 537      98  94.03234
## 538     104  86.58101
## 539      82  86.56917
## 540     105  88.22369
## 541      87  85.39937
## 542      96  81.25140
## 543     104  82.13859
## 544      92  80.30914
## 545     105  74.44826
## 546      98  76.10264
## 547      88  74.08154
## 548     103  69.92862
## 549      87  69.95132
## 550      73  74.53685
## 551      81  77.21791
## 552      63  68.76742
## 553      72  67.21485
## 554      88  78.58185
## 555      81  74.37172
## 556      79  77.15918
## 557      77  83.53078
## 558      93  84.49880
## 559      82  86.41669
## 560      67  75.22195
## 561      92  73.30565
## 562      98  76.43071
## 563      83  67.01121
## 564      56  78.15236
## 565      79  84.22717
## 566      87 103.62586
## 567      81  99.78573
## 568      78  92.94880
## 569      99  96.14162
## 570      99  93.83560
## 571      86  87.21972
## 572      98  80.56718
## 573      79  75.47100
## 574      75  85.48369
## 575      64  81.64591
## 576      71  80.88031
## 577      87  72.00168
## 578      71  98.77644
## 579      95  95.99900
## 580      81  95.74313
## 581      98  81.34501
## 582      93  90.01800
## 583      84  90.45721
## 584      63  93.97424
## 585      64  79.58646
## 586     111  83.30092
## 587      83  80.62590
## 588      63  74.64967
## 589      62  81.50964
## 590      56  77.75612
## 591      72  68.67546
## 592      59  67.09276
## 593     102  74.64771
## 594      93  74.22550
## 595      81  91.69689
## 596      94  88.52477
## 597      65  83.85911
## 598      73  85.75061
## 599      79  84.90506
## 600      98  78.19194
## 601      98  86.69231
## 602      86  84.65045
## 603      77  90.56613
## 604      79  88.21833
## 605     108  89.20599
## 606     102  96.33209
## 607      86  89.50356
## 608      76  96.00533
## 609      80  94.62374
## 610      64  93.39055
## 611      91  95.00998
## 612      81  80.83454
## 613      93  75.38982
## 614      71  76.75518
## 615      93  78.99632
## 616      65  84.62905
## 617      93  82.87983
## 618      92  89.69512
## 619      92  80.89995
## 620     101  77.82870
## 621      97  76.45063
## 622      98  78.45039
## 623      94  76.99874
## 624      79  87.95743
## 625      87  92.81547
## 626      62  80.81087
## 627      60  78.03260
## 628      70  81.21524
## 629      61  79.27102
## 630      57  80.71248
## 631      76  81.70592
## 632      87  78.63917
## 633      97  75.29746
## 634      68  72.33944
## 635      80  76.57523
## 636      88  81.96928
## 637      83  82.33040
## 638      67  85.74432
## 639      77  85.14898
## 640      84  84.98782
## 641      77  90.15115
## 642      70  87.60581
## 643      64  92.92149
## 644      85  95.31093
## 645     106  81.02289
## 646     100  76.88477
## 647      85  84.77585
## 648      93  88.93106
## 649      97  82.10155
## 650      82  89.52683
## 651     101  86.02564
## 652      88  79.37780
## 653      79  84.48734
## 654      85  82.93491
## 655      72  68.22435
## 656      84  75.99833
## 657     104  85.70357
## 658      85  79.57618
## 659      66  70.89438
## 660      43  89.49693
## 661      64  77.55810
## 662      76  81.40149
## 663      80  73.90587
## 664      64  73.55784
## 665      83  83.10389
## 666      78  78.39615
## 667      65  77.05779
## 668      66  80.72315
## 669      66  73.46132
## 670      69  61.90398
## 671      79  62.28685
## 672      79  73.70169
## 673      89  74.54397
## 674      82  73.84600
## 675      76  61.25132
## 676      85  79.47124
## 677      87  83.26075
## 678      89  71.60880
## 679      72  83.55334
## 680      91  75.68196
## 681      72  76.36565
## 682      77  80.20358
## 683      65  79.05666
## 684      58  77.81118
## 685      80  83.28782
## 686      93  74.22485
## 687      88  76.82253
## 688     102  77.12806
## 689      97  80.63736
## 690      88  77.08373
## 691      75  71.73085
## 692      91  74.57166
## 693     100  79.22417
## 694      88  86.49261
## 695      89  82.07615
## 696      79  86.23495
## 697      73  96.73694
## 698      78  83.63311
## 699      81  82.47753
## 700      88  79.80981
## 701      68  80.53284
## 702      93  81.83612
## 703      96  76.42994
## 704      95  73.20961
## 705      71  75.03780
## 706      78  74.59428
## 707      74  84.51842
## 708      56  84.17397
## 709      68  85.30279
## 710      81  78.78881
## 711      59  84.07513
## 712      91  79.38092
## 713      89  80.61640
## 714      79  78.04178
## 715      97  84.50836
## 716      77  83.54679
## 717      69  80.08120
## 718      53  82.91023
## 719      65  78.49132
## 720      56  83.35613
## 721      62  73.32258
## 722      64  71.67373
## 723      60  98.19085
## 724      85  98.01751
## 725      80  78.46787
## 726      78  89.91365
## 727      68  75.16995
## 728      70  75.50304
## 729      90  67.49380
## 730      83  79.75337
## 731      51  83.97999
## 732      61  85.34265
## 733      73  82.35342
## 734      92  80.79035
## 735     101  75.78706
## 736      59  74.82875
## 737      88  75.08221
## 738      96  82.44515
## 739      82  78.18402
## 740      83  76.62192
## 741      87  84.27091
## 742      99  78.70232
## 743      53  80.78674
## 744      79  76.73710
## 745     114  76.18862
## 746     103  68.60446
## 747      87  71.96965
## 748      92  84.90424
## 749     100  80.17055
## 750     108  83.99648
## 751      97  86.59310
## 752      71  75.47677
## 753      83  75.53225
## 754      84  86.19133
## 755      79  85.54998
## 756      90  89.02909
## 757      89  86.18981
## 758      89  97.91304
## 759      98  98.63173
## 760      96  92.47344
## 761     101 104.48665
## 762      88  97.33576
## 763      90 100.85301
## 764      87  92.93128
## 765      98  82.82575
## 766     101  87.45635
## 767     110  85.89462
## 768      56  82.10294
## 769      85  89.28083
## 770      63  92.29101
## 771      58  72.82231
## 772      52  72.27080
## 773      76  67.86196
## 774      56  62.46698
## 775      74  74.97552
## 776      59  82.33372
## 777      76  73.87224
## 778      99  76.05064
## 779      78  77.17043
## 780     102  82.98197
## 781      91  77.89406
## 782      85  88.99333
## 783     100  86.81349
## 784      88  86.70833
## 785      96  85.03963
## 786      66  73.59994
## 787      78  81.58437
## 788      94  75.50149
## 789      96  72.12338
## 790      72  68.86495
## 791      46  80.02704
## 792      69  77.39364
## 793      73  78.07165
## 794      85  77.63262
## 795      92  82.97429
## 796      62  79.14137
## 797      63  89.78282
## 798      71  89.27390
## 799      80  83.86970
## 800      90  77.30894
## 801      91  69.58990
## 802      81  72.98084
## 803      67  88.69279
## 804      97  87.92394
## 805      76  91.70302
## 806      67  82.67101
## 807      74  71.85679
## 808     102  68.10150
## 809     103  81.84717
## 810     101  78.12253
## 811      88  69.83695
## 812      93  76.46566
## 813      73  86.11409
## 814      95  90.88470
## 815      79  91.80391
## 816      80  92.11302
## 817      90  84.80266
## 818      97  74.65166
## 819      98  77.57825
## 820      57  76.78635
## 821      95  82.36152
## 822      98  82.50355
## 823      79  88.04743
## 824      78  87.72790
## 825      72  76.63681
## 826      75  75.42680
## 827      61  82.96084
## 828      71  78.21579
## 829      69  76.27439
## 830      92  73.79846
## 831      83  81.10167
## 832      89  75.42576
## 833      84  76.49613
## 834      91  74.59911
## 835      76  89.20900
## 836      74  87.22479
## 837      79  75.04804
## 838      66  71.78931
## 839      64  78.02374
## 840      74  76.61472
## 841      93  63.46132
## 842      93  73.67504
## 843      69  77.48161
## 844      78  80.45738
## 845      61  78.22131
## 846      88  80.05917
## 847      98  80.85200
## 848      92  80.57371
## 849      90  86.57068
## 850      91  85.80458
## 851      58  83.86479
## 852      83  83.11116
## 853      85  86.62546
## 854      87  86.96736
## 855      90  81.63054
## 856      88  84.51782
## 857      88  83.66754
## 858      74  90.72007
## 859      87  84.56464
## 860      77  79.70922
## 861      88  80.94532
## 862      86  79.55306
## 863      92  83.57741
## 864      50  76.30040
## 865      42  79.15809
## 866      79  72.63969
## 867      67  77.97984
## 868      82  80.06715
## 869      67  75.38970
## 870      86  80.66269
## 871      87  75.27283
## 872      83  83.24337
## 873      97  74.94358
## 874     106  81.51085
## 875      76  83.24421
## 876      87  84.04611
## 877     112  77.97604
## 878     110  75.08056
## 879     102  72.00019
## 880      83  78.40749
## 881      80  74.36827
## 882      78  69.61521
## 883      87  78.99502
## 884      72  76.21166
## 885      86  79.95868
## 886      85  85.97882
## 887      62  74.10177
## 888      55  74.43285
## 889      63  76.14842
## 890      67  76.55578
## 891      76  71.64883
## 892      64  88.44513
## 893      57  82.81897
## 894      76  79.19356
## 895      70  74.99922
## 896      83  84.37986
## 897      77  74.92339
## 898      89  81.98816
## 899      53  83.05794
## 900      89  79.14053
## 901      87  76.41732
## 902      80  77.00170
## 903      87  83.40517
## 904      75  77.80671
## 905      82  80.72405
## 906      78  84.92448
## 907      68  76.37162
## 908      68  80.14464
## 909      83  81.99346
## 910      67  79.12052
## 911      70  63.72078
## 912      80  60.35400
## 913      78  66.78161
## 914      88  60.68783
## 915      81  59.18272
## 916      90  70.53416
## 917      75  81.46170
## 918      87  71.45605
## 919      75  74.57813
## 920      85  71.72674
## 921      92  75.34780
## 922      98  81.52163
## 923      51  83.94570
## 924      53  91.68011
## 925     100  85.36356
## 926      70  81.99874
## 927      74  84.14420
## 928      81  73.31153
## 929      75  78.85476
## 930      90  76.00103
## 931      79  77.32058
## 932      68  73.77252
## 933      84  90.41431
## 934      88  85.21387
## 935      81  73.53234
## 936      74  76.49770
## 937      61  70.54324
## 938      67  84.21458
## 939      69  97.21267
## 940      65  80.64946
## 941     101  79.82280
## 942     106  76.58877
## 943      88  76.01548
## 944      96  75.33370
## 945      90  71.64498
## 946      69  84.87058
## 947      48  79.01173
## 948      83  81.63345
## 949      71  77.29707
## 950      77  79.07006
## 951      87  78.70672
## 952      69  98.16423
## 953      57  85.26197
## 954      81  90.39484
## 955      95  94.47383
## 956      91 104.75405
## 957      92  92.78122
## 958      91  93.66794
## 959      84  95.31631
## 960      73 102.26987
## 961      67  95.77832
## 962      87  93.74198
## 963      76  87.68429
## 964      67  85.41279
## 965      89  86.50211
## 966      80  86.91786
## 967      88  86.91658
## 968      78  92.97927
## 969      74 100.04520
## 970      67  92.21314
## 971     107  86.56469
## 972      52  90.68067
## 973      64  89.20259
## 974      94  95.19330
## 975      96  84.80754
## 976      50  85.47340
## 977      49  79.99373
## 978      54  66.40110
## 979      55  74.33150
## 980      45  80.55107
## 981      96  77.92324
## 982      81  75.59030
## 983      80  74.68523
## 984      89  73.58853
## 985      98  79.11850
## 986      89  86.74757
## 987      78  91.30102
## 988      83  80.72652
## 989      73  91.67552
## 990      80  85.58264
## 991      95  85.24275
## 992      98  79.61319
## 993     104  94.02997
## 994      82  91.34655
## 995     105  86.76379
## 996      87  83.15531
## 997      96  80.73357
## 998     104  78.38100
## 999      92  82.05406
## 1000    105  84.55653
## 1001     98  85.12766
## 1002     88  85.11712
## 1003    103  94.73952
## 1004     87  92.33759
## 1005     73  89.29616
## 1006     81  89.76105
## 1007     63  87.44643
## 1008     72  91.54011
## 1009     88  93.41918
## 1010     81  92.01233
## 1011     79  91.22298
## 1012     77  93.76596
## 1013     93  80.15303
## 1014     82  74.74933
## 1015     67  75.69253
## 1016     92  76.00785
## 1017     98  71.17943
## 1018     83  66.08970
## 1019     56  80.56114
## 1020     79  85.16664
## 1021     87  87.91888
## 1022     81  74.29519
## 1023     78  73.74423
## 1024     99  72.48966
## 1025     99  76.45746
## 1026     86  79.55431
## 1027     98  82.36934
## 1028     79  77.09259
## 1029     75  85.04632
## 1030     64  87.48039
## 1031     71  94.97976
## 1032     87  91.99768
## 1033     71  90.99592
## 1034     95  89.30324
## 1035     81  81.71456
## 1036     98  79.33073
## 1037     93  89.00786
## 1038     84  87.12255
## 1039     63  81.65797
## 1040     64  81.35012
## 1041    111  68.60934
## 1042     83  67.88631
## 1043     63  68.54235
## 1044     62  67.56090
## 1045     56  69.46154
## 1046     72  82.63933
## 1047     59  85.29402
## 1048    102  74.25824
## 1049     93  76.50022
## 1050     81  79.50063
## 1051     94  78.73351
## 1052     65  77.56332
## 1053     73  71.55565
## 1054     79  70.39303
## 1055     98  74.11267
## 1056     98  85.19926
## 1057     86  78.36045
## 1058     77  77.45106
## 1059     79  83.83186
## 1060    108  76.32044
## 1061    102  86.73581
## 1062     86  83.39038
## 1063     76  83.93669
## 1064     80  83.00696
## 1065     64  86.60338
## 1066     91  86.10335
## 1067     81  88.47441
## 1068     93  83.14315
## 1069     71  91.80580
## 1070     93  88.58660
## 1071     65  82.50243
## 1072     93  79.89829
## 1073     92  91.77266
## 1074     92  85.07039
## 1075    101  83.01118
## 1076     97  85.68794
## 1077     98  95.76558
## 1078     94  90.58290
## 1079     79  91.94511
## 1080     87  96.31662
## 1081     62  84.96631
## 1082     60  69.83334
## 1083     70  73.56281
## 1084     61  72.00398
## 1085     57  74.46932
## 1086     76  73.08326
## 1087     87  71.27250
## 1088     97  84.53501
## 1089     68  83.59480
## 1090     80  81.98597
## 1091     88  86.48434
## 1092     83  81.75761
## 1093     67  76.77313
## 1094     77  76.37420
## 1095     84  75.89710
## 1096     77  77.61684
## 1097     70  81.74684
## 1098     64  81.41920
## 1099     85  76.82860
## 1100    106  74.59177
## 1101    100  79.96919
## 1102     85  94.95448
## 1103     93  83.66543
## 1104     97  76.84529
## 1105     82  87.13900
## 1106    101  71.93989
## 1107     88  78.74975
## 1108     79  78.76011
## 1109     85  66.61349
## 1110     72  70.79383
## 1111     84  70.84328
## 1112    104  71.91145
## 1113     85  67.12428
## 1114     66  71.62438
## 1115     43  71.33810
## 1116     64  71.90173
## 1117     76  73.78630
## 1118     80  81.48361
## 1119     64  84.80836
## 1120     83  83.19514
## 1121     78  81.29440
## 1122     65  73.80876
## 1123     66  67.63890
## 1124     66  73.59700
## 1125     69  79.16576
## 1126     79  72.63248
## 1127     79  72.41918
## 1128     89  73.04657
## 1129     82  66.49953
## 1130     76  73.95977
## 1131     85  71.42417
## 1132     87  69.30896
## 1133     89  71.54921
## 1134     72  76.78362
## 1135     91  82.63552
## 1136     72  95.12850
## 1137     77  83.90767
## 1138     65  84.47296
## 1139     58  87.77827
## 1140     80  80.02842
## 1141     93  84.42900
## 1142     88  85.19428
## 1143    102  73.29018
## 1144     97  78.51938
## 1145     88  75.82602
## 1146     75  71.69911
## 1147     91  75.95149
## 1148    100  85.64490
## 1149     88  83.34346
## 1150     89  77.77329
## 1151     79  74.54778
## 1152     73  86.37762
## 1153     78  80.14170
## 1154     81  83.43499
## 1155     88  85.01173
## 1156     68  79.65990
## 1157     93  97.22181
## 1158     96  78.07314
## 1159     95  91.60057
## 1160     71  92.29360
## 1161     78  92.95479
## 1162     74  89.26299
## 1163     56  79.41539
## 1164     68  80.53341
## 1165     81  73.59357
## 1166     59  81.24245
## 1167     91  83.56699
## 1168     89  82.38249
## 1169     79  93.33796
## 1170     97  85.35170
## 1171     77  76.41227
## 1172     69  79.07561
## 1173     53  72.52526
## 1174     65  95.66395
## 1175     56  87.66176
## 1176     62  92.90300
## 1177     64  99.28524
## 1178     60  86.39980
## 1179     85  81.58923
## 1180     80  85.33114
## 1181     78  93.70586
## 1182     68  93.36064
## 1183     70  77.33035
## 1184     90  79.55576
## 1185     83  75.40082
## 1186     51  79.76631
## 1187     61  81.47024
## 1188     73  80.17441
## 1189     92  77.22722
## 1190    101  81.74268
## 1191     59  78.60004
## 1192     88  84.63944
## 1193     96  79.37534
## 1194     82  76.55477
## 1195     83  78.77538
## 1196     87  85.52381
## 1197     99  84.44809
## 1198     53  77.59152
## 1199     79  87.63939
## 1200    114  86.29028
## 1201    103  84.57447
## 1202     87  87.19745
## 1203     92  77.20342
## 1204    100  73.46173
## 1205    108  73.29697
## 1206     97  78.67011
## 1207     71  74.68087
## 1208     83  71.33847
## 1209     84  70.73626
## 1210     79  70.52728
## 1211     90  78.88400
## 1212     89  71.53207
## 1213     89  71.39736
## 1214     98  74.93688
## 1215     96  81.09400
## 1216    101  72.57754
## 1217     88  75.39038
## 1218     90  75.68757
## 1219     87  80.16107
## 1220     98  89.40135
## 1221    101  72.16460
## 1222    110  83.38028
## 1223     56  81.32313
## 1224     85  84.89702
## 1225     63  84.84828
## 1226     58  80.52925
## 1227     52  76.62719
## 1228     76  79.65161
## 1229     56  79.83045
## 1230     74  82.55894
## 1231     59  73.84234
## 1232     76  81.12999
## 1233     99  73.31931
## 1234     78  63.40639
## 1235    102  75.90363
## 1236     91  67.21770
## 1237     85  70.21542
## 1238    100  68.87801
## 1239     88  72.77913
## 1240     96  70.17411
## 1241     66  71.05437
## 1242     78  81.34791
## 1243     94  79.48590
## 1244     96  80.68455
## 1245     72  72.75448
## 1246     46  74.81253
## 1247     69  78.33119
## 1248     73  80.06916
## 1249     85  73.74964
## 1250     92  74.50480
## 1251     62  75.05737
## 1252     63  80.77484
## 1253     71  80.54898
## 1254     80  80.71122
## 1255     90  80.60354
## 1256     91  84.06425
## 1257     81  81.92056
## 1258     67  74.13072
## 1259     97  74.34099
## 1260     76  82.79877
## 1261     67  78.63570
## 1262     74  79.79308
## 1263    102  86.05803
## 1264    103  77.78018
## 1265    101  79.84836
## 1266     88  84.47589
## 1267     93  79.79337
## 1268     73  76.72617
## 1269     95  79.21975
## 1270     79  80.25741
## 1271     80  79.14570
## 1272     90  83.44643
## 1273     97  93.98953
## 1274     98  94.00703
## 1275     57  93.68490
## 1276     95  91.38872
## 1277     98  93.75317
## 1278     79  92.88496
## 1279     78  91.70101
## 1280     72  85.48907
## 1281     75  80.65960
## 1282     61  74.69654
## 1283     71  75.32830
## 1284     69  81.12020
## 1285     92  70.93756
## 1286     83  70.80932
## 1287     89  74.57660
## 1288     84  85.39189
## 1289     91  86.64623
## 1290     76  84.02929
## 1291     74  79.94081
## 1292     79  78.06523
## 1293     66  84.11563
## 1294     64  88.72233
## 1295     74  86.22934
## 1296     93  79.80524
## 1297     93  86.74477
## 1298     69  87.62938
## 1299     78  92.83543
## 1300     61  94.36469
## 1301     88  86.01547
## 1302     98  80.54337
## 1303     92  73.29776
## 1304     90  78.68654
## 1305     91  82.39342
## 1306     58  78.33648
## 1307     83  79.95985
## 1308     85  81.96134
## 1309     87  81.59286
## 1310     90  79.05989
## 1311     88  77.86516
## 1312     88  82.34401
## 1313     74  75.16981
## 1314     87  80.07663
## 1315     77  82.51050
## 1316     88  82.61926
## 1317     86  96.35798
## 1318     92  91.78498
## 1319     50  91.83003
## 1320     42  96.02732
## 1321     79  86.45282
## 1322     67  82.53612
## 1323     82  86.96077
## 1324     67  87.22632
## 1325     86  84.18313
## 1326     87  83.26233
## 1327     83  89.16195
## 1328     97  81.82784
## 1329    106  80.15164
## 1330     76  76.91678
## 1331     87  72.78148
## 1332    112  70.53999
## 1333    110  78.85112
## 1334    102  81.40689
## 1335     83  83.17569
## 1336     80  87.87173
## 1337     78  75.63225
## 1338     87  78.58231
## 1339     72  73.46579
## 1340     86  78.71896
## 1341     85  82.70960
## 1342     62  73.33920
## 1343     55  77.90761
## 1344     63  79.98687
## 1345     67  70.52942
## 1346     76  84.46694
## 1347     64  98.40246
## 1348     57  86.63365
## 1349     76  78.47395
## 1350     70  85.92126
## 1351     83  83.71443
## 1352     77  87.85776
## 1353     89  94.71885
## 1354     53  90.06946
## 1355     89  81.89370
## 1356     87  71.95118
## 1357     80  75.62014
## 1358     87  88.90679
## 1359     75  95.62316
## 1360     82  72.47442
## 1361     78  97.62852
## 1362     68  88.88781
## 1363     68  73.04341
## 1364     83  74.20625
## 1365     67  74.85814
## 1366     70  75.56058
## 1367     80  67.96074
## 1368     78  64.83668
## 1369     88  66.12139
## 1370     81  77.36962
## 1371     90  86.11549
## 1372     75  88.08887
## 1373     87  84.97467
## 1374     75  79.91790
## 1375     85  82.47619
## 1376     92  76.35949
## 1377     98  78.68785
## 1378     51  76.76146
## 1379     53  90.75052
## 1380    100  83.07884
## 1381     70  90.01429
## 1382     74  88.37454
## 1383     81  92.99445
## 1384     75  90.64963
## 1385     90  90.30104
## 1386     79  83.17853
## 1387     68  79.07535
## 1388     84  77.46767
## 1389     88  84.16275
## 1390     81  79.55429
## 1391     74  80.16002
## 1392     61  80.89242
## 1393     67  83.19119
## 1394     69  81.77316
## 1395     65  88.11476
## 1396    101  78.96010
## 1397    106  82.05215
## 1398     88  78.88763
## 1399     96  79.95473
## 1400     90  87.94550
## 1401     69  84.21109
## 1402     48  82.94389
## 1403     83  81.63757
## 1404     71  82.86589
## 1405     77  82.07041
## 1406     87  88.16516
## 1407     69  83.92208
## 1408     57  82.40573
## 1409     81  82.94323
## 1410     95  79.72355
## 1411     91  80.43830
## 1412     92  81.25349
## 1413     91  77.38370
## 1414     84  78.72661
## 1415     73  78.92403
## 1416     67  73.09722
## 1417     87  76.51427
## 1418     76  75.89040
## 1419     67  75.09768
## 1420     89  69.03655
## 1421     80  73.11790
## 1422     88  80.32345
## 1423     78  81.62448
## 1424     74  72.46322
## 1425     67  73.05070
## 1426    107  82.60617
## 1427     52  76.80164
## 1428     64  74.71730
## 1429     94  72.53451
## 1430     96  79.24098
## 1431     50  80.59521
## 1432     49  77.15610
## 1433     54  74.10392
## 1434     55  79.46361
## 1435     45  81.59949
## 1436     96  92.84525
## 1437     81  86.74162
## 1438     80  89.00812
## 1439     89  83.81009
## 1440     98  82.45990
## 1441     89  86.81278
## 1442     78  87.08329
## 1443     83  81.02582
## 1444     73  81.79036
## 1445     80  76.09764
## 1446     95  79.58062
## 1447     98  70.85242
## 1448    104  73.77270
## 1449     82  78.06725
## 1450    105  82.56246
## 1451     87  82.22541
## 1452     96  82.97301
## 1453    104  77.64168
## 1454     92  79.54539
## 1455    105  68.24060
## 1456     98  70.24206
## 1457     88  67.33869
## 1458    103  80.95609
## 1459     87  77.96052
## 1460     73  70.39466
## 1461     81  75.42098
## 1462     63  79.98365
## 1463     72  72.26114
## 1464     88  80.33432
## 1465     81  72.40334
## 1466     79  76.28209
## 1467     77  77.74120
## 1468     93  76.98433
## 1469     82  74.78655
## 1470     67  84.57421
## 1471     92  77.67434
## 1472     98  84.70901
## 1473     83  78.88817
## 1474     56  86.51928
## 1475     79  94.59262
## 1476     87  83.68060
## 1477     81  90.09426
## 1478     78  86.45699
## 1479     99  85.18612
## 1480     99  73.40551
## 1481     86  73.27707
## 1482     98  91.99847
## 1483     79  86.52118
## 1484     75  92.42330
## 1485     64  84.04087
## 1486     71  83.86156
## 1487     87  85.52067
## 1488     71  80.89763
## 1489     95  86.58041
## 1490     81  91.77189
## 1491     98  82.89224
## 1492     93  78.98014
## 1493     84  87.14506
## 1494     63  89.37809
## 1495     64  74.04740
## 1496    111  75.81540
## 1497     83  84.01966
## 1498     63  88.51943
## 1499     62  72.64024
## 1500     56  79.09215
## 1501     72  70.49611
## 1502     59  79.19742
## 1503    102  74.80890
## 1504     93  70.05598
## 1505     81  79.93610
## 1506     94  71.64301
## 1507     65  81.23458
## 1508     73  84.56800
## 1509     79  81.05502
## 1510     98  77.11747
## 1511     98  78.69532
## 1512     86  76.56764
## 1513     77  74.31725
## 1514     79  79.24925
## 1515    108  77.95751
## 1516    102  73.35033
## 1517     86  82.50775
## 1518     76  77.38719
## 1519     80  79.51796
## 1520     64  67.76209
## 1521     91  80.89661

mean (apply(predicted_values, 1, min)/apply(predicted_values, 1, max)) # calculate accuracy

## [1] 0.8572953

The prediction accuracy for the OLS Model5 is at 85.94% which is not bad for this purpose. But lets compare it to the Champion Model- The improved Ridge Regression.

predicted <- predict(Model6_Improved, newx = test_Ind)# predict on test data
predicted_values <- cbind (actual=test_baseball$TARGET_WINS, predicted)  # combine
predicted_values

##      actual        s0
## 1        70  70.53413
## 4        80  76.99308
## 7        78  75.10850
## 9        88  87.10650
## 13       81  79.77267
## 14       90  86.65858
## 16       75  78.21489
## 21       87  88.81181
## 26       75  77.83838
## 33       85  85.34758
## 34       92  91.07814
## 35       98  95.50240
## 37       51  59.95129
## 46       53  57.04574
## 54      100  91.14226
## 59       70  72.56263
## 63       74  75.19675
## 65       81  78.82389
## 69       75  74.14613
## 78       90  87.74708
## 80       79  80.58162
## 83       68  70.25118
## 87       84  81.23431
## 88       88  85.31067
## 92       81  78.59282
## 94       74  76.02486
## 99       61  67.21095
## 101      67  70.50401
## 105      69  74.78736
## 106      65  69.21803
## 109     101  94.93474
## 112     106  99.37726
## 115      88  85.76116
## 116      96  91.37478
## 117      90  88.17278
## 121      69  70.27787
## 127      48  56.72048
## 130      83  82.58744
## 138      71  73.60703
## 141      77  77.35759
## 143      87  83.42394
## 144      69  70.82723
## 151      57  63.19119
## 154      81  80.32676
## 156      95  88.86161
## 163      91  85.30826
## 170      92  88.12671
## 173      91  86.79622
## 179      84  84.28872
## 180      73  75.33200
## 181      67  72.39357
## 183      87  85.79437
## 184      76  78.93801
## 185      67  72.49274
## 186      89  88.09653
## 188      80  84.61430
## 189      88  89.62943
## 191      78  82.44216
## 192      74  78.44869
## 194      67  71.45831
## 204     107  98.34838
## 207      52  57.76360
## 208      64  65.56827
## 210      94  87.35202
## 216      96  88.54077
## 220      50  59.75506
## 221      49  58.48762
## 222      54  61.48032
## 224      55  61.30138
## 225      45  55.14053
## 231      96  93.63894
## 235      81  80.98433
## 238      80  78.81272
## 239      89  84.40315
## 250      98  92.23109
## 253      89  86.20442
## 254      78  81.52074
## 255      83  83.62963
## 258      73  73.13284
## 259      80  79.46066
## 268      95  95.23003
## 274      98  89.04356
## 276     104  94.35322
## 282      82  80.85499
## 284     105  96.77620
## 285      87  83.26177
## 288      96  90.74943
## 289     104  99.62744
## 293      92  87.65587
## 294     105  96.63034
## 295      98  94.27113
## 297      88  85.72481
## 303     103  96.82641
## 304      87  84.86256
## 305      73  73.78361
## 310      81  80.14813
## 314      63  68.81069
## 317      72  72.30197
## 319      88  84.56137
## 325      81  80.34307
## 326      79  79.45429
## 333      77  78.08399
## 334      93  88.85222
## 338      82  83.31288
## 342      67  72.14116
## 349      92  87.86002
## 351      98  90.43812
## 355      83  79.03901
## 366      56  65.13883
## 367      79  81.56761
## 368      87  88.31950
## 373      81  79.91958
## 376      78  76.45715
## 381      99  94.48347
## 386      99  93.27498
## 388      86  85.90281
## 390      98  90.31136
## 394      79  78.61356
## 397      75  77.03388
## 398      64  69.13544
## 402      71  71.39162
## 403      87  85.26137
## 409      71  71.51559
## 417      95  92.26627
## 419      81  83.18943
## 423      98  92.46634
## 427      93  87.72005
## 428      84  82.54620
## 434      63  67.08608
## 435      64  68.45397
## 438     111 100.83275
## 440      83  80.22935
## 444      63  68.65063
## 445      62  64.43612
## 446      56  62.67054
## 447      72  73.73673
## 449      59  64.67799
## 451     102  94.22575
## 453      93  86.36362
## 454      81  77.34596
## 455      94  86.52614
## 460      65  69.35549
## 463      73  74.98199
## 464      79  80.70505
## 469      98  91.90850
## 470      98  91.91461
## 471      86  82.47333
## 474      77  77.33002
## 477      79  77.02153
## 480     108 101.36029
## 481     102 100.00862
## 488      86  84.89780
## 495      76  78.75978
## 502      80  81.74727
## 503      64  68.72064
## 509      91  82.99405
## 514      81  79.47504
## 515      93  87.33406
## 520      71  76.04364
## 521      93  88.29802
## 522      65  68.10737
## 525      93  91.11869
## 528      92  91.27800
## 529      92  89.47355
## 535     101  97.17501
## 537      97  94.38511
## 538      98  92.56368
## 544      94  88.54121
## 546      79  78.52438
## 549      87  81.76061
## 550      62  66.57579
## 552      60  64.71219
## 556      70  73.55800
## 559      61  69.25300
## 563      57  62.78791
## 565      76  78.10097
## 568      87  88.53658
## 569      97  96.83056
## 573      68  70.68392
## 574      80  83.12310
## 576      88  85.00721
## 580      83  85.11009
## 588      67  71.69267
## 590      77  77.42194
## 592      84  78.99787
## 593      77  75.73106
## 595      70  75.55146
## 600      64  70.25671
## 603      85  87.83172
## 605     106 100.89122
## 606     100  98.25715
## 610      85  86.21820
## 615      93  86.96213
## 616      97  92.06917
## 622      82  81.23330
## 625     101  96.34546
## 628      88  85.54449
## 630      79  79.38547
## 632      85  83.25548
## 633      72  72.69895
## 638      84  84.83788
## 641     104  97.64760
## 650      85  87.11000
## 654      66  71.26735
## 656      43  53.52923
## 659      64  68.77313
## 660      76  79.40749
## 661      80  79.48389
## 664      64  69.36184
## 666      83  83.26684
## 668      78  78.62144
## 669      65  68.59003
## 670      66  66.96441
## 671      66  66.28451
## 674      69  72.37365
## 677      79  80.78054
## 678      79  78.08363
## 679      89  86.64477
## 680      82  80.78460
## 689      76  78.81555
## 693      85  83.50353
## 699      87  86.47769
## 700      89  86.70176
## 702      72  74.12514
## 709      91  86.99273
## 717      72  74.85640
## 718      77  80.54201
## 719      65  69.48703
## 729      58  61.83937
## 737      80  78.72859
## 750      93  89.70069
## 757      88  86.50834
## 760     102  96.99723
## 762      97  94.70708
## 765      88  85.71655
## 766      75  75.61513
## 767      91  88.42922
## 770     100  94.67405
## 776      88  87.00832
## 777      89  84.88470
## 784      79  81.59075
## 788      73  73.78489
## 790      78  76.52317
## 793      81  80.60735
## 803      88  88.00072
## 808      68  68.78425
## 813      93  89.54166
## 814      96  92.86832
## 816      95  91.48170
## 818      71  73.55274
## 825      78  77.71585
## 827      74  78.85615
## 830      56  64.25351
## 831      68  73.36450
## 833      81  80.60054
## 839      59  64.66963
## 846      91  87.00857
## 849      89  88.47074
## 850      79  81.18447
## 852      97  93.16708
## 857      77  80.23082
## 859      69  74.33196
## 868      53  62.62116
## 869      65  70.41686
## 871      56  63.01794
## 872      62  68.90861
## 873      64  67.83398
## 889      60  67.18931
## 892      85  85.24435
## 894      80  81.61858
## 899      78  83.02777
## 900      68  73.73707
## 901      70  74.88892
## 905      90  89.98500
## 907      83  81.73870
## 911      51  55.63974
## 914      61  62.66826
## 915      73  70.43528
## 924      92  90.48923
## 925     101  94.85381
## 929      59  65.15395
## 932      88  85.31113
## 933      96  96.53790
## 935      82  81.15640
## 938      83  83.15470
## 940      87  83.48769
## 941      99  91.14737
## 946      53  63.90310
## 950      79  78.30101
## 960     114 107.40924
## 964     103  94.92983
## 965      87  85.28770
## 967      92  87.66446
## 971     100  93.22618
## 973     108  99.12532
## 975      97  91.16455
## 977      71  72.75110
## 982      83  79.57500
## 985      84  82.33328
## 990      79  80.43937
## 994      90  89.45423
## 995      89  87.93766
## 1000     89  90.95326
## 1004     98  96.50287
## 1006     96  92.97704
## 1008    101  97.51556
## 1011     88  87.61073
## 1013     90  86.59208
## 1014     87  82.70013
## 1017     98  89.99315
## 1019    101  92.43209
## 1020    110  99.70525
## 1026     56  64.24170
## 1033     85  86.85605
## 1035     63  69.82702
## 1041     58  63.64000
## 1042     52  58.68170
## 1043     76  75.34484
## 1044     56  61.97257
## 1047     74  77.10928
## 1052     59  65.68318
## 1063     76  78.32770
## 1064     99  92.44416
## 1070     78  80.90417
## 1074    102  95.70958
## 1076     91  88.34149
## 1078     85  84.46482
## 1079    100  95.16721
## 1080     88  88.74787
## 1081     96  91.97239
## 1082     66  68.44144
## 1090     78  79.32060
## 1091     94  89.90802
## 1093     96  87.56335
## 1098     72  75.90608
## 1101     46  57.90269
## 1104     69  72.40214
## 1116     73  73.51963
## 1118     85  82.97670
## 1121     92  87.66890
## 1123     62  65.40437
## 1130     63  67.38607
## 1133     71  71.65268
## 1134     80  78.69926
## 1137     90  87.63451
## 1138     91  88.39833
## 1139     81  83.05860
## 1144     67  71.44723
## 1148     97  94.14461
## 1149     76  79.74062
## 1150     67  72.23940
## 1157     74  78.73195
## 1164    102  93.46506
## 1166    103  94.45086
## 1177    101  97.30206
## 1188     88  87.34925
## 1190     93  89.10515
## 1194     73  74.68791
## 1196     95  91.56593
## 1206     79  78.06603
## 1209     80  76.89921
## 1210     90  83.71684
## 1215     97  91.33505
## 1220     98  93.12816
## 1221     57  64.42125
## 1224     95  90.93255
## 1225     98  93.62773
## 1229     79  79.54110
## 1230     78  78.62824
## 1231     72  72.95698
## 1232     75  77.75992
## 1237     61  64.24644
## 1240     71  70.81103
## 1242     69  73.01879
## 1244     92  87.37569
## 1245     83  80.68888
## 1247     89  84.69363
## 1249     84  81.52825
## 1253     91  88.76071
## 1254     76  79.46025
## 1256     74  77.04982
## 1257     79  80.61267
## 1258     66  70.10862
## 1261     64  68.22669
## 1265     74  76.55678
## 1280     93  92.45762
## 1281     93  92.35706
## 1283     69  73.78459
## 1284     78  80.66665
## 1285     61  65.80174
## 1289     88  85.48517
## 1294     98  93.85148
## 1300     92  91.67806
## 1308     90  86.23284
## 1312     91  87.62536
## 1313     58  64.43965
## 1315     83  83.12994
## 1317     85  85.47956
## 1325     87  85.08414
## 1329     90  85.33739
## 1332     88  82.51889
## 1336     88  86.54725
## 1339     74  73.70848
## 1341     87  84.91302
## 1347     77  81.52463
## 1351     88  88.25820
## 1352     86  87.36930
## 1358     92  89.72743
## 1359     50  62.00884
## 1360     42  52.54768
## 1365     79  77.30854
## 1371     67  72.80553
## 1372     82  83.20568
## 1373     67  72.70512
## 1375     86  82.97777
## 1377     87  81.83669
## 1384     83  84.87732
## 1385     97  94.03447
## 1386    106  97.37123
## 1387     76  77.10574
## 1388     87  84.91752
## 1397    112 101.41846
## 1398    110  98.39779
## 1399    102  94.29028
## 1403     83  82.28840
## 1417     80  79.32141
## 1424     78  77.31367
## 1426     87  84.46718
## 1428     72  73.54342
## 1432     86  84.54383
## 1441     85  88.21591
## 1447     62  67.00138
## 1448     55  62.85740
## 1449     63  69.94316
## 1451     67  72.81948
## 1457     76  72.90065
## 1460     64  66.84871
## 1461     57  63.71301
## 1463     76  76.24981
## 1467     70  73.55343
## 1468     83  81.73602
## 1471     77  77.05264
## 1479     89  86.94894
## 1481     53  59.79608
## 1482     89  88.18588
## 1485     87  85.83743
## 1496     80  80.85115
## 1497     87  87.05447
## 1505     75  77.66813
## 1509     82  83.02379
## 1510     78  77.92556
## 1514     68  72.17659
## 1516     68  71.68182
## 1517     83  82.19070
## 1519     67  71.70301

Lets calculate the accuracy of using Model6 for our predictions

mean (apply(predicted_values, 1, min)/apply(predicted_values, 1, max)) # calculate accuracy

## [1] 0.9556918

The prediction accuracy of the improved Ridge Regression Model is 95.75%.

ModelName <- c("Model3", "Model5","Model6")
Model_Accuracy <- c("85.85%", "85.85%", "95.75%")
AccuracyCompared <- data.frame(ModelName,Model_Accuracy)
AccuracyCompared

ModelName	Model_Accuracy
Model3	85.85%
Model5	85.85%
Model6	95.75%

The prediction accuracy of the improved Ridge Regression Model6 is at 95.75% which is very good for this purpose.

0.5 Conclusion

The improved Model6 shows significant improvement from all the OLS Models when the R-Squared and the RMSE of the Models are compared. THis Model also predict TARGET WINS better than the OLS models because it is more stable and less prone to overfitting.

The chosen OLS Model3 and Model5 are due to the improved F-Statistic, positive variable coefficients and low Standard Errors. We will chose to make our predictions with the champion model the improved Ridge Regression Model6 because it beats all the OLs models when the model performance metrics are compared as well as the predictive ability of this model.

For Models 3 and 4, the variables were chosen just to test how the offfensive categories only would affect the model and how only defensive variables would affect the model. Based on the Coefficients for each model, the third model took the highest coefficient from each category model.

For offense, the two highest were HR and Triples. Which intuively does make sense because the HR and triple are two of the highest objectives a hitter can achieve when batting and thus the higher the totals in those categories the higher the runs scored which help a team win. And on the defensive side, the two highest cooeficients were Hits and WALKS. Which again just looking at it from a common sense point does make sense because as a pitcher, what they want to do is limit the numbers of times a batter gets on base whether by a hit or walk. Unless its an error, if a batter does not get a hit or walk then the outcome would be an out which would in essence limit the amount of runs scored by the opposing team.