Assignment-1

0.1 Introduction

In this assignment, we are tasked to explore, analyze and model a major league baseball dataset which contains around 2000 records where each record presents a baseball team from 1871 to 2006. Each observation provides the perforamce of the team for that particular year with all the statistics for the performance of 162 game season. The problem statement for the main objective is that “Can we predict the number of wins for the team with the given attributes of each record?”. In order to provide a solution for the problem, our goal is to build a linear regression model on the training data that creates this prediction.

0.1.1 About the Data

The data set are provided in csv format as moneyball-evaluation-data and moneyball-training-data where we will explore, preperate and create our model with the training data and further test the model with the evaluation data. Below is short description of the variables within the datasets.

**INDEX: Identification Variable(Do not use)

**TARGET_WINS: Number of wins

**TEAM_BATTING_H : Base Hits by batters (1B,2B,3B,HR)

**TEAM_BATTING_2B: Doubles by batters (2B)

**TEAM_BATTING_3B: Triples by batters (3B)

**TEAM_BATTING_HR: Homeruns by batters (4B)

**TEAM_BATTING_BB: Walks by batters

**TEAM_BATTING_HBP: Batters hit by pitch (get a free base)

**TEAM_BATTING_SO: Strikeouts by batters

**TEAM_BASERUN_SB: Stolen bases

**TEAM_BASERUN_CS: Caught stealing

**TEAM_FIELDING_E: Errors

**TEAM_FIELDING_DP: Double Plays

**TEAM_PITCHING_BB: Walks allowed

**TEAM_PITCHING_H: Hits allowed

**TEAM_PITCHING_HR: Homeruns allowed

**TEAM_PITCHING_SO: Strikeouts by pitchers

0.2 Data Exploration

0.2.1 Descriptive Statistics

# load libraries
library(ggplot2)
library(ggcorrplot)

## Warning: package 'ggcorrplot' was built under R version 3.6.2

library(psych)

## 
## Attaching package: 'psych'

## The following objects are masked from 'package:ggplot2':
## 
##     %+%, alpha

library(statsr)

## Loading required package: BayesFactor

## Loading required package: coda

## Loading required package: Matrix

## ************
## Welcome to BayesFactor 0.9.12-4.2. If you have questions, please contact Richard Morey (richarddmorey@gmail.com).
## 
## Type BFManual() to open the manual.
## ************

library(dplyr)

## 
## Attaching package: 'dplyr'

## The following objects are masked from 'package:stats':
## 
##     filter, lag

## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union

library(PerformanceAnalytics)

## Warning: package 'PerformanceAnalytics' was built under R version 3.6.2

## Loading required package: xts

## Warning: package 'xts' was built under R version 3.6.2

## Loading required package: zoo

## 
## Attaching package: 'zoo'

## The following objects are masked from 'package:base':
## 
##     as.Date, as.Date.numeric

## 
## Attaching package: 'xts'

## The following objects are masked from 'package:dplyr':
## 
##     first, last

## 
## Attaching package: 'PerformanceAnalytics'

## The following object is masked from 'package:graphics':
## 
##     legend

library(tidyr)

## Warning: package 'tidyr' was built under R version 3.6.2

## 
## Attaching package: 'tidyr'

## The following objects are masked from 'package:Matrix':
## 
##     expand, pack, unpack

library(reshape2)

## 
## Attaching package: 'reshape2'

## The following object is masked from 'package:tidyr':
## 
##     smiths

library(rcompanion)

## Warning: package 'rcompanion' was built under R version 3.6.2

## 
## Attaching package: 'rcompanion'

## The following object is masked from 'package:psych':
## 
##     phi

library(caret)

## Loading required package: lattice

library(MASS)

## 
## Attaching package: 'MASS'

## The following object is masked from 'package:dplyr':
## 
##     select

library(imputeTS)

## Warning: package 'imputeTS' was built under R version 3.6.2

## Registered S3 method overwritten by 'quantmod':
##   method            from
##   as.zoo.data.frame zoo

## 
## Attaching package: 'imputeTS'

## The following object is masked from 'package:zoo':
## 
##     na.locf

library(rsample)

## Warning: package 'rsample' was built under R version 3.6.2

library(huxtable)

## Warning: package 'huxtable' was built under R version 3.6.2

## 
## Attaching package: 'huxtable'

## The following object is masked from 'package:dplyr':
## 
##     add_rownames

## The following object is masked from 'package:ggplot2':
## 
##     theme_grey

library(glmnet)

## Warning: package 'glmnet' was built under R version 3.6.2

## Loaded glmnet 3.0-2

## 
## Attaching package: 'glmnet'

## The following object is masked from 'package:imputeTS':
## 
##     na.replace

library(sjPlot)

## Warning: package 'sjPlot' was built under R version 3.6.2

## Learn more about sjPlot with 'browseVignettes("sjPlot")'.

## 
## Attaching package: 'sjPlot'

## The following object is masked from 'package:huxtable':
## 
##     font_size

library(modelr)

## Warning: package 'modelr' was built under R version 3.6.2

## 
## Attaching package: 'modelr'

## The following object is masked from 'package:psych':
## 
##     heights

# Load data sets

baseball_eva <- read.csv("https://raw.githubusercontent.com/Emahayz/Data-621/master/moneyball-evaluation-data.csv")
baseball_train <- read.csv("https://raw.githubusercontent.com/Emahayz/Data-621/master/moneyball-training-data.csv")

We can start exploring our training data set by looking at basic descriptive statistics.

# look at training dataset structure
str(baseball_train)

## 'data.frame':    2276 obs. of  17 variables:
##  $ INDEX           : int  1 2 3 4 5 6 7 8 11 12 ...
##  $ TARGET_WINS     : int  39 70 86 70 82 75 80 85 86 76 ...
##  $ TEAM_BATTING_H  : int  1445 1339 1377 1387 1297 1279 1244 1273 1391 1271 ...
##  $ TEAM_BATTING_2B : int  194 219 232 209 186 200 179 171 197 213 ...
##  $ TEAM_BATTING_3B : int  39 22 35 38 27 36 54 37 40 18 ...
##  $ TEAM_BATTING_HR : int  13 190 137 96 102 92 122 115 114 96 ...
##  $ TEAM_BATTING_BB : int  143 685 602 451 472 443 525 456 447 441 ...
##  $ TEAM_BATTING_SO : int  842 1075 917 922 920 973 1062 1027 922 827 ...
##  $ TEAM_BASERUN_SB : int  NA 37 46 43 49 107 80 40 69 72 ...
##  $ TEAM_BASERUN_CS : int  NA 28 27 30 39 59 54 36 27 34 ...
##  $ TEAM_BATTING_HBP: int  NA NA NA NA NA NA NA NA NA NA ...
##  $ TEAM_PITCHING_H : int  9364 1347 1377 1396 1297 1279 1244 1281 1391 1271 ...
##  $ TEAM_PITCHING_HR: int  84 191 137 97 102 92 122 116 114 96 ...
##  $ TEAM_PITCHING_BB: int  927 689 602 454 472 443 525 459 447 441 ...
##  $ TEAM_PITCHING_SO: int  5456 1082 917 928 920 973 1062 1033 922 827 ...
##  $ TEAM_FIELDING_E : int  1011 193 175 164 138 123 136 112 127 131 ...
##  $ TEAM_FIELDING_DP: int  NA 155 153 156 168 149 186 136 169 159 ...

We have 2276 observations and 17 variables. All of our variables are integer type as expected.

# look at descriptive statistics
metastats <- data.frame(describe(baseball_train))
metastats <- tibble::rownames_to_column(metastats, "STATS")
metastats["pct_missing"] <- round(metastats["n"]/2276, 3)
head(metastats)

STATS	vars	n	mean	sd	median	trimmed	mad	min	max	range	skew	kurtosis	se	pct_missing
INDEX	1	2.28e+03	1.27e+03	736	1.27e+03	1.27e+03	953	1	2.54e+03	2.53e+03	0.00421	-1.22	15.4	1
TARGET_WINS	2	2.28e+03	80.8	15.8	82	81.3	14.8	0	146	146	-0.399	1.03	0.33	1
TEAM_BATTING_H	3	2.28e+03	1.47e+03	145	1.45e+03	1.46e+03	114	891	2.55e+03	1.66e+03	1.57	7.28	3.03	1
TEAM_BATTING_2B	4	2.28e+03	241	46.8	238	240	47.4	69	458	389	0.215	0.00616	0.981	1
TEAM_BATTING_3B	5	2.28e+03	55.2	27.9	47	52.2	23.7	0	223	223	1.11	1.5	0.586	1
TEAM_BATTING_HR	6	2.28e+03	99.6	60.5	102	97.4	78.6	0	264	264	0.186	-0.963	1.27	1

With the descriptive statistics, we are able to see mean, standard deviation, median, min, max values and percentage of each missing value of each variable. For example, when we look at TEAM_BATTING_H, we see that average 1469 Base hits by batters, with standard deviation of 144, median of 1454 with maximum base hits of 2554.

# Look for missing values
colSums(is.na(baseball_train))

##            INDEX      TARGET_WINS   TEAM_BATTING_H  TEAM_BATTING_2B 
##                0                0                0                0 
##  TEAM_BATTING_3B  TEAM_BATTING_HR  TEAM_BATTING_BB  TEAM_BATTING_SO 
##                0                0                0              102 
##  TEAM_BASERUN_SB  TEAM_BASERUN_CS TEAM_BATTING_HBP  TEAM_PITCHING_H 
##              131              772             2085                0 
## TEAM_PITCHING_HR TEAM_PITCHING_BB TEAM_PITCHING_SO  TEAM_FIELDING_E 
##                0                0              102                0 
## TEAM_FIELDING_DP 
##              286

# Percentage of missing values
missing_values <- metastats %>%
  filter(pct_missing < 1) %>%
  dplyr::select(STATS, pct_missing) %>%
  arrange(pct_missing)
missing_values

STATS	pct_missing
TEAM_BATTING_HBP	0.084
TEAM_BASERUN_CS	0.661
TEAM_FIELDING_DP	0.874
TEAM_BASERUN_SB	0.942
TEAM_BATTING_SO	0.955
TEAM_PITCHING_SO	0.955

When we look at the missing values within the training data set, we see that proportionaly against the total observations, TEAM_BATTING_HBP and TEAM_BESARUN_CS variables have the most missing values. We will be handling these missing values in our Data Preperation section.

0.2.2 Correlation and Distribution

# Look at correlation between variables

corr <- round(cor(baseball_train), 1)

ggcorrplot(corr,
           type="lower",
           lab=TRUE,
           lab_size=3,
           method="circle",
           colors=c("tomato2", "white", "springgreen3"),
           title="Correlation of variables in Training Data Set",
           ggtheme=theme_bw)

Team_Batting_H and Team_Batting_2B have the strongest positive correlation with Target_Wins. We also see that, there is a strong correlation between Team_Batting_H and Team_Batting_2B, Team_Pitching_B and TEAM_FIELDING_E. We will consider these findings on model creation as collinearity might complicate model estimation and we want to have explanotry variables to be independent from each other. We will try to avoid adding explanotry variables that are correlated to each other.

Let’s look at the correlations and distribution of the variables in more detail.

# Look at correlation from batting, baserunning, pitching and fielding perspective
Batting_df <- baseball_train[c(2:7, 10)] 
BaseRunning_df <- baseball_train[c(8:9)] 
Pitching_df <- baseball_train[c(11:14)] 
Fielding_df <- baseball_train[c(15:16)]

0.2.2.1 Batting

# Batting Correlations
chart.Correlation(Batting_df, histogram=TRUE, pch=19)

We can see that our response variable TARGET_WINS, TEAM_BATTING_H, TEAM_BATTING_2B, TEAM_BATTING_BB and TEAM_BASERUN_CS are normaly distributed. TEAM_BATTING_HR on the other hand is bimodal.

0.2.2.2 Baserunning

# baserunning Correlation

chart.Correlation(BaseRunning_df, histogram=TRUE, pch=19)

TEAM_BASERUN_SB is right skewed and TEAM_BATTING_SO is bimodal.

0.2.2.3 Pitching

#pitching correlations
chart.Correlation(Pitching_df, histogram=TRUE, pch=19)

TEAM_BATTING_HBP seems to be normally distributed however we shouldnt forget that we have a lot of missing values in this variable.

# fielding correlations
chart.Correlation(Fielding_df, histogram=TRUE, pch=19)

Let’s also look at the outliers and skewness for each varibale.

0.2.3 Outliers and Skewness

par(mfrow=c(3,3))
datasub_1 <- melt(baseball_train)

## No id variables; using all as measure variables

suppressWarnings(ggplot(datasub_1, aes(x= "value", y=value)) + 
                   geom_boxplot(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

## Warning: Removed 3478 rows containing non-finite values (stat_boxplot).

Based on the boxplot we created, TEAM_FIELDING_DP, TEAM_PITCHING_HR, TEAM_BATTING_HR and TEAM_BATTING_SO seem to have the least amount of outliers.

par(mfrow = c(3, 3))
datasub = melt(baseball_train) 

## No id variables; using all as measure variables

suppressWarnings(ggplot(datasub, aes(x= value)) + 
                   geom_density(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

## Warning: Removed 3478 rows containing non-finite values (stat_density).

metastats %>%
  filter(skew > 1) %>%
  dplyr::select(STATS, skew) %>%
  arrange(desc(skew))

STATS	skew
TEAM_PITCHING_SO	22.2
TEAM_PITCHING_H	10.3
TEAM_PITCHING_BB	6.74
TEAM_FIELDING_E	2.99
TEAM_BASERUN_CS	1.98
TEAM_BASERUN_SB	1.97
TEAM_BATTING_H	1.57
TEAM_BATTING_3B	1.11

We can see that the most skewed variable is TEAM_PITCHING_SO. We will correct the skewed variables in our data preperation section.

When we are creating a linear regression model, we are looking for the fitting line with the least sum of squares, that has the small residuals with minimized squared residuals. From our correlation analysis, we can see that the explatory variable that has the strongest correlation with TARGET_WINS is TEAM_BATTING_H. Let’s look at a simple model example to further expand our explaroty analysis.

0.2.4 Simple Model Example

# line that follows the best assocation between two variables

plot_ss(x = TEAM_BATTING_H, y = TARGET_WINS, data=baseball_train, showSquares = TRUE, leastSquares = TRUE)

## 
                                
## Call:
## lm(formula = y ~ x, data = data)
## 
## Coefficients:
## (Intercept)            x  
##    18.56233      0.04235  
## 
## Sum of Squares:  479178.4

When we are exploring to build a linear regression, one of the first thing we do is to create a scatter plot of the response and explanatory variable.

# scatter plot between TEAM_BATTING_H and TARGET_WINS

ggplot(baseball_train, aes(x=TEAM_BATTING_H, y=TARGET_WINS))+
  geom_point()

One of the conditions for least square lines or linear regression are Linearity. From the scatter plot between TEAM_BATTING_H and TARGET_WINS, we can see this condition is met. We can also create a scatterplot that shows the data points between TARGET_WINS and each variable.

baseball_train %>%
  gather(var, val, -TARGET_WINS) %>%
  ggplot(., aes(val, TARGET_WINS))+
  geom_point()+
  facet_wrap(~var, scales="free", ncol=4)

## Warning: Removed 3478 rows containing missing values (geom_point).

As we displayed earlier, hits walks and home runs have the strongest correlations with TARGET_WINS and also meets the linearity condition.

# create a simple example model
lm_sm <- lm(baseball_train$TARGET_WINS ~ baseball_train$TEAM_BATTING_H)
summary(lm_sm)

## 
## Call:
## lm(formula = baseball_train$TARGET_WINS ~ baseball_train$TEAM_BATTING_H)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -71.768  -8.757   0.856   9.762  46.016 
## 
## Coefficients:
##                                Estimate Std. Error t value Pr(>|t|)    
## (Intercept)                   18.562326   3.107523   5.973 2.69e-09 ***
## baseball_train$TEAM_BATTING_H  0.042353   0.002105  20.122  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 14.52 on 2274 degrees of freedom
## Multiple R-squared:  0.1511, Adjusted R-squared:  0.1508 
## F-statistic: 404.9 on 1 and 2274 DF,  p-value: < 2.2e-16

TARET_BATTING_H has the strongest correlation with TARGET_WINS response variable, however when we create a simple model just using TARGET_BATTING_H, we can only explain 15% of the variablity. (Adjusted R-squared: 0.1508). The remainder of the varibility can be explained with selected other variables within the training dataset.

#histogram of residuals for the simple model
hist(lm_sm$residuals)

# check for constant variability (honoscedasticity)

plot(lm_sm$residuals ~ baseball_train$TEAM_BATTING_H)

We do see that the residuals are distributed normally and variability around the regression line is roughly constant.

Based on our explatory analysis, we were able to see the correlation level between the possible explanatory variables and repsonse variable TARGET_WINS. Some of the variables such as TARGET_BATTING_H has somewhat strong positive correlation, however some of the variables such as TEAM_PITCHING_BB has weak positive relationship with TARGET_WINS. We also found out, hit by the pitcher(TEAM_BATTING_HBP) and caught stealing (TEAM_BASERUN_CS) variables are missing majority of the values. Skewness and distribution analysis gave us the insights that we have some variables that are right-tailed. Considering all of these insights, we will handle missing values, correct skewness and outliers and select our explaratory variables based on correlation in order to create our regression model.

0.3 Data Preparation

0.3.1 Objective

In this section, we will prepare the dataset for linear regression modeling. We accomplish this by handling missing values and outliers and by tranforming the data into more normal distributions. This section covers:

Identify and Handle Missing Data Correct Outliers *Adjust Skewed value - Box Cox Transformation

First, we will start by copying the dataset into a new variable, baseball_train_01, and we will remove the Index variable from the new dataset as well. We will now have 16 variables.

baseball_train_01 <- baseball_train

baseball_train_01 <-subset(baseball_train_01, select = -c(INDEX))

0.3.2 Identify and Handle Missing Data

0.3.2.1 Removal of Sparsely Populated Variables - MCAR

In the Data Exploration section, we identified these variables as having missing data values.The table below lists the variables with missing data. The variable, TEAM_BATTING_HBP, is sparsely populated. Since this data is Missing Completely at Random (MCAR) and is not related to any other variable, it is safe to completely remove the variable from the dataset.

missing_values

STATS	pct_missing
TEAM_BATTING_HBP	0.084
TEAM_BASERUN_CS	0.661
TEAM_FIELDING_DP	0.874
TEAM_BASERUN_SB	0.942
TEAM_BATTING_SO	0.955
TEAM_PITCHING_SO	0.955

baseball_train_01 <-subset(baseball_train_01, select = -c(TEAM_BATTING_HBP))

There are now 15 variables.

dim(baseball_train_01)

## [1] 2276   15

0.3.2.2 Imputation of Missing Values

For the remaining variables with missing values, we will impute the mean of the variable. The function, “na_mean” updates all missing values with the mean of the variable.

baseball_train_01 <- na_mean(baseball_train_01, option = "mean")  

Re-running the metastats dataframe on the new baseball_train_01 dataset shows that there are no missing values.

# look at descriptive statistics
metastats <- data.frame(describe(baseball_train_01))
metastats <- tibble::rownames_to_column(metastats, "STATS")
metastats["pct_missing"] <- round(metastats["n"]/2276, 3)

# Percentage of missing values
missing_values2 <- metastats %>%
  filter(pct_missing < 1) %>%
  dplyr::select(STATS, pct_missing) %>%
  arrange(pct_missing)
missing_values2

## Warning in max(nchar(as.character(col), type = "width")): no non-missing
## arguments to max; returning -Inf

## Warning in max(nchar(as.character(col), type = "width")): no non-missing
## arguments to max; returning -Inf

STATS

pct_missing

0.3.3 Correct Outliers

In this section, we created two functions that can identify outliers. The funcion, Identify_Outlier, uses the Turkey method, where outliers are identified by being below Q1-1.5IQR and above Q3+1.5IQR. The second function, tag_outlier, returns a binary list of values, “Acceptable” or “Outlier” that will be added to the dataframe.

Identify_Outlier <- function(value){

    interquartile_range = IQR(sort(value),na.rm = TRUE)
    q1 = matrix(c(quantile(sort(value),na.rm = TRUE)))[2]
    q3 = matrix(c(quantile(sort(value),na.rm = TRUE)))[4]
    lower = q1-(1.5*interquartile_range)
    upper = q3+(1.5*interquartile_range)
    
    bound <- c(lower, upper)
    
    return (bound)
}

tag_outlier <- function(value) {
    
   boundaries <- Identify_Outlier(value)
   tags <- c()
   counter = 1
    for (i in as.numeric(value))
    {

        if (i >= boundaries[1] & i <= boundaries[2]){
          tags[counter] <- "Acceptable"
        } else{
          tags[counter] <- "Outlier"
        }
      
      counter = counter +1
    }
   
   return (tags)
}

As seen in the box plots from the previous section, “TEAM_BASERUN_SB”, “TEAM_BASERUN_CS”, “TEAM_PITCHING_H”, “TEAM_PITCHING_BB”, “TEAM_PITCHING_SO”, and “TEAM_FIELDING_E” all have a high number of outliers. We will use the two functions above to tag those rows with extreme outliers.

tags<- tag_outlier(baseball_train_01$TEAM_BASERUN_SB)
baseball_train_01$TEAM_BASERUN_SB_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_BASERUN_CS)
baseball_train_01$TEAM_BASERUN_CS_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_PITCHING_H)
baseball_train_01$TEAM_PITCHING_H_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_PITCHING_BB)
baseball_train_01$TEAM_PITCHING_BB_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_PITCHING_SO)
baseball_train_01$TEAM_PITCHING_SO_Outlier <- tags

tags<- tag_outlier(baseball_train_01$TEAM_FIELDING_E)
baseball_train_01$TEAM_FIELDING_E_Outlier <- tags

Below, we filtered out all of the outliers and created a new dataframe, baseball_train_02

baseball_train_02 <- baseball_train_01 %>%
                filter(
                        TEAM_BASERUN_SB_Outlier != "Outlier" &
                        TEAM_BASERUN_CS_Outlier != "Outlier" &
                        TEAM_PITCHING_H_Outlier != "Outlier" &
                        TEAM_PITCHING_BB_Outlier != "Outlier" &
                        TEAM_PITCHING_SO_Outlier != "Outlier" &
                        TEAM_FIELDING_E_Outlier != "Outlier"
                )

Re-running the boxplots show data that has a better normal distribution except for the variable, TEAM_FIELDING_E which is still skewed. We will handle this next.

par(mfrow=c(3,3))
datasub_1 <- melt(baseball_train_02)

## Using TEAM_BASERUN_SB_Outlier, TEAM_BASERUN_CS_Outlier, TEAM_PITCHING_H_Outlier, TEAM_PITCHING_BB_Outlier, TEAM_PITCHING_SO_Outlier, TEAM_FIELDING_E_Outlier as id variables

suppressWarnings(ggplot(datasub_1, aes(x= "value", y=value)) + 
                   geom_boxplot(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

0.3.4 Adjust Skewed values

0.3.4.1 Box Cox Transformation

Removing the outliers tranformed each variable to a closer to a normal distribution and checking the skewness of the variables confirm this with the exception of TEAM_FIELDING_E. This variable is still skewed and not normal. In this section, we will use the Box Cox tranformation from the MASS library to normalize this variable.

metastats_02 <- data.frame(describe(baseball_train_02))

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in describe(baseball_train_02): NAs introduced by coercion

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to min; returning
## Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

## Warning in FUN(newX[, i], ...): no non-missing arguments to max; returning
## -Inf

metastats_02 <- tibble::rownames_to_column(metastats_02, "STATS") 
 
metastats_02 %>%
 filter(skew > 1 | skew < -1) %>%
  dplyr::select(STATS, skew) %>%
  arrange(desc(skew))

STATS	skew
TEAM_FIELDING_E	1.45

Looking at the histogram and QQ plots we can confirm that the variable, TEAM_FIELDING_E, is not normally distributed. It is skewed to the right.

plotNormalHistogram(baseball_train_02$TEAM_FIELDING_E)

qqnorm(baseball_train_02$TEAM_FIELDING_E,
       ylab="Sample Quantiles for TEAM_FIELDING_E")    
         qqline(baseball_train_02$TEAM_FIELDING_E,
           col="blue")

The following Box Cox transformation section is based on the tutorial at the link below:

[://rcompanion.org/handbook/I_12.html][Summary and Analysis of Extension Program Evaluation in R]

The Box Cox procedure uses a log-likelihood to find the lambda to use to transform a variable to a normal distribution.

TEAM_FIELDING_E <- as.numeric(dplyr::pull(baseball_train_02, TEAM_FIELDING_E))

#Transforms TEAM_FIELDING_E as a single vector 
Box = boxcox(TEAM_FIELDING_E ~ 1, lambda = seq(-6,6,0.1))

#Creates a dataframe with results
Cox = data.frame(Box$x, Box$y)

# Order the new data frame by decreasing y to find the best lambda.Displays the lambda with the greatest log likelihood.
Cox2 = Cox[with(Cox, order(-Cox$Box.y)),]
Cox2[1,] 

Box.x	Box.y
-0.8	-3.95e+03

#Extract that lambda and Transform the data
lambda = Cox2[1, "Box.x"]
T_box = (TEAM_FIELDING_E ^ lambda - 1)/lambda

We can now see that TEAM_FIELDING_E has a normal distribution.

plotNormalHistogram(T_box)

qqnorm(T_box, ylab="Sample Quantiles for TEAM_FIELDING_E")
qqline(T_box,
        col="blue")

baseball_train_02$TEAM_FIELDING_E <- T_box

The density plots below show that all of the variables for the dataset baseball_train_02 are now normally distributed. In the next section, we will use this dataset to build the models and discuss the coefficients of the models.

par(mfrow = c(3, 3))
datasub = melt(baseball_train_02) 
suppressWarnings(ggplot(datasub, aes(x= value)) + 
                   geom_density(fill='lightblue') + facet_wrap(~variable, scales = 'free') )

Viewing the dataframe shows that the dataset contains characters resulting from the transfromation of the outliers. These non numeric characters will impact our models especially if we build the intial baseline model with all the variables. We will need one more step to have our data ready for the models.

str(baseball_train_02)

## 'data.frame':    1521 obs. of  21 variables:
##  $ TARGET_WINS             : int  70 82 75 80 85 76 78 87 88 66 ...
##  $ TEAM_BATTING_H          : int  1387 1297 1279 1244 1273 1271 1305 1417 1563 1460 ...
##  $ TEAM_BATTING_2B         : int  209 186 200 179 171 213 179 226 242 239 ...
##  $ TEAM_BATTING_3B         : int  38 27 36 54 37 18 27 28 43 32 ...
##  $ TEAM_BATTING_HR         : int  96 102 92 122 115 96 82 108 164 107 ...
##  $ TEAM_BATTING_BB         : int  451 472 443 525 456 441 374 539 589 546 ...
##  $ TEAM_BATTING_SO         : num  922 920 973 1062 1027 ...
##  $ TEAM_BASERUN_SB         : num  43 49 107 80 40 72 60 86 100 92 ...
##  $ TEAM_BASERUN_CS         : num  30 39 59 54 36 34 39 69 53 64 ...
##  $ TEAM_PITCHING_H         : int  1396 1297 1279 1244 1281 1271 1364 1417 1563 1478 ...
##  $ TEAM_PITCHING_HR        : int  97 102 92 122 116 96 86 108 164 108 ...
##  $ TEAM_PITCHING_BB        : int  454 472 443 525 459 441 391 539 589 553 ...
##  $ TEAM_PITCHING_SO        : num  928 920 973 1062 1033 ...
##  $ TEAM_FIELDING_E         : num  1.23 1.23 1.22 1.23 1.22 ...
##  $ TEAM_FIELDING_DP        : num  156 168 149 186 136 159 141 136 172 146 ...
##  $ TEAM_BASERUN_SB_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_BASERUN_CS_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_PITCHING_H_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_PITCHING_BB_Outlier: chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_PITCHING_SO_Outlier: chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...
##  $ TEAM_FIELDING_E_Outlier : chr  "Acceptable" "Acceptable" "Acceptable" "Acceptable" ...

Subsetting - The code below will subset the data to have only numeric or integer values that will be used for our models. This will create baseball_train_03 dataframe.

baseball_train_03 <- baseball_train_02[c(1:15) ]
str(baseball_train_03)

## 'data.frame':    1521 obs. of  15 variables:
##  $ TARGET_WINS     : int  70 82 75 80 85 76 78 87 88 66 ...
##  $ TEAM_BATTING_H  : int  1387 1297 1279 1244 1273 1271 1305 1417 1563 1460 ...
##  $ TEAM_BATTING_2B : int  209 186 200 179 171 213 179 226 242 239 ...
##  $ TEAM_BATTING_3B : int  38 27 36 54 37 18 27 28 43 32 ...
##  $ TEAM_BATTING_HR : int  96 102 92 122 115 96 82 108 164 107 ...
##  $ TEAM_BATTING_BB : int  451 472 443 525 456 441 374 539 589 546 ...
##  $ TEAM_BATTING_SO : num  922 920 973 1062 1027 ...
##  $ TEAM_BASERUN_SB : num  43 49 107 80 40 72 60 86 100 92 ...
##  $ TEAM_BASERUN_CS : num  30 39 59 54 36 34 39 69 53 64 ...
##  $ TEAM_PITCHING_H : int  1396 1297 1279 1244 1281 1271 1364 1417 1563 1478 ...
##  $ TEAM_PITCHING_HR: int  97 102 92 122 116 96 86 108 164 108 ...
##  $ TEAM_PITCHING_BB: int  454 472 443 525 459 441 391 539 589 553 ...
##  $ TEAM_PITCHING_SO: num  928 920 973 1062 1033 ...
##  $ TEAM_FIELDING_E : num  1.23 1.23 1.22 1.23 1.22 ...
##  $ TEAM_FIELDING_DP: num  156 168 149 186 136 159 141 136 172 146 ...

0.4 Build Models

The first Model is using stepwise in Backward direction to eliminate variables, this is an automated process which is different from the manual variable selction process. We will not pay much attention to this process as the focus of the project is to manually identify and select those significant variables that will predict TARGET WINS.

Model <- step(lm(TARGET_WINS ~ ., data=baseball_train_03), direction = "backward")

## Start:  AIC=7313.35
## TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_3B + 
##     TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + TEAM_BASERUN_SB + 
##     TEAM_BASERUN_CS + TEAM_PITCHING_H + TEAM_PITCHING_HR + TEAM_PITCHING_BB + 
##     TEAM_PITCHING_SO + TEAM_FIELDING_E + TEAM_FIELDING_DP
## 
##                    Df Sum of Sq    RSS    AIC
## - TEAM_PITCHING_H   1       0.7 182710 7311.4
## - TEAM_PITCHING_HR  1     162.2 182872 7312.7
## - TEAM_BATTING_H    1     216.2 182926 7313.2
## <none>                          182709 7313.4
## - TEAM_BASERUN_CS   1     330.3 183040 7314.1
## - TEAM_BATTING_HR   1     338.0 183047 7314.2
## - TEAM_PITCHING_BB  1     363.7 183073 7314.4
## - TEAM_BATTING_BB   1     629.6 183339 7316.6
## - TEAM_PITCHING_SO  1    1242.9 183952 7321.7
## - TEAM_BATTING_SO   1    1857.6 184567 7326.7
## - TEAM_BATTING_2B   1    1864.9 184574 7326.8
## - TEAM_FIELDING_DP  1    6690.2 189400 7366.1
## - TEAM_BATTING_3B   1    7536.4 190246 7372.8
## - TEAM_BASERUN_SB   1    8080.4 190790 7377.2
## - TEAM_FIELDING_E   1   18743.7 201453 7459.9
## 
## Step:  AIC=7311.36
## TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_3B + 
##     TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + TEAM_BASERUN_SB + 
##     TEAM_BASERUN_CS + TEAM_PITCHING_HR + TEAM_PITCHING_BB + TEAM_PITCHING_SO + 
##     TEAM_FIELDING_E + TEAM_FIELDING_DP
## 
##                    Df Sum of Sq    RSS    AIC
## - TEAM_PITCHING_HR  1     173.3 182883 7310.8
## <none>                          182710 7311.4
## - TEAM_BASERUN_CS   1     331.1 183041 7312.1
## - TEAM_BATTING_HR   1     358.9 183069 7312.3
## - TEAM_PITCHING_SO  1    1259.1 183969 7319.8
## - TEAM_PITCHING_BB  1    1509.7 184220 7321.9
## - TEAM_BATTING_2B   1    1876.6 184587 7324.9
## - TEAM_BATTING_SO   1    1880.0 184590 7324.9
## - TEAM_BATTING_BB   1    2658.3 185368 7331.3
## - TEAM_BATTING_H    1    4833.0 187543 7349.1
## - TEAM_FIELDING_DP  1    6705.4 189416 7364.2
## - TEAM_BATTING_3B   1    7548.6 190259 7370.9
## - TEAM_BASERUN_SB   1    8142.6 190853 7375.7
## - TEAM_FIELDING_E   1   18841.1 201551 7458.6
## 
## Step:  AIC=7310.8
## TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + TEAM_BATTING_3B + 
##     TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + TEAM_BASERUN_SB + 
##     TEAM_BASERUN_CS + TEAM_PITCHING_BB + TEAM_PITCHING_SO + TEAM_FIELDING_E + 
##     TEAM_FIELDING_DP
## 
##                    Df Sum of Sq    RSS    AIC
## <none>                          182883 7310.8
## - TEAM_BASERUN_CS   1     418.7 183302 7312.3
## - TEAM_PITCHING_SO  1    1202.0 184085 7318.8
## - TEAM_PITCHING_BB  1    1537.3 184421 7321.5
## - TEAM_BATTING_2B   1    1928.3 184812 7324.8
## - TEAM_BATTING_SO   1    1977.9 184861 7325.2
## - TEAM_BATTING_BB   1    2678.0 185561 7330.9
## - TEAM_BATTING_H    1    4860.2 187744 7348.7
## - TEAM_BATTING_HR   1    5541.1 188424 7354.2
## - TEAM_BATTING_3B   1    7420.8 190304 7369.3
## - TEAM_FIELDING_DP  1    7423.8 190307 7369.3
## - TEAM_BASERUN_SB   1    9570.9 192454 7386.4
## - TEAM_FIELDING_E   1   18687.6 201571 7456.8

summary(Model)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_2B + 
##     TEAM_BATTING_3B + TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO + 
##     TEAM_BASERUN_SB + TEAM_BASERUN_CS + TEAM_PITCHING_BB + TEAM_PITCHING_SO + 
##     TEAM_FIELDING_E + TEAM_FIELDING_DP, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -45.468  -6.985  -0.128   7.454  34.637 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       1.368e+03  1.084e+02  12.624  < 2e-16 ***
## TEAM_BATTING_H    3.169e-02  5.006e-03   6.331 3.22e-10 ***
## TEAM_BATTING_2B  -4.198e-02  1.053e-02  -3.987 7.00e-05 ***
## TEAM_BATTING_3B   1.756e-01  2.244e-02   7.822 9.68e-15 ***
## TEAM_BATTING_HR   7.519e-02  1.112e-02   6.759 1.97e-11 ***
## TEAM_BATTING_BB   1.564e-01  3.328e-02   4.699 2.85e-06 ***
## TEAM_BATTING_SO  -8.768e-02  2.171e-02  -4.038 5.65e-05 ***
## TEAM_BASERUN_SB   6.625e-02  7.458e-03   8.884  < 2e-16 ***
## TEAM_BASERUN_CS  -6.510e-02  3.503e-02  -1.858 0.063350 .  
## TEAM_PITCHING_BB -1.130e-01  3.175e-02  -3.560 0.000382 ***
## TEAM_PITCHING_SO  6.484e-02  2.059e-02   3.148 0.001675 ** 
## TEAM_FIELDING_E  -1.085e+03  8.739e+01 -12.413  < 2e-16 ***
## TEAM_FIELDING_DP -1.121e-01  1.433e-02  -7.824 9.56e-15 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.01 on 1508 degrees of freedom
## Multiple R-squared:  0.3725, Adjusted R-squared:  0.3675 
## F-statistic: 74.59 on 12 and 1508 DF,  p-value: < 2.2e-16

The step backward variable selection process identified eleven significant variables with an R-squared of 37%, Residual Error of 11.01 and F-Statistic of 74.59. Notice that some of the coefficients are negative which means these Team will most likely result in negative wins. We will explore these coefficient a little further in this analysis.

0.4.1 OLS- MODEL 1

Using all the 15 Variables

Model1 <-lm(TARGET_WINS ~ ., data=baseball_train_03)
summary(Model1)

## 
## Call:
## lm(formula = TARGET_WINS ~ ., data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -45.067  -7.014  -0.101   7.499  34.361 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       1.376e+03  1.089e+02  12.634  < 2e-16 ***
## TEAM_BATTING_H    2.992e-02  2.242e-02   1.335   0.1821    
## TEAM_BATTING_2B  -4.140e-02  1.056e-02  -3.921 9.23e-05 ***
## TEAM_BATTING_3B   1.775e-01  2.252e-02   7.882 6.15e-15 ***
## TEAM_BATTING_HR   2.424e-01  1.452e-01   1.669   0.0953 .  
## TEAM_BATTING_BB   1.606e-01  7.051e-02   2.278   0.0229 *  
## TEAM_BATTING_SO  -1.072e-01  2.741e-02  -3.913 9.52e-05 ***
## TEAM_BASERUN_SB   6.361e-02  7.794e-03   8.161 6.94e-16 ***
## TEAM_BASERUN_CS  -5.853e-02  3.547e-02  -1.650   0.0992 .  
## TEAM_PITCHING_H   1.587e-03  2.058e-02   0.077   0.9386    
## TEAM_PITCHING_HR -1.617e-01  1.398e-01  -1.156   0.2478    
## TEAM_PITCHING_BB -1.166e-01  6.735e-02  -1.732   0.0836 .  
## TEAM_PITCHING_SO  8.366e-02  2.614e-02   3.201   0.0014 ** 
## TEAM_FIELDING_E  -1.092e+03  8.785e+01 -12.430  < 2e-16 ***
## TEAM_FIELDING_DP -1.086e-01  1.463e-02  -7.426 1.87e-13 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.01 on 1506 degrees of freedom
## Multiple R-squared:  0.3731, Adjusted R-squared:  0.3672 
## F-statistic: 64.01 on 14 and 1506 DF,  p-value: < 2.2e-16

This Model identified seven significant variables at = 0.05 with an R-squared of 37%, Residual Error of 11.01 and F-Statistic of 64.01. Although the F-Statistic reduced, this model does not improve significantly from the previous model.

Metrics1 <- data.frame(
  R2 = rsquare(Model1, data = baseball_train_03),
  RMSE = rmse(Model1, data = baseball_train_03),
  MAE = mae(Model1, data = baseball_train_03)
)
print(Metrics1)

##          R2     RMSE      MAE
## 1 0.3730717 10.96013 8.741105

0.4.2 OLS- MODEL 2

Using all the seven (7) significant variables from Model 1

Model2 <- lm(TARGET_WINS~TEAM_FIELDING_E + TEAM_BASERUN_SB + TEAM_BATTING_3B + TEAM_FIELDING_DP + TEAM_PITCHING_SO + TEAM_BATTING_SO + TEAM_BATTING_2B,data=baseball_train_03)
summary(Model2)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_FIELDING_E + TEAM_BASERUN_SB + 
##     TEAM_BATTING_3B + TEAM_FIELDING_DP + TEAM_PITCHING_SO + TEAM_BATTING_SO + 
##     TEAM_BATTING_2B, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -48.799  -8.299  -0.053   8.472  39.785 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       1.808e+03  1.158e+02  15.607  < 2e-16 ***
## TEAM_FIELDING_E  -1.410e+03  9.313e+01 -15.141  < 2e-16 ***
## TEAM_BASERUN_SB   5.429e-02  7.497e-03   7.242 7.02e-13 ***
## TEAM_BATTING_3B   1.788e-01  2.289e-02   7.808 1.08e-14 ***
## TEAM_FIELDING_DP -5.319e-02  1.525e-02  -3.488 0.000501 ***
## TEAM_PITCHING_SO -8.587e-03  9.176e-03  -0.936 0.349497    
## TEAM_BATTING_SO  -8.186e-03  9.308e-03  -0.880 0.379267    
## TEAM_BATTING_2B   4.475e-02  7.971e-03   5.614 2.35e-08 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.19 on 1513 degrees of freedom
## Multiple R-squared:  0.2288, Adjusted R-squared:  0.2253 
## F-statistic: 64.14 on 7 and 1513 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 22%, Residual Error of 12.19 and F-Statistic of 64.14. The R-Squared decreased and the Error increased slightly.

Metrics2 <- data.frame(
  R2 = rsquare(Model2, data = baseball_train_03),
  RMSE = rmse(Model2, data = baseball_train_03),
  MAE = mae(Model2, data = baseball_train_03)
)
print(Metrics2)

##          R2     RMSE      MAE
## 1 0.2288348 12.15572 9.731629

0.4.3 OLS- MODEL 3

All offensive categories which include hitting and base running

Model3 <-lm(TARGET_WINS~TEAM_BATTING_H + TEAM_BATTING_BB + TEAM_BATTING_HR + TEAM_BATTING_2B + TEAM_BATTING_SO + TEAM_BASERUN_CS + TEAM_BATTING_3B + TEAM_BASERUN_SB,data=baseball_train_03)
summary(Model3)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_BATTING_H + TEAM_BATTING_BB + 
##     TEAM_BATTING_HR + TEAM_BATTING_2B + TEAM_BATTING_SO + TEAM_BASERUN_CS + 
##     TEAM_BATTING_3B + TEAM_BASERUN_SB, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -49.812  -7.822   0.247   8.166  35.877 
## 
## Coefficients:
##                  Estimate Std. Error t value Pr(>|t|)    
## (Intercept)     15.763131   7.024077   2.244   0.0250 *  
## TEAM_BATTING_H   0.024765   0.005285   4.686 3.03e-06 ***
## TEAM_BATTING_BB  0.037681   0.003994   9.435  < 2e-16 ***
## TEAM_BATTING_HR  0.099319   0.011448   8.676  < 2e-16 ***
## TEAM_BATTING_2B -0.013435   0.010919  -1.230   0.2187    
## TEAM_BATTING_SO -0.010801   0.002767  -3.904 9.88e-05 ***
## TEAM_BASERUN_CS -0.068614   0.037166  -1.846   0.0651 .  
## TEAM_BATTING_3B  0.115379   0.022950   5.027 5.57e-07 ***
## TEAM_BASERUN_SB  0.076701   0.007431  10.321  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.73 on 1512 degrees of freedom
## Multiple R-squared:  0.2857, Adjusted R-squared:  0.2819 
## F-statistic: 75.58 on 8 and 1512 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 28%, Residual Error of 11.73 and F-Statistic of 75.58. Although the R-squared is not that great, the standard errors are more reasonable. We will hold onto this Model as performing better than the previous models for now.

Metrics3 <- data.frame(
  R2 = rsquare(Model3, data = baseball_train_03),
  RMSE = rmse(Model3, data = baseball_train_03),
  MAE = mae(Model3, data = baseball_train_03)
)
print(Metrics3)

##          R2     RMSE      MAE
## 1 0.2856527 11.69934 9.330048

0.4.4 OLS- MODEL 4

All defensive categories which include fielding and pitching

Model4 <- lm(TARGET_WINS~TEAM_PITCHING_H + TEAM_PITCHING_BB + TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E,data=baseball_train_03)
summary(Model4)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_PITCHING_H + TEAM_PITCHING_BB + 
##     TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_FIELDING_E, data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -48.818  -8.397   0.393   8.617  42.600 
## 
## Coefficients:
##                    Estimate Std. Error t value Pr(>|t|)    
## (Intercept)       7.915e+02  1.079e+02   7.335 3.61e-13 ***
## TEAM_PITCHING_H   2.420e-02  2.705e-03   8.947  < 2e-16 ***
## TEAM_PITCHING_BB  2.542e-02  3.943e-03   6.448 1.52e-10 ***
## TEAM_PITCHING_HR  1.503e-02  9.799e-03   1.533 0.125415    
## TEAM_PITCHING_SO -9.205e-03  2.369e-03  -3.886 0.000106 ***
## TEAM_FIELDING_E  -6.153e+02  8.770e+01  -7.016 3.44e-12 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 12.46 on 1515 degrees of freedom
## Multiple R-squared:  0.1932, Adjusted R-squared:  0.1905 
## F-statistic: 72.56 on 5 and 1515 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 19%, Residual Error of 12.46 and F-Statistic of 75.56.There is no significant improvement with this model.

Metrics4 <- data.frame(
  R2 = rsquare(Model4, data = baseball_train_03),
  RMSE = rmse(Model4, data = baseball_train_03),
  MAE = mae(Model4, data = baseball_train_03)
)
print(Metrics4)

##          R2     RMSE      MAE
## 1 0.1932003 12.43339 9.945165

0.4.5 OLS- MODEL 5

Using only the significant variables from Model 3

Model5 <- lm(TARGET_WINS~TEAM_PITCHING_H + TEAM_PITCHING_BB + TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_BATTING_3B + TEAM_BASERUN_SB,data=baseball_train_03)
summary(Model5)

## 
## Call:
## lm(formula = TARGET_WINS ~ TEAM_PITCHING_H + TEAM_PITCHING_BB + 
##     TEAM_PITCHING_HR + TEAM_PITCHING_SO + TEAM_BATTING_3B + TEAM_BASERUN_SB, 
##     data = baseball_train_03)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -51.002  -7.725   0.407   8.171  36.647 
## 
## Coefficients:
##                   Estimate Std. Error t value Pr(>|t|)    
## (Intercept)      44.113607   4.898047   9.006  < 2e-16 ***
## TEAM_PITCHING_H   0.002544   0.002951   0.862    0.389    
## TEAM_PITCHING_BB  0.029880   0.003774   7.917 4.66e-15 ***
## TEAM_PITCHING_HR  0.134928   0.009503  14.198  < 2e-16 ***
## TEAM_PITCHING_SO -0.016789   0.002522  -6.657 3.91e-11 ***
## TEAM_BATTING_3B   0.141192   0.023104   6.111 1.26e-09 ***
## TEAM_BASERUN_SB   0.077656   0.007190  10.800  < 2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 11.93 on 1514 degrees of freedom
## Multiple R-squared:  0.2606, Adjusted R-squared:  0.2576 
## F-statistic: 88.92 on 6 and 1514 DF,  p-value: < 2.2e-16

This Model identified five significant variables at = 0.05 with an R-squared of 26%, Residual Error of 11.93 and F-Statistic of 88.92. Although the R-squared is not better than than Model3, the F-statistic improved with smaller Standard Error.

Metrics5 <- data.frame(
  R2 = rsquare(Model5, data = baseball_train_03),
  RMSE = rmse(Model5, data = baseball_train_03),
  MAE = mae(Model5, data = baseball_train_03)
)
print(Metrics5)

##          R2     RMSE      MAE
## 1 0.2605772 11.90291 9.506094

0.4.6 Compare OLS Model Quality

anova(Model, Model1, Model2, Model3, Model4, Model5)

Res.Df	RSS	Df	Sum of Sq	F	Pr(>F)
1.51e+03	1.83e+05
1.51e+03	1.83e+05	2	174	0.717	0.488
1.51e+03	2.25e+05	-7	-4.2e+04	49.5	1.39e-63
1.51e+03	2.08e+05	1	1.66e+04	136	3.01e-30
1.52e+03	2.35e+05	-3	-2.69e+04	74	1.15e-44
1.51e+03	2.15e+05	1	1.96e+04	162	2.72e-35

tab_model(Model, Model1, Model2, Model3, Model4, Model5)

	TARGET_WINS			TARGET_WINS			TARGET_WINS			TARGET_WINS			TARGET_WINS			TARGET_WINS
Predictors	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p	Estimates	CI	p
(Intercept)	1368.26	1155.66 – 1580.87	<0.001	1376.10	1162.45 – 1589.76	<0.001	1807.57	1580.39 – 2034.75	<0.001	15.76	1.99 – 29.54	0.025	791.49	579.82 – 1003.16	<0.001	44.11	34.51 – 53.72	<0.001
TEAM_BATTING_H	0.03	0.02 – 0.04	<0.001	0.03	-0.01 – 0.07	0.182				0.02	0.01 – 0.04	<0.001
TEAM_BATTING_2B	-0.04	-0.06 – -0.02	<0.001	-0.04	-0.06 – -0.02	<0.001	0.04	0.03 – 0.06	<0.001	-0.01	-0.03 – 0.01	0.219
TEAM_BATTING_3B	0.18	0.13 – 0.22	<0.001	0.18	0.13 – 0.22	<0.001	0.18	0.13 – 0.22	<0.001	0.12	0.07 – 0.16	<0.001				0.14	0.10 – 0.19	<0.001
TEAM_BATTING_HR	0.08	0.05 – 0.10	<0.001	0.24	-0.04 – 0.53	0.095				0.10	0.08 – 0.12	<0.001
TEAM_BATTING_BB	0.16	0.09 – 0.22	<0.001	0.16	0.02 – 0.30	0.023				0.04	0.03 – 0.05	<0.001
TEAM_BATTING_SO	-0.09	-0.13 – -0.05	<0.001	-0.11	-0.16 – -0.05	<0.001	-0.01	-0.03 – 0.01	0.379	-0.01	-0.02 – -0.01	<0.001
TEAM_BASERUN_SB	0.07	0.05 – 0.08	<0.001	0.06	0.05 – 0.08	<0.001	0.05	0.04 – 0.07	<0.001	0.08	0.06 – 0.09	<0.001				0.08	0.06 – 0.09	<0.001
TEAM_BASERUN_CS	-0.07	-0.13 – 0.00	0.063	-0.06	-0.13 – 0.01	0.099				-0.07	-0.14 – 0.00	0.065
TEAM_PITCHING_BB	-0.11	-0.18 – -0.05	<0.001	-0.12	-0.25 – 0.02	0.084							0.03	0.02 – 0.03	<0.001	0.03	0.02 – 0.04	<0.001
TEAM_PITCHING_SO	0.06	0.02 – 0.11	0.002	0.08	0.03 – 0.13	0.001	-0.01	-0.03 – 0.01	0.349				-0.01	-0.01 – -0.00	<0.001	-0.02	-0.02 – -0.01	<0.001
TEAM_FIELDING_E	-1084.76	-1256.17 – -913.35	<0.001	-1091.94	-1264.26 – -919.62	<0.001	-1410.16	-1592.85 – -1227.48	<0.001				-615.31	-787.34 – -443.27	<0.001
TEAM_FIELDING_DP	-0.11	-0.14 – -0.08	<0.001	-0.11	-0.14 – -0.08	<0.001	-0.05	-0.08 – -0.02	0.001
TEAM_PITCHING_H				0.00	-0.04 – 0.04	0.939							0.02	0.02 – 0.03	<0.001	0.00	-0.00 – 0.01	0.389
TEAM_PITCHING_HR				-0.16	-0.44 – 0.11	0.248							0.02	-0.00 – 0.03	0.125	0.13	0.12 – 0.15	<0.001
Observations	1521			1521			1521			1521			1521			1521
R2 / R2 adjusted	0.372 / 0.367			0.373 / 0.367			0.229 / 0.225			0.286 / 0.282			0.193 / 0.191			0.261 / 0.258

0.4.7 RIDGE Regression- MODEL 6

The Ridge regression is an extension of linear regression where the loss function is modified to minimize the complexity of the model. This modification is done by adding a penalty parameter that is equivalent to the square of the magnitude of the coefficients.

Before implementing the RIDGE model, we will split the training dataset into 2 parts that is - training set within the training set and a test set that can be used for evaluation. By enforcing stratified sampling both our training and testing sets have approximately equal response “TARGET_WINS” distributions.

Transforming the variables into the form of a matrix will enable us to penalize the model using the ‘glmnet’ method in glmnet package.

#Split the data into Training and Test Set
baseball_train_set<- initial_split(baseball_train_03, prop = 0.7, strata = "TARGET_WINS")
train_baseball  <- training(baseball_train_set)
test_baseball   <- testing(baseball_train_set)

train_Ind<- as.matrix(train_baseball)
train_Dep<- as.matrix(train_baseball$TARGET_WINS)

test_Ind<- as.matrix(test_baseball)
test_Dep<- as.matrix(test_baseball$TARGET_WINS)

For the avoidance of multicollinearity, avoiding overfitting and predicting better, implementing RIDGE regression will become useful.

lambdas <- 10^seq(2, -3, by = -.1)
Model6 <- glmnet(train_Ind,train_Dep, nlambda = 25, alpha = 0, family = 'gaussian', lambda = lambdas)
summary(Model6)

##           Length Class     Mode   
## a0         51    -none-    numeric
## beta      765    dgCMatrix S4     
## df         51    -none-    numeric
## dim         2    -none-    numeric
## lambda     51    -none-    numeric
## dev.ratio  51    -none-    numeric
## nulldev     1    -none-    numeric
## npasses     1    -none-    numeric
## jerr        1    -none-    numeric
## offset      1    -none-    logical
## call        7    -none-    call   
## nobs        1    -none-    numeric

print(Model6, digits = max(3, getOption("digits") - 3),
           signif.stars = getOption("show.signif.stars"))

## 
## Call:  glmnet(x = train_Ind, y = train_Dep, family = "gaussian", alpha = 0,      nlambda = 25, lambda = lambdas) 
## 
##    Df   %Dev  Lambda
## 1  15 0.3009 100.000
## 2  15 0.3507  79.430
## 3  15 0.4043  63.100
## 4  15 0.4608  50.120
## 5  15 0.5193  39.810
## 6  15 0.5783  31.620
## 7  15 0.6366  25.120
## 8  15 0.6927  19.950
## 9  15 0.7453  15.850
## 10 15 0.7932  12.590
## 11 15 0.8357  10.000
## 12 15 0.8721   7.943
## 13 15 0.9025   6.310
## 14 15 0.9272   5.012
## 15 15 0.9466   3.981
## 16 15 0.9616   3.162
## 17 15 0.9728   2.512
## 18 15 0.9810   1.995
## 19 15 0.9869   1.585
## 20 15 0.9911   1.259
## 21 15 0.9940   1.000
## 22 15 0.9960   0.794
## 23 15 0.9974   0.631
## 24 15 0.9983   0.501
## 25 15 0.9989   0.398
## 26 15 0.9993   0.316
## 27 15 0.9995   0.251
## 28 15 0.9997   0.200
## 29 15 0.9998   0.158
## 30 15 0.9999   0.126
## 31 15 0.9999   0.100
## 32 15 0.9999   0.079
## 33 15 1.0000   0.063
## 34 15 1.0000   0.050
## 35 15 1.0000   0.040
## 36 15 1.0000   0.032
## 37 15 1.0000   0.025
## 38 15 1.0000   0.020
## 39 15 1.0000   0.016
## 40 15 1.0000   0.013
## 41 15 1.0000   0.010
## 42 15 1.0000   0.008
## 43 15 1.0000   0.006
## 44 15 1.0000   0.005
## 45 15 1.0000   0.004
## 46 15 1.0000   0.003
## 47 15 1.0000   0.003
## 48 15 1.0000   0.002
## 49 15 1.0000   0.002
## 50 15 1.0000   0.001
## 51 15 1.0000   0.001

The significant difference between the OLS and the Ridge Regresion is the hyperparameter tuning using lambda. The Ridge regression does not perform Feature Selection, but it predicts better and solve overfitting. Cross Validating the Ridge Regression will help us to identify the optimal lambda to penalize the model and enhance the predictability.

CrossVal_ridge <- cv.glmnet(train_Ind,train_Dep, alpha = 0, lambda = lambdas)
optimal_lambda <- CrossVal_ridge$lambda.min
optimal_lambda #The optimal lambda is 0.001 which we will use to penelize the Ridge Regression model.

## [1] 0.001

coef(CrossVal_ridge) # Shows the coefficients

## 16 x 1 sparse Matrix of class "dgCMatrix"
##                              1
## (Intercept)       3.341952e-01
## TARGET_WINS       9.998440e-01
## TEAM_BATTING_H   -4.649153e-06
## TEAM_BATTING_2B  -1.962509e-06
## TEAM_BATTING_3B   2.112129e-05
## TEAM_BATTING_HR  -4.411468e-04
## TEAM_BATTING_BB   1.149412e-04
## TEAM_BATTING_SO  -2.186509e-05
## TEAM_BASERUN_SB   2.014115e-05
## TEAM_BASERUN_CS  -2.713094e-05
## TEAM_PITCHING_H   6.630828e-06
## TEAM_PITCHING_HR  4.375479e-04
## TEAM_PITCHING_BB -1.065721e-04
## TEAM_PITCHING_SO  1.618392e-05
## TEAM_FIELDING_E  -2.627432e-01
## TEAM_FIELDING_DP -1.837557e-05

plot(CrossVal_ridge)

The plot shows that the errors increases as the magnitude of lambda increases, previously, we identified that the optimal lambda is 0.001 which is very obvious from the plot above. The coefficients are restricted to be small but not quite zero as Ridge Regression does not force the coefficient to zero. This indicates that the model is performing well so far. But let’s make it better using the optimal labmda.

eval_results <- function(true, predicted, df){
  SSE <- sum((predicted - true)^2)
  SST <- sum((true - mean(true))^2)
  R_square <- 1 - SSE / SST
  RMSE = sqrt(SSE/nrow(df))
data.frame(   
  RMSE = RMSE,
  Rsquare = R_square
)
  
}
# Prediction and evaluation on train data
predictions_train <- predict(Model6, s = optimal_lambda, newx = train_Ind)
eval_results(train_Dep, predictions_train, train_baseball)

RMSE	Rsquare
0.00181	1

We should be a little concern about the 100% R-squared performance for this Model. Although the Ridge Regression forces the coefficients towards zero to improve the Model performance and enhance the predictability, the very high peformance may require further investigation. Lets improve the model using a more reason lambda because optimal might not always be the best.

0.4.8 The Improved Ridge Regression

Model6_Improved <- glmnet(train_Ind,train_Dep, nlambda = 25, alpha = 0, family = 'gaussian', lambda = 6.310)
summary(Model6_Improved)

##           Length Class     Mode   
## a0         1     -none-    numeric
## beta      15     dgCMatrix S4     
## df         1     -none-    numeric
## dim        2     -none-    numeric
## lambda     1     -none-    numeric
## dev.ratio  1     -none-    numeric
## nulldev    1     -none-    numeric
## npasses    1     -none-    numeric
## jerr       1     -none-    numeric
## offset     1     -none-    logical
## call       7     -none-    call   
## nobs       1     -none-    numeric

coef(Model6_Improved)

## 16 x 1 sparse Matrix of class "dgCMatrix"
##                             s0
## (Intercept)       2.035211e+02
## TARGET_WINS       6.270221e-01
## TEAM_BATTING_H    5.662399e-03
## TEAM_BATTING_2B   2.221503e-03
## TEAM_BATTING_3B   2.758313e-02
## TEAM_BATTING_HR   8.601813e-03
## TEAM_BATTING_BB   6.570891e-03
## TEAM_BATTING_SO  -1.015369e-03
## TEAM_BASERUN_SB   1.397857e-02
## TEAM_BASERUN_CS  -7.423992e-03
## TEAM_PITCHING_H   1.916568e-03
## TEAM_PITCHING_HR  8.211906e-03
## TEAM_PITCHING_BB  5.082248e-03
## TEAM_PITCHING_SO -1.371691e-03
## TEAM_FIELDING_E  -1.550931e+02
## TEAM_FIELDING_DP -2.226452e-02

Let’s compute the Model’s Performance Metric to see how this model is doing.

eval_results <- function(true, predicted, df){
  SSE <- sum((predicted - true)^2)
  SST <- sum((true - mean(true))^2)
  R_square <- 1 - SSE / SST
  RMSE = sqrt(SSE/nrow(df))
data.frame(   
  RMSE = RMSE,
  Rsquare = R_square
)
  
}

# Prediction and evaluation on train data
predictions_train <- predict(Model6_Improved, s = lambda, newx = train_Ind)
eval_results(train_Dep, predictions_train, train_baseball)

RMSE	Rsquare
4.34	0.903

# Prediction and evaluation on test data
predictions_test <- predict(Model6_Improved, s = lambda, newx = test_Ind)
eval_results(test_Dep, predictions_test, test_baseball)

RMSE	Rsquare
4.27	0.903

The improved Model6 output shows that the RMSE and R-squared values for the Ridge Regression model on the training and test data are significantly improved. The Loss Function (RMSE) are severely reduced compared to the OLS models which indicates that the Ridge Regression is not overfitting. These performance is significantly improved compared to the OLS Models 1 to 5.

0.4.9 Model Performance Comparison

ModelName <- c("Model", "Model1","Model2","Model3","Model4","Model5","Model6")
Model_RSquared <- c("37%", "37%", "22%", "28%", "19%", "26% ", "90%")
Model_RMSE <- c("11.01", "10.96", "12.15", "11.69", "12.43", "11.93 ", "4.33")
Model_FStatistic <- c("74.59", "64.01", "64.14", "75.58", "72.56", "88.92 ", "NA")
Model_Performance <- data.frame(ModelName,Model_RSquared,Model_RMSE,Model_FStatistic)
Model_Performance

ModelName	Model_RSquared	Model_RMSE	Model_FStatistic
Model	37%	11.01	74.59
Model1	37%	10.96	64.01
Model2	22%	12.15	64.14
Model3	28%	11.69	75.58
Model4	19%	12.43	72.56
Model5	26%	11.93	88.92
Model6	90%	4.33	NA

0.4.10 Model Prediction

Based on the Model metrics above, we’re ready to make prediction and we will select our acceptable OLS Model3 and Model5 which has better F-Statistic, smaller standard errors and less negative coefficient as our best OLS models. We will also compare the prediction accuracy of these models to that of the improved Ridge Regression Model which is our champion Model for this exercise based on the very small RMSE and the highest R-squared of over 90%.

predicted <- predict(Model3, newx = test_baseball)# predict on test data
predicted_values <- cbind (actual=test_baseball$TARGET_WINS, predicted)  # combine

## Warning in cbind(actual = test_baseball$TARGET_WINS, predicted): number of
## rows of result is not a multiple of vector length (arg 1)

predicted_values

##      actual predicted
## 1        78  69.49875
## 2        88  67.56125
## 3        66  68.38378
## 4        90  73.25628
## 5        87  67.37030
## 6        70  66.86382
## 7        70  63.36361
## 8        82  76.58169
## 9        75  89.59170
## 10       85  76.54657
## 11       98  86.70081
## 12       51  77.31870
## 13       76  78.60647
## 14      111  84.73255
## 15       68  89.15530
## 16       58  86.01142
## 17       53  76.15138
## 18       56  76.18203
## 19       74  79.02243
## 20       81  72.03653
## 21       65  89.30377
## 22       68  84.16328
## 23       71  79.30964
## 24       86  76.00667
## 25       87  93.36918
## 26       94  79.72780
## 27       85  83.96472
## 28       74  81.83322
## 29       67  86.75976
## 30       76  83.85432
## 31       65  75.24767
## 32      106  96.12431
## 33       79  85.18416
## 34       93  87.23777
## 35       76  86.66397
## 36       74  80.90673
## 37       88  71.73633
## 38       86  81.22686
## 39       70  90.64517
## 40       69  94.18601
## 41       61  95.53100
## 42       67  80.08285
## 43       77  75.86419
## 44       86  66.74352
## 45       91  64.57578
## 46       91  60.91243
## 47       92  72.28834
## 48       91  68.48019
## 49       85  65.67250
## 50       84  72.48471
## 51       73  86.44011
## 52       67  80.65902
## 53       54  82.06254
## 54       76  78.70595
## 55       67  77.58369
## 56       85  72.97126
## 57       80  78.18561
## 58       79  72.24895
## 59       67  76.25716
## 60       78  77.02116
## 61       74  75.30225
## 62       86  76.46272
## 63      100  78.36297
## 64       83  68.74638
## 65       52  70.51741
## 66       94  66.00240
## 67      112  74.32965
## 68       45  68.02212
## 69       85  71.72855
## 70       96  69.41157
## 71       86  69.61566
## 72       89  70.79319
## 73       83  69.48860
## 74       73  65.80599
## 75       76  70.70940
## 76       95  82.73716
## 77       98  77.23526
## 78       75  81.04205
## 79       98  86.68577
## 80       93  81.16218
## 81       86  88.42305
## 82      104  85.99275
## 83      105  73.58515
## 84       98  82.22049
## 85       88  85.06236
## 86       74  86.09993
## 87      103  76.29960
## 88       73  79.91231
## 89       63  77.53938
## 90       59  84.34369
## 91       82  74.79848
## 92       88  70.02759
## 93       89  82.39423
## 94       81  79.28918
## 95       80  91.97048
## 96       97  78.96951
## 97       77  74.19733
## 98       71  72.42719
## 99       78  77.14336
## 100      82  76.15731
## 101      89  77.37089
## 102      65  72.05640
## 103      88  76.89373
## 104      88  75.92414
## 105     100  85.81217
## 106      93  75.59281
## 107      74  80.60366
## 108      63  85.62160
## 109      53  82.00590
## 110      79  84.74713
## 111      89  84.88357
## 112      77  86.74637
## 113      78  89.85103
## 114      74  88.78040
## 115      85  78.46174
## 116      94  84.86044
## 117      96  83.42672
## 118      89  82.79613
## 119      86  82.72617
## 120      95  66.91725
## 121      79  69.51332
## 122      92  78.66910
## 123      64  72.33942
## 124      90  67.50771
## 125      94  64.77165
## 126      77  69.79720
## 127      95  71.15595
## 128      93  77.34661
## 129      98  77.95811
## 130      90  86.54917
## 131      84  73.89007
## 132      69  79.45996
## 133      77  76.41158
## 134      64  81.88671
## 135      86  84.23277
## 136      56  81.62631
## 137      93  78.66292
## 138      64  78.01240
## 139      70  82.19463
## 140      68  84.27986
## 141      70  73.82318
## 142      98  75.50558
## 143      92  72.20240
## 144      89  71.73169
## 145     108  78.70550
## 146      89  82.60550
## 147      61  79.89101
## 148      73  80.98410
## 149      95  73.62353
## 150      66  74.78823
## 151      78  70.09769
## 152      69  69.79510
## 153      80  73.21664
## 154      91  76.42604
## 155      75  70.00951
## 156      93  79.24643
## 157      85  76.55529
## 158      82  77.45710
## 159      94  77.29383
## 160      97  74.83120
## 161      98  80.57564
## 162      97  75.27052
## 163      79  72.96984
## 164      81  88.01300
## 165      75  89.33125
## 166      62  86.25568
## 167      75  69.87604
## 168      84  89.64940
## 169      88  80.48480
## 170      82  81.38282
## 171      74  78.12840
## 172      68  80.46360
## 173      67  77.90473
## 174      66  86.93266
## 175      95  88.60549
## 176      83  86.22276
## 177      71  85.13130
## 178      95  80.44713
## 179      82  88.03475
## 180      97  80.82213
## 181      77  83.47961
## 182      53  72.80089
## 183     101  80.37391
## 184      88  79.72336
## 185     103  78.69135
## 186      79  85.57211
## 187      58  83.20429
## 188      74  89.48194
## 189      86  96.68228
## 190      83  87.75298
## 191      87  91.87313
## 192      88  86.26651
## 193      84  77.82142
## 194      75  76.04487
## 195      68  84.88891
## 196      65  81.55068
## 197      55  83.70493
## 198      54  80.38773
## 199      66 101.92399
## 200      65  81.10336
## 201      72  73.46680
## 202      69  93.36972
## 203      72  86.05010
## 204      79  85.59259
## 205      89  73.88114
## 206      82  74.83380
## 207      81  64.08681
## 208      64  65.84094
## 209      80  71.98560
## 210      76  79.13560
## 211      86  85.15069
## 212      65  84.82851
## 213      85  88.64454
## 214      72  87.36139
## 215      80  73.46671
## 216      76  73.32881
## 217      83  76.75376
## 218      92  80.18206
## 219      79  75.03975
## 220      65  75.66433
## 221      58  70.51948
## 222      69  74.54246
## 223      61  72.27715
## 224      79  70.35686
## 225      75  72.18585
## 226      80  79.91554
## 227      97  88.74324
## 228      82  87.01201
## 229      69  85.96054
## 230      66  89.84882
## 231     105  93.28619
## 232      86  91.12115
## 233      92  94.67752
## 234      99  73.79792
## 235      88  80.75316
## 236     102  76.99427
## 237     102  87.39336
## 238     110  77.79257
## 239     104  75.87232
## 240      75  87.87867
## 241      80  79.02455
## 242      95  79.53593
## 243      85  77.39581
## 244      95  71.37449
## 245      78  85.39949
## 246      90  78.33072
## 247      86  86.04030
## 248      71  79.55776
## 249      66  82.75494
## 250      93  86.19625
## 251      86  87.67022
## 252      71  85.00793
## 253      78  80.45297
## 254      68  87.60385
## 255      81  82.44495
## 256      73  77.50487
## 257      45  79.20244
## 258      78  69.31674
## 259      91  74.80593
## 260      89  83.14847
## 261      77  91.63208
## 262      69  93.48635
## 263      89  84.53632
## 264      92  86.35405
## 265      67  86.94776
## 266      56  82.27458
## 267      81  80.34078
## 268      77  97.84021
## 269      61  90.77786
## 270      70  86.29480
## 271      85  89.59173
## 272      91  81.45834
## 273      74  79.98754
## 274      95  76.10745
## 275      78  78.12297
## 276      70  76.91669
## 277      90  74.90087
## 278      41  87.24509
## 279      61  95.14666
## 280     100  90.93669
## 281      86  90.70110
## 282      98  81.29394
## 283      92  72.64682
## 284     101  82.98819
## 285      87  75.16897
## 286      91  78.10723
## 287      72  82.14880
## 288      78  84.91555
## 289      71  97.32176
## 290      88 101.10846
## 291      99  84.72247
## 292      97  75.83547
## 293      54  76.87742
## 294      79  84.10901
## 295      96  89.08411
## 296      99  81.50902
## 297     114  81.45233
## 298      92  78.28904
## 299      97  74.25636
## 300      96  75.53547
## 301      93  78.88184
## 302      80  77.90593
## 303      99  81.68807
## 304      79  76.22767
## 305      71  71.70319
## 306      88  74.06627
## 307      88  74.02593
## 308      97  79.72855
## 309      58  78.32084
## 310      66  75.75259
## 311      56  83.80749
## 312      67  79.14016
## 313      76  77.77473
## 314      88  77.15735
## 315      62  76.99388
## 316      59  68.85761
## 317      89  74.32405
## 318     102  76.06391
## 319      54  76.38482
## 320      77  77.40995
## 321      96  77.07957
## 322      65  81.59777
## 323      74  73.66385
## 324      91  79.18950
## 325      92  77.63615
## 326      85  79.20131
## 327     100  79.58513
## 328      96  78.45513
## 329      66  85.18691
## 330      87  87.36482
## 331      84  81.06456
## 332      94  86.55401
## 333      54  76.85508
## 334      75  80.08373
## 335      45  80.76698
## 336      67  78.96028
## 337      65  81.18968
## 338      92  84.35796
## 339      92  78.91002
## 340      67  75.96436
## 341      61  85.42552
## 342     101  80.55205
## 343      90  84.56468
## 344      78  82.79470
## 345      77  76.50017
## 346      65  78.80238
## 347      79  81.74452
## 348     105  81.81584
## 349     102  83.28464
## 350     117  75.40413
## 351      91  74.86536
## 352      85  75.32920
## 353      68  70.05289
## 354      54  72.91093
## 355      89  73.69728
## 356      99  84.47503
## 357      88  72.99975
## 358      90  83.43029
## 359      93  87.95245
## 360      72  78.31703
## 361      73  78.64492
## 362      87  81.71404
## 363      53  80.02600
## 364      74  77.59988
## 365      88  79.43621
## 366      74  79.28112
## 367      73  83.90819
## 368      78  93.54258
## 369      63  86.40102
## 370      61  82.58538
## 371      92  85.89380
## 372      84  79.81702
## 373      89  76.95876
## 374      79  70.17323
## 375      64  74.13669
## 376      57  72.30131
## 377      77  75.25157
## 378      64  76.54545
## 379      88  78.41872
## 380      76  81.53011
## 381     116  85.93305
## 382      93  83.95041
## 383      78  88.37161
## 384      98  86.31066
## 385      79  82.44512
## 386      90  83.91708
## 387      93  86.67909
## 388      76  87.44296
## 389      97  82.21262
## 390      98  76.52642
## 391      77  75.16841
## 392      64  64.24604
## 393      77  72.58621
## 394      87  77.46501
## 395      83  72.80068
## 396      88  78.05726
## 397      91  80.25288
## 398      88  76.11026
## 399      62  90.32384
## 400     103  74.10965
## 401      75  79.40059
## 402      86  70.55224
## 403      97  86.59573
## 404      90  87.70315
## 405      79  83.05623
## 406      55  78.62248
## 407      67  75.43148
## 408      82  83.51626
## 409      67  74.53313
## 410      55  75.06487
## 411      87  86.07172
## 412      64  87.84534
## 413      84  94.14527
## 414      82  96.04200
## 415      83  90.90022
## 416     106  86.34220
## 417      85  94.79885
## 418     110  86.76487
## 419     102  87.56303
## 420      94  85.09908
## 421      89  88.22749
## 422      93  93.78125
## 423      80  90.14901
## 424      92  97.39963
## 425      93  88.67552
## 426      93  87.20702
## 427     102  80.27788
## 428      76  85.93653
## 429      78  74.39622
## 430      81  74.75851
## 431      87  72.61288
## 432      83  90.56274
## 433      93  89.47333
## 434     105  78.69775
## 435     100  77.35048
## 436      63  79.53762
## 437      70  83.39287
## 438      61  83.62948
## 439      61  76.57712
## 440      62  77.10248
## 441      77  83.34904
## 442      70  78.56847
## 443      83  69.57909
## 444      77  73.96331
## 445      71  63.06532
## 446      53  69.99296
## 447      89  76.64262
## 448      80  79.98629
## 449      55  73.50146
## 450      82  77.92409
## 451      78  82.13105
## 452      74  77.90857
## 453      88  72.95778
## 454      83  69.73774
## 455      71  69.46233
## 456      78  67.52634
## 457      88  72.19872
## 458      66  78.59388
## 459      90  73.35113
## 460      87  71.86038
## 461      70  75.50214
## 462      70  70.66007
## 463      82  73.52882
## 464      75  88.14247
## 465      85  88.25647
## 466      98  81.56281
## 467      51  85.11315
## 468      76  80.24222
## 469     111  81.24244
## 470      68  82.69428
## 471      58  73.51443
## 472      53  75.48643
## 473      56  88.59721
## 474      74  74.75932
## 475      81  69.56123
## 476      65  88.86014
## 477      68  71.28452
## 478      71  86.22500
## 479      86  88.95228
## 480      87  92.69140
## 481      94 101.10893
## 482      85  93.85807
## 483      74  84.70288
## 484      67  82.88195
## 485      76  84.57159
## 486      65  73.01808
## 487     106  78.50309
## 488      79  87.77309
## 489      93  90.41206
## 490      76  81.01700
## 491      74  81.18505
## 492      88  78.76486
## 493      86  79.22418
## 494      70  89.33282
## 495      69  81.16533
## 496      61  79.74078
## 497      67  92.39093
## 498      77  89.28796
## 499      86  77.12440
## 500      91  79.77105
## 501      91  75.68524
## 502      92  87.43997
## 503      91  76.52938
## 504      85  84.78700
## 505      84  80.50877
## 506      73  77.35972
## 507      67  72.72118
## 508      54  80.34685
## 509      76  72.55936
## 510      67  70.06595
## 511      85  83.25037
## 512      80  80.59516
## 513      79  81.71215
## 514      67  78.13880
## 515      78  82.04003
## 516      74  88.64843
## 517      86  84.59548
## 518     100  90.67361
## 519      83  84.94869
## 520      52  82.43145
## 521      94  77.98885
## 522     112  72.89823
## 523      45  83.43799
## 524      85  86.92602
## 525      96  87.18417
## 526      86  77.09404
## 527      89  89.40353
## 528      83  93.34790
## 529      73  86.96778
## 530      76  82.14507
## 531      95  83.43017
## 532      98  75.10548
## 533      75  74.03722
## 534      98  74.16809
## 535      93  94.69567
## 536      86  85.56016
## 537     104  93.39981
## 538     105  85.25712
## 539      98  87.29422
## 540      88  87.46496
## 541      74  83.61357
## 542     103  79.45141
## 543      73  79.45387
## 544      63  78.43478
## 545      59  73.38522
## 546      82  76.03883
## 547      88  72.47713
## 548      89  66.23869
## 549      81  67.67506
## 550      80  74.29160
## 551      97  76.15938
## 552      77  68.53339
## 553      71  64.18492
## 554      78  76.55550
## 555      82  72.31920
## 556      89  75.70448
## 557      65  86.25488
## 558      88  87.13625
## 559      88  85.66963
## 560     100  75.01403
## 561      93  72.48383
## 562      74  76.99319
## 563      63  67.65273
## 564      53  78.88975
## 565      79  85.81330
## 566      89  99.38599
## 567      77 102.30433
## 568      78  93.87881
## 569      74 100.37487
## 570      85  98.32826
## 571      94  89.11290
## 572      96  78.88624
## 573      89  73.97117
## 574      86  86.87866
## 575      95  82.38174
## 576      79  82.41774
## 577      92  70.81168
## 578      64  94.27943
## 579      90  96.56763
## 580      94  96.17381
## 581      77  83.56929
## 582      95  91.26643
## 583      93  93.49057
## 584      98  95.36925
## 585      90  80.85944
## 586      84  84.39534
## 587      69  82.97421
## 588      77  77.45613
## 589      64  82.72740
## 590      86  77.65384
## 591      56  67.80243
## 592      93  66.62810
## 593      64  74.11153
## 594      70  75.35735
## 595      68  91.41911
## 596      70  89.37015
## 597      98  82.88452
## 598      92  82.28763
## 599      89  84.44431
## 600     108  77.11652
## 601      89  89.87563
## 602      61  87.62389
## 603      73  93.76432
## 604      95  87.78726
## 605      66  93.25463
## 606      78  97.35486
## 607      69  92.79773
## 608      80  98.47135
## 609      91  94.72234
## 610      75  93.80173
## 611      93  96.01043
## 612      85  82.86228
## 613      82  76.13348
## 614      94  77.17624
## 615      97  77.23491
## 616      98  84.72118
## 617      97  84.54481
## 618      79  91.45792
## 619      81  81.48643
## 620      75  76.05063
## 621      62  75.66432
## 622      75  77.26623
## 623      84  77.11791
## 624      88  85.23634
## 625      82  94.58567
## 626      74  82.27350
## 627      68  76.21672
## 628      67  80.88749
## 629      66  77.09902
## 630      95  79.78842
## 631      83  80.55472
## 632      71  78.32830
## 633      95  74.66971
## 634      82  70.44247
## 635      97  75.76856
## 636      77  81.25690
## 637      53  85.21570
## 638     101  87.45149
## 639      88  83.24715
## 640     103  85.12337
## 641      79  90.08131
## 642      58  85.69963
## 643      74  91.72563
## 644      86  95.57921
## 645      83  80.82969
## 646      87  76.76201
## 647      88  84.40290
## 648      84  88.69413
## 649      75  83.20373
## 650      68  92.68599
## 651      65  82.30209
## 652      55  79.07586
## 653      54  86.73771
## 654      66  81.29509
## 655      65  66.71945
## 656      72  73.26650
## 657      69  86.38450
## 658      72  80.74925
## 659      79  72.12318
## 660      89  86.47275
## 661      82  79.04706
## 662      81  82.85552
## 663      64  74.31326
## 664      80  75.84208
## 665      76  84.08627
## 666      86  79.05523
## 667      65  79.31311
## 668      85  80.23662
## 669      72  75.09644
## 670      80  61.44278
## 671      76  60.04693
## 672      83  73.69484
## 673      92  75.61348
## 674      79  75.09419
## 675      65  60.58337
## 676      58  79.31909
## 677      69  85.07393
## 678      61  71.05731
## 679      79  82.34659
## 680      75  75.70126
## 681      80  76.36214
## 682      97  79.97192
## 683      82  79.96235
## 684      69  77.60617
## 685      66  75.63627
## 686     105  72.69408
## 687      86  78.08013
## 688      92  77.26828
## 689      99  81.21731
## 690      88  75.48051
## 691     102  71.40040
## 692     102  74.64094
## 693     110  78.67222
## 694     104  88.47661
## 695      75  82.65871
## 696      80  89.85068
## 697      95  97.41705
## 698      85  84.06305
## 699      95  83.32637
## 700      78  79.06699
## 701      90  81.32051
## 702      86  81.98676
## 703      71  72.57010
## 704      66  73.44789
## 705      93  74.81811
## 706      86  74.98640
## 707      71  84.21973
## 708      78  84.56636
## 709      68  82.75465
## 710      81  77.97886
## 711      73  84.76507
## 712      45  79.18286
## 713      78  82.33910
## 714      91  78.44898
## 715      89  81.50181
## 716      77  84.18849
## 717      69  80.75256
## 718      89  87.77617
## 719      92  79.01648
## 720      67  84.53152
## 721      56  73.76380
## 722      81  74.64648
## 723      77  97.05403
## 724      61  95.97254
## 725      70  77.70426
## 726      85  91.18758
## 727      91  75.73231
## 728      74  73.62204
## 729      95  65.45411
## 730      78  78.44786
## 731      70  84.83279
## 732      90  85.78446
## 733      41  82.58456
## 734      61  80.45664
## 735     100  74.78727
## 736      86  70.95567
## 737      98  73.82233
## 738      92  84.23530
## 739     101  79.76443
## 740      87  75.55942
## 741      91  83.90318
## 742      72  78.55669
## 743      78  80.46496
## 744      71  76.10448
## 745      88  77.89772
## 746      99  71.32829
## 747      97  73.19595
## 748      54  84.23038
## 749      79  80.71645
## 750      96  82.88104
## 751      99  86.58766
## 752     114  77.59884
## 753      92  76.13671
## 754      97  87.43058
## 755      96  86.48412
## 756      93  90.45556
## 757      80  86.15989
## 758      99  97.07526
## 759      79  95.61839
## 760      71  91.30071
## 761      88 102.70882
## 762      88  94.41229
## 763      97  97.02811
## 764      58  89.96791
## 765      66  82.18003
## 766      56  83.80303
## 767      67  85.00570
## 768      76  80.53210
## 769      88  87.92249
## 770      62  94.71954
## 771      59  73.29979
## 772      89  71.60731
## 773     102  67.21755
## 774      54  61.79597
## 775      77  75.69141
## 776      96  84.94305
## 777      65  76.45704
## 778      74  76.87543
## 779      91  79.08576
## 780      92  82.82991
## 781      85  77.53815
## 782     100  88.68762
## 783      96  86.93289
## 784      66  86.40456
## 785      87  85.95538
## 786      84  71.33692
## 787      94  81.90683
## 788      54  74.62795
## 789      75  72.43715
## 790      45  68.46631
## 791      67  80.61536
## 792      65  77.72868
## 793      92  78.76628
## 794      92  79.50862
## 795      67  83.21869
## 796      61  78.19842
## 797     101  89.64355
## 798      90  89.13728
## 799      78  82.46795
## 800      77  77.71627
## 801      65  68.72122
## 802      79  73.81261
## 803     105  90.70552
## 804     102  88.53050
## 805     117  93.36524
## 806      91  81.87482
## 807      85  69.53519
## 808      68  65.83305
## 809      54  80.91972
## 810      89  75.66819
## 811      99  69.78134
## 812      88  74.65439
## 813      90  86.20933
## 814      93  90.52296
## 815      72  89.88804
## 816      73  91.22499
## 817      87  85.30952
## 818      53  76.50755
## 819      74  77.81370
## 820      88  76.53768
## 821      74  80.85762
## 822      73  83.55077
## 823      78  86.28180
## 824      63  90.36860
## 825      61  76.09565
## 826      92  75.87291
## 827      84  87.02377
## 828      89  78.48445
## 829      79  76.31062
## 830      64  73.67242
## 831      57  82.00670
## 832      77  76.92192
## 833      64  77.12030
## 834      88  75.08059
## 835      76  87.08930
## 836     116  84.10366
## 837      93  69.87058
## 838      78  67.95321
## 839      98  76.27043
## 840      79  77.82358
## 841      90  61.48364
## 842      93  73.93711
## 843      76  80.09102
## 844      97  80.39881
## 845      98  77.94942
## 846      77  79.27261
## 847      64  80.39794
## 848      77  79.57612
## 849      87  88.63689
## 850      83  85.94038
## 851      88  83.89021
## 852      91  84.32137
## 853      88  88.01176
## 854      62  89.30777
## 855     103  84.21572
## 856      75  89.72324
## 857      86  88.68351
## 858      97  93.65513
## 859      90  86.63118
## 860      79  80.90752
## 861      55  83.25118
## 862      67  82.34711
## 863      82  83.24225
## 864      67  77.00634
## 865      55  79.45941
## 866      87  70.73381
## 867      64  76.64514
## 868      84  80.04985
## 869      82  77.07574
## 870      83  81.67504
## 871     106  74.98745
## 872      85  84.02452
## 873     110  72.21179
## 874     102  77.78425
## 875      94  81.11352
## 876      89  82.50825
## 877      93  76.61177
## 878      80  74.25842
## 879      92  69.83767
## 880      93  78.11892
## 881      93  75.86576
## 882     102  70.01321
## 883      76  82.94723
## 884      78  80.03326
## 885      81  82.57406
## 886      87  87.68125
## 887      83  74.67282
## 888      93  67.15518
## 889     105  76.66567
## 890     100  77.28677
## 891      63  73.35303
## 892      70  85.23277
## 893      61  82.52631
## 894      61  80.73811
## 895      62  74.93185
## 896      77  84.73855
## 897      70  75.75555
## 898      83  81.92132
## 899      77  87.57509
## 900      71  81.64124
## 901      53  78.23981
## 902      89  79.85345
## 903      80  83.07927
## 904      55  77.58223
## 905      82  83.27506
## 906      78  84.80112
## 907      74  77.35284
## 908      88  83.92642
## 909      83  80.88043
## 910      71  79.19680
## 911      78  61.31942
## 912      88  58.38833
## 913      66  66.35593
## 914      90  60.64071
## 915      87  59.60618
## 916      70  71.52196
## 917      70  82.97232
## 918      82  69.80963
## 919      75  73.80638
## 920      85  71.90841
## 921      98  76.54061
## 922      51  81.68224
## 923      76  85.81771
## 924     111  92.23216
## 925      68  84.05760
## 926      58  79.86707
## 927      53  83.63096
## 928      56  72.09129
## 929      74  76.10990
## 930      81  76.51728
## 931      65  79.08606
## 932      68  75.26965
## 933      71  92.94321
## 934      86  85.87283
## 935      87  73.31166
## 936      94  77.15060
## 937      85  70.75125
## 938      74  83.97144
## 939      67  95.72627
## 940      76  77.69374
## 941      65  79.19794
## 942     106  76.35351
## 943      79  78.32832
## 944      93  75.91017
## 945      76  70.76003
## 946      74  86.96236
## 947      88  78.23717
## 948      86  81.18502
## 949      70  75.78016
## 950      69  75.92134
## 951      61  77.63943
## 952      67  97.27400
## 953      77  85.27491
## 954      86  88.53154
## 955      91  93.95050
## 956      91 104.40963
## 957      92  93.08013
## 958      91  94.25311
## 959      85  96.39318
## 960      84 102.13149
## 961      73  93.15151
## 962      67  93.26443
## 963      54  88.38960
## 964      76  83.12927
## 965      67  87.16569
## 966      85  86.20118
## 967      80  84.48924
## 968      79  93.72888
## 969      67 100.22576
## 970      78  90.75092
## 971      74  85.92259
## 972      86  90.08918
## 973     100  88.66595
## 974      83  94.18213
## 975      52  85.07843
## 976      94  87.16259
## 977     112  76.98871
## 978      45  66.83786
## 979      85  71.05327
## 980      96  80.22158
## 981      86  77.96724
## 982      89  74.09699
## 983      83  75.16087
## 984      73  75.85082
## 985      76  78.57326
## 986      95  86.20376
## 987      98  90.85388
## 988      75  82.07620
## 989      98  92.37084
## 990      93  84.76236
## 991      86  86.21000
## 992     104  82.30628
## 993     105  93.02867
## 994      98  92.67147
## 995      88  87.26267
## 996      74  85.61029
## 997     103  81.43227
## 998      73  78.74050
## 999      63  85.55504
## 1000     59  85.67697
## 1001     82  89.49447
## 1002     88  89.33847
## 1003     89  97.19323
## 1004     81  94.41711
## 1005     80  90.90383
## 1006     97  89.17507
## 1007     77  90.21799
## 1008     71  93.50255
## 1009     78  93.87559
## 1010     82  95.81350
## 1011     89  90.93701
## 1012     65  92.44312
## 1013     88  78.34708
## 1014     88  72.71619
## 1015    100  76.46792
## 1016     93  74.59291
## 1017     74  72.01169
## 1018     63  63.68687
## 1019     53  80.15956
## 1020     79  87.11331
## 1021     89  92.71264
## 1022     77  72.71623
## 1023     78  73.56595
## 1024     74  69.61434
## 1025     85  73.36825
## 1026     94  79.86398
## 1027     96  79.38112
## 1028     89  76.32710
## 1029     86  87.36658
## 1030     95  87.46441
## 1031     79  94.51520
## 1032     92  93.38030
## 1033     64  91.96627
## 1034     90  89.23678
## 1035     94  82.81230
## 1036     77  80.88571
## 1037     95  87.73838
## 1038     93  86.40998
## 1039     98  81.97909
## 1040     90  82.52483
## 1041     84  68.20531
## 1042     69  66.22217
## 1043     77  70.54708
## 1044     64  67.31787
## 1045     86  70.66388
## 1046     56  84.08191
## 1047     93  85.42203
## 1048     64  72.46110
## 1049     70  76.83792
## 1050     68  75.06820
## 1051     70  75.85712
## 1052     98  76.59533
## 1053     92  70.17289
## 1054     89  68.75619
## 1055    108  73.16580
## 1056     89  82.69221
## 1057     61  78.45319
## 1058     73  74.46707
## 1059     95  84.53133
## 1060     66  72.54941
## 1061     78  85.11678
## 1062     69  83.35557
## 1063     80  82.59932
## 1064     91  83.53339
## 1065     75  86.85838
## 1066     93  84.46278
## 1067     85  88.67685
## 1068     82  83.37239
## 1069     94  88.58651
## 1070     97  89.59870
## 1071     98  84.64656
## 1072     97  81.64766
## 1073     79  92.61935
## 1074     81  86.27243
## 1075     75  82.85143
## 1076     62  85.66203
## 1077     75  95.84800
## 1078     84  90.95834
## 1079     88  95.81290
## 1080     82  98.69009
## 1081     74  85.71811
## 1082     68  69.76796
## 1083     67  75.60889
## 1084     66  71.34770
## 1085     95  72.41132
## 1086     83  72.15901
## 1087     71  71.27513
## 1088     95  84.49067
## 1089     82  82.25327
## 1090     97  81.79879
## 1091     77  85.53387
## 1092     53  80.98372
## 1093    101  73.90174
## 1094     88  73.92732
## 1095    103  71.53401
## 1096     79  74.23172
## 1097     58  81.61689
## 1098     74  82.79164
## 1099     86  77.45100
## 1100     83  76.58674
## 1101     87  78.73276
## 1102     88  96.77330
## 1103     84  87.60659
## 1104     75  76.93790
## 1105     68  87.38424
## 1106     65  72.88084
## 1107     55  80.21365
## 1108     54  79.76677
## 1109     66  66.28857
## 1110     65  69.89638
## 1111     72  67.14770
## 1112     69  70.31318
## 1113     72  63.48091
## 1114     79  71.00934
## 1115     89  70.69567
## 1116     82  70.69689
## 1117     81  73.50918
## 1118     64  78.85426
## 1119     80  83.60694
## 1120     76  83.10800
## 1121     86  80.02899
## 1122     65  71.49187
## 1123     85  66.74572
## 1124     72  73.57565
## 1125     80  80.18532
## 1126     76  72.35786
## 1127     83  72.86273
## 1128     92  71.97395
## 1129     79  65.32023
## 1130     65  73.37091
## 1131     58  70.95516
## 1132     69  67.70542
## 1133     61  71.73850
## 1134     79  76.98511
## 1135     75  84.88781
## 1136     80  94.03477
## 1137     97  83.22815
## 1138     82  84.15055
## 1139     69  89.26872
## 1140     66  79.14093
## 1141    105  84.17269
## 1142     86  84.81615
## 1143     92  72.06287
## 1144     99  77.75736
## 1145     88  78.77255
## 1146    102  72.63020
## 1147    102  78.03885
## 1148    110  90.17251
## 1149    104  78.48975
## 1150     75  79.38583
## 1151     80  77.11963
## 1152     95  90.17322
## 1153     85  81.64387
## 1154     95  83.99443
## 1155     78  85.64188
## 1156     90  82.44665
## 1157     86  95.51429
## 1158     71  77.02829
## 1159     66  89.46990
## 1160     93  94.55289
## 1161     86  95.36864
## 1162     71  91.57210
## 1163     78  80.87037
## 1164     68  82.79777
## 1165     81  75.87495
## 1166     73  80.97836
## 1167     45  84.59811
## 1168     78  81.97436
## 1169     91  94.32579
## 1170     89  85.66276
## 1171     77  74.16725
## 1172     69  77.35892
## 1173     89  71.56327
## 1174     92  89.51002
## 1175     67  83.62805
## 1176     56  96.01226
## 1177     81  99.63880
## 1178     77  86.66543
## 1179     61  85.08412
## 1180     70  88.86202
## 1181     85  96.16164
## 1182     91  94.35191
## 1183     74  77.78130
## 1184     95  79.97524
## 1185     78  76.64023
## 1186     70  81.35226
## 1187     90  82.58319
## 1188     41  83.45274
## 1189     61  80.12935
## 1190    100  82.74097
## 1191     86  78.38666
## 1192     98  85.76137
## 1193     92  80.03589
## 1194    101  75.26520
## 1195     87  79.94641
## 1196     91  86.44159
## 1197     72  84.42915
## 1198     78  76.77597
## 1199     71  85.49227
## 1200     88  85.43649
## 1201     99  82.01907
## 1202     97  85.49127
## 1203     54  76.48413
## 1204     79  72.47560
## 1205     96  74.71764
## 1206     99  79.17754
## 1207    114  75.89540
## 1208     92  72.52410
## 1209     97  72.99217
## 1210     96  73.95031
## 1211     93  80.28342
## 1212     80  76.84542
## 1213     99  72.66814
## 1214     79  79.43934
## 1215     71  84.21485
## 1216     88  76.13103
## 1217     88  75.85239
## 1218     97  80.34523
## 1219     58  81.40989
## 1220     66  89.32086
## 1221     56  71.65314
## 1222     67  83.47910
## 1223     76  79.31333
## 1224     88  83.99140
## 1225     62  86.04019
## 1226     59  80.58368
## 1227     89  78.55305
## 1228    102  81.19871
## 1229     54  80.75316
## 1230     77  83.34638
## 1231     96  72.71053
## 1232     65  83.05962
## 1233     74  75.30919
## 1234     91  61.84184
## 1235     92  75.49493
## 1236     85  66.04304
## 1237    100  67.75688
## 1238     96  69.12003
## 1239     66  71.72482
## 1240     87  69.96031
## 1241     84  70.90493
## 1242     94  82.19781
## 1243     54  78.83265
## 1244     75  79.97341
## 1245     45  73.89507
## 1246     67  74.58934
## 1247     65  77.85825
## 1248     92  80.11585
## 1249     92  72.59577
## 1250     67  73.24053
## 1251     61  74.94112
## 1252    101  80.93204
## 1253     90  82.70670
## 1254     78  83.81551
## 1255     77  81.01255
## 1256     65  83.63764
## 1257     79  83.91016
## 1258    105  75.41719
## 1259    102  77.28019
## 1260    117  85.05006
## 1261     91  76.05888
## 1262     85  78.48064
## 1263     68  86.08551
## 1264     54  75.94236
## 1265     89  79.40133
## 1266     99  84.21428
## 1267     88  76.93622
## 1268     90  76.13965
## 1269     93  80.08179
## 1270     72  80.51164
## 1271     73  78.57486
## 1272     87  82.35286
## 1273     53  91.28096
## 1274     74  96.60549
## 1275     88  94.87582
## 1276     74  92.31089
## 1277     73  94.05118
## 1278     78  94.20321
## 1279     63  96.22611
## 1280     61  87.91910
## 1281     92  83.99419
## 1282     84  78.77420
## 1283     89  75.09218
## 1284     79  82.55089
## 1285     64  71.98056
## 1286     57  69.20131
## 1287     77  77.96775
## 1288     64  87.78023
## 1289     88  86.35932
## 1290     76  82.21422
## 1291    116  79.65022
## 1292     93  75.35074
## 1293     78  81.20123
## 1294     98  90.70037
## 1295     79  84.24705
## 1296     90  78.67993
## 1297     93  87.97745
## 1298     76  87.69785
## 1299     97  92.90616
## 1300     98  96.57243
## 1301     77  85.69885
## 1302     64  78.76782
## 1303     77  73.07410
## 1304     87  77.54960
## 1305     83  83.26963
## 1306     88  78.98654
## 1307     91  79.07793
## 1308     88  80.37722
## 1309     62  79.71387
## 1310    103  78.50491
## 1311     75  76.75080
## 1312     86  80.27969
## 1313     97  73.46866
## 1314     90  79.25616
## 1315     79  82.00955
## 1316     55  79.89839
## 1317     67  91.02800
## 1318     82  88.08337
## 1319     67  88.85369
## 1320     55  94.45356
## 1321     87  84.35462
## 1322     64  78.05740
## 1323     84  83.95127
## 1324     82  86.33623
## 1325     83  82.90238
## 1326    106  81.67246
## 1327     85  85.51611
## 1328    110  80.18255
## 1329    102  79.54560
## 1330     94  75.54402
## 1331     89  73.65885
## 1332     93  71.01422
## 1333     80  81.45969
## 1334     92  82.86082
## 1335     93  80.62995
## 1336     93  88.25696
## 1337    102  76.46766
## 1338     76  80.34835
## 1339     78  72.91620
## 1340     81  76.90302
## 1341     87  83.18826
## 1342     83  70.94189
## 1343     93  76.87526
## 1344    105  80.45797
## 1345    100  69.39293
## 1346     63  84.77448
## 1347     70  85.62956
## 1348     61  83.98628
## 1349     61  80.12358
## 1350     62  86.59676
## 1351     77  87.14948
## 1352     70  89.34894
## 1353     83  96.16870
## 1354     77  89.87599
## 1355     71  82.54057
## 1356     53  72.35155
## 1357     89  75.48538
## 1358     80  89.39629
## 1359     55  90.79489
## 1360     82  72.17537
## 1361     78  98.82305
## 1362     74  91.07166
## 1363     88  74.70489
## 1364     83  74.08408
## 1365     71  75.45591
## 1366     78  75.76548
## 1367     88  67.74708
## 1368     66  63.73702
## 1369     90  63.09795
## 1370     87  79.36250
## 1371     70  85.89266
## 1372     70  88.93737
## 1373     82  86.42517
## 1374     75  79.24316
## 1375     85  80.67956
## 1376     98  74.32748
## 1377     51  77.51726
## 1378     76  74.68399
## 1379    111  90.58364
## 1380     68  84.36898
## 1381     58  89.61974
## 1382     53  87.27046
## 1383     56  91.72583
## 1384     74  90.46183
## 1385     81  92.84914
## 1386     65  83.81388
## 1387     68  78.44059
## 1388     71  78.52910
## 1389     86  84.60661
## 1390     87  80.35381
## 1391     94  81.01220
## 1392     85  81.58834
## 1393     74  83.27561
## 1394     67  83.42052
## 1395     76  88.09058
## 1396     65  80.71739
## 1397    106  82.57078
## 1398     79  79.27823
## 1399     93  79.45690
## 1400     76  87.45363
## 1401     74  82.67362
## 1402     88  83.26842
## 1403     86  79.96190
## 1404     70  83.17930
## 1405     69  81.28235
## 1406     61  88.91157
## 1407     67  79.89535
## 1408     77  82.59148
## 1409     86  83.88969
## 1410     91  79.11848
## 1411     91  80.02279
## 1412     92  79.38053
## 1413     91  79.62617
## 1414     85  81.86965
## 1415     84  81.52487
## 1416     73  75.59362
## 1417     67  76.78916
## 1418     54  74.30279
## 1419     76  76.30224
## 1420     67  69.65634
## 1421     85  74.26352
## 1422     80  83.82491
## 1423     79  84.75257
## 1424     67  73.73032
## 1425     78  75.06767
## 1426     74  84.63083
## 1427     86  78.97639
## 1428    100  76.45999
## 1429     83  71.29329
## 1430     52  81.13600
## 1431     94  80.70558
## 1432    112  78.14821
## 1433     45  71.04719
## 1434     85  80.36170
## 1435     96  81.90376
## 1436     86  93.33645
## 1437     89  88.19413
## 1438     83  90.25974
## 1439     73  84.66770
## 1440     76  82.96837
## 1441     95  89.12204
## 1442     98  87.95702
## 1443     75  82.57955
## 1444     98  82.70475
## 1445     93  79.25230
## 1446     86  80.20210
## 1447    104  71.72168
## 1448    105  74.42225
## 1449     98  79.63366
## 1450     88  81.94346
## 1451     74  82.75808
## 1452    103  80.78277
## 1453     73  76.62972
## 1454     63  77.85689
## 1455     59  68.36665
## 1456     82  69.11303
## 1457     88  66.24626
## 1458     89  81.84655
## 1459     81  78.35787
## 1460     80  69.94233
## 1461     97  75.76441
## 1462     77  81.02432
## 1463     71  74.52049
## 1464     78  81.01784
## 1465     82  71.46302
## 1466     89  75.61764
## 1467     65  78.34152
## 1468     88  77.83881
## 1469     88  77.22750
## 1470    100  86.79195
## 1471     93  77.71251
## 1472     74  82.62507
## 1473     63  81.35461
## 1474     53  89.93796
## 1475     79  96.51608
## 1476     89  86.16285
## 1477     77  91.33961
## 1478     78  86.93554
## 1479     74  84.63384
## 1480     85  72.70103
## 1481     94  71.08248
## 1482     96  91.35231
## 1483     89  86.00672
## 1484     86  91.94352
## 1485     95  85.43193
## 1486     79  83.53370
## 1487     92  85.50160
## 1488     64  80.42214
## 1489     90  87.39467
## 1490     94  93.21013
## 1491     77  83.54572
## 1492     95  77.31713
## 1493     93  89.01612
## 1494     98  89.92811
## 1495     90  76.09057
## 1496     84  77.41702
## 1497     69  85.67225
## 1498     77  87.78315
## 1499     64  71.42951
## 1500     86  78.50359
## 1501     56  70.81870
## 1502     93  80.91072
## 1503     64  75.32227
## 1504     70  68.16678
## 1505     68  79.64992
## 1506     70  71.17387
## 1507     98  79.90700
## 1508     92  84.00078
## 1509     89  82.16401
## 1510    108  77.16445
## 1511     89  76.00898
## 1512     61  78.13104
## 1513     73  72.61470
## 1514     95  77.78654
## 1515     66  77.61568
## 1516     78  72.10171
## 1517     69  81.84125
## 1518     80  77.74045
## 1519     91  78.32464
## 1520     75  67.84336
## 1521     93  80.92785

mean (apply(predicted_values, 1, min)/apply(predicted_values, 1, max)) # calculate accuracy

## [1] 0.859728

The prediction accuracy here is at 85.85%

predicted <- predict(Model5, newx = test_baseball)# predict on test data
predicted_values <- cbind (actual=test_baseball$TARGET_WINS, predicted)  # combine

## Warning in cbind(actual = test_baseball$TARGET_WINS, predicted): number of
## rows of result is not a multiple of vector length (arg 1)

predicted_values

##      actual predicted
## 1        78  67.44342
## 2        88  67.45113
## 3        66  70.07451
## 4        90  75.43397
## 5        87  67.72684
## 6        70  67.72587
## 7        70  63.76240
## 8        82  77.57821
## 9        75  87.50168
## 10       85  75.33778
## 11       98  86.57140
## 12       51  76.10532
## 13       76  79.30210
## 14      111  85.05514
## 15       68  89.33395
## 16       58  86.35917
## 17       53  75.87327
## 18       56  75.27661
## 19       74  76.92580
## 20       81  72.38257
## 21       65  91.24888
## 22       68  81.21417
## 23       71  76.85482
## 24       86  75.73465
## 25       87  92.95906
## 26       94  79.74796
## 27       85  82.59460
## 28       74  82.93132
## 29       67  84.01025
## 30       76  83.67314
## 31       65  74.98318
## 32      106  94.40116
## 33       79  85.35736
## 34       93  86.31272
## 35       76  85.20792
## 36       74  79.86968
## 37       88  71.39335
## 38       86  79.65920
## 39       70  88.48478
## 40       69  92.89590
## 41       61  95.70531
## 42       67  80.15908
## 43       77  77.64883
## 44       86  67.18263
## 45       91  64.94270
## 46       91  62.58915
## 47       92  71.88851
## 48       91  67.42428
## 49       85  66.89260
## 50       84  74.11101
## 51       73  84.19595
## 52       67  77.19389
## 53       54  81.04087
## 54       76  78.77630
## 55       67  78.14143
## 56       85  76.04199
## 57       80  79.46042
## 58       79  73.69400
## 59       67  74.54247
## 60       78  75.74929
## 61       74  74.97222
## 62       86  75.44410
## 63      100  79.02010
## 64       83  69.75831
## 65       52  70.64981
## 66       94  69.12150
## 67      112  74.71038
## 68       45  69.62466
## 69       85  70.17964
## 70       96  70.25523
## 71       86  69.23863
## 72       89  71.09084
## 73       83  72.67250
## 74       73  67.57287
## 75       76  73.96325
## 76       95  82.29090
## 77       98  76.84995
## 78       75  80.96384
## 79       98  86.00466
## 80       93  80.99199
## 81       86  90.30120
## 82      104  86.68144
## 83      105  76.35323
## 84       98  83.38313
## 85       88  86.70403
## 86       74  88.15920
## 87      103  77.29176
## 88       73  78.74435
## 89       63  78.89828
## 90       59  84.86922
## 91       82  76.52274
## 92       88  68.53117
## 93       89  80.63921
## 94       81  79.04574
## 95       80  91.06636
## 96       97  78.71918
## 97       77  73.94550
## 98       71  72.47404
## 99       78  76.97092
## 100      82  78.30645
## 101      89  77.84933
## 102      65  73.92133
## 103      88  87.30513
## 104      88  76.38672
## 105     100  87.67499
## 106      93  77.46469
## 107      74  82.05258
## 108      63  85.24247
## 109      53  87.16002
## 110      79  83.45556
## 111      89  83.35626
## 112      77  86.73727
## 113      78  91.21914
## 114      74  88.04261
## 115      85  78.51105
## 116      94  81.95540
## 117      96  82.76242
## 118      89  81.70924
## 119      86  85.27418
## 120      95  69.20964
## 121      79  69.37010
## 122      92  78.11881
## 123      64  69.72151
## 124      90  66.82142
## 125      94  65.28600
## 126      77  71.84567
## 127      95  71.77897
## 128      93  76.57889
## 129      98  79.01450
## 130      90  85.26364
## 131      84  75.40463
## 132      69  82.54799
## 133      77  78.50081
## 134      64  82.48798
## 135      86  84.16834
## 136      56  81.26085
## 137      93  80.88645
## 138      64  75.53196
## 139      70  80.44320
## 140      68  84.92377
## 141      70  75.02694
## 142      98  75.21066
## 143      92  71.99137
## 144      89  71.21597
## 145     108  81.47611
## 146      89  80.90290
## 147      61  80.66912
## 148      73  82.13911
## 149      95  74.68290
## 150      66  73.79684
## 151      78  69.14518
## 152      69  69.07586
## 153      80  74.64032
## 154      91  75.73128
## 155      75  71.89865
## 156      93  78.09027
## 157      85  76.71291
## 158      82  77.55373
## 159      94  77.24322
## 160      97  75.43189
## 161      98  81.07324
## 162      97  75.80984
## 163      79  75.01335
## 164      81  87.32782
## 165      75  87.38260
## 166      62  85.36363
## 167      75  72.33313
## 168      84  88.77234
## 169      88  80.72687
## 170      82  81.51611
## 171      74  79.82869
## 172      68  79.48619
## 173      67  79.26894
## 174      66  88.37324
## 175      95  86.43100
## 176      83  85.22346
## 177      71  83.65356
## 178      95  80.09002
## 179      82  88.30086
## 180      97  79.69794
## 181      77  84.31400
## 182      53  74.81675
## 183     101  80.47708
## 184      88  79.70448
## 185     103  78.06843
## 186      79  85.14572
## 187      58  82.05263
## 188      74  94.71854
## 189      86  96.65872
## 190      83  87.64550
## 191      87  90.21192
## 192      88  87.23107
## 193      84  78.75851
## 194      75  79.00789
## 195      68  81.25164
## 196      65  81.88150
## 197      55  84.03268
## 198      54  82.40478
## 199      66  95.78563
## 200      65  80.30579
## 201      72  75.01089
## 202      69  95.34271
## 203      72  87.50287
## 204      79  87.19443
## 205      89  76.17791
## 206      82  76.06067
## 207      81  64.91472
## 208      64  65.72480
## 209      80  72.39191
## 210      76  77.65363
## 211      86  84.15721
## 212      65  82.04471
## 213      85  86.08161
## 214      72  87.63339
## 215      80  75.05263
## 216      76  74.98438
## 217      83  83.25936
## 218      92  82.71286
## 219      79  75.53272
## 220      65  76.88389
## 221      58  71.86619
## 222      69  76.06333
## 223      61  74.51141
## 224      79  70.23775
## 225      75  73.52272
## 226      80  79.16581
## 227      97  86.89115
## 228      82  87.13819
## 229      69  85.46662
## 230      66  89.75357
## 231     105  91.54361
## 232      86  90.59831
## 233      92  95.02468
## 234      99  75.40648
## 235      88  79.71284
## 236     102  75.96297
## 237     102  85.91518
## 238     110  78.85678
## 239     104  76.84806
## 240      75  86.14862
## 241      80  78.05120
## 242      95  77.47512
## 243      85  78.23876
## 244      95  73.50749
## 245      78  86.78095
## 246      90  79.08171
## 247      86  84.72774
## 248      71  78.34657
## 249      66  82.76116
## 250      93  86.34586
## 251      86  87.92397
## 252      71  84.11055
## 253      78  77.99811
## 254      68  85.45186
## 255      81  77.80467
## 256      73  75.25132
## 257      45  76.95882
## 258      78  68.38314
## 259      91  73.50657
## 260      89  94.02116
## 261      77  93.71058
## 262      69  89.96523
## 263      89  80.98936
## 264      92  84.84977
## 265      67  85.19842
## 266      56  79.47680
## 267      81  79.10396
## 268      77  95.64288
## 269      61  86.71087
## 270      70  85.77351
## 271      85  88.98923
## 272      91  80.79937
## 273      74  77.98964
## 274      95  75.28088
## 275      78  77.35333
## 276      70  76.26755
## 277      90  75.33738
## 278      41  88.17904
## 279      61  96.27179
## 280     100  89.92458
## 281      86  92.67221
## 282      98  84.05349
## 283      92  74.83770
## 284     101  85.95691
## 285      87  78.72426
## 286      91  78.71273
## 287      72  81.26346
## 288      78  85.39316
## 289      71  94.85380
## 290      88  98.58835
## 291      99  82.07467
## 292      97  75.18221
## 293      54  76.37619
## 294      79  82.90378
## 295      96  88.14256
## 296      99  81.52590
## 297     114  82.87604
## 298      92  78.59107
## 299      97  75.43887
## 300      96  75.87870
## 301      93  77.95901
## 302      80  77.42188
## 303      99  79.91021
## 304      79  75.70473
## 305      71  72.30603
## 306      88  74.72546
## 307      88  77.04007
## 308      97  84.57445
## 309      58  79.03250
## 310      66  75.61020
## 311      56  85.12809
## 312      67  83.01153
## 313      76  81.21832
## 314      88  77.58116
## 315      62  76.16731
## 316      59  71.52026
## 317      89  75.22951
## 318     102  75.48431
## 319      54  76.87937
## 320      77  75.93558
## 321      96  75.95209
## 322      65  82.46585
## 323      74  74.48463
## 324      91  77.49127
## 325      92  75.66515
## 326      85  77.13311
## 327     100  79.35974
## 328      96  78.58116
## 329      66  85.79704
## 330      87  87.45633
## 331      84  82.37835
## 332      94  86.72390
## 333      54  75.70592
## 334      75  80.05905
## 335      45  80.56484
## 336      67  81.56194
## 337      65  79.82098
## 338      92  87.10452
## 339      92  79.65431
## 340      67  76.07465
## 341      61  83.48986
## 342     101  80.22453
## 343      90  82.98359
## 344      78  82.75474
## 345      77  75.27901
## 346      65  79.49034
## 347      79  82.43166
## 348     105  82.29389
## 349     102  84.23164
## 350     117  76.59806
## 351      91  74.96025
## 352      85  76.25467
## 353      68  68.62133
## 354      54  73.26311
## 355      89  75.04697
## 356      99  81.57823
## 357      88  76.97103
## 358      90  84.95317
## 359      93  88.65480
## 360      72  81.29242
## 361      73  80.11348
## 362      87  81.05779
## 363      53  78.21721
## 364      74  78.12708
## 365      88  77.36967
## 366      74  77.29128
## 367      73  81.79287
## 368      78  87.54212
## 369      63  85.17605
## 370      61  81.07061
## 371      92  85.04887
## 372      84  78.68070
## 373      89  77.38031
## 374      79  70.92452
## 375      64  74.03955
## 376      57  71.75593
## 377      77  75.89851
## 378      64  76.24042
## 379      88  78.59571
## 380      76  82.86759
## 381     116  86.95993
## 382      93  84.03568
## 383      78  87.55968
## 384      98  85.98648
## 385      79  82.14632
## 386      90  85.69631
## 387      93  87.07823
## 388      76  86.90617
## 389      97  81.14569
## 390      98  74.98691
## 391      77  74.22010
## 392      64  66.23871
## 393      77  70.80897
## 394      87  79.02572
## 395      83  74.46659
## 396      88  76.17838
## 397      91  78.85746
## 398      88  75.16575
## 399      62  90.13430
## 400     103  76.43702
## 401      75  80.42516
## 402      86  71.76322
## 403      97  85.62887
## 404      90  88.54009
## 405      79  85.41391
## 406      55  80.87090
## 407      67  76.33751
## 408      82  85.13585
## 409      67  76.30060
## 410      55  72.82338
## 411      87  85.53497
## 412      64  88.16092
## 413      84  94.67713
## 414      82  92.65560
## 415      83  90.99936
## 416     106  85.43587
## 417      85  92.80923
## 418     110  89.13903
## 419     102  87.66494
## 420      94  86.26550
## 421      89  87.30838
## 422      93  95.64263
## 423      80  89.77173
## 424      92  96.58498
## 425      93  89.08766
## 426      93  84.87436
## 427     102  79.09352
## 428      76  83.13129
## 429      78  74.78158
## 430      81  75.01630
## 431      87  75.37997
## 432      83  91.42830
## 433      93  89.53917
## 434     105  80.57424
## 435     100  77.44170
## 436      63  80.56396
## 437      70  86.13667
## 438      61  86.17377
## 439      61  76.58950
## 440      62  76.80901
## 441      77  83.22035
## 442      70  78.71702
## 443      83  68.39029
## 444      77  74.51458
## 445      71  65.97702
## 446      53  69.94125
## 447      89  78.07337
## 448      80  79.56762
## 449      55  75.47085
## 450      82  78.95959
## 451      78  82.04844
## 452      74  78.93698
## 453      88  75.55269
## 454      83  73.11614
## 455      71  71.16156
## 456      78  68.87201
## 457      88  74.10248
## 458      66  80.17785
## 459      90  77.30115
## 460      87  72.49663
## 461      70  77.63991
## 462      70  72.55387
## 463      82  75.26255
## 464      75  89.56768
## 465      85  89.34414
## 466      98  81.88434
## 467      51  84.87743
## 468      76  81.65693
## 469     111  82.50675
## 470      68  81.01440
## 471      58  75.36897
## 472      53  74.83014
## 473      56  86.95051
## 474      74  75.64894
## 475      81  71.91194
## 476      65  88.80437
## 477      68  72.45152
## 478      71  85.64352
## 479      86  87.18982
## 480      87  88.23807
## 481      94  97.61437
## 482      85  94.15000
## 483      74  85.46822
## 484      67  81.29913
## 485      76  83.24641
## 486      65  73.97669
## 487     106  78.98370
## 488      79  87.86617
## 489      93  90.77691
## 490      76  82.77272
## 491      74  81.84398
## 492      88  78.91267
## 493      86  78.56951
## 494      70  90.81276
## 495      69  82.15841
## 496      61  77.31243
## 497      67  91.95794
## 498      77  87.79896
## 499      86  76.99782
## 500      91  79.89077
## 501      91  74.67528
## 502      92  86.66090
## 503      91  76.67202
## 504      85  84.36278
## 505      84  82.37365
## 506      73  77.49743
## 507      67  72.37151
## 508      54  75.22012
## 509      76  73.18528
## 510      67  71.40753
## 511      85  78.79972
## 512      80  77.39860
## 513      79  81.06421
## 514      67  78.92919
## 515      78  83.29334
## 516      74  92.13062
## 517      86  86.88210
## 518     100  86.09691
## 519      83  83.65993
## 520      52  80.00122
## 521      94  75.79357
## 522     112  71.29807
## 523      45  80.14395
## 524      85  85.04878
## 525      96  86.42506
## 526      86  77.88073
## 527      89  89.99265
## 528      83  93.12574
## 529      73  86.35583
## 530      76  82.66900
## 531      95  86.55700
## 532      98  76.00762
## 533      75  75.10227
## 534      98  76.24862
## 535      93  93.95561
## 536      86  86.80163
## 537     104  94.03234
## 538     105  86.58101
## 539      98  86.56917
## 540      88  88.22369
## 541      74  85.39937
## 542     103  81.25140
## 543      73  82.13859
## 544      63  80.30914
## 545      59  74.44826
## 546      82  76.10264
## 547      88  74.08154
## 548      89  69.92862
## 549      81  69.95132
## 550      80  74.53685
## 551      97  77.21791
## 552      77  68.76742
## 553      71  67.21485
## 554      78  78.58185
## 555      82  74.37172
## 556      89  77.15918
## 557      65  83.53078
## 558      88  84.49880
## 559      88  86.41669
## 560     100  75.22195
## 561      93  73.30565
## 562      74  76.43071
## 563      63  67.01121
## 564      53  78.15236
## 565      79  84.22717
## 566      89 103.62586
## 567      77  99.78573
## 568      78  92.94880
## 569      74  96.14162
## 570      85  93.83560
## 571      94  87.21972
## 572      96  80.56718
## 573      89  75.47100
## 574      86  85.48369
## 575      95  81.64591
## 576      79  80.88031
## 577      92  72.00168
## 578      64  98.77644
## 579      90  95.99900
## 580      94  95.74313
## 581      77  81.34501
## 582      95  90.01800
## 583      93  90.45721
## 584      98  93.97424
## 585      90  79.58646
## 586      84  83.30092
## 587      69  80.62590
## 588      77  74.64967
## 589      64  81.50964
## 590      86  77.75612
## 591      56  68.67546
## 592      93  67.09276
## 593      64  74.64771
## 594      70  74.22550
## 595      68  91.69689
## 596      70  88.52477
## 597      98  83.85911
## 598      92  85.75061
## 599      89  84.90506
## 600     108  78.19194
## 601      89  86.69231
## 602      61  84.65045
## 603      73  90.56613
## 604      95  88.21833
## 605      66  89.20599
## 606      78  96.33209
## 607      69  89.50356
## 608      80  96.00533
## 609      91  94.62374
## 610      75  93.39055
## 611      93  95.00998
## 612      85  80.83454
## 613      82  75.38982
## 614      94  76.75518
## 615      97  78.99632
## 616      98  84.62905
## 617      97  82.87983
## 618      79  89.69512
## 619      81  80.89995
## 620      75  77.82870
## 621      62  76.45063
## 622      75  78.45039
## 623      84  76.99874
## 624      88  87.95743
## 625      82  92.81547
## 626      74  80.81087
## 627      68  78.03260
## 628      67  81.21524
## 629      66  79.27102
## 630      95  80.71248
## 631      83  81.70592
## 632      71  78.63917
## 633      95  75.29746
## 634      82  72.33944
## 635      97  76.57523
## 636      77  81.96928
## 637      53  82.33040
## 638     101  85.74432
## 639      88  85.14898
## 640     103  84.98782
## 641      79  90.15115
## 642      58  87.60581
## 643      74  92.92149
## 644      86  95.31093
## 645      83  81.02289
## 646      87  76.88477
## 647      88  84.77585
## 648      84  88.93106
## 649      75  82.10155
## 650      68  89.52683
## 651      65  86.02564
## 652      55  79.37780
## 653      54  84.48734
## 654      66  82.93491
## 655      65  68.22435
## 656      72  75.99833
## 657      69  85.70357
## 658      72  79.57618
## 659      79  70.89438
## 660      89  89.49693
## 661      82  77.55810
## 662      81  81.40149
## 663      64  73.90587
## 664      80  73.55784
## 665      76  83.10389
## 666      86  78.39615
## 667      65  77.05779
## 668      85  80.72315
## 669      72  73.46132
## 670      80  61.90398
## 671      76  62.28685
## 672      83  73.70169
## 673      92  74.54397
## 674      79  73.84600
## 675      65  61.25132
## 676      58  79.47124
## 677      69  83.26075
## 678      61  71.60880
## 679      79  83.55334
## 680      75  75.68196
## 681      80  76.36565
## 682      97  80.20358
## 683      82  79.05666
## 684      69  77.81118
## 685      66  83.28782
## 686     105  74.22485
## 687      86  76.82253
## 688      92  77.12806
## 689      99  80.63736
## 690      88  77.08373
## 691     102  71.73085
## 692     102  74.57166
## 693     110  79.22417
## 694     104  86.49261
## 695      75  82.07615
## 696      80  86.23495
## 697      95  96.73694
## 698      85  83.63311
## 699      95  82.47753
## 700      78  79.80981
## 701      90  80.53284
## 702      86  81.83612
## 703      71  76.42994
## 704      66  73.20961
## 705      93  75.03780
## 706      86  74.59428
## 707      71  84.51842
## 708      78  84.17397
## 709      68  85.30279
## 710      81  78.78881
## 711      73  84.07513
## 712      45  79.38092
## 713      78  80.61640
## 714      91  78.04178
## 715      89  84.50836
## 716      77  83.54679
## 717      69  80.08120
## 718      89  82.91023
## 719      92  78.49132
## 720      67  83.35613
## 721      56  73.32258
## 722      81  71.67373
## 723      77  98.19085
## 724      61  98.01751
## 725      70  78.46787
## 726      85  89.91365
## 727      91  75.16995
## 728      74  75.50304
## 729      95  67.49380
## 730      78  79.75337
## 731      70  83.97999
## 732      90  85.34265
## 733      41  82.35342
## 734      61  80.79035
## 735     100  75.78706
## 736      86  74.82875
## 737      98  75.08221
## 738      92  82.44515
## 739     101  78.18402
## 740      87  76.62192
## 741      91  84.27091
## 742      72  78.70232
## 743      78  80.78674
## 744      71  76.73710
## 745      88  76.18862
## 746      99  68.60446
## 747      97  71.96965
## 748      54  84.90424
## 749      79  80.17055
## 750      96  83.99648
## 751      99  86.59310
## 752     114  75.47677
## 753      92  75.53225
## 754      97  86.19133
## 755      96  85.54998
## 756      93  89.02909
## 757      80  86.18981
## 758      99  97.91304
## 759      79  98.63173
## 760      71  92.47344
## 761      88 104.48665
## 762      88  97.33576
## 763      97 100.85301
## 764      58  92.93128
## 765      66  82.82575
## 766      56  87.45635
## 767      67  85.89462
## 768      76  82.10294
## 769      88  89.28083
## 770      62  92.29101
## 771      59  72.82231
## 772      89  72.27080
## 773     102  67.86196
## 774      54  62.46698
## 775      77  74.97552
## 776      96  82.33372
## 777      65  73.87224
## 778      74  76.05064
## 779      91  77.17043
## 780      92  82.98197
## 781      85  77.89406
## 782     100  88.99333
## 783      96  86.81349
## 784      66  86.70833
## 785      87  85.03963
## 786      84  73.59994
## 787      94  81.58437
## 788      54  75.50149
## 789      75  72.12338
## 790      45  68.86495
## 791      67  80.02704
## 792      65  77.39364
## 793      92  78.07165
## 794      92  77.63262
## 795      67  82.97429
## 796      61  79.14137
## 797     101  89.78282
## 798      90  89.27390
## 799      78  83.86970
## 800      77  77.30894
## 801      65  69.58990
## 802      79  72.98084
## 803     105  88.69279
## 804     102  87.92394
## 805     117  91.70302
## 806      91  82.67101
## 807      85  71.85679
## 808      68  68.10150
## 809      54  81.84717
## 810      89  78.12253
## 811      99  69.83695
## 812      88  76.46566
## 813      90  86.11409
## 814      93  90.88470
## 815      72  91.80391
## 816      73  92.11302
## 817      87  84.80266
## 818      53  74.65166
## 819      74  77.57825
## 820      88  76.78635
## 821      74  82.36152
## 822      73  82.50355
## 823      78  88.04743
## 824      63  87.72790
## 825      61  76.63681
## 826      92  75.42680
## 827      84  82.96084
## 828      89  78.21579
## 829      79  76.27439
## 830      64  73.79846
## 831      57  81.10167
## 832      77  75.42576
## 833      64  76.49613
## 834      88  74.59911
## 835      76  89.20900
## 836     116  87.22479
## 837      93  75.04804
## 838      78  71.78931
## 839      98  78.02374
## 840      79  76.61472
## 841      90  63.46132
## 842      93  73.67504
## 843      76  77.48161
## 844      97  80.45738
## 845      98  78.22131
## 846      77  80.05917
## 847      64  80.85200
## 848      77  80.57371
## 849      87  86.57068
## 850      83  85.80458
## 851      88  83.86479
## 852      91  83.11116
## 853      88  86.62546
## 854      62  86.96736
## 855     103  81.63054
## 856      75  84.51782
## 857      86  83.66754
## 858      97  90.72007
## 859      90  84.56464
## 860      79  79.70922
## 861      55  80.94532
## 862      67  79.55306
## 863      82  83.57741
## 864      67  76.30040
## 865      55  79.15809
## 866      87  72.63969
## 867      64  77.97984
## 868      84  80.06715
## 869      82  75.38970
## 870      83  80.66269
## 871     106  75.27283
## 872      85  83.24337
## 873     110  74.94358
## 874     102  81.51085
## 875      94  83.24421
## 876      89  84.04611
## 877      93  77.97604
## 878      80  75.08056
## 879      92  72.00019
## 880      93  78.40749
## 881      93  74.36827
## 882     102  69.61521
## 883      76  78.99502
## 884      78  76.21166
## 885      81  79.95868
## 886      87  85.97882
## 887      83  74.10177
## 888      93  74.43285
## 889     105  76.14842
## 890     100  76.55578
## 891      63  71.64883
## 892      70  88.44513
## 893      61  82.81897
## 894      61  79.19356
## 895      62  74.99922
## 896      77  84.37986
## 897      70  74.92339
## 898      83  81.98816
## 899      77  83.05794
## 900      71  79.14053
## 901      53  76.41732
## 902      89  77.00170
## 903      80  83.40517
## 904      55  77.80671
## 905      82  80.72405
## 906      78  84.92448
## 907      74  76.37162
## 908      88  80.14464
## 909      83  81.99346
## 910      71  79.12052
## 911      78  63.72078
## 912      88  60.35400
## 913      66  66.78161
## 914      90  60.68783
## 915      87  59.18272
## 916      70  70.53416
## 917      70  81.46170
## 918      82  71.45605
## 919      75  74.57813
## 920      85  71.72674
## 921      98  75.34780
## 922      51  81.52163
## 923      76  83.94570
## 924     111  91.68011
## 925      68  85.36356
## 926      58  81.99874
## 927      53  84.14420
## 928      56  73.31153
## 929      74  78.85476
## 930      81  76.00103
## 931      65  77.32058
## 932      68  73.77252
## 933      71  90.41431
## 934      86  85.21387
## 935      87  73.53234
## 936      94  76.49770
## 937      85  70.54324
## 938      74  84.21458
## 939      67  97.21267
## 940      76  80.64946
## 941      65  79.82280
## 942     106  76.58877
## 943      79  76.01548
## 944      93  75.33370
## 945      76  71.64498
## 946      74  84.87058
## 947      88  79.01173
## 948      86  81.63345
## 949      70  77.29707
## 950      69  79.07006
## 951      61  78.70672
## 952      67  98.16423
## 953      77  85.26197
## 954      86  90.39484
## 955      91  94.47383
## 956      91 104.75405
## 957      92  92.78122
## 958      91  93.66794
## 959      85  95.31631
## 960      84 102.26987
## 961      73  95.77832
## 962      67  93.74198
## 963      54  87.68429
## 964      76  85.41279
## 965      67  86.50211
## 966      85  86.91786
## 967      80  86.91658
## 968      79  92.97927
## 969      67 100.04520
## 970      78  92.21314
## 971      74  86.56469
## 972      86  90.68067
## 973     100  89.20259
## 974      83  95.19330
## 975      52  84.80754
## 976      94  85.47340
## 977     112  79.99373
## 978      45  66.40110
## 979      85  74.33150
## 980      96  80.55107
## 981      86  77.92324
## 982      89  75.59030
## 983      83  74.68523
## 984      73  73.58853
## 985      76  79.11850
## 986      95  86.74757
## 987      98  91.30102
## 988      75  80.72652
## 989      98  91.67552
## 990      93  85.58264
## 991      86  85.24275
## 992     104  79.61319
## 993     105  94.02997
## 994      98  91.34655
## 995      88  86.76379
## 996      74  83.15531
## 997     103  80.73357
## 998      73  78.38100
## 999      63  82.05406
## 1000     59  84.55653
## 1001     82  85.12766
## 1002     88  85.11712
## 1003     89  94.73952
## 1004     81  92.33759
## 1005     80  89.29616
## 1006     97  89.76105
## 1007     77  87.44643
## 1008     71  91.54011
## 1009     78  93.41918
## 1010     82  92.01233
## 1011     89  91.22298
## 1012     65  93.76596
## 1013     88  80.15303
## 1014     88  74.74933
## 1015    100  75.69253
## 1016     93  76.00785
## 1017     74  71.17943
## 1018     63  66.08970
## 1019     53  80.56114
## 1020     79  85.16664
## 1021     89  87.91888
## 1022     77  74.29519
## 1023     78  73.74423
## 1024     74  72.48966
## 1025     85  76.45746
## 1026     94  79.55431
## 1027     96  82.36934
## 1028     89  77.09259
## 1029     86  85.04632
## 1030     95  87.48039
## 1031     79  94.97976
## 1032     92  91.99768
## 1033     64  90.99592
## 1034     90  89.30324
## 1035     94  81.71456
## 1036     77  79.33073
## 1037     95  89.00786
## 1038     93  87.12255
## 1039     98  81.65797
## 1040     90  81.35012
## 1041     84  68.60934
## 1042     69  67.88631
## 1043     77  68.54235
## 1044     64  67.56090
## 1045     86  69.46154
## 1046     56  82.63933
## 1047     93  85.29402
## 1048     64  74.25824
## 1049     70  76.50022
## 1050     68  79.50063
## 1051     70  78.73351
## 1052     98  77.56332
## 1053     92  71.55565
## 1054     89  70.39303
## 1055    108  74.11267
## 1056     89  85.19926
## 1057     61  78.36045
## 1058     73  77.45106
## 1059     95  83.83186
## 1060     66  76.32044
## 1061     78  86.73581
## 1062     69  83.39038
## 1063     80  83.93669
## 1064     91  83.00696
## 1065     75  86.60338
## 1066     93  86.10335
## 1067     85  88.47441
## 1068     82  83.14315
## 1069     94  91.80580
## 1070     97  88.58660
## 1071     98  82.50243
## 1072     97  79.89829
## 1073     79  91.77266
## 1074     81  85.07039
## 1075     75  83.01118
## 1076     62  85.68794
## 1077     75  95.76558
## 1078     84  90.58290
## 1079     88  91.94511
## 1080     82  96.31662
## 1081     74  84.96631
## 1082     68  69.83334
## 1083     67  73.56281
## 1084     66  72.00398
## 1085     95  74.46932
## 1086     83  73.08326
## 1087     71  71.27250
## 1088     95  84.53501
## 1089     82  83.59480
## 1090     97  81.98597
## 1091     77  86.48434
## 1092     53  81.75761
## 1093    101  76.77313
## 1094     88  76.37420
## 1095    103  75.89710
## 1096     79  77.61684
## 1097     58  81.74684
## 1098     74  81.41920
## 1099     86  76.82860
## 1100     83  74.59177
## 1101     87  79.96919
## 1102     88  94.95448
## 1103     84  83.66543
## 1104     75  76.84529
## 1105     68  87.13900
## 1106     65  71.93989
## 1107     55  78.74975
## 1108     54  78.76011
## 1109     66  66.61349
## 1110     65  70.79383
## 1111     72  70.84328
## 1112     69  71.91145
## 1113     72  67.12428
## 1114     79  71.62438
## 1115     89  71.33810
## 1116     82  71.90173
## 1117     81  73.78630
## 1118     64  81.48361
## 1119     80  84.80836
## 1120     76  83.19514
## 1121     86  81.29440
## 1122     65  73.80876
## 1123     85  67.63890
## 1124     72  73.59700
## 1125     80  79.16576
## 1126     76  72.63248
## 1127     83  72.41918
## 1128     92  73.04657
## 1129     79  66.49953
## 1130     65  73.95977
## 1131     58  71.42417
## 1132     69  69.30896
## 1133     61  71.54921
## 1134     79  76.78362
## 1135     75  82.63552
## 1136     80  95.12850
## 1137     97  83.90767
## 1138     82  84.47296
## 1139     69  87.77827
## 1140     66  80.02842
## 1141    105  84.42900
## 1142     86  85.19428
## 1143     92  73.29018
## 1144     99  78.51938
## 1145     88  75.82602
## 1146    102  71.69911
## 1147    102  75.95149
## 1148    110  85.64490
## 1149    104  83.34346
## 1150     75  77.77329
## 1151     80  74.54778
## 1152     95  86.37762
## 1153     85  80.14170
## 1154     95  83.43499
## 1155     78  85.01173
## 1156     90  79.65990
## 1157     86  97.22181
## 1158     71  78.07314
## 1159     66  91.60057
## 1160     93  92.29360
## 1161     86  92.95479
## 1162     71  89.26299
## 1163     78  79.41539
## 1164     68  80.53341
## 1165     81  73.59357
## 1166     73  81.24245
## 1167     45  83.56699
## 1168     78  82.38249
## 1169     91  93.33796
## 1170     89  85.35170
## 1171     77  76.41227
## 1172     69  79.07561
## 1173     89  72.52526
## 1174     92  95.66395
## 1175     67  87.66176
## 1176     56  92.90300
## 1177     81  99.28524
## 1178     77  86.39980
## 1179     61  81.58923
## 1180     70  85.33114
## 1181     85  93.70586
## 1182     91  93.36064
## 1183     74  77.33035
## 1184     95  79.55576
## 1185     78  75.40082
## 1186     70  79.76631
## 1187     90  81.47024
## 1188     41  80.17441
## 1189     61  77.22722
## 1190    100  81.74268
## 1191     86  78.60004
## 1192     98  84.63944
## 1193     92  79.37534
## 1194    101  76.55477
## 1195     87  78.77538
## 1196     91  85.52381
## 1197     72  84.44809
## 1198     78  77.59152
## 1199     71  87.63939
## 1200     88  86.29028
## 1201     99  84.57447
## 1202     97  87.19745
## 1203     54  77.20342
## 1204     79  73.46173
## 1205     96  73.29697
## 1206     99  78.67011
## 1207    114  74.68087
## 1208     92  71.33847
## 1209     97  70.73626
## 1210     96  70.52728
## 1211     93  78.88400
## 1212     80  71.53207
## 1213     99  71.39736
## 1214     79  74.93688
## 1215     71  81.09400
## 1216     88  72.57754
## 1217     88  75.39038
## 1218     97  75.68757
## 1219     58  80.16107
## 1220     66  89.40135
## 1221     56  72.16460
## 1222     67  83.38028
## 1223     76  81.32313
## 1224     88  84.89702
## 1225     62  84.84828
## 1226     59  80.52925
## 1227     89  76.62719
## 1228    102  79.65161
## 1229     54  79.83045
## 1230     77  82.55894
## 1231     96  73.84234
## 1232     65  81.12999
## 1233     74  73.31931
## 1234     91  63.40639
## 1235     92  75.90363
## 1236     85  67.21770
## 1237    100  70.21542
## 1238     96  68.87801
## 1239     66  72.77913
## 1240     87  70.17411
## 1241     84  71.05437
## 1242     94  81.34791
## 1243     54  79.48590
## 1244     75  80.68455
## 1245     45  72.75448
## 1246     67  74.81253
## 1247     65  78.33119
## 1248     92  80.06916
## 1249     92  73.74964
## 1250     67  74.50480
## 1251     61  75.05737
## 1252    101  80.77484
## 1253     90  80.54898
## 1254     78  80.71122
## 1255     77  80.60354
## 1256     65  84.06425
## 1257     79  81.92056
## 1258    105  74.13072
## 1259    102  74.34099
## 1260    117  82.79877
## 1261     91  78.63570
## 1262     85  79.79308
## 1263     68  86.05803
## 1264     54  77.78018
## 1265     89  79.84836
## 1266     99  84.47589
## 1267     88  79.79337
## 1268     90  76.72617
## 1269     93  79.21975
## 1270     72  80.25741
## 1271     73  79.14570
## 1272     87  83.44643
## 1273     53  93.98953
## 1274     74  94.00703
## 1275     88  93.68490
## 1276     74  91.38872
## 1277     73  93.75317
## 1278     78  92.88496
## 1279     63  91.70101
## 1280     61  85.48907
## 1281     92  80.65960
## 1282     84  74.69654
## 1283     89  75.32830
## 1284     79  81.12020
## 1285     64  70.93756
## 1286     57  70.80932
## 1287     77  74.57660
## 1288     64  85.39189
## 1289     88  86.64623
## 1290     76  84.02929
## 1291    116  79.94081
## 1292     93  78.06523
## 1293     78  84.11563
## 1294     98  88.72233
## 1295     79  86.22934
## 1296     90  79.80524
## 1297     93  86.74477
## 1298     76  87.62938
## 1299     97  92.83543
## 1300     98  94.36469
## 1301     77  86.01547
## 1302     64  80.54337
## 1303     77  73.29776
## 1304     87  78.68654
## 1305     83  82.39342
## 1306     88  78.33648
## 1307     91  79.95985
## 1308     88  81.96134
## 1309     62  81.59286
## 1310    103  79.05989
## 1311     75  77.86516
## 1312     86  82.34401
## 1313     97  75.16981
## 1314     90  80.07663
## 1315     79  82.51050
## 1316     55  82.61926
## 1317     67  96.35798
## 1318     82  91.78498
## 1319     67  91.83003
## 1320     55  96.02732
## 1321     87  86.45282
## 1322     64  82.53612
## 1323     84  86.96077
## 1324     82  87.22632
## 1325     83  84.18313
## 1326    106  83.26233
## 1327     85  89.16195
## 1328    110  81.82784
## 1329    102  80.15164
## 1330     94  76.91678
## 1331     89  72.78148
## 1332     93  70.53999
## 1333     80  78.85112
## 1334     92  81.40689
## 1335     93  83.17569
## 1336     93  87.87173
## 1337    102  75.63225
## 1338     76  78.58231
## 1339     78  73.46579
## 1340     81  78.71896
## 1341     87  82.70960
## 1342     83  73.33920
## 1343     93  77.90761
## 1344    105  79.98687
## 1345    100  70.52942
## 1346     63  84.46694
## 1347     70  98.40246
## 1348     61  86.63365
## 1349     61  78.47395
## 1350     62  85.92126
## 1351     77  83.71443
## 1352     70  87.85776
## 1353     83  94.71885
## 1354     77  90.06946
## 1355     71  81.89370
## 1356     53  71.95118
## 1357     89  75.62014
## 1358     80  88.90679
## 1359     55  95.62316
## 1360     82  72.47442
## 1361     78  97.62852
## 1362     74  88.88781
## 1363     88  73.04341
## 1364     83  74.20625
## 1365     71  74.85814
## 1366     78  75.56058
## 1367     88  67.96074
## 1368     66  64.83668
## 1369     90  66.12139
## 1370     87  77.36962
## 1371     70  86.11549
## 1372     70  88.08887
## 1373     82  84.97467
## 1374     75  79.91790
## 1375     85  82.47619
## 1376     98  76.35949
## 1377     51  78.68785
## 1378     76  76.76146
## 1379    111  90.75052
## 1380     68  83.07884
## 1381     58  90.01429
## 1382     53  88.37454
## 1383     56  92.99445
## 1384     74  90.64963
## 1385     81  90.30104
## 1386     65  83.17853
## 1387     68  79.07535
## 1388     71  77.46767
## 1389     86  84.16275
## 1390     87  79.55429
## 1391     94  80.16002
## 1392     85  80.89242
## 1393     74  83.19119
## 1394     67  81.77316
## 1395     76  88.11476
## 1396     65  78.96010
## 1397    106  82.05215
## 1398     79  78.88763
## 1399     93  79.95473
## 1400     76  87.94550
## 1401     74  84.21109
## 1402     88  82.94389
## 1403     86  81.63757
## 1404     70  82.86589
## 1405     69  82.07041
## 1406     61  88.16516
## 1407     67  83.92208
## 1408     77  82.40573
## 1409     86  82.94323
## 1410     91  79.72355
## 1411     91  80.43830
## 1412     92  81.25349
## 1413     91  77.38370
## 1414     85  78.72661
## 1415     84  78.92403
## 1416     73  73.09722
## 1417     67  76.51427
## 1418     54  75.89040
## 1419     76  75.09768
## 1420     67  69.03655
## 1421     85  73.11790
## 1422     80  80.32345
## 1423     79  81.62448
## 1424     67  72.46322
## 1425     78  73.05070
## 1426     74  82.60617
## 1427     86  76.80164
## 1428    100  74.71730
## 1429     83  72.53451
## 1430     52  79.24098
## 1431     94  80.59521
## 1432    112  77.15610
## 1433     45  74.10392
## 1434     85  79.46361
## 1435     96  81.59949
## 1436     86  92.84525
## 1437     89  86.74162
## 1438     83  89.00812
## 1439     73  83.81009
## 1440     76  82.45990
## 1441     95  86.81278
## 1442     98  87.08329
## 1443     75  81.02582
## 1444     98  81.79036
## 1445     93  76.09764
## 1446     86  79.58062
## 1447    104  70.85242
## 1448    105  73.77270
## 1449     98  78.06725
## 1450     88  82.56246
## 1451     74  82.22541
## 1452    103  82.97301
## 1453     73  77.64168
## 1454     63  79.54539
## 1455     59  68.24060
## 1456     82  70.24206
## 1457     88  67.33869
## 1458     89  80.95609
## 1459     81  77.96052
## 1460     80  70.39466
## 1461     97  75.42098
## 1462     77  79.98365
## 1463     71  72.26114
## 1464     78  80.33432
## 1465     82  72.40334
## 1466     89  76.28209
## 1467     65  77.74120
## 1468     88  76.98433
## 1469     88  74.78655
## 1470    100  84.57421
## 1471     93  77.67434
## 1472     74  84.70901
## 1473     63  78.88817
## 1474     53  86.51928
## 1475     79  94.59262
## 1476     89  83.68060
## 1477     77  90.09426
## 1478     78  86.45699
## 1479     74  85.18612
## 1480     85  73.40551
## 1481     94  73.27707
## 1482     96  91.99847
## 1483     89  86.52118
## 1484     86  92.42330
## 1485     95  84.04087
## 1486     79  83.86156
## 1487     92  85.52067
## 1488     64  80.89763
## 1489     90  86.58041
## 1490     94  91.77189
## 1491     77  82.89224
## 1492     95  78.98014
## 1493     93  87.14506
## 1494     98  89.37809
## 1495     90  74.04740
## 1496     84  75.81540
## 1497     69  84.01966
## 1498     77  88.51943
## 1499     64  72.64024
## 1500     86  79.09215
## 1501     56  70.49611
## 1502     93  79.19742
## 1503     64  74.80890
## 1504     70  70.05598
## 1505     68  79.93610
## 1506     70  71.64301
## 1507     98  81.23458
## 1508     92  84.56800
## 1509     89  81.05502
## 1510    108  77.11747
## 1511     89  78.69532
## 1512     61  76.56764
## 1513     73  74.31725
## 1514     95  79.24925
## 1515     66  77.95751
## 1516     78  73.35033
## 1517     69  82.50775
## 1518     80  77.38719
## 1519     91  79.51796
## 1520     75  67.76209
## 1521     93  80.89661

mean (apply(predicted_values, 1, min)/apply(predicted_values, 1, max)) # calculate accuracy

## [1] 0.8610993

The prediction accuracy for the OLS Model5 is at 85.94% which is not bad for this purpose. But lets compare it to the Champion Model- The improved Ridge Regression.

predicted <- predict(Model6_Improved, newx = test_Ind)# predict on test data
predicted_values <- cbind (actual=test_baseball$TARGET_WINS, predicted)  # combine
predicted_values

##      actual        s0
## 7        78  75.03711
## 9        88  87.01636
## 10       66  70.96167
## 14       90  86.78105
## 21       87  88.85227
## 22       70  74.80807
## 24       70  73.34972
## 25       82  83.90338
## 26       75  77.82982
## 33       85  85.38910
## 35       98  95.56051
## 37       51  59.96291
## 38       76  78.49513
## 39      111 100.63697
## 43       68  70.89286
## 44       58  61.66102
## 46       53  57.04949
## 50       56  61.10418
## 63       74  75.01580
## 65       81  78.67340
## 73       65  68.51614
## 75       68  69.64439
## 76       71  74.30808
## 77       86  83.92501
## 81       87  86.16102
## 84       94  89.06941
## 90       85  83.60543
## 94       74  75.92110
## 97       67  69.60139
## 103      76  78.82769
## 106      65  69.25997
## 112     106  99.45416
## 118      79  81.24117
## 119      93  88.17171
## 120      76  73.66306
## 123      74  73.75321
## 124      88  81.49834
## 135      86  84.96702
## 140      70  74.32101
## 144      69  70.78080
## 148      61  67.96736
## 150      67  70.08708
## 157      77  77.27762
## 158      86  83.41425
## 163      91  85.55094
## 169      91  86.47062
## 170      92  88.18314
## 173      91  86.83936
## 178      85  83.20055
## 179      84  84.28171
## 180      73  75.26937
## 181      67  72.30857
## 182      54  61.43096
## 184      76  79.02501
## 185      67  72.45972
## 187      85  85.12688
## 188      80  84.76213
## 190      79  83.44257
## 194      67  71.60719
## 195      78  80.76420
## 196      74  77.06805
## 197      86  83.57190
## 205     100  91.14015
## 206      83  80.45242
## 207      52  57.68847
## 210      94  87.32754
## 213     112 102.79642
## 225      45  55.15122
## 230      85  86.24841
## 231      96  93.64028
## 233      86  87.52974
## 239      89  84.43616
## 255      83  83.60073
## 258      73  73.21070
## 260      76  80.87881
## 268      95  95.25110
## 269      98  95.54130
## 271      75  78.28698
## 280      98  93.76895
## 281      93  90.14201
## 286      86  83.00154
## 289     104  99.55793
## 294     105  96.57130
## 295      98  94.18231
## 297      88  85.77068
## 299      74  74.93959
## 303     103  96.71589
## 305      73  73.72188
## 314      63  68.86051
## 315      59  65.23369
## 316      82  77.33838
## 319      88  84.56275
## 322      89  86.16204
## 325      81  80.29325
## 327      80  79.26768
## 329      97  92.53020
## 330      77  79.46911
## 331      71  74.80862
## 336      78  79.30623
## 338      82  83.46643
## 341      89  88.86301
## 343      65  73.02589
## 344      88  87.10304
## 345      88  84.81484
## 352     100  90.93449
## 353      93  85.09738
## 360      74  76.55836
## 361      63  68.65867
## 364      53  61.48733
## 367      79  81.47885
## 371      89  87.44385
## 375      77  76.84541
## 376      78  76.41159
## 377      74  74.39786
## 379      85  83.07092
## 380      94  90.24674
## 382      96  91.87284
## 383      89  88.02348
## 385      86  84.43969
## 391      95  88.13087
## 394      79  78.59321
## 395      92  85.66815
## 398      64  69.02299
## 399      90  88.95943
## 412      94  91.20603
## 413      77  83.15905
## 417      95  92.31902
## 422      93  91.08178
## 423      98  92.44188
## 424      90  88.96385
## 428      84  82.49343
## 429      69  70.77145
## 431      77  75.48418
## 435      64  68.39124
## 437      86  84.09055
## 446      56  62.43464
## 453      93  86.40273
## 456      64  67.10247
## 457      70  70.64746
## 459      68  70.96312
## 461      70  73.56430
## 470      98  91.84934
## 472      92  86.43395
## 473      89  88.37428
## 480     108 101.46663
## 485      89  87.64842
## 486      61  66.47212
## 493      73  76.12512
## 497      95  93.46618
## 499      66  70.80571
## 500      78  78.78698
## 501      69  71.43561
## 507      80  77.89068
## 509      91  83.14196
## 510      75  72.93749
## 515      93  87.62065
## 523      85  84.21848
## 524      82  83.49894
## 536      94  90.29993
## 537      97  94.43574
## 538      98  92.62975
## 539      97  91.61566
## 546      79  78.48345
## 547      81  78.26172
## 548      75  74.04307
## 550      62  66.57698
## 553      75  73.23095
## 558      84  83.83066
## 576      88  84.88218
## 583      82  85.97682
## 586      74  77.89110
## 587      68  74.89603
## 588      67  71.65880
## 591      66  67.64450
## 594      95  87.44251
## 597      83  82.61035
## 598      71  76.04592
## 611      95  94.33950
## 613      82  80.39077
## 616      97  92.10884
## 619      77  78.10838
## 620      53  62.04609
## 625     101  96.37482
## 628      88  85.58182
## 629     103  94.40436
## 630      79  79.49821
## 634      58  62.83692
## 635      74  74.18933
## 637      86  84.92152
## 639      83  82.63007
## 643      87  87.32796
## 645      88  86.19682
## 648      84  84.93321
## 649      75  78.05461
## 651      68  73.83253
## 652      65  70.68442
## 655      55  60.78924
## 663      54  62.16574
## 671      66  66.24674
## 672      65  68.55525
## 673      72  73.58779
## 674      69  72.42452
## 675      72  70.55399
## 677      79  80.81004
## 679      89  86.73092
## 680      82  80.74043
## 681      81  80.28609
## 682      64  69.66643
## 683      80  79.75968
## 689      76  78.82215
## 690      86  83.18917
## 691      65  68.74806
## 693      85  83.53562
## 702      72  74.15006
## 706      80  79.18263
## 710      76  77.13721
## 711      83  82.70913
## 713      92  88.97052
## 715      79  80.65417
## 719      65  69.42058
## 729      58  61.89778
## 730      69  71.57611
## 731      61  69.22148
## 733      79  79.03961
## 735      75  74.87910
## 738      80  80.14746
## 739      97  91.00801
## 741      82  82.56460
## 744      69  72.36972
## 747      66  69.83938
## 751     105  98.62205
## 752      86  84.23502
## 754      92  89.41307
## 756      99  95.38238
## 757      88  86.63050
## 758     102  98.21484
## 760     102  97.05952
## 761     110 104.83533
## 763     104  99.26512
## 766      75  75.68135
## 771      80  77.91938
## 772      95  87.72689
## 775      85  83.14244
## 789      95  87.55607
## 790      78  76.57855
## 794      90  86.07544
## 799      86  84.67367
## 802      71  74.05465
## 811      66  68.20371
## 813      93  89.49641
## 815      86  84.74147
## 818      71  73.44951
## 825      78  77.80901
## 829      68  71.96929
## 833      81  80.75377
## 836      73  77.47239
## 841      45  52.26375
## 845      78  77.29350
## 846      91  87.05190
## 849      89  88.36922
## 857      77  80.09991
## 859      69  74.26901
## 863      89  86.47908
## 865      92  87.86884
## 866      67  69.78199
## 871      56  63.00110
## 883      81  81.77057
## 887      77  75.91352
## 888      61  66.81055
## 890      70  73.16614
## 892      85  85.28716
## 893      91  89.58622
## 895      74  76.07859
## 896      95  92.00404
## 899      78  82.94302
## 901      70  74.80678
## 905      90  89.92446
## 910      41  53.66875
## 914      61  62.57269
## 916     100  90.71328
## 918      86  82.30690
## 922      98  92.45750
## 924      92  90.58234
## 925     101  94.94783
## 926      87  84.76503
## 927      91  88.51374
## 928      72  74.05413
## 930      78  79.03157
## 931      71  73.41550
## 932      88  85.37928
## 941      99  91.12406
## 943      97  89.99370
## 945      54  59.84998
## 950      79  78.41076
## 955      96  94.10060
## 959      99  96.13333
## 960     114 107.47269
## 967      92  87.86477
## 975      97  91.17138
## 976      96  91.69353
## 980      93  88.52896
## 983      80  77.83020
## 988      99  93.27990
## 990      79  80.42502
## 998      71  73.02867
## 999      88  87.60258
## 1005     88  88.94530
## 1010     97  95.72285
## 1023     58  64.27910
## 1024     66  67.37750
## 1036     56  64.78611
## 1040     67  72.28783
## 1043     76  75.13695
## 1046     88  86.95097
## 1050     62  66.45400
## 1052     59  65.72218
## 1056     89  85.71438
## 1057    102  93.40480
## 1060     54  61.14283
## 1061     77  78.83631
## 1067     96  92.28392
## 1071     65  71.74706
## 1072     74  76.71142
## 1076     91  88.37653
## 1077     92  89.58216
## 1078     85  84.48986
## 1079    100  95.00794
## 1081     96  92.02783
## 1082     66  68.32324
## 1086     87  82.17474
## 1089     84  82.61644
## 1091     94  89.92985
## 1100     54  62.72216
## 1102     75  81.17730
## 1112     45  54.93649
## 1114     67  69.36467
## 1115     65  68.36067
## 1121     92  87.72829
## 1127     92  85.34645
## 1131     67  69.54043
## 1132     61  65.79245
## 1136    101  96.58305
## 1137     90  87.70293
## 1146     78  78.16252
## 1152     77  82.13712
## 1153     65  72.61402
## 1158     79  76.79913
## 1160    105  98.87103
## 1164    102  93.48714
## 1167    117 104.32583
## 1168     91  87.19217
## 1170     85  84.47892
## 1172     68  70.82733
## 1173     54  60.51890
## 1178     89  86.48616
## 1179     99  93.79642
## 1188     88  87.17542
## 1189     90  86.86075
## 1190     93  89.01343
## 1191     72  74.83482
## 1194     73  74.68507
## 1200     87  85.95239
## 1203     53  61.93625
## 1208     74  72.33440
## 1214     88  84.04302
## 1223     74  76.29071
## 1228     73  76.57395
## 1230     78  78.79765
## 1235     63  67.16567
## 1237     61  64.31163
## 1244     92  87.33599
## 1246     84  81.48820
## 1247     89  84.77387
## 1257     79  80.80011
## 1261     64  68.15035
## 1262     57  64.42651
## 1269     77  78.57385
## 1271     64  69.84801
## 1273     88  89.77412
## 1276     76  80.83139
## 1279    116 108.89136
## 1281     93  92.33233
## 1284     78  80.51975
## 1294     98  93.68402
## 1296     79  78.14688
## 1299     90  89.11274
## 1301     93  89.97517
## 1302     76  76.79456
## 1303     97  88.33447
## 1304     98  90.49289
## 1310     77  78.36077
## 1316     64  70.11849
## 1319     77  80.37727
## 1325     87  85.11975
## 1326     83  82.28502
## 1328     88  84.81495
## 1331     91  85.19752
## 1336     88  86.54688
## 1342     62  66.39315
## 1346    103  96.25300
## 1348     75  78.29751
## 1352     86  87.44785
## 1353     97  95.72093
## 1361     90  88.85385
## 1365     79  77.29926
## 1368     55  58.48840
## 1371     67  72.96048
## 1372     82  83.36419
## 1373     67  72.68941
## 1374     55  63.44108
## 1377     87  81.89851
## 1378     64  67.91359
## 1380     84  83.49812
## 1381     82  83.21167
## 1384     83  84.82787
## 1386    106  97.45435
## 1392     85  83.93294
## 1398    110  98.33059
## 1399    102  94.33953
## 1400     94  91.16584
## 1401     89  87.80385
## 1405     93  88.63952
## 1408     80  80.38671
## 1409     92  88.40008
## 1415     93  88.59284
## 1416     93  86.69133
## 1419    102  93.22559
## 1422     76  78.04900
## 1424     78  77.24801
## 1425     81  79.35412
## 1426     87  84.41423
## 1436     83  84.60137
## 1439     93  89.55473
## 1442    105  98.72252
## 1443    100  93.25404
## 1449     63  69.89372
## 1450     70  74.59612
## 1452     61  68.01101
## 1453     61  66.31212
## 1455     62  65.82437
## 1466     77  77.58788
## 1467     70  73.57520
## 1469     83  81.32210
## 1473     77  78.83679
## 1476     71  76.31154
## 1481     53  59.79117
## 1482     89  88.17955
## 1496     80  80.85315
## 1504     55  60.27143
## 1509     82  83.02252
## 1510     78  77.96093
## 1511     74  76.54894
## 1512     88  85.42037
## 1518     83  82.33936
## 1521     71  75.75057

Lets calculate the accuracy of using Model6 for our predictions

mean (apply(predicted_values, 1, min)/apply(predicted_values, 1, max)) # calculate accuracy

## [1] 0.9564973

The prediction accuracy of the improved Ridge Regression Model is 95.75%.

ModelName <- c("Model3", "Model5","Model6")
Model_Accuracy <- c("85.85%", "85.85%", "95.75%")
AccuracyCompared <- data.frame(ModelName,Model_Accuracy)
AccuracyCompared

ModelName	Model_Accuracy
Model3	85.85%
Model5	85.85%
Model6	95.75%

The prediction accuracy of the improved Ridge Regression Model6 is at 95.75% which is very good for this purpose.

0.5 Conclusion

The improved Model6 shows significant improvement from all the OLS Models when the R-Squared and the RMSE of the Models are compared. This Model also predict TARGET WINS better than the OLS models because it is more stable and less prone to overfitting.

The chosen OLS Model3 and Model5 are due to the improved F-Statistic, positive variable coefficients and low Standard Errors. We will chose to make our predictions with the champion model the improved Ridge Regression Model6 because it beats all the OLS models when the model performance metrics are compared as well as the predictive ability of this model.

For Models 3 and 4, the variables were chosen just to test how the offfensive categories only would affect the model and how only defensive variables would affect the model. Based on the Coefficients for each model, the third model took the highest coefficient from each category model.

For offense, the two highest were HR and Triples. Which intuively does make sense because the HR and triple are two of the highest objectives a hitter can achieve when batting and thus the higher the totals in those categories the higher the runs scored which help a team win. And on the defensive side, the two highest cooeficients were Hits and WALKS. Which again just looking at it from a common sense point does make sense because as a pitcher, what they want to do is limit the numbers of times a batter gets on base whether by a hit or walk. Unless its an error, if a batter does not get a hit or walk then the outcome would be an out which would in essence limit the amount of runs scored by the opposing team.