1. Date Exploration
The baseball training dataset contains 2,276 observations of 17
variables detailing various teams’ performances per year from 1871 to
2006. The description of the columns is shown below. Due to the
relatively long period, we expect to see outliers and missing data as
the league modified official game rules; these rule changes undoubtedly
caused teams and players to change their tactics in response.
Additionally, the number of single base hits is noticeably missing from
the columns. However, we will derive this value as the number of other
types of hits (doubles, triples, home runs) can be subtracted from total
hits. Lastly, other columns representing game number (out of 162),
inning number (1-9), and matching opponent columns would have been
vastly useful for predictions. One last noticeable omission from the
original dataset is of the number of single base hits. However, this
value can possibly be calculated as a difference between other types of
hits (doubles, triples, home runs) and total hits.
1.1 Summary Statistics
The table below shows us some valuable descriptive statistics for the
training data. The data set contains all integers. We can see that many
of the variables have a minimum of 0 but not all. The means and medians
of each variable are all relatively close in value for each individual
variable. This tells us that most data is free from extreme outliers as
they tend to skew the mean relative to the median.
One interesting piece of information is the min/max of the
TARGET_WINS variable. The minimum is 0 meaning there are
teams that did not win a single game. The maximum is 146 which indicates
no team in the training dataset had a perfect season, as we know from
the data a season consists of 162 games.
Also of note is the number of missing values from certain variables.
Most notably the TEAM_BATTING_HBP (batters hit by pitch
variable). With 91% of the data missing we will remove this variable
from our dataset because there simply is not enough information to
impute a sensible value. The column TEAM_BASERUN_CS (caught
stealing) had 34% of the missing data, we may consider removing it
later. The missing data for these two columns may be due to a change
official rules or tactics before the modern era of baseball.
1.2 Distribution and Box Plots
Next, we’ll visually check for normal distributions and box plots in
both the dependent and independent variables. The density plot below
shows normalcy in most features except for extremely right skewed
features such as hits allowed (PITCHING_H) or errors (FIELDING_E).
Homeruns by batters (BATTING_HR) and strikeouts by batters (BATTING_SO)
variables seem bimodal. It implies the existence of two distinct
clusters within the baseball season data, where teams tended to score
more in one of the clusters.
Box plots for these further show a high number of outliers exist outside
of the interquartile ranges so their effects should be carefully
considered and we may deal with non-unimodal distributions.
Lastly, the function featurePlot() will show the relationship between
independent variables and the target variable TARGET_WINS.
In general, while our graphs display certain intriguing connections
among the variables, they also expose noteworthy problems with the data.
For example, the dataset contains a team that has not won any games,
which appears improbable. By checking the web data, we found that it
actually happened 2 times: 1872 and 1873. Or that the pitching data
contains numerous instances of 0’s, several teams have 0 strikeouts by
their pitchers over the season, which is highly improbable. Also, there
is as a team achieving 20,000 strikeouts. There will be further steps to
work with outliers and 0’s.
1.3 Correlation Matrix
Plotting the correlations between TARGET_WINS and the
variables (excluding INDEX and
TEAM_BATTING_HBP) we can see that very few variables are
strongly correlated with the target variable. Columns with correlations
close to zero are unlikely to offer significant insights into the
factors that contribute to a team’s victories.
To avoid multicolinearity, we should not include features that have
strong correlation. Comparing offensive (any column starting with
BATTING or BASERUN) to defensive stats
unexpectedly shows some correlation, pointing to potential problems.
Qualitatively, the matrix implies some teams or players are exceptional
both at hitting (offensive) and fielding (defensive). Furthermore, a
typical team’s number of batted home runs and allowed home runs has a
correlation of nearly 1.0. This is an unexpected correlation but can be
explained by noticing most games are decided by a difference of one or
two runs (whether the games are high scoring or not). Any final models
should include one of these two home run variables. Alternatively, the
correlation between a team’s hits (BATTING_H) and hits
allowed (PITCHING_H) is around 0.3 which is seems
reasonable.
There are some other strong correlations that are less obvious such
as Errors (TEAM_FIELDING_E) being strongly negatively
correlated with walks by batters (TEAM_BATTING_BB), strike
outs (TEAM_BATTING_SO). All combined together, teams that
get a lot of hits do not generally make fielding errors.
Digging a little deeper we can see there is a Pearson correlation
coefficient of -0.6559708 for errors and walks by batters which
indicates a strong negative correlation between the two variables.
Looking at errors compared with team pitching hits allowed we see a
correlation of 0.667759 which indicates a strong positive
correlation.
|
|
Coefficient
|
|
TARGET_WINS
|
1.0000000
|
|
TEAM_BATTING_H
|
0.4699467
|
|
TEAM_BATTING_2B
|
0.3129840
|
|
TEAM_BATTING_3B
|
-0.1243459
|
|
TEAM_BATTING_HR
|
0.4224168
|
|
TEAM_BATTING_BB
|
0.4686879
|
|
TEAM_BATTING_SO
|
-0.2288927
|
|
TEAM_BASERUN_SB
|
0.0148364
|
|
TEAM_BASERUN_CS
|
-0.1787560
|
|
TEAM_BATTING_HBP
|
0.0735042
|
|
TEAM_PITCHING_H
|
0.4712343
|
|
TEAM_PITCHING_HR
|
0.4224668
|
|
TEAM_PITCHING_BB
|
0.4683988
|
|
TEAM_PITCHING_SO
|
-0.2293648
|
|
TEAM_FIELDING_E
|
-0.3866880
|
|
TEAM_FIELDING_DP
|
-0.1958660
|
Lastly, lets take a closer look at the missing data. We’ve already
determined that the batter hit by pitch (TEAM_BATTING_HBP)
variable is missing 91% of its data but what of the other variables. We
will just drop the column from the further analysis.
Using the plot below we can visualize the missingness of the
remaining variables. There are 5 variables that contain varying degrees
of missing data. We will use the information to fill in the missing
values in our data preparation step.
TEAM_BASERUN_CS appears to be missing the second most
amount of values but at only 772 missing values out of 2276 this is much
less of a concern than the HBP variable we identified earlier. The
remaining variables that are missing data have less than 25% of their
data missing so should be safe to impute.
2. Data Preparation
2.1 Missing Data
Notes / Questions
As discussed above, we will drop the INDEX and
TEAM_BATTING_HBP variables as the
TEAM_BATTING_HBP variable is missing 91% of its data and
the INDEX variable is just an identification Variable.
We’ll also derive a new column for single base hits derived from
subtracting double, triples and home runs from the total number of
hits.
For further work with NA’s, we create flags to suggest if a variable
was missing.
2.2 Drop Outliers
To to impute the missing values in the trainDf data, the
mice library is used. To utilize MICE, one must make the
assumption that the missing values are missing at random, indicating
that the missingness can be explained by variables that have complete
information. The MICE algorithm then performs several iterations over
the data, as suggested by its name, and generates data to complete the
missing values. We check what impute method we use for each column.
pmm is predictive mean matching, replacing missing data
with column means.
Let’s also take a look at the density plots pre and post-imputation
to make sure densities look similar. Unfortunately, for
TEAM_BASERUN_SB, TEAM_BATTING_SO, and
TEAM_FIELDING DP they do not. But in the case of
TEAM_BATTING_SO our distribution becomes roughly more
normal, so it may be beneficial. For the TEAM_BASERUN_SB
and TEAM_BASERUN_CS and TEAM_FIELDING_DP we
may need to consider alternative methods.
3. Building models
Notes / Questions
At this juncture, with a thorough comprehension of our dataset and
having completed the data cleaning process, we can initiate the
construction of our multiple linear regression models. We will build
four separate linear models and compare their performance.
3.1 Model 1: Baseline
For the first model, we will select all the variables from the
original un-cleaned dataset. We may use this model as a base model to
compare with.
3.2 Model 2: Removed N/A Values
The second model will be based on the cleaned dataset without missing
values and non-normal distributions. We chose variables
TEAM_PITCHING_HR, TEAM_PITCHING_BB,
TEAM_BATTING_2B, TEAM_BATTING_3B,
TEAM_FIELDING_E based on the intuition and the
understanding of the data.
3.3 Model 3: Removed Outliers
The third model will use the cleaned dataset with outliers omitted.
To identify outliers, we used historical data from the Lahman’s Baseball
Database. In baseball’s early history, few games were played in a season
(less than 20); thus, the calculation used to normalize the statistics
to a 162-game season tends to create outliers. We used the min and max
from the modern era (post-1900) as a filter to alleviate the issues
created by outlier values.
The initial model will include all variables from the cleaned data
set with outliers omitted. Stepwise variable selection based on the AIC
score is used to filter the model features. The resulting model has a
significant p-values for the model and all predictor variables.
|
|
TARGET_WINS
|
|
Predictors
|
Estimates
|
CI
|
p
|
df
|
|
(Intercept)
|
56.66
|
43.45 – 69.87
|
<0.0001
|
1443.00
|
|
TEAM BATTING 2B
|
-0.03
|
-0.05 – -0.02
|
<0.0001
|
1443.00
|
|
TEAM BATTING 3B
|
0.21
|
0.16 – 0.25
|
<0.0001
|
1443.00
|
|
TEAM BATTING BB
|
0.09
|
0.07 – 0.11
|
<0.0001
|
1443.00
|
|
TEAM BATTING SO
|
-0.02
|
-0.03 – -0.02
|
<0.0001
|
1443.00
|
|
TEAM BASERUN SB
|
0.03
|
0.02 – 0.05
|
0.0001
|
1443.00
|
|
TEAM BASERUN CS
|
0.05
|
0.01 – 0.09
|
0.0057
|
1443.00
|
|
TEAM PITCHING HR
|
0.13
|
0.11 – 0.15
|
<0.0001
|
1443.00
|
|
TEAM PITCHING BB
|
-0.05
|
-0.06 – -0.03
|
<0.0001
|
1443.00
|
|
TEAM FIELDING E
|
-0.15
|
-0.17 – -0.13
|
<0.0001
|
1443.00
|
|
TEAM FIELDING DP
|
-0.12
|
-0.14 – -0.09
|
<0.0001
|
1443.00
|
|
TEAM BATTING 1B
|
0.04
|
0.03 – 0.04
|
<0.0001
|
1443.00
|
|
Observations
|
1455
|
|
R2 / R2 adjusted
|
0.439 / 0.435
|
|
AIC
|
10708.883
|
The linear modeling assumption are evaluated using a diagnostic plot,
the Breusch–Pagan Test for Heteroscedasticity and Variance Inflation
Factor (VIF) to assess colinearity.
The initial review of the diagnostic plots for this model shows some
deviation from linear modeling assumptions at the boundaries of the
prediction range, but the overall QQ plot is almost a flat line; the
residuals are flat and exhibit homoscedasticity of variance in the 65 -
95 fitted value range;
The Breusch–Pagan Test for Heteroscedasticity assumes the following
Null and Alternate hypothesis.
- H0 - Residuals are distributed with equal variance (i.e.,
homoscedasticity)
- H1 - Residuals are distributed with unequal variance (i.e.,
heteroscedasticity)
For this model iteration, we reject the null hypothesis and conclude
that this model violates the homoscedasticity function.
studentized Breusch-Pagan test
data: lm3step BP = 30.179, df = 11, p-value = 0.001485
The Variance Inflation Factor (VIF) calculation detects collinearity
with the TEAM_PITCHING_BB and TEAM_BATTING_BB variables. Both variables
have a VIF score over 8 and are 0.93 correlated.
For the refined model we will drop TEAM_PITCHING_BB and
TEAM_BATTING_BB from the model. The p-value
TEAM_BASERUN_CS is not significant therefore it will be
dropped from the model as well. This model has a lower \(AdjR^2=0.35\) but should align better with
the linear regression assumptions.
|
|
TARGET_WINS
|
|
Predictors
|
Estimates
|
CI
|
p
|
df
|
|
(Intercept)
|
76.17
|
64.38 – 87.96
|
<0.0001
|
1795.00
|
|
TEAM BATTING 3B
|
0.20
|
0.16 – 0.23
|
<0.0001
|
1795.00
|
|
TEAM BATTING SO
|
-0.02
|
-0.03 – -0.02
|
<0.0001
|
1795.00
|
|
TEAM BASERUN SB
|
0.08
|
0.06 – 0.09
|
<0.0001
|
1795.00
|
|
TEAM PITCHING HR
|
0.14
|
0.12 – 0.15
|
<0.0001
|
1795.00
|
|
TEAM FIELDING E
|
-0.12
|
-0.13 – -0.10
|
<0.0001
|
1795.00
|
|
TEAM FIELDING DP
|
-0.10
|
-0.13 – -0.08
|
<0.0001
|
1795.00
|
|
TEAM BATTING 1B
|
0.02
|
0.02 – 0.03
|
<0.0001
|
1795.00
|
|
Observations
|
1803
|
|
R2 / R2 adjusted
|
0.354 / 0.352
|
|
AIC
|
13642.626
|
The diagnostic plots for the new model appear to be closer aligned
with the linear regression assumptions. For this model iteration, we
will again reject the null hypothesis and conclude that this model
violates the homoscedasticity function. And finally the Variance
Inflation Factor (VIF) calculation does not detects collinearity across
the remaining variables. Overall we will reject model 3 because the
residuals are not homoscedastic.
studentized Breusch-Pagan test
data: lm3step.final BP = 40.152, df = 7, p-value = 1.177e-06
|
|
model 3
|
model 3 (StepAIC)
|
model 3 (Final)
|
|
Predictors
|
Estimates
|
CI
|
p
|
Estimates
|
CI
|
p
|
Estimates
|
CI
|
p
|
|
(Intercept)
|
56.77
|
43.51 – 70.02
|
<0.0001
|
56.66
|
43.45 – 69.87
|
<0.0001
|
76.17
|
64.38 – 87.96
|
<0.0001
|
|
TEAM BATTING 2B
|
-0.04
|
-0.12 – 0.03
|
0.2397
|
-0.03
|
-0.05 – -0.02
|
<0.0001
|
|
|
|
|
TEAM BATTING 3B
|
0.20
|
0.11 – 0.28
|
<0.0001
|
0.21
|
0.16 – 0.25
|
<0.0001
|
0.20
|
0.16 – 0.23
|
<0.0001
|
|
TEAM BATTING HR
|
-0.02
|
-0.31 – 0.27
|
0.8959
|
|
|
|
|
|
|
|
TEAM BATTING BB
|
0.09
|
-0.07 – 0.25
|
0.2878
|
0.09
|
0.07 – 0.11
|
<0.0001
|
|
|
|
|
TEAM BATTING SO
|
-0.01
|
-0.08 – 0.07
|
0.8913
|
-0.02
|
-0.03 – -0.02
|
<0.0001
|
-0.02
|
-0.03 – -0.02
|
<0.0001
|
|
TEAM BASERUN SB
|
0.04
|
0.02 – 0.05
|
0.0001
|
0.03
|
0.02 – 0.05
|
0.0001
|
0.08
|
0.06 – 0.09
|
<0.0001
|
|
TEAM BASERUN CS
|
0.05
|
0.01 – 0.09
|
0.0058
|
0.05
|
0.01 – 0.09
|
0.0057
|
|
|
|
|
TEAM PITCHING H
|
0.01
|
-0.06 – 0.08
|
0.8072
|
|
|
|
|
|
|
|
TEAM PITCHING HR
|
0.14
|
-0.12 – 0.40
|
0.2870
|
0.13
|
0.11 – 0.15
|
<0.0001
|
0.14
|
0.12 – 0.15
|
<0.0001
|
|
TEAM PITCHING BB
|
-0.05
|
-0.20 – 0.11
|
0.5598
|
-0.05
|
-0.06 – -0.03
|
<0.0001
|
|
|
|
|
TEAM PITCHING SO
|
-0.01
|
-0.09 – 0.06
|
0.6954
|
|
|
|
|
|
|
|
TEAM FIELDING E
|
-0.15
|
-0.17 – -0.13
|
<0.0001
|
-0.15
|
-0.17 – -0.13
|
<0.0001
|
-0.12
|
-0.13 – -0.10
|
<0.0001
|
|
TEAM FIELDING DP
|
-0.12
|
-0.14 – -0.09
|
<0.0001
|
-0.12
|
-0.14 – -0.09
|
<0.0001
|
-0.10
|
-0.13 – -0.08
|
<0.0001
|
|
TEAM BATTING 1B
|
0.03
|
-0.05 – 0.10
|
0.4665
|
0.04
|
0.03 – 0.04
|
<0.0001
|
0.02
|
0.02 – 0.03
|
<0.0001
|
|
Observations
|
1455
|
1455
|
1803
|
|
R2 / R2 adjusted
|
0.439 / 0.434
|
0.439 / 0.435
|
0.354 / 0.352
|
|
AIC
|
10714.711
|
10708.883
|
13642.626
|
4. Selecting Models
Next, we apply the models to the evaluation dataset to make
predictions. However, to ensure that the models 2-4 work properly, we
will fill in the missing values in the evaluation data set using the
same imputation method - mice, for model 4 we will also make necessary
transformations.
CHANGE SINCE THE MODELS WILL BE DIFFERENT The table
below contains the predicted TARGET_WINS for each model.
Upon initial examination, it is noticeable that the first model is
generating predictions with negative values. This is not a realistic
outcome as it is impossible to have a negative number of wins.
Therefore, this model is not particularly valuable. However, the second
and third models, which utilize cleaned and imputed data, do not
encounter this problem of producing large negative predictions.
Generally, both the AIC-generated model and the second model are
yielding comparable results.
Money Ball Dataset
|
lm1
|
lm2
|
aic
|
aic4
|
|
49.79
|
72.82
|
62.31
|
64.67
|
|
60.34
|
73.94
|
68.84
|
71.42
|
|
65.66
|
72.98
|
73.96
|
76.51
|
|
79.89
|
78.48
|
81.81
|
84.14
|
|
-3970.00
|
47.55
|
2.09
|
136.88
|
|
-1633.38
|
50.35
|
12.09
|
79.35
|
|
25.95
|
61.91
|
51.31
|
70.22
|
|
-134.29
|
61.55
|
57.11
|
69.57
|
|
26.47
|
79.90
|
70.21
|
77.57
|
|
33.99
|
73.04
|
69.26
|
71.90
|
|
42.44
|
72.21
|
60.17
|
66.32
|
|
68.81
|
85.46
|
82.52
|
86.88
|
|
94.21
|
89.44
|
85.99
|
90.18
|
|
91.46
|
78.53
|
82.77
|
89.20
|
|
85.54
|
75.27
|
89.98
|
90.72
|
We can also see when plotting the predictions that there doesn’t seem
to be much obvious difference between the models aside from the clearly
outrageous outliers generated by the first model, and the 4th model
showing a slightly tighter range in wins.
We can the graphs below to check the validity of our models. All
models suffer from a lack of linearity which indicates that a linear
regression model may not be the greatest technique for predicting values
from this data with the given variables. The models that included all
the most variables (model 1, model 3, model 4), suffer from co-linearity
issues.
Model 1
Model 2
Model 3
Model 4
Appendix A: Lahman’s Baseball Database
Despite significant efforts to compensate for poor data quality, the
resulting models are poor predictors of win totals. Moreover, the poor
data quality is inconsistent with the overall state of baseball
statistics. When it comes to the major sports, baseball has the most
mature statistics available. Therefore finding better data is the best
course of action for developing a better predictive model.
We were able to locate a cleaner version of the same data set
provided in the class. The Lahman’s Baseball Database includes the same
variables as the sample database with fewer errors and additional
reference data that would allow us to connect the database to other
sources.
https://www.seanlahman.com/baseball-archive/statistics/
A significant advantage of Lahman’s data set over the data set
provided in class is that it includes information about the year and the
team. This data is valuable when considering how baseball has changed
over the years. The modern era in baseball is often delineated by the
turn of the century. However, when looking at the past 120 years of
baseball history, it is easy to pinpoint rule changes, evolutions in
playing strategy, and league structure that have fundamentally impacted
the game.
When comparing statistics across time, it is common to use many of
the breakdowns below to add context to the analysis:
- The Dead Ball Era (1901 - 1920)
- World War 2 (1941 - 1945)
- Segregation Era (1901 - 1947ish)
- Post-War Era/Yankees Era (1945 - late 50s/early 60s)
- Westward Expansion (1953 - 1961)
- Dead Ball 2 (The Sixties, roughly)
- Designated Hitter Era (1973 - current, AL only)
- Free Agency/Arbitration Era (1975 - current)
- Steroid Era (unknown, but late 80s - 2005 seems likely)
- Wild Card Era (1994 - current)
Surveying these periods would suggest that a more granular model has
the potential to perform better.
Although we could have chosen any number of the time periods above,
exploring the statistical outliers highlights that many of these values
correspond to the pre-1969 period. This delineation has some historical
support. As Jayson Stark of ESPN argues in this article (https://www.espn.com/mlb/columns/story?columnist=stark_jayson&id=2471349)
In 1969 the MLB underwent several rule changes and changes to the league
structure that impacted win totals and team statistics. 1969 was the
first year of division play and the expanded postseason. The Pitcher’s
Mound was lowered five inches. The Strike zone shrinks. Five-person
rotations kicking in. The save was invented. And more expansion to the
unbalanced schedules.
Thus using 1900 as the beginning of the modern era and 1969 as an
additional breakpoint, the dataset can be divided into three segments.
The density profiles for the predictor variables approach a normal
distribution when grouped by the three segments we identified. To
support data exploration, we added a era_cat field to the data set.
In general the Lahman’s data set contains fewer data gaps and the
variables are more consistently distributed. There are some missing
values in the data set including the Caught Stealing
(CS/TEAM_BASERUN_CS) variable is missing 27.9%; the Batters Hit by Pitch
(HBP/TEAM_BATTING_HBP) variable is missing 38.8%; and the Sacrifice
Flies (SF) variable is missing 51.6% of the values.
Most of the variables in the data set show some level of skewness,
with the following variables having a Kurtosis measure of greater than
3, TEAM_BATTING_H, TEAM_BASERUN_SB, TEAM_BASERUN_CS, TEAM_PITCHING_H,
and TEAM_FIELDING_E
The Lahman data set contains several variables with bimodal
distributions, including, TEAM_BATTING_HR, TEAM_BATTING_SO,
TEAM_BATTING_HR, and TEAM_PITCHING_SO.
When we group the statistics by the three categories presented
earliers, we see a much cleaner density plot across all variables. There
are few signs of bimodal distributions, and the skewness of individual
variables is reduced greatly.
The training data set exibits the following characteristics.
DATA PREPARATION
The use of the era_cat variable allows us to group the data set into
3 categories that variables that approach normal distribution.
Since segmenting the data set using the era_cat variable creates 3
categories of variables that approach the normal distribution, we will
focus our data preparation step on missing values. The documentation for
MICE package recommends that a 5% threshold should be observed for
safely imputing missing values. Based on this rule of thumb we will drop
the ‘TEAM_BASERUN_CS’ and the ‘TEAM_BATTING_HBP’ variables.
The remaining variables with missing data
(‘TEAM_BATTING_BB’,‘TEAM_BATTING_SO’ and ‘TEAM_BASERUN_SB’) will be
impuned using the MICE package. Several methods can be used but for
simplicity we selected m=5 the default method.
yearID era_cat TARGET_WINS TEAM_BATTING_H
"" "" "" ""
TEAM_BATTING_2B TEAM_BATTING_3B TEAM_BATTING_HR TEAM_BATTING_BB “” “”
“” “pmm” TEAM_BATTING_SO TEAM_BASERUN_SB TEAM_PITCHING_H
TEAM_PITCHING_HR “pmm” “pmm” “” “” TEAM_PITCHING_BB TEAM_PITCHING_SO
TEAM_FIELDING_E TEAM_FIELDING_DP “” “” “” “”
Build Model
The first model includes all the variables from the original data set
with the exception of the yearID. It could be argued that year might
have an impact on the numbers, however we are accounting for year with
the era_cat variable. The initial model has an \(AdjR^2 = 0.8078\), and the model selected
by stepAIC includes the same variables and has the same \(AdjR^2=8078\).
Call: lm(formula = TARGET_WINS ~ . - yearID, data = trainingTeam_df,
by = era_cat)
Residuals: Min 1Q Median 3Q Max -41.931 -4.084 0.041 4.117 69.904
Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 73.928822 3.451424 21.420 < 2e-16
era_cat1900-1969 2.334654 0.889034 2.626 0.00869 era_cat1969+
1.709035 1.004416 1.702 0.08898 .
TEAM_BATTING_H 0.055323 0.001982 27.919 < 2e-16
TEAM_BATTING_2B 0.010082 0.004848 2.080 0.03768 *
TEAM_BATTING_3B 0.010279 0.009500 1.082 0.27934
TEAM_BATTING_HR 0.104601 0.005724 18.272 < 2e-16
TEAM_BATTING_BB 0.047301 0.001851 25.552 < 2e-16
TEAM_BATTING_SO -0.011012 0.001446 -7.615 3.78e-14
TEAM_BASERUN_SB 0.023154 0.002559 9.049 < 2e-16
TEAM_PITCHING_H -0.056791 0.001451 -39.132 < 2e-16
TEAM_PITCHING_HR -0.091167 0.006374 -14.302 < 2e-16
TEAM_PITCHING_BB -0.057994 0.001858 -31.213 < 2e-16
TEAM_PITCHING_SO 0.009377 0.001394 6.727 2.17e-11
TEAM_FIELDING_E -0.002432 0.001788 -1.360 0.17396
TEAM_FIELDING_DP 0.051356 0.007505 6.843 9.81e-12 * — Signif.
codes: 0 ‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ’ ’
1
Residual standard error: 6.718 on 2373 degrees of freedom Multiple
R-squared: 0.8139, Adjusted R-squared: 0.8127 F-statistic: 691.7 on 15
and 2373 DF, p-value: < 2.2e-16
Call: lm(formula = TARGET_WINS ~ era_cat + TEAM_BATTING_H +
TEAM_BATTING_2B + TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO +
TEAM_BASERUN_SB + TEAM_PITCHING_H + TEAM_PITCHING_HR + TEAM_PITCHING_BB
+ TEAM_PITCHING_SO + TEAM_FIELDING_E + TEAM_FIELDING_DP, data =
trainingTeam_df, by = era_cat)
Residuals: Min 1Q Median 3Q Max -41.265 -4.045 0.045 4.136 69.568
Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 73.492304 3.427890 21.440 < 2e-16
era_cat1900-1969 2.251512 0.885740 2.542 0.0111
era_cat1969+ 1.438109 0.972743 1.478 0.1394
TEAM_BATTING_H 0.056184 0.001815 30.959 < 2e-16
TEAM_BATTING_2B 0.009789 0.004841 2.022 0.0433
TEAM_BATTING_HR 0.103579 0.005646 18.345 < 2e-16
TEAM_BATTING_BB 0.047327 0.001851 25.567 < 2e-16
TEAM_BATTING_SO -0.010805 0.001433 -7.538 6.75e-14
TEAM_BASERUN_SB 0.023901 0.002464 9.701 < 2e-16
TEAM_PITCHING_H -0.056764 0.001451 -39.118 < 2e-16
TEAM_PITCHING_HR -0.092139 0.006311 -14.600 < 2e-16
TEAM_PITCHING_BB -0.057846 0.001853 -31.216 < 2e-16
TEAM_PITCHING_SO 0.009269 0.001391 6.666 3.26e-11
TEAM_FIELDING_E -0.002638 0.001778 -1.484 0.1380
TEAM_FIELDING_DP 0.050746 0.007484 6.781 1.50e-11 *** — Signif. codes: 0
‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ’ ’ 1
Residual standard error: 6.718 on 2374 degrees of freedom Multiple
R-squared: 0.8138, Adjusted R-squared: 0.8127 F-statistic: 740.9 on 14
and 2374 DF, p-value: < 2.2e-16
The TEAM_BATTING_3B variable was dropped due to a low p-values,
giving us the final model with an \(AdjR^2=0.8078\).
Call: lm(formula = TARGET_WINS ~ era_cat + TEAM_BATTING_H +
TEAM_BATTING_2B + TEAM_BATTING_HR + TEAM_BATTING_BB + TEAM_BATTING_SO +
TEAM_BASERUN_SB + TEAM_PITCHING_H + TEAM_PITCHING_HR + TEAM_PITCHING_BB
+ TEAM_PITCHING_SO + TEAM_FIELDING_E + TEAM_FIELDING_DP, data =
trainingTeam_df, by = era_cat)
Residuals: Min 1Q Median 3Q Max -41.265 -4.045 0.045 4.136 69.568
Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 73.492304 3.427890 21.440 < 2e-16
era_cat1900-1969 2.251512 0.885740 2.542 0.0111
era_cat1969+ 1.438109 0.972743 1.478 0.1394
TEAM_BATTING_H 0.056184 0.001815 30.959 < 2e-16
TEAM_BATTING_2B 0.009789 0.004841 2.022 0.0433
TEAM_BATTING_HR 0.103579 0.005646 18.345 < 2e-16
TEAM_BATTING_BB 0.047327 0.001851 25.567 < 2e-16
TEAM_BATTING_SO -0.010805 0.001433 -7.538 6.75e-14
TEAM_BASERUN_SB 0.023901 0.002464 9.701 < 2e-16
TEAM_PITCHING_H -0.056764 0.001451 -39.118 < 2e-16
TEAM_PITCHING_HR -0.092139 0.006311 -14.600 < 2e-16
TEAM_PITCHING_BB -0.057846 0.001853 -31.216 < 2e-16
TEAM_PITCHING_SO 0.009269 0.001391 6.666 3.26e-11
TEAM_FIELDING_E -0.002638 0.001778 -1.484 0.1380
TEAM_FIELDING_DP 0.050746 0.007484 6.781 1.50e-11 *** — Signif. codes: 0
‘’ 0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ’ ’ 1
Residual standard error: 6.718 on 2374 degrees of freedom Multiple
R-squared: 0.8138, Adjusted R-squared: 0.8127 F-statistic: 740.9 on 14
and 2374 DF, p-value: < 2.2e-16
There are a number of issues with the model diagnostics. The
Linearity, and QQ plots show deviation from the straight line at the
boundaries. In addition, the reference line for the Homogeneity of
Variance graph is not flat. This indicates that residual variance is not
consistent across the model values. Further exploration could be
conducted to see if there are smaller date ranges and prediction ranges
that generate a model that better aligns with the linear regression
assumptions.
PREDICT
The next step is to use the selected model to predict the testing
data set. The Yardstick package was used to calculate model performance,
including the \(R^2\). With an \(R^2=0.8318\) the performance on the testing
data set is consistent with the model summary. In the Residual
vs. TARGET_WINS graph, there appears to be wins range between 60 and 110
that generates more accurate forecasts. However, it should be noted that
the residuals appear to drift downwards. TARGET_WINS less than the mean
are overestimated and TARGET_WINS greater than the mean is over
estimated.
yearID era_cat TARGET_WINS TEAM_BATTING_H
"" "" "" ""
TEAM_BATTING_2B TEAM_BATTING_3B TEAM_BATTING_HR TEAM_BATTING_BB “” “”
“” “” TEAM_BATTING_SO TEAM_BASERUN_SB TEAM_PITCHING_H TEAM_PITCHING_HR
“pmm” “pmm” “” “” TEAM_PITCHING_BB TEAM_PITCHING_SO TEAM_FIELDING_E
TEAM_FIELDING_DP “” “” “” “”
|
|
.metric
|
.estimator
|
.estimate
|
|
1
|
mape
|
standard
|
7.0172
|
|
2
|
smape
|
standard
|
6.8136
|
|
3
|
mase
|
standard
|
0.3122
|
|
4
|
mpe
|
standard
|
-1.0803
|
|
5
|
rmse
|
standard
|
6.7128
|
|
6
|
rsq
|
standard
|
0.7940
|
Appendix B: Pythagorean Model
Bill James developed the Pythagorean winning percentage. The concept
strives to calculate the number of games a team should win based on the
total offense and the number of runs allowed. Since this model includes
total runs scored vs. total runs allowed the expectation is that this
model will be a good predictor of a team’s wins in a given season.
\(Win Percentage = (Runs Scored)^2 / [
(Runs Scored)^2 + (Runs Allowed)^2]\)
As we expected, the Pythagorean model performs well \(Adj R^2\) = 0.9112. However, this linear
model differs slightly from the formal definition since it includes an
intercept of 5.21 and a coefficient for the Pythagorean factor of
151.39. The formal definition of the model would have an intercept of 0
and a coefficient of 162.
Call: lm(formula = TARGET_WINS ~ pythPercent, data =
trainingTeam_df)
Residuals: Min 1Q Median 3Q Max -38.657 -3.006 0.090 3.103 18.395
Coefficients: Estimate Std. Error t value Pr(>|t|)
(Intercept) 5.016 0.490 10.24 <2e-16 pythPercent 151.702
0.964 157.36 <2e-16 — Signif. codes: 0 ‘’
0.001 ’’ 0.01 ’’ 0.05 ‘.’ 0.1 ’ ’ 1
Residual standard error: 4.609 on 2387 degrees of freedom Multiple
R-squared: 0.9121, Adjusted R-squared: 0.912 F-statistic: 2.476e+04 on 1
and 2387 DF, p-value: < 2.2e-16
The diagnostic plots show residuals that are normally distributed
with a linear relationship to the fitted values. The homogeneity of
variance is curved, indicating that the variance of residuals is not
consistent.
The next step is to use the Pythagorean Model to predict the testing
data set. The Yardstick package was used to calculate model performance,
including the \(R^2\). With an \(R^2=0.9131\) the performance on the testing
data set is consistent with the model summary. In the Residual
vs. TARGET_WINS graph there appears to be a wins range between 50 and
130 that generates more accurate forecasts.
|
|
.metric
|
.estimator
|
.estimate
|
|
1
|
mape
|
standard
|
Inf
|
|
2
|
smape
|
standard
|
4.7570
|
|
3
|
mase
|
standard
|
0.2086
|
|
4
|
mpe
|
standard
|
-Inf
|
|
5
|
rmse
|
standard
|
4.4718
|
|
6
|
rsq
|
standard
|
0.9078
|
Unsurprising the Pythagorean Model performs well when it comes to win
projections. There is clear relationship between the total yearly run
differential and the number of games won. It is a simple model but
efficient.
