Glimpse of 2019 Dataset (Only First 100 Games):

Codebook (defines baseball and dataset-specific terms):

The dataset above encompasses and emphasizes pitch-level data, which includes pertinent information about the trajectory of each pitch. The dataset is joined together with matching data for the at-bats and games that sync up with each pitch. I utilized the join function to merge the datasets as part of the data cleaning process. This ensures a complete picture of the various game situations.

Each observation represents a specific pitch while all the column variables describe characteristics of the result of the pitch and the pitch itself at that point in time. This includes but is not limited to the current score, game situation (score, number of outs, etc.), pitch type, pitch speed, batter’s count, and result of the pitch. The data table above displays a quick glimpse at the first 25 MLB games recorded for the 2019 season and all pitches thrown in each of those games. However, there are 2,408 distinct games included in the entire dataset for the 2019 regular season. I am not using the entire dataset due to size and time issues when processing, but 25 games including all 30 different MLB teams at least should be enough variability to produce an decently accurate model. The data was scraped from the official site MLB.com but was imported from Kaggle for simplicity (https://www.kaggle.com/pschale/mlb-pitch-data-20152018?select=pitches.csv).

The main reason I chose this data is because I am current baseball player and enthusiast and find great value in using predictive analysis to predict player decisions, especially when it comes to predicting pitching strategies. In fact, I am very interested in pursuing a career in sports data analysis or sports technology, so I thought a great way to start building experience would be to complete this report using MLB data.

Distribution of Outcome Variables

Using the full dataset described above, I chose to explore the distribution of multiple predicted outcome variables: pitch_type, break_angle, start_speed, px, and pz (see codebook for further clarification). First, I perform a quick skim of the data and note any potential problems (such as missingness).

Variable type: character

Variable type: factor

Variable type: numeric

We notice here that there are 6,654 observations in which there is missing data for all our outcome variables. This is likely due to the failure to record the metrics for some pitches. In this case, we can drop the observations from the main dataset since they do not provide much useful information.

Also, it will be useful to see the distribution of the continuous outcome variables:

## Warning: Removed 2 rows containing missing values (geom_bar).

## Warning: Removed 2 rows containing missing values (geom_bar).

Furthermore, I found the distribution of the discrete outcome variable pitch_type:

The last bar distribution plot tells us that the fastball is the most common type of pitch (as expected). It is also no surprise that most of the pitches end up being close to the middle of the strike zone as seen from the px and pz histogram plots. What is more interesting is that the most common speed of pitches are around 90 - 95 mph and most common break angles are below 10 degrees (close to a straight fastball but not quite).

Q-Q Plots

Q-Q plots are primarily utilized as a graphical tool to help us assess if a set of data came from some theoretical normal distribution. If we assume our outcome variable is normally distributed, a Q-Q plot can verify this assumption:

Break Angles

Pitch Speed

Pitch Location X (outside, middle, or inside)

Pitch Location Z (high, middle, or low)

The Q-Q plots tell us that both p_x and p_z have normal distributions while start_speed and break_angle do not.

Correlation Between Variables

3-0 COUNT

2-0 COUNT, 3-1 COUNT

1-0 COUNT, 2-1 COUNT, 3-2 COUNT

EVEN COUNT

0-1 COUNT, 1-2 COUNT

0-2 COUNT

In summary, the above plots show that there is a slight relationship between at-bat count, inning number, and score differential with the pitching type and metrics. In order to explore this relationship more in-depth, I decided to generate the best model possible to predict the break angle and speed of a pitch given the inning, batter’s count differential, score differential, and number of outs. This is addressed in the next section.

Assessment of Both Models

Random Forest tuning plots, shows how node size and number of predictors affects the RMSE value which is value used to evaluate how good a model is at predictions.

k-nearest neighbor model tuning plots for both break angle and speed of pitch, show the effects of number of neighbors on the RMSE value

Tuning results for pitch speed (outcome variable)

## # A tibble: 20 x 10
##    model_type  mtry min_n .metric .estimator  mean     n std_err .config
##    <chr>      <int> <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>  
##  1 rf             2    21 rmse    standard    5.95    50  0.0224 Prepro~
##  2 rf             2    30 rmse    standard    5.95    50  0.0220 Prepro~
##  3 rf             2    40 rmse    standard    5.96    50  0.0222 Prepro~
##  4 rf             2    11 rmse    standard    5.96    50  0.0233 Prepro~
##  5 rf             3    40 rmse    standard    5.97    50  0.0228 Prepro~
##  6 rf             4    40 rmse    standard    5.97    50  0.0231 Prepro~
##  7 rf             3    30 rmse    standard    5.98    50  0.0230 Prepro~
##  8 rf             4    30 rmse    standard    5.99    50  0.0233 Prepro~
##  9 rf             2     2 rmse    standard    5.99    50  0.0246 Prepro~
## 10 rf             3    21 rmse    standard    5.99    50  0.0232 Prepro~
## 11 rf             4    21 rmse    standard    6.01    50  0.0234 Prepro~
## 12 rf             3    11 rmse    standard    6.04    50  0.0245 Prepro~
## 13 rf             4    11 rmse    standard    6.08    50  0.0253 Prepro~
## 14 rf             3     2 rmse    standard    6.16    50  0.0276 Prepro~
## 15 rf             4     2 rmse    standard    6.24    50  0.0293 Prepro~
## 16 knn           NA    NA rmse    standard    6.28    50  0.0241 Prepro~
## 17 knn           NA    NA rmse    standard    6.34    50  0.0246 Prepro~
## 18 knn           NA    NA rmse    standard    6.44    50  0.0252 Prepro~
## 19 knn           NA    NA rmse    standard    6.80    50  0.0305 Prepro~
## 20 knn           NA    NA rmse    standard    8.05    50  0.0391 Prepro~
## # ... with 1 more variable: neighbors <int>

**Tuning results for break angle (outcome variable)*

## # A tibble: 20 x 10
##    model_type  mtry min_n .metric .estimator  mean     n std_err .config
##    <chr>      <int> <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>  
##  1 rf             2    40 rmse    standard    12.9    50  0.0405 Prepro~
##  2 rf             2    30 rmse    standard    12.9    50  0.0407 Prepro~
##  3 rf             3    40 rmse    standard    12.9    50  0.0419 Prepro~
##  4 rf             4    40 rmse    standard    13.0    50  0.0417 Prepro~
##  5 rf             2    21 rmse    standard    13.0    50  0.0414 Prepro~
##  6 rf             3    30 rmse    standard    13.0    50  0.0417 Prepro~
##  7 rf             4    30 rmse    standard    13.0    50  0.0418 Prepro~
##  8 rf             2    11 rmse    standard    13.0    50  0.0413 Prepro~
##  9 rf             3    21 rmse    standard    13.1    50  0.0422 Prepro~
## 10 rf             4    21 rmse    standard    13.1    50  0.0425 Prepro~
## 11 rf             2     2 rmse    standard    13.1    50  0.0417 Prepro~
## 12 rf             3    11 rmse    standard    13.2    50  0.0426 Prepro~
## 13 rf             4    11 rmse    standard    13.4    50  0.0426 Prepro~
## 14 knn           NA    NA rmse    standard    13.5    50  0.0435 Prepro~
## 15 rf             3     2 rmse    standard    13.5    50  0.0452 Prepro~
## 16 knn           NA    NA rmse    standard    13.7    50  0.0439 Prepro~
## 17 rf             4     2 rmse    standard    13.7    50  0.0468 Prepro~
## 18 knn           NA    NA rmse    standard    13.9    50  0.0483 Prepro~
## 19 knn           NA    NA rmse    standard    14.8    50  0.0564 Prepro~
## 20 knn           NA    NA rmse    standard    17.5    50  0.0638 Prepro~
## # ... with 1 more variable: neighbors <int>

We can find the lowest RMSE score for both models and compare which one is lower. RMSE is the square root of the mean of the square of all of the error in the dataset given and is an excellent performance measure or error metric for numerical predictions. The lower the RMSE the more ideal.

Tuned Random Forest model for pitching speed

## # A tibble: 1 x 8
##    mtry min_n .metric .estimator  mean     n std_err .config              
##   <int> <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>                
## 1     2    21 rmse    standard    5.95    50  0.0224 Preprocessor1_Model07

Tuned Random Forest model for break angle

## # A tibble: 1 x 8
##    mtry min_n .metric .estimator  mean     n std_err .config              
##   <int> <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>                
## 1     2    40 rmse    standard    12.9    50  0.0405 Preprocessor1_Model13

Tuned k-nearest neighbor model for pitching speed

## # A tibble: 1 x 7
##   neighbors .metric .estimator  mean     n std_err .config             
##       <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
## 1        15 rmse    standard    6.28    50  0.0241 Preprocessor1_Model5

Tuned k-nearest neighbor model for break angle

## # A tibble: 1 x 7
##   neighbors .metric .estimator  mean     n std_err .config             
##       <int> <chr>   <chr>      <dbl> <int>   <dbl> <chr>               
## 1        15 rmse    standard    13.5    50  0.0435 Preprocessor1_Model5

## == Workflow [trained] ==========================================================
## Preprocessor: Recipe
## Model: rand_forest()
## 
## -- Preprocessor ----------------------------------------------------------------
## 0 Recipe Steps
## 
## -- Model -----------------------------------------------------------------------
## Ranger result
## 
## Call:
##  ranger::ranger(x = maybe_data_frame(x), y = y, mtry = min_cols(~2L,      x), min.node.size = min_rows(~21L, x), importance = ~"impurity",      num.threads = 1, verbose = FALSE, seed = sample.int(10^5,          1)) 
## 
## Type:                             Regression 
## Number of trees:                  500 
## Sample size:                      5712 
## Number of independent variables:  4 
## Mtry:                             2 
## Target node size:                 21 
## Variable importance mode:         impurity 
## Splitrule:                        variance 
## OOB prediction error (MSE):       34.85219 
## R squared (OOB):                  0.1211724

## == Workflow [trained] ==========================================================
## Preprocessor: Recipe
## Model: rand_forest()
## 
## -- Preprocessor ----------------------------------------------------------------
## 0 Recipe Steps
## 
## -- Model -----------------------------------------------------------------------
## Ranger result
## 
## Call:
##  ranger::ranger(x = maybe_data_frame(x), y = y, mtry = min_cols(~2L,      x), min.node.size = min_rows(~40L, x), importance = ~"impurity",      num.threads = 1, verbose = FALSE, seed = sample.int(10^5,          1)) 
## 
## Type:                             Regression 
## Number of trees:                  500 
## Sample size:                      5711 
## Number of independent variables:  4 
## Mtry:                             2 
## Target node size:                 40 
## Variable importance mode:         impurity 
## Splitrule:                        variance 
## OOB prediction error (MSE):       167.0845 
## R squared (OOB):                  0.02129066

After finalizing the workflow, I discovered that the Random Forest model produced the most optimal RMSE score for the training data. More specifically, 500 trees were used with an mtry value of 2 and target node size of 40. We can thus use this information to produce the most optimal estimated RMSE values for both speed of the pitch and break angle. This model will then be used to be tested on the testing set I created earlier in the report.

## # A tibble: 1 x 3
##   .metric .estimator .estimate
##   <chr>   <chr>          <dbl>
## 1 rmse    standard        5.89

## # A tibble: 1 x 3
##   .metric .estimator .estimate
##   <chr>   <chr>          <dbl>
## 1 rmse    standard        12.8

Model Performance Evaluation

The RMSE values of 6.018 and 12.948 are less than ideal since typically we see RMSE values between 0 and 1. This is likely due to the fact that it may have been more beneficial to log transform the outcome variables in the dataset since RMSE score is correlated with the range of the dependent variable being measured. Nevertheless, the random forest model actually performed slightly better on the test dataset, with a 0.07 lower RMSE value for the speed of the pitch and 0.03 lower RMSE value for the break angle, which was surprising to me considering I expected a little bit of overfitting.