MATH2349 Semester 1, 2019

Assignment 3

Required packages

library(fitzRoy) # for data
library(tidyr)
library(dplyr)
library(plyr)
library(mlr)
library(MVN)
library(forecast)

Executive Summary

This report contains data of AFL seasons 2010-2018. Two datasets, one containing detailed stats and the other match results, were merged and tidied to enable easy analysis or modelling.

This involved ensuring varibales were of the correct class and changing a couple of variables to the correct class. Then, the data was converted to a tidy format and was scanned for any obvious errors such as missing or incorrect values.

Following this, the data was checked for outliers and these values were removed. Finally, the variables for goals was transformed using a square root transformation to make the data symmetrical and normal as oppsed to positively skewed.

Data

The data is from the fitzRoy package in R, which is a package to source AFL data. The player_stats data is sourced from footywire.com via the fitzRoy package and contains statistics of individual players in each AFL match from 2010-2018. These stats are grouped so that each row is the total for each team in each match.

The match_results data is sourced using the fitzRoy package and contains the match results of every VFL/AFL match since 1897 from afltables.com. This is then filtered to include the same date range as the player data (from 2010-2018).

The two datasets are merged so that the team totals for each stat are combined with the result of the match. This means that each row contains the data of a team’s performance for each match from 2010-2018 and their total stats for that match. Some team names need to be changed in each of the original datasets before merging to ensure all levels are of the same name.

The variables used in the final dataset are as follows: * Game: ID for individual match * Date: date of the match * Round: Name of round match was played in * Venue: Name of ground match was played at * Margin: Margin in the match (i.e. points scored minus points against) * Season: The year the match was played * Round.Type: Whether the match was played in the regular season or the finals * Round.Number: The number of the round * Status: Whether team was playing at home or away * Behinds: Number of behinds kicked in match * Goals: Number of goals kicked in match * Points: Team score for the match * Team: Team’s name * The following variables are team stats: CP (contested possessions), UP (uncontested possessions), ED (effective disposals), DE (disposal efficiency), CM (contested marks), GA (goal assists), MI5 (marks inside 50), One.Percenters, BO (bounces), K (kicks), HB (handballs), D (disposals), M (marks), T (tackles), HO (hitouts), I50 (inside 50s), CL (clearances), CG (clangers), R50 (rebound 50s), FF (frees for), FA (frees against), AF (AFL Fantasy points), SC (Supercoach points).

Further information on how these team stats are collected can be found here: http: //www.afl.com.au/news/2017-12-28/stats-glossary-every-stat-explained

player.match.stats <- fitzRoy::player_stats
head(player.match.stats)

results <- fitzRoy::match_results
head(results)

results2010 <- results %>% filter(Season > 2009)
grouped.player.stats <- player.match.stats %>% dplyr::group_by(Match_id, Team, Date) %>% dplyr::summarise(CP = sum(CP),
                   UP = sum(UP),
                   ED = sum(ED),
                   DE = mean(DE),
                   CM = sum(CM),
                   GA = sum(GA),
                   MI5 = sum(MI5),
                   One.Percenters = sum(One.Percenters),
                   BO = sum(BO),
                   K = sum(K),
                   HB = sum(HB),
                   D = sum(D),
                   M = sum(M),
                   T = sum(T),
                   HO = sum(HO),
                   I50 = sum(I50),
                   CL = sum(CL),
                   CG = sum(CG),
                   R50 = sum(R50),
                   FF = sum(FF),
                   FA = sum(FA),
                   AF = sum(AF),
                   SC = sum(SC))
grouped.player.stats$Team <- as.factor(grouped.player.stats$Team)
grouped.player.stats$Team <- revalue(grouped.player.stats$Team, c("Brisbane"="Brisbane Lions"))
head(grouped.player.stats)

results2010 <- convert_results(results2010)
results2010$Team <- as.factor(results2010$Team)
results2010$Team <- revalue(results2010$Team, c("Footscray"="Western Bulldogs"))
Results_player <- left_join(results2010, grouped.player.stats, by = c("Date","Team"))

Column `Team` joining factors with different levels, coercing to character vector

Results_player$Match_id <- NULL
head(Results_player)

Understand

Firstly, the structure of the data frame is analysed to see the class of each variable. The goals, behinds and points all need to be changed from character to numeric. The team variable also needs to be changed to be a factor and the levels checked to see that their are 18 and all 18 teams are represented. Other than that, the variables are all of the correct class. The data includes multiple variable types: numeric, date, character, integer and factor.

str(Results_player)

Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   3660 obs. of  36 variables:
 $ Game          : num  13578 13578 13579 13579 13580 ...
 $ Date          : Date, format: "2010-03-25" "2010-03-25" ...
 $ Round         : chr  "R1" "R1" "R1" "R1" ...
 $ Venue         : chr  "M.C.G." "M.C.G." "M.C.G." "M.C.G." ...
 $ Margin        : num  -56 56 31 -31 -56 56 -8 8 32 -32 ...
 $ Season        : num  2010 2010 2010 2010 2010 2010 2010 2010 2010 2010 ...
 $ Round.Type    : chr  "Regular" "Regular" "Regular" "Regular" ...
 $ Round.Number  : int  1 1 1 1 1 1 1 1 1 1 ...
 $ Status        : chr  "Home" "Away" "Home" "Away" ...
 $ Behinds       : chr  "10" "12" "11" "16" ...
 $ Goals         : chr  "9" "18" "19" "13" ...
 $ Points        : chr  "64" "120" "125" "94" ...
 $ Team          : chr  "Richmond" "Carlton" "Geelong" "Essendon" ...
 $ CP            : int  119 129 130 150 124 126 144 130 138 123 ...
 $ UP            : int  170 302 304 190 224 285 226 204 232 258 ...
 $ ED            : int  198 336 324 224 246 302 268 241 265 282 ...
 $ DE            : num  66 76.6 76.1 67 72 ...
 $ CM            : int  7 13 8 12 10 11 8 8 12 7 ...
 $ GA            : int  2 13 13 4 6 9 9 10 12 7 ...
 $ MI5           : int  7 16 17 11 10 16 10 5 20 9 ...
 $ One.Percenters: int  36 54 49 49 42 46 49 43 60 44 ...
 $ BO            : int  3 9 9 9 10 6 4 7 4 7 ...
 $ K             : int  166 245 206 198 175 247 196 199 209 198 ...
 $ HB            : int  129 187 230 144 176 166 183 140 169 194 ...
 $ D             : int  295 432 436 342 351 413 379 339 378 392 ...
 $ M             : int  66 136 95 73 83 133 73 81 85 90 ...
 $ T             : int  59 78 74 74 83 75 62 77 66 52 ...
 $ HO            : int  26 44 54 31 36 25 50 36 38 35 ...
 $ I50           : int  45 57 54 48 44 62 48 45 62 55 ...
 $ CL            : int  33 34 34 47 29 31 44 33 33 37 ...
 $ CG            : int  45 53 55 43 47 48 40 44 42 38 ...
 $ R50           : int  36 34 30 31 39 34 30 33 43 40 ...
 $ FF            : int  23 18 13 25 23 20 29 18 15 14 ...
 $ FA            : int  17 23 25 13 20 23 18 29 14 15 ...
 $ AF            : int  1250 1938 1774 1503 1516 1863 1532 1490 1607 1543 ...
 $ SC            : int  1321 1986 1775 1523 1470 1829 1661 1637 1691 1609 ...

#Results_player$Game <- as.factor(Results_player$Game)
Results_player$Behinds <- as.numeric(Results_player$Behinds)
Results_player$Goals <- as.numeric(Results_player$Goals)
Results_player$Points <- as.numeric(Results_player$Points)
Results_player$Team <- as.factor(Results_player$Team)
levels(Results_player$Team)

 [1] "Adelaide"         "Brisbane Lions"   "Carlton"          "Collingwood"     
 [5] "Essendon"         "Fremantle"        "Geelong"          "Gold Coast"      
 [9] "GWS"              "Hawthorn"         "Melbourne"        "North Melbourne" 
[13] "Port Adelaide"    "Richmond"         "St Kilda"         "Sydney"          
[17] "West Coast"       "Western Bulldogs"

Tidy & Manipulate Data I

The data was not originially in a tidy format as each of the team statistics could be rearranged into one column.

To convert the data to the tidy data format, the totals for each team statistic are rearranged to be in two different columns, one which contains the type of statistic and the other containing the team’s total for that stat.

After the data is reshaped like this, it conforms to tidy data principles: * Each variable has its own column * Each observation has its own row * Each value has its own cell

Results_player <- Results_player %>% gather(Statistic, Stat_Total, 14:36)
head(Results_player)

Tidy & Manipulate Data II

A new variable, ‘Result’, is created to display whether a team has won, drawn or lost the match. To do this, the margin of the match is looked at and if positive the result is given as W, if 0 then D and if negative L and assigned as a factor. This means that each observation now contains the result of the match as simple, win, loss or draw which could be useful for further analysis or modelling.

Results_player<- Results_player %>% mutate(
  Result = as.factor(ifelse(Margin > 0, "W", ifelse(Margin == 0, "D", "L"))
))
head(Results_player)

Scan I

There are no missing values. There are 18 team, 29 Round, 2 Status and 2 Round.Type levels as is to be expected. The number of venues (22) is also a reasonable number and would be expected. The mean and median of zero for the margin is also good as it shows it is likely each match has both teams represented in the data. Overall, there are no obvious errors or inconsistencies in the data that need to be dealt with.

summarizeColumns(Results_player)

NAs introduced by coercion

Scan II

First a boxplot of the stat totals are looked at to see if there are any outliers, but it is clear that each level needs to be looked at individually. So a boxplot is made for each of these stats, which shows there are potential univariate outliers for each of these. However, because this is multivariate data, a multivariate technique was used to identify outliers. 44 outliers were identified in this way, the decision was made to remove these from the data because there are only a small number of them in relation to the dataset as a whole.

Results_player$Stat_Total %>%  boxplot(main="Box Plot of All Stat Totals", ylab="Number", col = "grey")

boxplot(Results_player$Stat_Total ~ Results_player$Statistic, main="Stat Total By Stat Type", ylab = "Number", xlab = "Statistic")

#Results_player <- arrange(Results_player, Game)
Results_player_wide <- Results_player %>% spread(Statistic, Stat_Total)
Results_player_numeric <- Results_player_wide[,15:37]
Results_player_numeric_clean <- mvn(data = Results_player_numeric, multivariateOutlierMethod = "quan", showOutliers = TRUE, showNewData = TRUE)

The covariance matrix has become singular during
the iterations of the MCD algorithm.
There are 3660 observations (in the entire dataset of 3660 obs.) lying on
the hyperplane with equation a_1*(x_i1 - m_1) + ... + a_p*(x_ip - m_p) = 0
with (m_1, ..., m_p) the mean of these observations and coefficients a_i
from the vector a <- c(0, 0, 0, 0, 0, 0, -0.5773503, 0, 0, 0, 0, 0,
0.5773503, 0, 0, 0.5773503, 0, 0, 0, 0, 0, 0, 0)

Outliers$multivariateOutliers

Results_player_wide_clean <- Results_player_wide[-c(1:44),]
dim(Results_player_wide)

[1] 3660   37

dim(Results_player_wide_clean)

[1] 3616   37

Results_player_clean <- Results_player_wide_clean %>% gather(Statistic, Stat_Total, 15:37)

Transform

The goals variable was used to transform. Initially, the histogram shows that it has a positive skew and so a transformation may be beneficial to make this variable easier to analyse.

First, a boxcox transformation was attempted to make the goals variable normal.

hist(Results_player_wide_clean$Goals)

boxcox_Goals<- BoxCox(Results_player_wide_clean$Goals,lambda = "auto")
hist(boxcox_Goals)

BoxCox transformation didn’t work in making the goals variable symmetrical and so another type of transformation will be attempted to make the goals variable symmetrical.

The square root transformation was chosen because of the positive skewing and this had a much better effect, results in a relatively normal distribution.

sqrt_Goals <- sqrt(Results_player_wide_clean$Goals)
hist(sqrt_Goals)