Multi-Check ELO

Note: For this to work, you’ll first need to make sure that you’ve created all of the functions we created in our ELO labs.

The Multi-Check Function

MultiCheckElo <- function(K.vector, Kmargin.vector, R.vector, gw.power.vector, start.season=2009, 
              end.season=2016, start.check.season=2012){
  
  full.results <- data.frame(K = as.numeric(), Kmargin=as.numeric(), R = as.numeric(), 
                             gw.power = as.numeric(), rmse=as.numeric(), logloss=as.numeric())
  for (k in seq_along(K.vector)){
    for (km in seq_along(Kmargin.vector)){
      for (r in seq_along(R.vector)){
        for (g in seq_along(gw.power.vector)){
          results <- CheckELO(K=K.vector[k], Kmargin=Kmargin.vector[km], 
                              R=R.vector[r], start.season=start.season, end.season=end.season, 
                              gw.power=gw.power.vector[g])
          results.sum <- results %>% filter(season>=start.check.season) %>% 
            summarize(rmse = mean(RMSE), logloss = mean(LogLoss)) %>% as.numeric()
          full.results[nrow(full.results)+1, ] <- c(K.vector[k], Kmargin.vector[km], R.vector[r], gw.power.vector[g], results.sum)
          
        }
      }
    }
  }
  return(full.results)
}

Putting it into Action

Now, you can let R evaluate a vector of values for each of the parameters we’re interesting in: K, Kmargin, R and gw.power. This function will check every combination of possible parameter values you list:

full.results <- MultiCheckElo(K.vector=c(30,35), Kmargin.vector=c(10,15), R.vector=c(0,0.1), gw.power.vector=0.1, start.season=2009, 
                          end.season=2016, start.check.season=2012)

View(full.results)
full.results %>% top_n(10, desc(logloss)) %>% arrange(logloss)

Sample Results (from the code above):

#    K Kmargin   R gw.power      rmse   logloss
# 1 35      15 0.0      0.1 0.4353586 0.5603182
# 2 30      15 0.0      0.1 0.4354802 0.5605334
# 3 35      10 0.0      0.1 0.4354711 0.5606060
# 4 30      10 0.0      0.1 0.4357794 0.5611965
# 5 35      15 0.1      0.1 0.4401201 0.5695179
# 6 35      10 0.1      0.1 0.4403768 0.5701256
# 7 30      15 0.1      0.1 0.4403667 0.5702177
# 8 30      10 0.1      0.1 0.4408119 0.5712367

• Note that there’s both a start.season and start.check.season, so that you can start computing ELO in an earlier year but filter out the first few years (which are likely less accurate) where ELO is just building up steam.

• Before trying this with a set of long vectors you should think about how long it will take. My laptop checked the 2x2x2x1 = 8 combinations listed above in 1.6 minutes which means that it could check 300 combinations in an hour. Our server will take longer, particularly if several of us our running this simultaneously. I strongly urge you not to attempt more than 300 combinations in one night. Of course, the numbers of possible parameters multiply so if you check 5 values of K, 5 values of Kmargin, 4 values of R and 3 values of gw.power that’s already 5x5x4x3 = 300 combinations.