Comparative performance of models to forecast match wins in professional tennis

Predicting Wins in Tennis

• Forecasting the winner of a competition is a fascination of all sports

• Tennis has a long history of interest (and frustration) with predicting match outcomes

• For most of its history, tennis has lacked formal models to predict wins

• Around 2000, the first quantitative approaches for prediction began to appear

• Since then a number of models have been published

• In this talk, I will categorize models into 3 broad classes:

  • Point-based
  • Regression-based
  • Other

Though these forecasts are fascinating, what is really needed is a comparison of the forecasting accuracy of various methods. Then we can judge which forecasts to use in our fantasy leagues, draft decisions, or player personnel decisions. For example, who predicts future baseball performance better: Ron Shandler or Bill James? Perhaps in the future databases of accuracy of sports forecasts (and stock pickers) will be commonplace on the Internet.

W. L. Winston, Mathletics: How Gamblers, Managers, and Sports Enthusiasts Use Mathematics in Baseball, Basketball, and Football

• As sports analysts and fans, we think a lot about performance and how athletes stack up against their competitors

• Yet, attention to the comparative performance of prediction models in sport is often absent (at least from scholarly literature)

• Tennis is no exception

• The purpose of this study was to identify published models to predict the winner of a professional singles match in tennis

• The study design had 3 phases:

  1. Search for prediction models
  2. Fitting model parameters from the same set of match training data
  3. Evaluating model performance against an independent set of match data

• Review of abstracts for relevant articles identified from Google Scholar and their citations

• 17 articles were screened with exclusion criteria

• 10 models were included in the validation analysis

• Causes for Exclusion

  • Not sufficient information to replicate (2 articles)
  • Not a forecasting model (2 articles)
  • Concerned previously published model (2 articles)
  • Required within-match updating (1 article)

• Four of the included models are or are extensions of the basic iid model proposed by Newton and Keller, which assumes that points won on serve and return are independent and identically distributed (iid) Bernoulli trials

• Given the probability of winning a point on serve and return, explicit formulae can be derived for the probability of winning a game, tiebreak, set, and match

• The models differ in how the probabilities of serve and return are estimated

• Four models propose a regression relationship between predictors and the probability of winning a match

• Unlike point-based models, these model a match win directly and make no assumptions about characteristics of points within a match

• The models differ in the set of predictors included though ALL include player rank/seeding in some form

• There were 2 additional models that do not fit within either the point- or regression-based classes

ROI = Return on investment

• Parameters were based on match outcomes for the 52 weeks prior to the match being predicted
• For predictors that specifically required a longer period than 1-year (e.g. 'career wins'), the specified time period was used
• Match data included all tournaments above the Challenger level

• Calibration

• Discrimination

  • AUC
  • Mean probability difference

• Prediction models in tennis are not created equal

• Regression-based models generally had better accuracy than point-based models

• Calibration was generally good but there was a wide range in discriminatory performance

• The Prize probit model had excellent accuracy and superior discriminatory power compared to competitors

• Factors like prize earnings and head-to-head outcomes have important predictive value even after account for player rankings