Predicting Quarterback Efficiency in the NFL
Author: Steven Patton | Twitter: @PattonAnalytics
August 12, 2022
Introduction
The past two Super Bowls have been won by quarterbacks that changed teams prior to the NFL season. Tom Brady left for Tampa Bay as a free agent (’20) and Los Angeles traded for Matthew Stafford (’21) en route to becoming World Champs. Should the immediate success of these two quarterbacks lead us to believe that Russell Wilson can do the same with a change of scenery in Denver? Could Matt Ryan’s fresh start with Indianapolis end with a trip to Glendale, AZ?
Heading into the ’21 NFL Draft, Cincinnati faced a tough decision; protect their young quarterback (coming off an ACL injury) or add an additional receiving threat. The controversy produced a meme that foreshadowed the end of Super Bowl LVI. \[\\[0.0005in]\]
Pass Protection v. Pass Catchers
Miami added wide receiver, Tyreek Hill, and left tackle, Terron Armstead, this offseason in hopes of improving the supporting cast around Tua Tagovailoa. Las Vegas traded for Derek Carr’s former college teammate, Davante Adams, but didn’t address their offensive line concerns. While Super Bowl appearances would be a success for the Dolphins and Raiders, the Bengals’ blueprint may not be a sustainable path to follow.
This raises an important question, does a team gain more by investing in the protection around the quarterback or the receivers he is throwing to? Team spending for these position groups may be correlated with quarterback performance. A quarterback’s NFL experience and prior efficiency could also provide some explanation on future production.
Framework
This analysis looks to predict Expected Points Added (EPA) per Drive for a quarterback with a minimum of 250 plays in the regular season from 2015 to 2021. Data is gathered from three sources; Over the Cap, nflfastR and nflreadr. There are a total of 237 seasons produced by 73 unique quarterbacks.
The salary cap hits for a team’s wide receivers and offensive line are represented in inflated salary cap dollars for 2022. A quarterback’s NFL experience is represented with the number of years in the NFL prior to the season. If the team hired a new coach or a quarterback changed teams they are identified with binary metrics.
The quarterback’s TOTAL EPA from the previous season is used to help explain his EPA per drive the following year. This helps eliminate variance for quarterbacks with very little playing time the year prior. Quarterbacks that did not play in the NFL the year prior are given a value of zero. \[\\[0.0005in]\] \[ EPA \sim B_{0} + B_{1}Exp + B_{2}lagEPA + B_{3}nHC + B_{4}nTM + B_{5}WR + B_{6}OL \] \[\\[0.0005in]\] EPA is a continuous variable that follows a normal distribution. For an accurate prediction a linear regression model is used to fit the data. The initial assumptions are that a quarterback’s experience, previous year’s efficiency, and positional spending will have a positive impact on EPA. A negative relationship will be expected if a quarterback changed teams or the team hired a new head coach.
Results
Before diving into the marginal effects of the explanatory metrics, we’ll look at how well the model adheres to the Ordinary Least Squares (OLS) assumptions. The graphic below visualizes how well the metrics predict the observed EPA per Drive (top left). The graphic also includes residual plots for linearity (top right), homogeneity (middle left), leverage (middle right), collinearity (bottom left) and normality (bottom right).\[\\[0.0005in]\]
\[\\[0.0005in]\] All the variables in the model are statistically significant at the 5% level. This means we can interpret the coefficients by unit increases of the explanatory metrics. For every additional year of NFL experience, a quarterback is expected to have an increase of 0.012 EPA. For every 50 expected points added the year prior a quarterback produces 0.1 EPA per drive more the following year.
A quarterback’s EPA drops on average by 0.184 per drive if the team hires a new head coach. If the quarterback changes teams; his EPA per drive is expected to fall by 0.172. These are the only variables in the model that have a negative relationship with a quarterback’s efficiency.
If a team invests 10 million dollars more towards their wide receivers or offensive line, the quarterback sees an increase of 0.09 and 0.14 EPA per drive, respectively. This means that quarterback’s see a higher return on EPA per drive when a team spends more on the offensive line in a given year. The marginal effects for each metric are displayed: \[\\[0.0005in]\] \[ EPA \sim -0.261 + 0.012Exp + 0.002lagEPA - 0.184nHC - 0.172nTM + 0.009WR + 0.014OL \] \[\\[0.0005in]\] The \(R^{2}\) and adjusted \(R^{2}\) (top left) are numeric representations of how well the model fits the data. Pearson’s correlation coefficient (bottom right) depicts the relationship between the predicted and the actual values of EPA. The average quarterback since 2015 has produced roughly ~ 0.32 EPA per Drive. \[\\[0.0005in]\]
Individual Performances
Quarterbacks who won the Most Valuable Player (MVP) award from 2015 to 2021 are displayed below. The player’s EPA and Predicted EPA (PEPA) per drive are listed and their rank in EPA Over Expectation is highlighted. The ranking is based on the 237 quarterback seasons observed. EPA over Expectation is the difference between EPA and PEPA per drive.
Before their MVP seasons, the model expected Newton, Mahomes, and Jackson to be average starters in the NFL (Mean EPA per Drive ~ 0.32). In comparison, Ryan, Brady and Rodgers (2) were predicted to have above average seasons prior to their MVP campaigns. \[\\[0.0005in]\]
| Most Valuable Players in the NFL since 2015 | ||||||
|---|---|---|---|---|---|---|
| EPA, PEPA, and Over Expectation Rank (OER) | ||||||
| Season | Name | Team | Drives | EPA | PEPA | OER |
| 2015 | C.Newton |  |
183 | 0.61 | 0.27 | 44 |
| 2016 | M.Ryan |  |
169 | 1.10 | 0.61 | 20 |
| 2017 | T.Brady |  |
165 | 1.00 | 0.66 | 45 |
| 2018 | P.Mahomes |  |
161 | 1.53 | 0.34 | 1 |
| 2019 | L.Jackson |  |
142 | 1.39 | 0.28 | 2 |
| 2020 | A.Rodgers |  |
148 | 1.44 | 0.66 | 4 |
| 2021 | A.Rodgers | |
149 | 1.00 | 0.94 | 96 |
\[\\[0.0005in]\] Aaron Rodger’s MVP season last year ranked 96th in EPA Over Expectation, even though his EPA per Drive ranked 16th. The model accounts for the prior year’s efficiency, which partially explains why Rodgers was expected to have another stellar year. This highlights how difficult it is to repeat as MVP. The graph below displays the top 25 seasons in EPA over Expectation. Matthew Stafford was the only quarterback from 2021 to rank. \[\\[0.0005in]\]
\[\\[0.0005in]\] There have been five quarterback seasons during this time span that averaged at least 1.3 EPA per Drive. Patrick Mahomes is responsible for three of those seasons!! Mahomes (3), Rodgers (2), Prescott (2) and Tannehill (2) are the only players to have multiple seasons ranked in the top 25.
40% of the quarterbacks who had season(s) in the top 25 also had season(s) in the bottom 25. Notable quarterbacks who had years remarkably under expectation were Rodgers, Ryan, Prescott, and Tannehill. The remaining quarterbacks that found themselves in both extremes; Roethlisberger, Wentz, Goff, and Trubisky.\[\\[0.0005in]\]
\[\\[0.0005in]\]Several of these quarterbacks were traded (Tannehill, Rosen, Darnold, Wentz, & Mayfield) or retired (Manning & Roethlisberger) shortly after these underwhelming seasons. The Ravens, known for being an analytically driven front office, drafted Lamar Jackson after Joe Flacco’s forgettable 2017 campaign.
Four out of the five quarterbacks who finished in the bottom 25 last year are no longer the starter for that team. The Browns traded Mayfield for a conditional fifth round pick to Carolina, the Commanders traded for Wentz in an attempt to upgrade from Heinicke, Dalton signed with the Saints in free agency and Roethlisberger retired from the Steelers. Fields, the only rookie of the bunch, is still the starter with the Bears.
Future Predictions
The predictions for the starting quarterbacks this season are calculated by plugging in the marginal effects from the model. According to Vegas Insider, the MVP betting favorites currently are:
- Josh Allen (+650)
- Patrick Mahomes (+750)
- Tom Brady (+800)
- Aaron Rodgers (+850)
- Justin Herbert (+900)
The model expects these MVP frontrunners to finish among the top seven in Predicted EPA. The projections should correlate with ~ 61% of the season’s final results and explain ~ 37% of the variance in EPA per Drive. Lag EPA represents the total EPA the quarterback produced the year prior. The binary metrics that represent whether the quarterback has changed teams or the team hired a new head coach are not displayed.\[\\[0.0005in]\]
| Predicted EPA per Drive for Projected NFL Starters (2022) | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Years Experience (Exp), Prior Efficiency (lag EPA), Active Cap Hits in Millions | |||||||||||||||
| Name | Team | Exp | Lag EPA | WR | OL | PEPA | Name | Team | Exp | Lag EPA | WR | OL | PEPA | ||
| 1 | T.Brady |  |
24 | 164.38 | 31.7 | 44.4 | 1.12 | 17 | Z.Wilson |  |
1 | −69.69 | 29.2 | 48.6 | 0.55 |
| 2 | J.Herbert |  |
2 | 142.50 | 41.0 | 33.2 | 0.88 | 18 | G.Smith |  |
9 | 5.02 | 28.2 | 29.0 | 0.52 |
| 3 | K.Murray |  |
3 | 76.00 | 30.1 | 47.5 | 0.86 | 19 | J.Winston |  |
7 | 43.53 | 30.9 | 36.6 | 0.52 |
| 4 | J.Allen |  |
4 | 137.39 | 22.6 | 42.1 | 0.85 | 20 | K.Cousins |  |
10 | 76.22 | 23.5 | 29.9 | 0.46 |
| 5 | P.Mahomes | |
5 | 164.96 | 20.7 | 37.7 | 0.84 | 21 | T.Lance |  |
1 | 6.88 | 18.0 | 37.2 | 0.45 |
| 6 | M.Stafford |  |
13 | 129.00 | 29.7 | 30.1 | 0.84 | 22 | B.Mayfield | |
4 | 20.59 | 27.6 | 38.3 | 0.44 |
| 7 | A.Rodgers | |
17 | 149.06 | 19.1 | 29.9 | 0.83 | 23 | D.Jones |  |
3 | 8.89 | 41.8 | 31.6 | 0.43 |
| 8 | J.Hurts |  |
2 | 59.69 | 21.9 | 51.1 | 0.79 | 24 | R.Wilson |  |
10 | 28.29 | 18.5 | 47.7 | 0.39 |
| 9 | D.Prescott |  |
6 | 87.16 | 17.2 | 42.8 | 0.74 | 25 | D.Carr |  |
8 | 68.27 | 21.6 | 25.7 | 0.34 |
| 10 | R.Tannehill |  |
10 | 83.46 | 20.8 | 33.2 | 0.68 | 26 | J.Brissett |  |
6 | −8.45 | 16.1 | 40.1 | 0.33 |
| 11 | M.Jones | |
1 | 72.80 | 38.7 | 30.6 | 0.67 | 27 | D.Mills |  |
1 | −41.19 | 21.8 | 43.8 | 0.29 |
| 12 | L.Jackson | |
4 | 43.56 | 11.7 | 49.2 | 0.67 | 28 | M.Mariota | |
7 | 6.52 | 15.2 | 33.5 | 0.27 |
| 13 | J.Burrow |  |
2 | 109.15 | 26.9 | 31.3 | 0.66 | 29 | T.Tagovailoa |  |
2 | 26.99 | 25.8 | 28.1 | 0.26 |
| 14 | J.Goff |  |
6 | −23.77 | 17.8 | 50.6 | 0.63 | 30 | T.Lawrence |  |
1 | −35.04 | 34.4 | 31.9 | 0.25 |
| 15 | C.Wentz |  |
6 | 43.48 | 31.2 | 43.3 | 0.61 | 31 | M.Trubisky |  |
5 | −5.52 | 20.5 | 23.7 | 0.13 |
| 16 | M.Ryan |  |
14 | 12.16 | 14.9 | 49.6 | 0.59 | 32 | J.Fields |  |
1 | −51.45 | 16.3 | 29.6 | 0.03 |
| Table: @PattonAnalytics | Data: @nflreadr, @nflfastR & @Jason_OTC | |||||||||||||||
\[\\[0.0005in]\] Six projected starters for this upcoming season were drafted last year. Five of those six quarterbacks were drafted in the first round. Their PEPA rankings are fairly indicative of their supporting casts. The Patriots, Jets and 49ers have invested in the protection and weapons around their young quarterback. The offensive line and wide receivers for the Texans, Jaguars and Bears have more question marks than answers.
- Mac Jones (11)
- Zach Wilson (17)
- Trey Lance (21)
- Davis Mills (27)
- Trevor Lawrence (30)
- Justin Fields (32)
Jalen Hurts is primed to have a big year behind arguably the best offensive line in football. The Eagles have spent first round picks the last three years on wide receivers; Jalen Reagor (’20), DeVonta Smith (’21) and AJ Brown (’22). While Reagor is struggling to make a roster spot, drafting DeVonta and trading for AJ over the past two years have been phenomenal additions by Howie Roseman.
Shortcomings
Active cap hits in this model do not account for injuries and suspensions. Tom Brady’s starting center, Ryan Jensen, is doubtful to start the season and his star receiver, Chris Godwin, is coming off an ACL injury. These injuries will effect the performance of the offense in Tampa Bay. The Cardinal’s star wide receiver, DeAndre Hopkins, is suspended for the first six games of the season which should have a negative impact on Kyler Murray’s production.
These projections also fail to identify a team’s net gain on investment for different position groups. The model assumes the active salary cap hits for wide receivers and the offensive line are efficiently allocated. This assumption unfortunately isn’t true. For example, Joe Burrow’s top two wide receivers, Chase and Higgins, are both on rookie contracts. Their salary cap hits are significantly smaller than the production they’ve provided.
The Dolphins and Raiders acquired elite wide receivers in the NFL and signed them to massive extensions. The cap hits for Tyreek Hill and Davante Adams in ‘22 are fairly low compared to their contracts’ Annual Average Value (AAV). These additions are probably undervalued which means Tua Tagovailoa and Derek Carr should perform over expectation.
Team spending on position groups is one way to evaluate a team’s abilities by it’s parts. Another way could be utilizing metrics from Next Gen Stats (NGS). Average Yards Over Expecation and Average Separation for a wide receiver room from the previous year may yield a more accurate prediction of a quarterback’s efficiency. Likewise, Pass Block Win Rate (PBWR) from ESPN correlates with cap expenditure, but could provide a more insightful impact on performance moving forward. These statistics from NGS (since 2016) and ESPN (since 2019) have only been around for a few years. The lack of data is the reason why they are omitted in this analysis.
Concluding Thoughts
The Dolphins, Raiders, Broncos and Colts offseason acquisitions mirror teams that have made the Super Bowl in the past couple of years. The Dolphins and Raiders decided to invest in pass catchers over pass protection. While these teams have a surplus of receiving threats, the only team spending less on their offensive line are the Steelers. Their projected starter, Trubisky, is ranked next to last in Predicted EPA (PEPA). On average, quarterbacks don’t do well behind cheap offensive lines. The Bengals’ Super Bowl run should be viewed as an outlier, not a recipe for team building.
The Broncos’ believed they were “a QB away” and traded for Russell Wilson this past spring. The Buccanneers and Rams won championships with this blueprint. However, both of those teams had proven coaches in Bruce Arians and Sean McVay. Nathaniel Hackett was just hired as the head coach in Denver. In a competitive division, a new system, and Tim Patrick’s ACL injury, immediate success shouldn’t be expected for Wilson and Co. They have the foundation to make a run in the next couple of years, but growing pains seem imminent this year.
Indianapolis’ acquisition of Matt Ryan should not be overlooked. They are a well coached team under Frank Reich with one of the better offensive lines in the NFL. They have a rising star in Michael Pittman, a bell cow in Jonathan Taylor and several play makers on defense. If a quarterback was to make a Super Bowl run after switching teams, Ryan and the Colts (+2380) should be the betting favorite over Wilson and the Broncos (+1620). \[\\[0.0005in]\]