Completion percentage over expected, or CPOE, has been used as a proxy for quarterback accuracy since the NFL started tracking air yards. Completion percentage is highly dependent on throw selection, as we know that passes that travel a longer distance through the air have a lower chance of being completed. CPOE attempts to contextualize completion percentage by adjusting for factors like air yards, down, distance, and whether the quarterback is under pressure when making the throw.
CPOE has been shown to be relatively stable from year-to-year, meaning that it could be capturing a skill that is repeatable and not just noise. This is important because it allows us to better evaluate quarterback play and predict future performance. However, CPOE is not perfect and there are still improvements that can be made to make it a more stable metric. In this article, we will explore some of the limitations of CPOE and propose some ways to make it a more stable metric.
As data scientists, we do not condone filtering just for the sake of filtering. However, we believe that included in the CPOE calculation is a set of plays which, from the perspective of measuring QB accuracy, could be considered as noise and thus removed from the set of plays used to calculate average CPOE. Some such plays can be attributed to circumstance and include drops, spectacular catches, trick plays, hail marys, and others (full list below).
It is important to note that there is some subjectivity in what a drop is, what is a spectacular catch, etc., but we are still interested in observing the impact of filtering out this subset of plays to observe the impact on the predictive power of CPOE:
- Drops
- Spectacular catches
- Throw aways
- Trick plays
- QB spikes
- Laterals
- Screens
- Prevent defense plays
- Hail Marys
FTN has been kind enough to share some of their charting data with the open source nflverse community, which makes this analysis possible. However, the charting data is only available for the 2022 and 2023 seasons, which means our historical analysis is limited. Still, we can observe the stability of CPOE for quarterbacks between the 2022 and 2023 seasons. For this analysis, we used the completion probability model from nflfastR.
Filtering out the noisy plays from the two seasons decreased the play count by 20%, from 41,116 to 32,928 plays. Only quarterbacks that surpassed a threshold of 150 pass attempts (~4 games worth of throws) in each of the 2022 and 2023 seasons were included in this analysis. We compare the correlation from the unfiltered CPOE to the filtered CPOE and observe that the correlation increases from 0.46 to 0.51. This suggests that removing noisy plays makes CPOE a more stable metric.
Figure 1. Comparing CPOE between 2022 and 2023 for eligible QBs, including postseason throws.
Figure 2. Comparing filtered CPOE between 2022 and 2023 for eligible QBs, includes postseason throws.
Additionally, we can see which quarterbacks have the largest improvements in CPOE when filtering out noisy plays. Daniel Jones comfortably led the league with +2.9% average CPOE increase over the 2022 and 2023 seasons. Note this is not a list of the top overall CPOE for quarterbacks but instead shows which quarterbacks benefit the most from the CPOE adjustment. 
Figure 3. Top 15 CPOE changes after filtering noisy plays.
Here is an alternate view to show the rank changes for 2023 CPOE based on the filtering mentioned in this article.
Figure 4. Rank changes in 2023 CPOE after filtering.
This is a small sample size, but it is a promising start to making CPOE a more stable metric. We look forward to another season of charting data for the 2024 season to determine if the same methodology results in a stronger correlation between the 2023 and 2024 seasons.
