Understanding Efficiency Margin

So, last week a few of my tweets in response to Jesse Newell, the Kansas basketball beat writer for The Kansas City Star, received a little buzz on the old Twitter dot com website.

The reaction came from mostly from peeved Missouri fans, who wanted Newell called out for omitting the Tigers from his top-25 ballot. Here’s his reasoning: MU’s resume was impressive — at the time the Tigers were 5-0 — but every analytics guru’s algorithm placed them well outside the rankings. Newell’s methodology has caused a stir, but I understand his broad thinking. We don’t evaluate resumes alone. Team quality should be a factor.

In reality, I share a lot in common philosophically with Newell when it comes to evaluating teams. Advanced metrics in sports allow us to objectively determine who is good, who is average, who is inflated by a weak schedule, and who has benefited from a few lucky bounces.

Where we differ is the application of those metrics. In discussing that with him, I went off on a tangent, one based on some quick and dirty and math calculating the Tigers’ adjusted efficiency margin if you stripped out preseason expectations. In a vacuum, though, that number and how I calculated it might seem confusing.

As a I result, I though it would be worth walking you and other readers through that process and understanding what adjusted efficiency margin (AdjEM) is trying to quantify.

As someone who never liked math in school, I feel like I should lead with this: understanding how these metrics work is no different than understanding points per game, or any other more traditional statistic.

While it sounds complex, efficiency margin is a simple concept: it boils down the difference between how many points you score and how many points you allow to a per possession basis.

In a prior piece on usage rate, I talked about possessions and how they’re calculated:

Whenever the offensive team has the ball, they’re in possession, right? Right.

That’s basically it? Well, not really.

If only it was easy to just count the number of times a team has the ball? So you kind of do that:

Possessions = Field Goal Attempts - Offensive Rebounds + Turnovers + (0.44 x Free throw attempts)

Free throw attempts are always tricky because not every free throw attempt is the same. Sometimes, you’re fouled on a made field goal. Sometimes, you might get fouled on a missed three point attempt. So Dean Oliver, the father of the modern analytics movement and author of Basketball on Paper, figured out to multiply free throw attempts by 0.44 to provide an estimate of team possession.

I should note here that KenPom, arguably College Basketball’s most famous analytics guy, uses a multiplier of 0.475. Why? There are a few more free-throw attempts at the college level. If Pomeroy uses it, so do I.

RAW Efficiency and Efficiency Margin

Raw efficiency works like this: How many points did you score given a specific number of overall possessions?

From raw efficiency, we have the basis for points per possession. Calculating it is simply plug and chug. Let’s look at the raw inputs for MU:

• FGA: 441
• ORB: 84
• TOs: 117
• FTA: 204

Next, let’s do some math.

• Step 1: FGA-ORB or 441-84 = 357
• Step 2: (FGA-ORB)+TOs or 357+117 = 474
• Step 3: FTA x 0.475 or 204 x 0.475 = 96.9
• Step 4: ((FGA-ORB)+TOs) + (FGA x 0.5745) or 474+96.9 = 570.9

With rounding, we’ve calculated that MU has amassed 571 possessions in eight games — or 71.4 per game. We also know MU’s tallied up 584 points. To find raw efficiency. We simply divide points (584) by possessions (571), giving us 1.023 points per possession.

And just like that we have Missouri’s offensive efficiency. KenPom (and other analytics sites) use a tempo-neutral possession number of 100 to make it easier to compare teams. Basically, multiply raw efficiency (1.023 PPP) by 100, and you find MU averages 102.4 points per 100 possessions.

Now, we go through same series of steps for the Tigers’ opponents.

The possession tally should be pretty even, and it is. MU’s foes have played 569.28 of them so far, and they’ve racked up 533 points. Dividing points by possessions tells us the Tigers only allow 0.936 points per possession. And when we multiply by 100 possessions? You guessed it. MU’s raw defensive efficiency is 93.6

With those two numbers in hand, we just do basic subtraction to find efficiency margin.

• Step 1: Offensive Efficiency - Defensive Efficiency or 102.3-93.6 = 8.7

Voila, we know MU’s raw efficiency margin is 8.7 points per 100 possessions.

So how do we get these danged Adjusted numbers?

Jay Biggerstaff-USA TODAY Sports

Adjusted efficiency margin tries to capture another simply but obvious observation: a possession against Tennessee isn’t the same as one against Mississippi Valley State. Meanwhile, trying to stop Iowa isn’t the same as halting...Mississippi Valley State. What Pomeroy’s site does is factor all possessions against all teams to decipher who is good, who is bad, and who is meh.

The more data — possessions — you play, the easier it is to discern who uses them...efficiently.

This is why non-conference schedules matter. They’re a chance for teams to cross-pollinate. By playing diverse slates, you see teams of varying strengths and weaknesses generate a diverse pool of data. When that happens, we can use it to get a sense for they theoretically play against one another.

Mizzou and another team might have similar raw efficiency margins (plus-8.8) but faced wildly different schedules. To help you see this in real life, here is a handy chart showing MU’s schedule so far, total possessions played, margin against each team, and each opponent’s ranking in KenPom as of Jan. 3.

The median ranking is 66.25. What adjusted efficiency margin wants to capture is how much better (or worse) you might be on an average possession against an average opponent on a neutral floor. That average opponent would have an AdjEM of 0.0 — they would allow just as many points as they score on a possession.

Again, our big pool of possessions helps. You look across Division I and find the team whose raw efficiency margin is closest to 0.0 in adjusted efficiency. In this case, it’s...Oral Roberts. The Golden Eagles are sporting a 0.03 AdjEM at the moment.

Pomeroy also understands that early on in the season, there’s not enough possessions to create a representative sample. So, they weight adjusted efficiency margin with data from the prior season. By the time non-conference play ends, that old data has cycled out. That’s why MU’s current AdjEM is 17.16.

So, we know MU’s efficiency margin is 8.7 points per 100 possessions. Meanwhile, the 66th-ranked team in KenPom has a raw efficiency margin of 11.87. Again, we do simple math: add the efficiency margins. What we see is MU’s adjusted efficiency margin is 20.57.

At the moment, any efficiency margin greater than 20.0 would rank among the top 20. For Missouri, it would ranking them 16th, slightly ahead of Virginia. We can also see how preseason expectations — old data — acts as a drag. If we’ve calculated adjusted efficiency as 20.57 and Pomeroy lists it at 17.16, the difference between them (-3.41) is the value of preseason weighting.

Denny Medley-USA TODAY Sports

What does this have to do with Jesse Newell?

I’ve shown you have to calculate raw efficiency margin and how an adjusted version is finding a median across all of college basketball. And that leads me back to my divergence with Newell.

First, polls are mostly dumb. Decades ago, before cable TV, the internet and Dean Oliver, tapping into a network or reporters to vote was logical. That collection of scribes would, in theory, see the best teams in their respective footprint. Pooling their ballots and assigning points for each place helped create a weekly snapshot of the sport.

Now? You and I can watch almost any team we want via a streaming service. We’ve got data galore, too. Yes, the games and results still matter. We’re not running sim mode. But what greater access and data has done is lend us more context around those outcomes.

If polls are crude, then preseason polls are worse. You have zero results. It’s an exercise in projection, one where the standard depends on the voter.

So, what Newell does — lean heavily on analytic ratings — is smart, right? I just showed you all that math and stuff.

Well, here’s the thing: Newell still has to use his eyes and account for results as they come in. Especially this year. Analytics get better as the season goes along and possessions pile up. But they can be slightly erratic early on. (Again, that’s why Pomeroy uses old data as a weight.) What you see right now is KenPom capturing new data points to replace old ones.

That volatility is likely worse this season, which has seen the non-conference season turned on its head by the ongoing coronavirus pandemic. Scheduling is all over the place. And that’s after an offseason that threw practice and player development into chaos. Put simply, this isn’t a normal non-conference season. And that’s going to be reflected in Pomeroy’s ratings.

To be sure, they still have value, especially in looking at specific facets of a team’s performance, such as effective field-goal percentage or how well they keep opponents off the glass. But the top-line number — a team’s adjusted efficiency margin — needs to be taken with a grain of salt.

By simply punting and letting analytics do his work, Newell’s ballot isn’t doing what it intends. If he factored in how adjusted efficiency is calculated and the environment in which that’s happening, he would probably base more of his judgments on the actual outcomes of games and, as the season progresses, steadily shift toward giving Pomeroy’s ratings a bigger role in his decision-making process.

Some teams have canceled their seasons. Others nixed non-conference play. And a team like Missouri has sliced off roughly five games. In other words, we’re not seeing the same level of cross-pollination.

Meanwhile, veteran teams with lots of returning production and continuity in their rotation — like Missouri, Northwestern and Virginia Tech — have thrived. They’re defying preseason expectations. The poll, blunt as it might be, should reflect it. You don’t have to think Missouri is an elite team. They likely aren’t. But it’s still very good, and, with the proper understanding of efficiency margins, the numbers bear it out.