How the NCAA Selection Committee is using analytics incorrectly

If you read this site enough, you probably know a few things about me. One is, I love basketball. It’s a passion. Two is, I’m a pretty big fan of analytics in basketball. After all, I write a Study Hall post after every game.

Analytics has taken over most of the major sports, and its application towards basketball has led to a revolution in the way the game is played. Efficiency has become the name of the game. The more efficient you are on both sides of the floor, the better off you’ll be. If you have questions about Efficiency Margin, and what it’s all about, I recommend this article: Understanding Efficiency Margin

So it really makes me happy to see analytics becoming more prevalent in today’s conversations. It also makes me incredibly sad to see them misapplied. Obviously, the goal in any sport is to not be the most efficient team you can be. It’s to win the most games you can, right? Ultimately, that’s why they have games. You can achieve wins through efficiency, which is why the focus is such a big part of it, but what matters more than anything is the results.

So I think the application of analytics is often misguided. Metrics tell a part of the story, but if they don’t net results then what's the point?

Let’s start here: Missouri isn’t a particularly efficient basketball team

I’d be willing to bet few understand this better than I do. Their raw efficiency margin is narrow — see the post linked above if you want the calculations — and it’s easy to look at the fact, Mizzou has scored 1841 points on the season and given up 1798. After 25 basketball games, that adds up to an average score of 73.6 to 71.9, or a difference of 1.7 points. Now I’ve added up their possessions and got 1785, or 71.4 per game.

Offensively, that’s 1.03 points per possession, and Defensively, they’re at 1.01 points per possession. So the Efficiency Margin is +0.02. And as Matt told you in this post, Mizzou’s efficiency margin is what sinks its NET ranking.

Mizzou’s best wins are by narrow margins, their worst losses... aren’t. When you take the totality of the season’s possessions, Missouri looks a lot like an above average high major basketball team. A tournament team, but not really a high seed. In league play they were perfectly average, 7th in Offensive efficiency, 8th in Defensive efficiency. But as somebody who’s written about the numbers surrounding this team all year long (and really for the last 2-3 years with a largely similar group of players) I feel like I don’t need to go into the Tigers’ flaws here. We know them well enough. But what we know about them is they’re still pretty good in spite of their flaws.

Things could have gone a little better in league play, but half of their losses were to NCAA tournament teams, and 3 of the other four were on the road. In fact, Mizzou’s 6-4 road record was as good as Iowa (a 2 seed), and better than Arkansas (a 3 seed), Purdue (a 4 seed), Texas Tech (a 6 seed), Florida (a 7 seed), and LSU (an 8 seed).

Jay Biggerstaff-USA TODAY Sports

The question I have is: What is the NCAA tournament if not a results-based reward system set up to determine an annual National Champion?

Over the years, the selection process has been changed and refined and a few years ago the NCAA introduced the NET rankings. The NET’s formula isn’t public, but it’s apparently based upon a variety of things including: Winning percentage, Opponent winning percentage, Efficiency margin, Strength of Schedule, and more. It’s used in place of banking on the advanced metrics used by Sagarin, KenPom, and ESPN’s BPI in an attempt to eschew the largely outmoded RPI.

When they announced NET, I applauded. It’s a step forward! Moving away from the disastrous and easily gamed RPI meant you couldn’t creatively schedule your way into an at-large bid, you had to play your way in.

Enter Mizzou. Specifically, this season’s Tigers. Prior to the complications of scheduling due to COVID, Cuonzo Martin and his program were scheduling aggressively (as he’s largely done his entire career), and when things were tossed out the window before the season, Martin didn’t budge in his attempt to build a tough non-con slate. Missouri kept Illinois (a preseason top 10 team), worked in a neutral court matchup against Oregon, scheduled a road game at Wichita State, and then scheduled really good mid majors to fill out the rest. Here’s where their non-con opponents finished in the NET rankings:

• Illinois: 3
• Oregon: 33
• Wichita State: 72
• L*berty: 86
• Oral Roberts: 158
• Bradley: 174

They also were scheduled to play in the Big12 - SEC Challenge, and pulled the Big 12’s 3rd worst team in the TCU Horned Frogs (granted, the TCU game could’ve gone better, but that’s why they actually have these pesky contests). TCU finished 141 in NET, or the 5th worst team Mizzou played this year. Knowing you are going to face a competitive SEC slate, Mizzou scheduled what ended up being just one Quad 4 game, and two Quad 3 games.

So here is Mizzou’s team card for NET:

“Don’t take bad losses!” You say?

Technically, Missouri had no bad losses (Quad 3 and Quad 4). They had some ugly losses in Quad 2, but they were only ugly in that the efficiency wasn’t great and so the scoring margin was wider than you’d like to see. But here’s a list of teams with a bad loss (Q3 or Q4) on their resume who received higher seeds than Missouri did:

• Alabama (2 seed)
• Houston (2 seed)
• Virginia (4 seed)
• Purdue (4 seed)
• Oklahoma State (4 seed)
• Florida State (5 seed)
• Colorado (5 seed) — THREE Quad 3 losses
• Creighton (5 seed) — TWO Quad 3 losses
• Florida (7 seed)
• Clemson (7 seed)
• North Carolina (8 seed)
• Oklahoma (8 seed)

That’s 12 teams seeded higher than Missouri with bad losses, while Mizzou had none. Looking above, you can see that Mizzou now had no Q3 or Q4 losses while being 7-6 in Quad 1. Seven Quad 1 wins is damn good. Only 13 teams had as many wins in Quad 1 as Mizzou, and two of those teams didn’t have a winning record in Q1 (Kansas and West Virginia). Meaning, only 11 teams have as many Quad 1 wins while still maintaining a positive win percentage. Here’s the list of 11 and their records in Q1:

Illinois (12-5), Oklahoma State (10-6), Ohio State (9-7), Iowa (8-6), Texas (8-6), Alabama (8-4), Michigan (8-3), Baylor (8-2), Gonzaga (8-0), and Creighton (7-3).

This is a proverbial who’s who on the seed lines. The worst seed here is Creighton at 5. West Virginia (7-7 in Q1) is a 3-seed. Kansas (7-8 in Q1) is a 3-seed as well.

Christopher Hanewinckel-USA TODAY Sports

Wins as good as teams lined up in the seed lines of 3 - 5, no bad losses

All of this and I was still thinking Mizzou was going to end up on a 7 line. Why? Strength of record. Which is honestly one of my favorite metrics, from ESPN:

Strength of Record (SOR) is a measure of team accomplishment based on how difficult a team’s W-L record is to achieve. Game predictions account for opponent strength, pace of play, site, travel distance, day’s rest and altitude, and are used to simulate the season 10,000 times to produce season projections. Numbers update daily.

If you use ESPN’s Strength of Record it’s easy to build out Tiers for seeding.

• 1-4 were all the 1 seeds: Gonzaga, Illinois, Baylor, Michigan
• 5-8 were 3 of the four 2 seeds: Alabama, Iowa, Ohio State
• 9-12 were 3 of the four 3 seeds: Kansas, Arkansas, Texas
• 13-16 had a 2 seed (Houston), 3 seed (West Virginia), 4 seed (Purdue), and a 6 seed (USC)

Oklahoma State is 8th, and was seeded 5th in the Midwest. Virginia is 12th and was a 4 seed in the West. So if you go through the top of the Strength of Record rankings, it largely hems to where teams were seeded. Only as you go down the list farther does it start to separate. Based upon Strength of Record, who was under seeded?

Oklahoma State, USC, Clemson, Oregon, LSU, and Missouri

Who was over seeded?

Houston, West Virginia, Florida State, Tennessee, and Colorado

So combining Quad 1 wins, Quad 2 losses, and Strength of Record, you have a team profiling towards a 6 seed or at worst a 7 seed. And here’s where I have a bone to pick with the approach of the committee.

Analytics as predictors vs. Results based rankings

This comes down to the entire AP Poll vs Jesse Newell conversation. I don’t fault Newell for using analytics, in fact I encourage it. And I’m not writing this to disparage analytics. As I said above, I’m a big proponent of them. But it’s how you apply them. If you’re only using analytics then the results barely matter, and the entire reason we play the games is for the results.

When you look at the results, it doesn’t make sense.

If you’re the kind of Mizzou fan Jerry Palm talks about, you might still need some convincing:

First, stop being like this.

Second, most of these fans are very insular. It’s easy to feel like your team is trash if you don’t watch a ton of other teams, other than maybe the top 10 teams in the land. Did Missouri lose more games than they probably should have, especially down the stretch?

Yes!

Are we happy those losses happened?

No!

Do MOST of the teams in the NCAA tournament have similar losses?

You betcha!

Here’s another team sheet:

7-7 in Quad 1 — Mizzou was 7-6

4-4 in Quad 1A (that’s very good games) — Mizzou was 5-3

3-3 in Quad 2 — Mizzou was 2-3

6-0 in Quad 3 — Mizzou was the same

2-0 in Quad 4 — Mizzou was 1-0

If the Tigers would have completed their schedule against Vanderbilt and Texas A&M with wins - both at home - would’ve been Q3 games. Then Mizzou and the above resume would’ve both been 18-9. Both 7 Quad 1 wins, Mizzou slightly worse in Q2. But identical results in Q3 and Q4. Really, the only difference in the results in efficiency margin.

The above resume? Third seeded West Virginia.

Mizzou is a 9 seed.

The committee got it wrong.

To wrap this up, I’m in no way saying Mizzou should be a 3 seed or a 4 seed. But results matter. Losses count the same if you lose by one on a full court heave at the buzzer or get the door blown off you, and wins count the same if you’re on the other side of that outcome. You can do everything right and win, and you can do everything wrong and still win. The results are what they are.

Analytics are a just a part of the picture. They allow us to get a much better idea on what could happen, and they’re important in telling a deeper part of the story about how you played. But they don’t dictate the result.

When you look at a KenPom team page, it gives you percentages to win. Missouri has a 46% chance to win against Oklahoma per his models. There are a million different potential outcomes to any given basketball game, and the estimation is they basically split most of the time. Mizzou losing by 50 doesn’t prove the committee right in their seeding any more than Mizzou making a Final Four makes them wrong.

The committee should use analytics. But they should start with the results. Figure out their basic tiers, and then apply the analytics to make the final determining spots. Doing anything else dismisses the most important part of basketball... winning and losing.