BlogPoll: Determining the method for future madness

A few weeks ago, we were contacted by Michigan blogger and SEC public enemy No. 1 Brian Cook about becoming a pollster in the 2008 version of the BlogPoll. We're happy to announce that Rock M Nation, alongside our comrades at Mizzourah, will be representing Mizzou in the polls for the upcoming season. For those unfamiliar with the BlogPoll, here's a synopsis straight out of the creator's mouth:

WHAT IS THE BLOGPOLL? It's basically the AP poll except with bloggers. It's a poll and so fall prey to all the things that polls fall prey to, but if you're so inclined these are the reasons the BlogPoll might be superior to other polls out there. . . .

Biases are disclosed and closely monitored. Every voter has to have a favorite team; voters without are laughed at and told to watch something soulless. The poll closely tracks each voter's level of bias and uses stern disapproval to keep would-be homers in check. (Would-be negative nancies are not quite so easily dissuaded.)

The poll's goals are clearly stated. The AP poll is full of voters who vote team X super high in the preseason because of its schedule; this is strongly discouraged by the BlogPoll. Preseason polls are supposed to be exclusively about how good a team is thought to be, and postseason polls are supposed to be exclusively about how much a team has accomplished on the field. . . .

It's weirder. The poll has some definite wackos in it, but they are relatively few and act as a net positive, forcing more mainstream voters to argue things like "Kansas probably shouldn't be #20 at 11-0" or "why rank Hawaii at all?["] . . .

It's more fun. No one really cares, so we can just vote and not have garments rent.


However, the transition from anonymously ripping pollsters to becoming one comes with problems of its own - chiefly, what method to use and how to approach the ranking of teams. Since we got in to Top 5 talk in the Links today, I figure it's time I open things up for the community to help decide how Rock M Nation should go about creating its ballot.

Follow the jump for analysis and descriptions of the different methods and options.

The various methods and processes have been well-chronicled around the SB Nation network, so I'll be leaning heavily on their descriptions to help better explain the options. The best description, as usual, comes from our soon-to-be-departed friend at Sunday Morning Quarterback.

METHOD NO. 1: RESUME

Quite simply put, what have they done? This method is highly regarded by most pollsters, as noted by well-crafted arguments made by the folks at Dawg Sports and Black Shoe Diaries. Again, I turn to SMQ to do my dirty work:

Strengths: Consistency. Attempts to use "evidence" rather than perception or past history to eliminate abstraction, and treats every team equally and entirely as a team - doesn't give any boosts or demerits to teams based on the recent past or personnel... If Boise State defeats a I-AA team in its opener by a two touchdowns more than Georgia defeats a I-AA team, as was the case the first week of this season, the "Resume" voter would rank Boise higher in the second week even if he believed Georgia was the "better" team, because there's no way to measure UGA's perceived superiority - it's just an abstract notion based on past teams, not the current reality. When Michigan State was an impressive 3-0, the "Power Poll" voter might have said "I don't believe in the Spartans, they always fall apart," and stayed away from MSU, but the "Resume" voter, even if he believed in an eminent collapse, would criticize and reward based solely on those three games... All that's considered is what's happened on the field to date, which is all that can be measured, and which is all anyone will have to go on in the final ranking in January, when it counts.

Drawbacks: ...Even if a voter is using a statistical method (see below), subjectivity and abstraction creep in when considering how much credit or punishment is deserved for a particular win, especially early in the season, or, on the same lines, how to account statistically for strength of schedule. It's OK that the same win or loss on a resume changes in value as the season goes along according to changes in perception about a particular opponent, but that's still dreaded perception... Early this year, in trying to come up with a way to account for strength of schedule on various resumes, SMQ started making a list that assigned a basically arbitrary value to each team as part of a group of similarly-valued teams, until it dawned on him to ask, "If this is what I actually think of these teams, why don't I just use this list?"

The resume method makes the most sense to me towards the end of the season once the resumes begin to look complete, but adhering strictly to this process in the first few weeks of the season reeks of statistical issues stemming from sample sizes that are way too small. This method is, however, pretty much unanimously accepted as the process for postseason balloting.

METHOD NO. 2: POWER POLL

It's the classic "Who wins on a neutral field" argument, one brought up earlier today in the Top 5 debate in the Mizzou Links. An even easier way to put it is by simply calling it "the eye test." It is extremely vulnerable to personal bias, and it's hard for anyone, expert or not, to irrefutably claim that they can just look at college football and pick the Top 25 teams by playing hypothetical neutral sight games in their head. Again, SMQ:

Strengths: Simple, direct, and the most flexible, because its based the most on perception and opinion. Can incorporate both a "resume" and a "futures" element that takes into account where a team has come from and where it's going compared to another, similar team; i.e., if two teams look like they're in the same spot at the same point in the season, like undefeated West Virginia and undefeated Rutgers, for example, a notion of "strength" can take into account not only WVU's more successful past, but also its likely more successful future as the conference schedule stiffens towards the end of the season...

Drawbacks: Haphazard. There are really no internal rules to dictate consistency, which is a bitch when perception does not reflect reality, and an overemphasis is placed on a team's history (meaning past seasons) rather than its present. Ratings on "strength" are abstract, almost by definition non-quantifiable, and easily wrecked by idiosyncrasies in the illogical infinite regress of who beat who - in 2005, for instance, a victory chain can be drawn to show how Division III Averett University could have beaten Ohio State, which is proof (the chain, that is, which can be drawn to and from any team in any division) that merely beating a team is not a pure indicator of "strength." So other very malleable notions like "talent" must be brought into the picture to determine a prospective ten-win team from an eight-game winner. It's a real instinctual, gut-feeling guessing game up here, when one of the first rules of the process should be that your eyes and gut are not always reliable sources. Also leads to the dreaded "drop-em-when-they-lose" syndrome, which is excessively loyal to preconceived notions and pretty much just unfairly stubborn.

I tend to favor this method by a hair, but that's built out of plain hubris and the desire to not get bogged down with "facts." But I certainly see the flaws from subjective biases creating a major issue for this type of process, flaws that several of the other processes can account for. The question I have is, since the BlogPoll is specifically engineered to make biases transparent, then doesn't it become incumbent upon the voter to determine how much his or her own bias influences the poll? Whoever is the most biased will have to answer to the angry mob.

METHOD NO. 3: FUTURES

Polling based on the schedule that lies ahead. The question asked is "Is Team A more likely to win than Team B over the final X games of the season?" The question becomes highly dangerous though, as who predicted Stanford to upset USC? The fact that we can't predict college football is what makes the sport so great and the polls so hard. SMQ, take it away:

Strengths: Ruthless pragmatism. The "Futures" voter probably didn't get carried away with Florida because of the minefield it had ahead of it, and is probably a lot less excited by Southern Cal with California, Oregon and Notre Dame awaiting than the Trojan-loving computers are. On the other end, Arkansas' stock shot up like a rocket with its remaining schedule after it beat Auburn. [Ed. note: This SMQ post is from Oct. 2006]

Drawbacks: Highly speculative by definition. Rewards soft scheduling, and creates bubbles around teams prepared to devour the empty calories in delicious cupcakes. Instills a hollow, frontrunning mentality.

The great question here becomes situations like Kansas and Hawaii last year. Kansas severely lacked resume credibility and couldn't muster a whole lot of power poll credibility (because, like "you're Missouri," they were reminded, "they're Kansas"), but was rewarded for a fortunate future conference schedule up until the last game of the season. Last season, Hawaii, again lacking credibility in the first two methods, becomes the odds on favorite in every game. Does Hawaii become No. 1 because the road ahead is easier than a team with a tougher road? This form of polling essentially boils down to "How well can you predict the future?" rather than "Who is the best team in college football?" This method is highly discouraged in preseason polling by the BlogPoll.

METHOD NO. 4: STATISTICAL

Just like the BCS, the computers do all the talking here. It seems like if there was ever a method suited to Rock M's Beyond the Box Score roots, this would be it. But don't cold hard numbers completely ignore the fact that college football is a game played by warm-blooded humans driven by psychological motivation? Hit us, SMQ:

Strengths: Able to process huge amounts of relevant information that puny human brains could never consider alone, and reach subsequently enlightening conclusions. When SMQ raged against the machines Monday, frequent commenter and resident stat guru Paul Kislanko argued "the only thing worse than using computers is using the human polls," and said by the end of the season, when teams are more connected by common opponents and opponents of opponents, etc., results like six I-AA teams ranked ahead of No. 63 Miami of Florida would be eliminated. So, clearly, they're not beholden to flawed human perceptions and biases, either - you know, an acrobatic, game-winning 20-yard catch that earns a kid an impressive highlight and all-conference honors is just another 20-yard catch in the books. Stupid mortals!

Drawbacks: Puny human brains are telling the computers what factors to consider and how much to consider them to reach said conclusions. SMQ, as one who's tried to devise his own low-tech, purely stat or other number-based projections, didn't say "faux objective" for nothing: the formula for input itself has all kinds of built-in biases that can be rigged (intentionally, for you conspiracy theorists, but more likely unintentionally) to favor certain types of teams. It doesn't matter what the formula is - unless, that is, it's something exceedingly simple like pure winning percentage, in which case it can't account for the all-important strength of schedule variances. Strength of schedule itself is the biggest stick in the craw here, because it skews the relevance of every other possible number, and the most difficult element to measure by numbers alone; many computer rankings, like Jeff Sagarin's, for instance, use "Record vs. Top 10" and "Record vs. Top 30," but this seems more than a little "Chicken or Egg?" If the rankings haven't been generated yet, how can you tell who's in the top 10 or top 30? After those numbers are figured in, and the top 10 and top 30 change, do the inputs to those categories change again to reflect the difference? And do they change again after that? And again, ad infinitum? The "finish line" to such changes is subjective. There's also the huge problem of grouping at the margins (No. 11 is grouped with No. 29 rather than No. 10, for example), which brings us back to the arbitrary nature of such decisions.

The numbers can help you argue your way out of a lot of claims of bias, but they alone do not tell the entire story.

THE CONCLUSION (OR, RATHER, THE QUESTIONS)

So, the four primary methods, although I'm sure many others exist, all account for distinct advantages and weakness as they pertain to weekly submission of BlogPoll ballots. But the question remains, which one to use? That question spawns follow-ups:

 

  • Can you use one method all season long? 
  • Can you use a power poll until the sample is large enough to warrant a resume approach? 
  • Should a preseason poll be based on a power poll or should it try to be predictive by using a futures approach?
  • Is the BlogPoll indicative of the Top 25 teams so farthat week, or who we think will be the Top 25 teams at the season's end?

 

This is where I turn to our readers - both Rock M Nation readers AND all the other BlogPoll balloters out there (and there are a lot of you with a lot to say on the subject). How should we go about ranking teams? Vote in the poll and elaborate in the comments.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

Join Rock M Nation

You must be a member of Rock M Nation to participate.

We have our own Community Guidelines at Rock M Nation. You should read them.

Join Rock M Nation

You must be a member of Rock M Nation to participate.

We have our own Community Guidelines at Rock M Nation. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9347_tracker