“Imagine a map where every city has a road to it and to a neighboring city,” said Massey, who first applied his ratings system to a high school national poll in 2008. “So from Atlanta to Chattanooga to Nashville to Indianapolis, you could find roads that connect all of those towns. It’s the same thing with schools. It’s a big network that’s pushed and pulled by who wins and loses, so you can compare California to Tennessee to Georgia pretty well if you have years of data.”
CalPreps founder Ned Freeman, who with the assistance of cofounder Eric Maddy, began applying his Freeman Ratings on a national level in 2003, acknowledged that sometimes there are roadblocks in Massey’s analogy.
“You run into issues like self-contained units,” said Freeman, who developed his first ratings at age 7 using NFL scores. “Some parts of the country, like Long Island, don’t play off the island. Illinois has parts like that, too. They’ll play within their league but not against anyone else. That’s the kiss of death for a power ratings guy.”
Where instances like the one Freeman described can limit ratings, subjective pollsters can take them into account.
At USA Today, Indianapolis Star sports editor Matt Glenesk is in his first football season in charge of the Super 25. As the Gannett-owned paper continues to downsize, the poll is no longer done on the national level, but instead divvied out to a network of regional editors, led by Glenesk, who sorts out the rankings through conference calls, his team’s regional rankings and feedback from local reporters on the ground.
“There’s no formulas or mathematics,” Glenesk said. “Just local expertise with an eye test. My reporters at the Austin American-Statesman know Texas better than anyone else. We’ve got 15 (Gannett-owned) papers in Florida that know the best teams in the state better than any formula would. So we lean on our localized expertise and build out the rankings from there.”
Steve Montoya, who heads the national rankings for MaxPreps, along with Zack Poff, run something close to a hybrid subjective/computer model. While their top-50 national poll is subjective and formed mostly through game coverage, watching NFHS Network broadcasts and Hudl, MaxPreps utilizes an algorithm-based ratings system to determine state rankings, and they sometimes use those as a reference when considering the national rankings.
“We’re ranking 50 teams, but we’re keeping an eye on an additional 50 that could jump in any week,” Montoya said. “Say there’s a team that moved into the top five in one of our state rankings. For example, Norcross was off to a 3-0 start, so let’s take a look at them. Or there’s another team that just smoked a team we were looking at, so who is this team?”
When it comes to the effectiveness of subjective polling over objective, a case can be made for both.
“It’s not like the college level where everyone in the country is playing each other, so it’s always a fun argument to have as far as which state is the best,” Montoya said. “Our rankings give that perspective.”
Said Freeman: “The two approaches validate and learn from each other. People look at the power ratings, and if a team is emerging that isn’t in the subjective rankings, they’ll keep an eye on that team. In most cases, we both end up at the same place, so it’s hard to find too much of a difference.”