What are your thoughts, ideas and concerns about the
allocation of nationals bids to regions based on regular season results/rankings? If you disagree with the mechanism described in the plan, how would you suggest regions be awarded bids to national events?
The main concern here is that out-of-Region record is used to assign bids. If teams from the NW region take each other out in the quarterfinals of a tournament but are undefeated out of Region, that shouldn't penalize them.
ReplyDeleteWell, the bids will be based not on placement finish at tier 1 tournaments, but on a power ranking based only on official rostered games. So if NW teams take each other out early in an event, but only lose to one another, then all the NW teams will have really strong power rankings, and they will snag a bunch of the strength bids to nationals.
ReplyDeleteThis "power ranking" needs to be tested well on previous years data. Thankfully we have several years of it available.
ReplyDeleteMy inclination is to not count point differential, unlike RRI. Focus on W-L, strength of schedule, out of region record, and weight certain events higher (probably the National-level event(s)).
100% agreed on testing, Ryan. It's an absolute necessity and it WILL happen before any system is implemented.
ReplyDeletePersonally, I am inclined to count some point differential, but with a cutoff. So 15-8 is better than 15-14, but 15-2 is not much better (or maybe not better at all) than 15-8. I want to recognize a clear win versus a nail-biter, but I don't want to reward running up the score. If we find that using limited point differential doesn't improve the predictive power of the algorithm (e.g. using the rankings from the 2006 preseason tournaments to "predict" the games of the 2006 college series) then we won't use it at all, of course.
W-L & SOS will definitely be part of it, as they are in basically any power ranking. Weighting the tier 1 events more heavily than the tier 2 events is likely.
I completely agree with Tarr on having a point differential but with a cutoff, or cutoffs?
ReplyDeleteI think everybody would agree (same winning team in this situation) a loser of a 15-14 game is better than the loser of a 15-8 and that losing 15-1 is worse than 15-8. Theoretically, extremely since I can't figure a way to do this, you could have different weights on losses of 1/2/3/4 and then jump to like 8/9. So you would fall into a close lose of 1/2/3/4 to a lose, but not a blowout (without differention) of 5/6/7/8 and then have a blowout section of 9+. The problem that I have with this would be the creation of the cutoff. So while a 15-10 loser is better than the losers of 15-6 and 15-7, more solution then creates a false seperation between the losers of 15-6 and 15-7 (which there really isn't)
So while I think a cutoff is a good idea, you still run into problems of the seperation line (just like driving...Is 61 really going to kill you will 60 is completely safe?) So maybe just do a W-L but somehow account if the game is within 3? or at least a universe game
Scurry, I like how you talk yourself into 5 different things over the course of your post.
ReplyDeleteThere's no question that, by truncating scores (so that, for instance, we consider any margin of 15-8 or larger as "at least 15-8"), we lose information. If we lose information, we slightly weaken the predictive power of the algorithm, in the abstract.
But the thing is, the world isn't abstract. In the real world, if we treat 15-4 differently from 15-8, we create an incentive to play every point of every game as if it were a crucial point. This changes the way teams approach tournaments. We don't want the algorithm to affect teams' decision-making in this way.
Okay, getting back to the point of this section, which is the allocation of Nationals bids to the various Regional tournaments - I feel like we're adding a serious layer of complexity, this idea of playing for a bid for your conference, instead of playing directly for yourself, that is unnecessary.
ReplyDeleteIf we trust that the regular season results are reliable enough to allocate Nationals bids to Regions, then why not allocate them directly to the Conferences? It's like you put all this stock in the regular season, then you might throw it all away on the basis of performance at one event, because incentives are funny and unclear.
Better, I think, to really encourage teams to play in big events, as many as they can, to get the best sense of how good they are, then allocate the bids to the conferences accordingly (maybe the major conferences get one guaranteed bid to D1?) and have at it!
Maybe that's outside the scope of this blog, to suggest a radical restructuring like this, but I can see all sorts of fun events, like a 'conference vs. conference' tourney, where two conferences travel and only play teams from the other one all weekend long. Stuff like that will be great for the algorithms to chew over, while building conference spirit and making the season very fun.
Tarr, I assume this must have been considered at some point - you can mail me off list if you want to tell me why I'm an idiot.
No OD, you're not an idiot at all. This had occurred to me, too. The national->regional->conference, two step allocation of bids does add a layer of complexity that is arguably unnecessary.
ReplyDeleteWhen it comes to D2 and D3, I completely agree that we should be looking wide-angle at the whole season. That was largely the point of my monster comment at the start of the "Conference Bid Allocation" thread. In those cases, regionals results are just another set of data points for the algorithm to use, albeit with a heavier weight than other tier 1/tier 2 games.
If we abandon "games to go" for D1, though, then we elevate regular season to preeminent importance. The issue is that every conference can't have a D1 nationals bid. If teams don't have a chance to earn a bid for their conference at regionals, then conference assignment becomes destiny for any team that doesn't participate in tier 1.
Since we want to keep things inclusive, I think allowing tier 2 teams to earn D1 bids for their conferences at regionals is a necessity. I do think that, 20 years down the line, the regional stage may become unnecessary and we could just divvy bids directly to the conferences. But we're not there yet.
Tarr,
ReplyDeleteI see your point - without increasing the size of D1 nats to allow one bid per conference, a team in a bad conference could be SOL. Even if they prove themselves good enough, by participating in Tier 1 events throughout the season, the conference might not be good enough to earn a D1 bid.
To my mind, this seemingly necessary (right now) level of complexity is a big strike against the conference plan...
Why do you see it that way? Sure, it's an additional level of conceptual complexity, but think about the actual effect on the teams. (Or read through the "sample seasons" in the plan details.)
ReplyDeleteTeams pass through three stages of UPA-managed postseason play. In stead of sectionals, regionals, nationals, we have regionals, conference championships, nationals.
Teams that were, in the past, eliminated at sectionals, may or may not play regionals but will play conference championships.
Teams that were, in the past, eliminated at regionals, will almost surely play regionals, play conference championships, and may qualify for a nationals event.
Teams that were, in the past, making nationals, will will almost surely play regionals, play conference championships, and qualify for a nationals event.
What's the problem? Who is burdened by an extra level of complexity?
I think that this extra layer is necessary. Without regionals, we are putting TOOOOO much weight on the regular season. I think that Ultimate is not at a point yet where we are in a position to do that. Not enough resources/funding/time/everything to make regular season games THAT important, and to require teams to play a lot of games so as to make the results accurate.
ReplyDeleteBut i think what is more important is what was talked about earlier. I think we REALLY need to hammer down this "power ranking" system before this is given the thumbs up. The whole thing sort of hinges on this.
Alex,
ReplyDeleteYes, the power ranking will be crucial. I don't know how much stock you put in my thinking, but rest assured, I've put a lot of thought into how it will work, and how we should fine-tune it.