NCAA committee ditches RPI in favor of new metric

The NCAA Basketball Committee has burned its Bible.

The Ratings Percentage Index, commonly known as the RPI, has been a ratings formula in use since 1981 to measure college basketball teams. The committee that selects the field for the 68-team NCAA Tournament has relied on the RPI for all kinds of guidance.

But the committee has officially scrapped the RPI for a new evaluation method called the NCAA Evaluation Tool, which will be known as the NET. According to an NCAA release, the new system was approved in July after months of consultation with the committee, the National Association of Basketball Coaches, analytics experts and Google Cloud Professional Services.

The NET relies on game results, strength of schedule, game location, scoring margin, net offensive and defensive efficiency, and the quality of wins and losses.

The NCAA did not release the exact formula, which frankly is a must. The NCAA for many years was secretive with the RPI formula, which creates only distrust. If this is a good tool, give analysts the formula. Most of us don’t have the time or the brains to figure it out, but some do, and those analysts are invaluable in making sure the process remains above reproach.

The NCAA release stated that “the NET was built to create a ranking system that was as accurate as possible while also evaluating team performance fairly. To ensure fairness, certain types of data were omitted from the model. Of key importance, game date and order were omitted to give equal importance to both early- and late-season games. In addition, a cap of 10 points was applied to the winning margin to prevent rankings from encouraging unsportsmanlike play, such as needlessly running up the score in a game where the outcome was certain.”

Giving early-season games an equal weight to late-season games is wise. If early games don’t have the same clout, they are reduced to quasi-exhibitions, a feeling that already is a problem with college basketball’s nonconference scheduling.

But margin of victory, especially capped at 10 points, is not a good metric to use. A six-point final might be closer and more dramatic than a one-point finish, depending on how the fouling goes in the final seconds.

Another troubling element to the metric is that it was tested as a game-predictor. “To make sense of team performance data, late-season games (including from the NCAA tournament) were used as test sets to develop a ranking model leveraging machine-learning techniques. The model, which used team performance data to predict the outcome of games in test sets, was optimized until it was as accurate as possible. The resulting model is the one that will be used as the NET going forward.”

Wait. Game prediction is not what the RPI or the Net or any analytics ranking should be designed for. Analyzing what has occurred — not what should occur.

“What has been developed is a contemporary method of looking at teams analytically, using results-based and predictive metrics that will assist the men’s basketball committee as it reviews games throughout the season,” said Dan Gavitt, senior vice president of basketball for the NCAA. “While no perfect rankings exist, using the results of past tournaments will help ensure that the rankings are built on an objective source of truth.”

The NET will be used in conjunction with the quadrant system, which was introduced a year ago and is intended to give added weight to games played away from home. The system produced four quadrants:

Quadrant 1: Home vs. 1-30, Neutral 1-50, Away 1-75

Quadrant 2: Home vs. 31-75, Neutral 51-100, Away 76-135

Quadrant 3: Home vs. 76-160, Neutral 101-200, Away 135-240

Quadrant 4: Home vs. 161-351, Neutral 201-351, Away 241-351

The NET will replace the RPI when grouping records in the quadrant system on team sheets. For instance, last Selection Sunday, OSU had a record of 5-12 in Quadrant 1 games, 3-2 in Quad 2 games, 3-0 in Quad 3 and 8-0 in Quad 4. That was using the RPI to determine the ranking. From now on, the NET will be used.

“While the quadrant system was widely deemed an improvement to the selection process, the NET is another significant step in addressing the recommendations the NCAA received from the NABC’s ad hoc committee, whose purpose was to make recommendations regarding the selection, seeding and bracketing of teams,” said the NCAA’s release.

The NCAA said other metrics added last year would remain: the Kevin Pauga Index and ESPN’s results-oriented metric, the Strength of Record. The team sheets also included three predictive metrics: those by basketball analytics experts Ken Pomeroy and Jeff Sagarin, as well as ESPN’s Basketball Power Index.

“The committee has had helpful metrics it has used over the years, and will continue to use the team sheets, but those will now be sorted by the NCAA Evaluation Tool,” Gavitt said. “As has always been the case, the committee won’t solely focus on metrics to select at-large teams and seed the field. There will always be a subjective element to the tournament selection process, too.”

But the RPI, which basically combines a team’s record (25 percent), with a team’s opponents’ record (50 percent) and a team’s opponents’ opponents’ record (25 percent), will remain in use by other sports, including women’s basketball.