In CEO’s report on racial disparities in UW admissions, they highlight an extremely misleading statistical concept — that of “odds ratios” — to leave the false impression that black and Latino applicants to UW are hundreds of times more likely to win acceptance than whites. They also dump more than a thousand students of color out of their applicant sample, inflating admissions percentages for blacks and Latinos by excluding weak and unqualified applicants from that pool and distorting statistics on Asians by excluding all applicants of Southeast Asian origin from their study.

In addition to all that, they engage in a variety of petty manipulations of data, as when they scale their admissions rates chart to begin at 50% rather than 0%, thus dramatically enhancing the visual impact of the graph at the expense of accuracy and readability.

Strangely missing in all this statistical sleight-of-hand is any straightforward statement of the magnitude of the supposed advantage that black and Latino applicants have over whites. At no point in the report do they compare — for instance — the chances of admission of two students, each at the midpoint of the applicant pool, one white, one black. (Neither do they directly compare the chances of admissions of students by criteria other than race under which white applicants have a structural advantage — those of legacy admits vs. non-legacies, for instance.)

At one point they inch toward such a comparison, with a chart listing the number of students of various races rejected with SATs or ACT scores and class rank higher than the median black admittee, but since that chart fails to list how many students in that category were accepted from each race, it’s impossible to translate the chart into actual comparative data.

In fact, there is only one section of their report in which they offer a direct comparison of the chances of admission of two groups of students, and it’s a comparison whose terms have been cherry-picked to provide the impression that they are hoping to leave.

In the report’s section on “Probabilities of Admission” they provide a chart comparing the chances of admission for groups of white, black, Latino, and Asian students — one chart each for in-state and out-of-state applicants. So far so good.

But each chart compares only a small sliver of the actual applicant pool. Beyond the exclusions I mentioned in previous posts, these charts leave out female applicants, who represent well over half of total applicants. They leave out the substantial fraction who took the SAT rather than the ACT. They leave out all legacies, a mostly white group with significant advantages in the admissions process. And as in the previous chart they set the bar for comparison at the median ACT score for black admittees.

There’s a basic principle in statistics that the farther away from the middle you get, the weirder your numbers are going to turn out. If you compare the chances of two students near the middle of the pack, you’re going to get stats on their odds of admission that reflect the fact that they’re similarly situated. But if you go looking for outliers, things start to get wacky.

To understand how this works, let’s do a thought experiment. Imagine that only one student whose first and last names both begin with the letter Z was admitted to Wisconsin in a particular year, and that this student happened, by chance, to have the second-worst grades and test scores of the entire entering class. Of all those students whose numbers were worse, only one was admitted, while 2000 were turned down. And among those 2000, by coincidence, there was a second student with a ZZ name.

Among ZZ-named students with grades and test scores as bad as or worse than our admittee, then, one out of two was admitted, giving that group odds of admission of one in two, or 50%. Among non-ZZ students with similar grades and test scores, only one in 2000 was admitted, giving  admission odds of 0.05%. ZZ-named students at that grade/score level, in other words, were one thousand times more likely to be admitted than non-ZZs.

And what does this tell us? Pretty much nothing. If that ZZ student happened to be 100th from the bottom rather than second, the exact same formula would show that ZZs had odds twenty times better than non-ZZs, instead of a thousand times better. One-hundredth from the bottom and second are damn near identical in terms of actual numbers, but we’re so far out on the statistical distribution tail that even a slight change in real-world data produces huge swings in the reported odds.

The folks at CEO understand this. They understand that because the vast majority of UW’s applicants are white, and because black applicants tend to have somewhat lower test scores, choosing the black admittees’ median as your starting point will produce more dramatic contrasts than using the median of all applicants. They also understand that the smaller you make the pool, the more random variation you get. And so they made the pool small and unrepresentative.

To be clear, I don’t know what the numbers would look like if CEO were to crunch the data in a useful way. I don’t know how many times more likely to gain admission a black or Latino applicant with an application at the middle of the total pool would be than a white student with identical numbers. I suspect that such a student would have a considerable advantage.

But here’s the thing. CEO does know the answer to this question. They do have the data. They know what admissions rates look like if you compare students of different races from the middle of the pack, just as they know what the plain-language version of their misleading “odds ratio” claim would be.

They know all this stuff. They’re just choosing not to share.