Scalia said it in about the most racist way possible, but the fact is that there is a mismatch happening between students and Institutions. Back in Louisiana I'd see area high school students choose to use TOPS to go to LSU, nearly fail out, and then come back to a smaller regional campus and succeed. They'd be getting an education just as high quality as they'd get at LSU, just in an environment better suited to help them succeed.
What Scalia should have pointed out is that if you let in an under-prepared student to a large R1 campus with little in the way of undergrad support structures they will fail. Take the same student and put them in a smaller Liberal Arts or Regional Public campus and they succeed. Students need to find the right environment for them. The policies under discussion don't always help students make that choice as it encourages the campus to try to recruit students they know are under-prepared to meet goals that don't further the individual student's education.
That isn't a race issue. It's the same issue facing students from poor economic backgrounds, first generation college students, legacy students, student athletes, etc.