Did you actually listen to what he said? He said inclusion of those who had been left out should not disadvantage others who had not discriminated, that quotas and including people only because of their skin color disadvantaged people by denying them the dignity and perception of merit and deserving of their place in the organization. In other words he definitely said the right thing to do is not exclude people who had previously been excluded but he opposed affirmative action as it diminished their real achievements. He holds that same opinion now.
In simplest terms he was simply saying that black people, etc. who qualify should be admitted to the university and/or be hired and not denied a position instead of being excluded as was once the case.
Scene from the movie "Changing Places" starring Eddie Murphy who plays Billy Ray Valentine. Billy Ray is in a bathroom stall and overhears his employers, Randolph and Mortimer, discussing him:
[Billy Ray overhears the Dukes talking in the bathroom]
Randolph:
Pay up, Mortimer. I've won the bet.
Mortimer:
Here, one dollar.
Randolph:
[chuckling] We took a perfectly useless psychopath like Valentine, and turned him into a successful executive. And during the same time, we turned an honest, hard-working man into a violently, deranged, would-be killer! [laughs] Now, what are we going to do about taking Winthorpe back and returning Valentine to the ghetto?
Mortimer:
I don't want Winthorpe back, after what he's done.
Randolph:
You mean, keep Valentine on as managing director?
Mortimer:
Do you really believe I would have a n*gger run our family business, Randolph?
[Valentine's eyes widen with outrage]
Randolph:
Of course not. Neither would I.