They already did. PG County, MD, 1973. SCOTUS-ordered desegregation busing.
As the busing ensued, the white conservatives, mostly democrats then, moved beyond the beltway and eventually out of the county altogether. As schools became more black, they faltered. The blacks blamed it on schools outside the beltway getting more attention. Eventually whites outside the beltway fled, too, and that argument disappeared as those schools became all black and faltered as well.
For the past 33 years, the PG County school system, which had been one of the best in the state, has been 2nd worst in the state and the DC area. The few pockets that are mostly white now (and very insulated) have successful elementary schools but those populations aren't enough to counter the failure of the subsequent mostly-black middle and high schools. At that point in their kids' education, the democrats in those isolated white locales either move away or send their kids private. Too much diversity.