The Fifth Amendment to the U.S. Constitution states the following: "No person shall be...deprived of...property, without due process of law; nor shall private property be taken for public use, without just compensation."
In other words, the Government cannot take your property without following some legal procedure, and you have to be compensated for it. Think of "eminent domain." Government can take what it needs, but it has to compensate the owner for what was taken.
Like it or not, slaves were legal property in the United States. When Abraham Lincoln sought to "free" the slaves it had the same effect (if successful) as if he had effectively torn up the deeds to all of the real estate in the states "in rebellion."
Keep in mind that it was Lincoln's position (and the position of the USA) that the confederate states remained part of the U.S., and their rebellion had no legal effect. So the citizens of the CSA were in fact still citizens of the U.S.
One could argue that Lincoln couldn't free the slaves (i.e., take the slave-owners' property) because it was not for "public use." End of story. But he did purport to free the slaves.
So according to the Fifth Amendment, the slave owners were entitled to be compensated for the loss of their property at the hands of the Federal Government. Reparations?
But on the other hand slavery was certainly a form of imprisonment, and one could argue very strongly that any laws establishing or permitting slavery were illegitimate on their face. Because as we know from the Declaration of Independence, we get our rights from The Creator, and not from government.
Therefore the imprisonment of slavery was "false imprisonment" under U.S. tort law, and the slaves should have a cause of action against the slave owners for false imprisonment.
So really, I think the descendants of the former slaves and the descendants of the former slave owners should duke it out between themselves, and leave everyone else (and the Government) out of it.
My ancestors were in Ireland and England and Poland when all this crap was going on, so personally, I don't give a schidt one way or another.