...by 1860, there were more millionaires (slaveholders all) living in the lower Mississippi Valley than anywhere else in the United States. In the same year, the nearly 4 million American slaves were worth some $3.5 billion, making them the largest single financial asset in the entire U.S. economy, worth more than all manufacturing and railroads combined. So, of course, the war was rooted in these two expanding and competing economies—but competing over what? What eventually tore asunder America's political culture was slavery's expansion into the Western territories.
Battle Cry of Freedom
Get a clue...
Of course there were rich people in the South. As you posted, a large source of capital wealth in the South was human beings. Thousands of rich slaveowners went bust and lost everything after the Civil War as they had loans secured on their slaves. Even though the slaves were released, the loans were not, and many had their land seized by banks. That's the primary economic motivation of the Civil War. One can talk all they want about tariffs, but that pales to the capital risked by the Southern slaveowners in human beings. It's understandable why so many fought to leave the union. If someone was going to take away everything you had, wouldn't you fight too?
quoted for great wisdom.
One of the great problems of southern economics was the slaves were a huge asset, they were, by and large, an unproductive asset. According to Mark Twain, and I am sure it can be verified by other sources, the average field hand was worth $800. That makes the average hand worth about 30 cents a day, when northern labor worth 75 cents for fresh immigrants fresh of the boat, which is what Sheridans dad got. And that is low Irish labor.
There is also the problem that southern management was expensive. Wages in the south were low, interest rates were high. Slave labor is unproductive. And it blighted all labor around it.
Which answers the question of why northern workers were fighting to end the practice. They had no love for the slaves. They detested them, by and large. They were fighting to end slavery, which impoverished the white worker. And froze the white worker out of the labor pool. The poor white worker in the north was fighting for his life.