That's true. It was endemic. It had to be. Can't justify keeping an entire race in slavery in perpetuity without it.
No, it isn't true.
The truth is that Southerners detested manual labor. They confounded hard work with "n*gger work". They were lazy and stupid and backward.
The North was not afraid of hard work. They were much more industrious as a result.
Tocqueville: Book I Chapter 18
Time, however, continued to advance, and the Anglo-Americans, spreading beyond the coasts of the Atlantic Ocean, penetrated farther and farther into the solitudes of the West. They met there with a new soil and an unwonted climate; they had to overcome obstacles of the most various character; their races intermingled, the inhabitants of the South going up towards the North, those of the North descending to the South. But in the midst of all these causes the same result occurred at every step; in general, the colonies in which there were no slaves became more populous and more prosperous than those in which slavery flourished. The farther they went, the more was it shown that slavery, which is so cruel to the slave, is prejudicial to the master.
But this truth was most satisfactorily demonstrated when civilization reached the banks of the Ohio. The stream that the Indians had distinguished by the name of Ohio, or the Beautiful River, waters one of the most magnificent valleys which have ever been made the abode of man. Undulating lands extend upon both shores of the Ohio, whose soil affords inexhaustible treasures to the laborer; on either bank the air is equally wholesome and the climate mild, and each of them forms the extreme frontier of a vast state: that which follows the numerous windings of the Ohio upon the left is called Kentucky; that upon the right bears the name of the river. These two states differ only in a single respect: Kentucky has admitted slavery, but the state of Ohio has prohibited the existence of slaves within its borders. Thus the traveler who floats down the current of the Ohio to the spot where that river falls into the Mississippi may be said to sail between liberty and servitude; and a transient inspection of surrounding objects will convince him which of the two is more favorable to humanity.
Upon the left bank of the stream the population is sparse; from time to time one descries a troop of slaves loitering in the half-desert fields; the primeval forest reappears at every turn; society seems to be asleep, man to be idle, and nature alone offers a scene of activity and life.
From the right bank, on the contrary, a confused hum is heard, which proclaims afar the presence of industry; the fields are covered with abundant harvests; the elegance of the dwellings announces the taste and activity of the laborers; and man appears to be in the enjoyment of that wealth and contentment which is the reward of labor.
The state of Kentucky was founded in 1775, the state of Ohio only twelve years later; but twelve years are more in America than half a century in Europe; and at the present day the population of Ohio exceeds that of Kentucky by two hundred and fifty thousand souls. These different effects of slavery and freedom may readily be understood; and they suffice to explain many of the differences which we notice between the civilization of antiquity and that of our own time.
Upon the left bank of the Ohio labor is confounded with the idea of slavery, while upon the right bank it is identifies with that of prosperity and improvement; on the one side it is degraded, on the other it is honored. On the former territory no white laborers can be found, for they would be afraid of assimilating themselves to the Negroes; all the work is done by slaves; on the latter no one is idle, for the white population extend their activity and intelligence to every kind of employment. Thus the men whose task it is to cultivate the rich soil of Kentucky are ignorant and apathetic, while those who are active and enlightened either do nothing or pass over into Ohio, where they may work without shame.
It is true that in Kentucky the planters are not obliged to pay the slaves whom they employ, but they derive small profits from their labor, while the wages paid to free workmen would be returned with interest in the value of their services. The free workman is paid, but he does his work quicker than the slave; and rapidity of execution is one of the great elements of economy. The white sells his services, but they are purchased only when they may be useful; the black can claim no remuneration for his toil, but the expense of his maintenance is perpetual; he must be supported in his old age as well as in manhood, in his profitless infancy as well as in the productive years of youth, in sickness as well as in health. Payment must equally be made in order to obtain the services of either class of men: the free workman receives his wages in money; the slave in education, in food, in care, and in clothing. The money which a master spends in the maintenance of his slaves goes gradually and in detail, so that it is scarcely perceived; the salary of the free workman is paid in a round sum and appears to enrich only him who receives it; but in the end the slave has cost more than the free servant, and his labor is less productive.38
The influence of slavery extends still further: it affects the character of the master and imparts a peculiar tendency to his ideas and tastes. Upon both banks of the Ohio the character of the inhabitants is enterprising and energetic, but this vigor is very differently exercised in the two states. The white inhabitant of Ohio, obliged to subsist by his own exertions, regards temporal prosperity as the chief aim of his existence; and as the country which he occupies presents inexhaustible resources to his industry, and ever varying lures to his activity, his acquisitive ardor surpasses the ordinary limits of human cupidity: he is tormented by the desire of wealth, and he boldly enters upon every path that fortune opens to him; he becomes a sailor, a pioneer, an artisan, or a cultivator with the same indifference, and supports with equal constancy the fatigues and the dangers incidental to these various professions; the resources of his intelligence are astonishing, and his avidity in the pursuit of gain amounts to a species of heroism.
But the Kentuckian scorns not only labor but all the undertakings that labor promotes; as he lives in an idle independence, his tastes are those of an idle man; money has lost a portion of its value in his eyes; he covets wealth much less than pleasure and excitement; and the energy which his neighbor devotes to gain turns with him to a passionate love of field sports and military exercises; he delights in violent bodily exertion, he is familiar with the use of arms, and is accustomed from a very early age to expose his life in single combat. Thus slavery prevents the whites not only from becoming opulent, but even from desiring to become so.
It is true. Northern states had what they called the "black codes" which where far harsher than segregation in the South and often didn't allow blacks to live in the state.
Just read what Lincoln had to say on the subject of blacks before the war:
"I will say, then, that I AM NOT NOR HAVE EVER BEEN in favor of bringing about in any way the social and political equality of the black and white races---that I am not, nor ever have been, in favor of making voters or jurors of Negroes, nor of qualifying them to hold office, nor to intermarry with White people; and I will say in addition to this that there is a physical difference between the White and black races which will ever FORBID the two races living together on terms of social and political equality. And inasmuch as they cannot so live, while they do remain together, there must be the position of superior and inferior, and I, as much as any other man, am in favor of having the superior position assigned to the White race."
— 4th Lincoln-Douglas debate, September 18th, 1858; COLLECTED WORKS Vol. 3, pp. 145-146
"What I would most desire would be the separation of the white and black races."
— Spoken at Springfield, Illinois on July 17th, 1858; from ABRAHAM LINCOLN: COMPLETE WORKS, 1894, Vol. 1, page 273
"See our present condition---the country engaged in war! Our White men cutting one another's throats! And then consider what we know to be the truth. But for your race among us there could not be war, although many men engaged on either side do not care for you one way or another.
"Why should the people of your race be colonized, and where? Why should they leave this country? This is, perhaps, the first question for proper consideration. You and we are different races. We have between us a broader difference than exists between almost any other two races. Whether it is right or wrong I need not discuss, but this physical difference is a great disadvantage to us both, as I think your race suffer very greatly, many of them by living among us, while ours suffer from your presence. In a word, we suffer on each side. If this be admitted, it affords a reason at least why we should be separated. It is better for both, therefore, to be separated."
— Spoken at the White House to a group of black community leaders, August 14th, 1862, from COLLECTED WORKS OF ABRAHAM LINCOLN, Vol 5, page 371