If he stayed with the original plan he would done a huge amount of damage to the Pennsylvania and New Jersey rail net and river ports, and threatened Philly and NYC both. Might have been enough to unseat Lincoln; he barely retained any power after the 1862 mid-terms, and only held on by having his private army controlling ans stuffing the ballot boxes in the border states. Many northern voters feared he was turning the war into an anti-slavery crusade, something they didn't support at all. They had been led to believe it was about keeping all blacks out of the new territories and out of the Midwest and northeastern states. Lincoln and the GOP had run a campaign based on white nationalism., and won on that basis.
This is clear from the aftermath, where almost all blacks remained in the South after the war. During the war, the Union forced blacks to keep the plantations running and weren't allowed to leave without written permission, and the rest were put in 'contraband camps' to keep them from fleeing north. Most northern states already had Black Codes for decades; Lincoln's own state of Illinois made it nearly impossible for blacks to make a legal living or buy property.
