That's exactly what you're arguing, as you're claiming that a single state, all by itself, can strip the federal government of powers, change the number of states in the union, and seize the constitutionally delegated powers and territory for itself.
As James Madison made clear, that's not authority that a single party has. With your assumptions being a 'fallacy', adverse to 'common sense and common justice'.
I partake of the wonder that the men you name should view secession in the light mentioned. The essential difference between a free Government and Governments not free, is that the former is founded in compact, the parties to which are mutually and equally bound by it. Neither of them therefore can have a greater fight to break off from the bargain, than the other or others have to hold them to it. And certainly there is nothing in the Virginia resolutions of –98, adverse to this principle, which is that of common sense and common justice.
The fallacy which draws a different conclusion from them lies in confounding a single party, with the parties to the Constitutional compact of the United States. The latter having made the compact may do what they will with it. The former as one only of the parties, owes fidelity to it, till released by consent, or absolved by an intolerable abuse of the power created. In the Virginia Resolutions and Report the plural number, States, is in every instance used where reference is made to the authority which presided over the Government. As I am now known to have drawn those documents, I may say as I do with a distinct recollection, that the distinction was intentional.
James Madison
James Madison understands the constitution better than you do.
You can't get around that.