So, the founders, in the Constitution, said only citizens should be counted? Uh NO.
Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers, which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons.
Rather one was a citizen or not was not even a question of the first Census. The total number of people enumerated was just shy of four million. Without a doubt, hundreds of thousands of them were not citizens, were foreign born, and had not been naturalized. Probably, more than half were female. The majority of those counted did not own property. And then there is that damn pesky 14th amendment.
Representatives shall be apportioned among the several States according to their respective numbers, counting the whole number of persons in each State, excluding Indians not taxed.
The Constitution, including the amendments, is the supreme law of the land. You don't get to violate it to get what you want. Every member of Congress, of the Senate, the President, and the Vice-president, swear an oath, to DEFEND the Constitution, not ignore it when it suits their purposes.