He wasn't their father. The father was the neighbor who wasn't paying a red cent. Whether the kids would have been upset or not doesn't matter. The man shouldn't be paying a penny for kids he didn't father.
He should not be forced to pay support for kids that he did not Father. If he willingly chooses to step up, it's been done before.