I think you're a generous person, the type moochers exploit. I also think your instincts and judgment are superior to those you humbly asked for advice, including Frannie.
These people typically make more money than they would at a warehouse or restaurant, with frequent bonus days where someone is feeling especially generous and gives them a $100 bill or multiple people give them $20's.
They only "work" half a day or so, set their own schedule, don't get drug-tested or have to explain missed days. The American people are reliably generous. What a country!
The public assumes these people are homeless, when that's usually not the case. They often have cars and homes. The begger gig beats working for a living.
These people are exploitative. They don't feel embarrassed. I don't give them anything and I don't feel the need to explain why to them or feel guilty. They are counting on good people feeling awkward. I'm also not unfriendly or rude. In fact, I often simply say "no thanks" and move on.
Your mention of needing permits to solicit money from strangers sparked the idea that perhaps that should be applied by local authorities and enforced on these people, too.
Don't worry about them further. I'm sure your decency and generosity can be put to better use elsewhere.