2014 was a record year for new billionaires (268). In the world, there are a total 1675 billionaires with a combined wealth of $6.4 trillion. Incidentally, only 172 billionaires are women.
Especially if you're just talking about Americans, this is a small club. But, billionaires are better described as international citizens. They hang out in Manhattan, LA, London, Morocco, Dubai, Montana, etc.. They have no peer, but with one another, and they tend to exist in these billionaire circles apart from our world, just rubbing elbows with one another.
They chat with one another about the building and preservation of their empires, and they come up with strategies to screw us over and enslave us. And if one of the billionaires upsets the golden apple cart, he can be excommunicated just as with any clique. It's all just normal human nature, but on a billionaire level with near unlimited resources to discreetly carry out their plans.
It's like Adam Smith said... "People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the public."