What privileges, special or otherwise, that are granted to a person is no concern of mine.
When I say that the idiocy of others has no power over me I mean precisely that. I don't choose to let them.
When the entire world went collectively insane during COVID, I refused to participate. I didn't wear a mask, I didn't receive the jab, I didn't lock myself in the house. I did what I wanted. I didn't reproach anyone else for doing exactly what they wanted. I didn't demand anyone not wear a mask or not get a shot.
The reciprocal premise of not letting anyone else have power over me is that I cannot and will not attempt to extend power over anyone else.
In Ayn Rand's novel, "The Fountainhead", the hero, Howard Roark, is confronted by the antagonist, Ellsworth Toohey, a man who very nearly destroyed his professional career. When Roark was down and things seemed to be hopeless for him, Toohey encountered him and said, "We're alone, no one will know, tell me exactly what you think of me".
Roark responded, "But, I don't think of you".