Whistling does affect me physically. I find whistling to be physically painful. And I'm not being sarcastic or facetious, either. It actually hurts. I can't stand listening to a radio, either. I've been harrassed at work for years about it. There is this noise the eminates from a radio that is just like fingers on a chalk board to me. It doesn't matter what station it's on. It has nothing to do with what kind of music is playing. There is this noise, and it varies in pitch and tempo and it drives me crazy. It's all I can hear and I can't concentrate on anything else. I can hear a radio at 500 yards. Asking people to turn it down doesn't work and just pisses them off. People take their radios very personally.
I once walked into a hardware store with my sister to buy paint. We were standing at the display and I was trying to pick out the color. The radio in the store was so loud I couldn't hear myself think. I remarked about it to my sister and she acted shocked. She didn't even notice it, she said.
I don't ask people to stop whistling or turn their radios down around me anymore. Human nature is such as that soon as you tell someone that something bothers you, they will go out of their way to do it to you. If you don't beleive me, try it. Tell 10 people something bothers you and I guarantee that within your next five encounters with that person, every single one of them will have done exactly what you told them bothers you. Just make sure it's not something that really does bother you, because they will do it.
No, now I work at home as much as I can, and if I'm in a store or out in public when somebody start whistling, I just leave as fast as I can.