I can't tell if most people dislike hate crime legislation period or just dislike it because in their perception it doesn't protect white people (or white men, since gender is protected).
So which is it? There should be no hate crime legislation or it should protect white people, too?
Speaking from the legal perspective, I never really understood the value of adding time for someone's motive. In criminal cases, motive is not part of the case. I don't understand why you should bring in some other idea to specific cases categorized as "hate" crimes.
Crimes are not usually acts of love, so people shouldn't get too confused. If there is a specific action you would like to increase a penalty for, I think that's fine. Like in Virginia, we have Project Exile and the like, you commit a crime and have a gun on you, you sit an extra five years.
But, if you are punishing a specific action with more time, I think it should serve a "compelling state interest" like punishing people who use firearms when they commit a crime. If the action is merely obnoxious, I wouldn't see it as a compelling interest. There are many things that are obnoxious and offensive.
That's my take anyway. If you are not satisfied with the level of punishment you give a certain crime, then increase it across the board.