Swasticas and KKK hoods probably weren't generally viewed as racist and vile in the beginning (except among blacks).
That may well be true, but... the swastica (once a symbol of fertility) has come stand for the holocaust, and the hood stands for thousands of murders and decades of state-sanctioned terror of tens of millions. Objectively, MAGA stands for nothing of that magnitude. That may still come, but they aren't there yet. In a way, I find the owner's attempt at throwing them all into one category is belittling the other two symbols. And all, I suspect, to enhance his name recognition.
I do understand: That hat serves no purpose other than poking a thumb into every liberal's eye - which has to count as the main, if not the only, source of fun for MAGA hatters and similarly imbecilic crackpots. Overreaction to same I still find ill-advised.