That may be, but that it be doesn't make it the right direction in which we should allow our society to progress. I'll be damned and dead before I encourage or support moving away from the principle of the Golden Rule. And since this is a thread about moral philosophy, I think that is a relevant to consider for the point of morality is to address and resolve what should be not what is or was.
You are a little confused about the Golden Rule. It has nothing to do with respecting other people's rights to destroy each other.
??? No, I am most definitely not confused about it.
Would you want someone else to destroy you? I am going to assume your answer and everyone else's is "no." Therefore, don't destroy someone other than yourself.
If I am of a mind to destroy someone, I must ask myself, "Would I want them to destroy me?" My answer is "no." Therefore I must refrain from destroying them.
Is it really that difficult to see that the rule applies to every action quite easily? All the reasons why I might initially conceive to destroy another don't matter. What does matter is the act I want to commit, destroying them. All the reasons why my answer is "no" don't matter. What matters is that my answer is "no."
Why is it really just that simple? Because all those reasons -- the reasons why I want to destroy another and reasons why I don't want to be destroyed by another -- can change, but the act itself, every act in fact, once performed cannot be unperformed. One may be able to make restitution for having performed it, but act, once done, is done; it is at that instant a fact of history. When someone figures out how to reverse the flow of time itself I will almost surely have to reconsider the value of the Golden Rule. For now, however, nobody has accomplished that.