My feeling on this, and indeed many other of our "rights" is that we as a people need to realize that we unfortunately live in a time in which we can't simply trust the average person enough to make a blatant statement that all of our rights are absolute. Every single one of the original Amendments which guaranteed our rights now have restrictions on said rights, and rightfully so.
First Amendment - Taken literally, the government can NOT interfere with my right of free speech nor my freedom of religeon; yet we don't allow people to scream fire in a theater, nor even talk about the Bible, for example.
It just seems odd that everyone is willing to except the fact that these limitations MUST exist except when it comes to a right that THEY care about.
Sorry, but our forefathers never meant for us to literally be able to do anything we want without regards to the consequences.