Excellent point. Saying "I would gladly knock him into next week" is not a threat. It's an expression of speech. Saying "I'm getting my gun and I'll shoot you son of. . ." could also be an expression of speech but is also possibly a threat and should be taken more seriously.
Schumer's threat to SCOTUS justices: "“I want to tell you, Gorsuch. I want to tell you, Kavanaugh. You have released the whirlwind, and you will pay the price. You won’t know what hit you if you go forward with these awful decisions. . .” was also one of those statements that is hard not to see as an actual threat but also could just be a figure of speech.
Things like that should put people on a watch list but cannot be criminalized without evidence of intent. Take that silly unstable and unusable gallows at the Jan 6 rally. Bad form absolutely, but also clearly symbolic with zero evidence of intent. Not illegal. Just dumb.
Obvious intent or actual actions to harm persons or property should always be illegal and subject to consequences of law though due process and equal protection under the law must be ethically exercised even in those cases.
But expressed opinion, even the most hateful language, should never be criminalized.