The libertarian philosophy fails on its ignorance concerning human nature. Individuals will prey on one another if given the opportunity. Our history as mankind has demonstrated that immutable fact.
Government must retain the right to authorized violence, and it must retain the right to define what level of individual violence is permissible in self dense.
I guess this says it all about your position. Let me say this though about your first point. Libertarianism isn't the absence of law, that's anarchy.
Libertarians reject aggression so there would be no authorized use of violence. Aggressors would forfeit the right to exemption from violence from their victim. Aggressors would also be subject to the agreed upon penalties for their crimes.
Individuals prey on one another now and we are far from a libertarian ideal. There wouldn't be any less restriction on personal aggression in a libertarian society and the punishment could be more severe and much swifter.