Hands free does fix a lot of the problem, especially with voice activation (like on my phone). In fact, I find that it keeps me awake and alert when I can carry on a conversation while driving. Holding a phone, on the other hand, has the added distraction of trying to operate the wheel, turn signals, windshield wipers, and, in my case, a manual transmission all with just one hand.
As for your 'speed kills' comment, I should really remind you that speed not only makes collisions more dangerous (do you really think you're not more likely to die hitting a phone pole at 90 as opposed to 60?), but driving too fast, especially under adverse conditions, makes you far more likely to spin out of control if you have to make a sudden stop. During nighttime hours, it's also a really bad idea to drive so fast that your stopping distance is greater than the range of your headlights, for obvious reasons.
Lots of stuff everyone does distracts from driving. I, personally, go the speed of the prevailing traffic and no faster, avoid any distractions (even eschewing talking when in heavy traffic or adverse conditions). If I used any mind-altering substances, I would be sure to be sober before hopping behind the wheel. Anybody who tries to multitask while driving, speeds (and not just to keep up with traffic), drives intoxicated, or several of these other things that can make driving more dangerous, but says that it's not really dangerous "because he can handle it" has been lied to, usually by himself.