Many years ago I read that on average a person had to work out the equivalent of 7 years to add about five years to their lifespan. To me that's a 2 year loss so I quit and have never worried about it since. I feel fine. Stress is a major killer; my brother is a health nut, but is also always pushing himself, has had serious ulcer problems, high blood pressure, tumors, knees and ankles going bad, goes at everything like he's going to war, no matter how small or large the project is, and generally annoys everybody around him.
I'm convinced personality type plays a bigger role in later health issues than all the diets, exercise fads and other stuff combined. I know as many people who don't do anything that might cause them to break out in a sweat doing just as fine health wise as those who have their entire lifestyles built around 'fitness'. True, many aren't, many will definitely benefit, but from personal experience I don't see a lot of difference statistically; lots of 90+ year old school teachers with life long sedentary lifestyles out there, and dead 50 year old joggers and crippled 'fitness junkies' and the like. I can't help but laugh at the nuts who go out jogging or cycling in urban smog, for instance, and think they're getting 'healthy'.