Because of the secondary smoke effects what a smoker does affects other people. In your own house, if both adults are smokers, then there should be no problem. However if there are kids in there I see a problem. This is why I support public prohibition on smoking. In the house ... presents a very hard dilemma. Is subjecting your kids to secondary smoke child abuse? It is causing them physical damage to their lungs and such.
I believe a person should be able to do what they want on their property without fear of the government or nosy neighbors as long as what they do does not hurt other people. If they want to kill themselves, go ahead, but don't take other people down that road.
Of course condos and rental properties have all kinds of rules, no pets and stuff like that, so go ahead and make them smoke free. People have the right and freedom to move somewhere else that will allow smoking so it is not an undue burden.