The Kellerman, et al (1993) study in the NEJM attempts to use the case-control method (CCM) to show that gun ownership increases homicide in the home. The limitations of the CCM, and serious flaws in the study methodology, result in invalidation of the study's conclusions.
The CCM has a number of limitations in what it can accomplish, and has a number of conditions (assumptions) which must be satisfied for it to be able to satisfactorily accomplish even the limited goals for which it is suitable. The biggest limitation is that the CCM can't demonstrate causation. The CCM finds 'associations' between studied factors and the 'outcome' which defines the 'cases'. These 'associations' may suggest that there is a causal relationship, and may then be used to justify a study of causal relationships, but it is incorrect to jump from the discovery of an association to a conclusion of causation. Other weak points in the CCM have to do with susceptibility to biases in the selection of the cases, and with confounding factors which can affect the choice of the controls. These can easily lead to spurious associations when there actually are none, or to associations which are reversed in direction from what actually exists.
The Kellerman, et al (1993) study has been widely quoted as demonstrating that there is a causal relationship between handguns in the home and homicides. The paper itself doesn't go that far, but it uses suggestive language, which suggests that there is more than merely an 'association'. The flaws in the paper are such as to make the the reader suspicious of the association found. Showing flaws in the methods does not prove that the paper is wrong, but it causes a loss of confidence in the results. Conclusions which are not properly supported must be considered invalid until proper support becomes available, if ever. It is the responsibility of the authors to support their conclusions. It isn't the responsibility of the readers to go out to collect data to prove that the flaws in the paper lead to incorrect conclusions.
The detailed treatment of these flaws, with supporting data, examples and methods is necessarily quite long, but it does illustrate that the Kellerman, et al paper is based on unsupported assumptions and that the conclusions must be viewed with suspicion or rejected as being unsupported.
-- Henry E. Schaffer, Ph. D.
Serious Flaws in Kellerman