A detailed look at how bad the National Crime Victimization Survey is......read this to understand how bad the National Crime Victimization Survey actually is at determining defensive gun uses....
A. THE NATIONAL CRIME VICTIMIZATION SURVEY (NCVS)
However consistent the evidence may be concerning the effectiveness of armed victim resistance, there are some who minimize its significance by insisting that it is rare.[15] This assertion is invariably based entirely on a single source of information, the National Crime Victimization Survey (NCVS).
Data from the NCVS imply that each year there are only about 68,000 defensive uses of guns in connection with assaults and robberies,[16] or about 80,000 to 82,000 if one adds in uses linked with household burglaries.[17] These figures are less than one ninth of the estimates implied by the results of at least thirteen other surveys, summarized in Table 1, most of which have been previously reported.[18] The NCVS estimates imply that about 0.09 of 1% of U.S. households experience a defensive gun use (DGU) in any one year, compared to the Mauser survey's estimate of 3.79% of households over a five year period, or about 0.76% in any one year, assuming an even distribution over the five year period, and no repeat uses.[19]
The strongest evidence that a measurement is inaccurate is that it is inconsistent with many other independent measurements or observations of the same phenomenon; indeed, some would argue that this is ultimately the only way of knowing that a measurement is wrong. Therefore, one might suppose that the gross inconsistency of the NCVS-based estimates with all other known estimates, each derived from sources with no known flaws even remotely substantial enough to account for nine-to-one, or more, discrepancies, would be sufficient to persuade any serious scholar that the NCVS estimates are unreliable.
Apparently it is not, since the Bureau of Justice Statistics continues to disseminate their DGU estimates as if they were valid,[20] and scholars continue to cite the NCVS estimates as being at least as reasonable as those from the gun surveys.[21] Similarly, the editors of a report on violence conducted for the prestigious National Academy of Sciences have uncritically accepted the validity of the NCVS estimate as being at least equal to that of all of the alternative estimates.[22] In effect, even the National Academy of Sciences gives no more weight to estimates from numerous independent sources than to an estimate derived from a single source which is, as explained below, singularly ill-suited to the task of estimating DGU frequency.
This sort of bland and spurious even-handedness is misleading. For example, Reiss and Roth withheld from their readers that there were at least nineother estimates contradicting the NCVS-based estimate; instead they vaguely alluded only to "a number of surveys,"[23] as did Cook,[24] and they down played the estimates from the other surveys on the basis of flaws which they only speculated those surveys might have. Even as speculations, these scholars' conjectures were conspicuously one-sided, focusing solely on possible flaws whose correction would bring the estimate down, while ignoring obvious flaws, such as respondents (Rs) forgetting or intentionally concealing DGUs, whose correction would push the estimate up. Further, die speculations, even if true, would be wholly inadequate to account for more than a small share of the enormous nine-to-one or more discrepancy between the NCVS-based estimates and all other estimates. For example, the effects of telescoping can be completely cancelled out by the effects of memory loss and other recall failure, and even if they are not, they cannot account for more than a tiny share of a discrepancy of nine-to-one or more.
Equally important, those who take the NCVS-based estimates seriously have consistently ignored the most pronounced limitations of the NCVS for estimating DGU frequency. The NCVS is a non anonymous national survey conducted by a branch of the federal government, the U.S. Bureau of the Census. Interviewers identify themselves to Rs as federal government employees, even displaying, in face-to-face contacts, an identification card with a badge. Rs are told that the interviews are being conducted on behalf of the U.S. Department of justice, the law enforcement branch of the federal government. As a preliminary to asking questions about crime victimization experiences, interviewers establish the address, telephone number, and full names of all occupants, age twelve and over, in each household they contact.[25] In short, it is made very clear to Rs that they are, in effect, speaking to a law enforcement arm of the federal government, whose employees know exactly who the Rs and their family members are, where they live, and how they can be re contacted.
Even under the best of circumstances, reporting the use of a gun for self-protection would be an extremely sensitive and legally controversial matter for either of two reasons. As with other forms of forceful resistance, the defensive act itself, regardless of the characteristics of any weapon used, might constitute an unlawful assault or at least the R might believe that others, including either legal authorities or the researchers, could regard it that way. Resistance with a gun also involves additional elements of sensitivity. Because guns are legally regulated, a victim's possession of the weapon, either in general or at the time of the DGU, might itself be unlawful, either in fact or in the mind of a crime victim who used one. More likely, lay persons with a limited knowledge of the extremely complicated law of either self-defense or firearms regulation are unlikely to know for sure whether their defensive actions or their gun possession was lawful.
It is not hard for gun-using victims interviewed in the NCVS to withhold information about their use of a gun, especially since they are never directly asked whether they used a gun for self-protection. They are asked only general questions about whether they did anything to protect themselves.[26] In short, Rs are merely given the opportunity to volunteer the information that they have used a gun defensively. All it takes for an R to conceal a DGU is to simply refrain from mentioning it, i.e., to leave it out of what may be an otherwise accurate and complete account of the crime incident.
Further, Rs in the NCVS are not even asked the general self-protection question unless they already independently indicated that they had been a victim of a crime. This means that any DGUs associated with crimes the Rs did not want to talk about would remain hidden.
It has been estimated that the NCVS may catch less than one-twelfth of spousal assaults and one-thirty-third of rapes,[27] thereby missing nearly all DGUs associated with such crimes.
In the context of a non anonymous survey conducted by the federal government, an R who reports a DGU may believe that he is placing himself in serious legal jeopardy. For example, consider the issue of the location of crimes. For all but a handful of gun owners with a permit to carry a weapon in public places (under 4% of the adult population even in states like Florida, where carry permits are relatively easy to get)[28], the mere possession of a gun in a place other than their home, place of business, or in some states, their vehicle, is a crime, often a felony. In at least ten states, it is punishable by a punitively mandatory minimum prison sentence.[29] Yet, 88% of the violent crimes which Rs reported to NCVS interviewers in 1992 were committed away from the victim's home,[30] i.e., in a location where it would ordinarily be a crime for the victim to even possess a gun, never mind use it defensively. Because the question about location is asked before the self-protection questions,[31] the typical violent crime victim R has already committed himself to having been victimized in a public place before being asked what he or she did for self-protection. In short, Rs usually could not mention their defensive use of a gun without, in effect, confessing to a crime to a federal government employee.
Even for crimes that occurred in the victim's home, such as a burglary, possession of a gun would still often be unlawful or of unknown legal status; because the R had not complied with or could not be sure he had complied with all legal requirements concerning registration of the gun's acquisition or possession, permits for purchase, licensing of home possession, storage requirements, and so on. In light of all these considerations, it may be unrealistic to assume that more than a fraction of Rs who have used a gun defensively would be willing to report it to NCVS interviewers.
The NCVS was not designed to estimate how often people resist crime using a gun. It was designed primarily to estimate national victimization levels; it incidentally happens to include a few self-protection questions which include response categories covering resistance with a gun. Its survey instrument has been carefully refined and evaluated over the years to do as good a job as possible in getting people to report illegal things which otherpeople have done to them. This is the exact opposite of the task which faces anyone trying to get good DGU estimates--to get people to admit controversial and possibly illegal things which the Rs themselves have done. Therefore, it is neither surprising, nor a reflection on the survey's designers, to note that the NCVS is singularly ill-suited for estimating the prevalence or incidence of DGU.
It is not credible to regard this survey as an acceptable basis for establishing, in even the roughest way, how often Americans use guns for self-protection.