You Can't Trust Official Statistics

[beretta304]

All of these things are true, to widely varying degrees, about all the economic stats. Unemployment data is notoriously invalid and unreliable in the U.S. and everywhere. They change the small-print, hidden operational definitions frequently, normally in part to keep the official rates down.

The mass of Labor Dept. statisticians have nothing to do with this. The few top political appointees and "expert" committees they appoint make the decisions, as is usually the case with official stats.

If you read the fine print of the "experts," you can follow these things. Who does? The top "experts" working with the political leaders of all these official statistical organizations can vary all kinds of "small variables" hardly noticed by non-experts – "seasonal adjustments," periods considered, the wordings and personal characteristics of telephone pollsters, decisions to recall or not, etc., etc.


You Can't Trust Official Statistics by Jack D. Douglas

 

[oldfart]

All of these things are true, to widely varying degrees, about all the economic stats. Unemployment data is notoriously invalid and unreliable in the U.S. and everywhere. They change the small-print, hidden operational definitions frequently, normally in part to keep the official rates down.

OK I read the Jack Douglas article and I think I understand the points he is trying to make. I am not sure what points you are trying to make. I have some experience in statistical analysis in general and statistical analysis in population studies (Douglas's field) and labor statistics. I'm not being defensive or snarky, I just trying to see what points you are making so I can respond intelligently.

You are correct that some data is more amenable to statistical analysis than other data. Economic statistics can vary quite a bit. I'm not sure in what sense you think unemployment data is "notoriously invalid and unreliable." Validity and reliability have specific meanings in statistics and are not the same thing. Reliability is a measure of stability of results when multiple samples are taken or different researchers try to duplicate the same process. Validity is a measure of whether an analysis or process measures what it says it measures and not something else. Unemployment statistics do not seem to me to be a lot worse than other economic statistics. I see a lot more problems with the validity of indexes that measure inflation, for example. Where do you think the unemployment statistics are msleading? Is it a problem with gauging intent and defining who is looking for work but not employed?

The mass of Labor Dept. statisticians have nothing to do with this. The few top political appointees and "expert" committees they appoint make the decisions, as is usually the case with official stats.

I'm guessing you are commenting on the definitions and how they are changed and with methods like seasonal adjustments and the annual "baseline" adjustment. Am I correct? If so, I don't think you can avoid these adjustments, but they should be transparent so that peer review can bring problems to light. Are there some particular changes you feel were not transparent enough or that details of the methodology were not sufficiently public?

If you read the fine print of the "experts," you can follow these things. Who does? The top "experts" working with the political leaders of all these official statistical organizations can vary all kinds of "small variables" hardly noticed by non-experts – "seasonal adjustments," periods considered, the wordings and personal characteristics of telephone pollsters, decisions to recall or not, etc., etc.

I'll agree with you that statistical analysis can be abused, which is a good reason to insist on transparency. Polling techniques are especially subject to bias. You are correct that data gathering is a bit acane for the average citizen. But that's why I think the best way to keep the "experts" honest is to have other trained people and ordinary citizens who make the effort to inform themselves act as watchdogs and sound the alarm when something fishy turns up.