Psychiatry statistics smell fishy? There's a reason for that

[Altostrata]

Epidemiology Mis-counts: Systematic Bias Leads To Misleading Rates

By Allen Frances, MD September 16, 2011 Psychiatric Times

 

The entire field of psychiatric epidemiology has a systematic bias that leads it to misleadingly report what are highly inflated rates of psychiatric disorder. Psychiatric symptoms in mild form are widely distributed in the general population—from time to time, almost everyone will have some depression or anxiety. But symptoms alone do not define psychiatric disorder. As indicated in an earlier blog, the symptoms must also cause clinically significant distress or impairment. Epidemiologic studies all routinely ignore this crucial requirement. They mistakenly diagnose as psychiatric disorder symptoms that are mild, transient, and lacking in clinical significance.

 

How can an entire field of scientific endeavor go so far astray? The reason for the error comes down to simple dollar and cent cost considerations. Epidemiologic studies sampling the general population require very large n's with the performance of many thousands of interviews. It would be too expensive to employ clinicians in so extensive an endeavor—so the studies rely on the cheap labor provided by lay interviewers who have no clinical experience and no discretion in judging whether symptoms are clinically meaningful. They make their diagnoses of psychiatric disorders based on symptom counts alone with no consideration of whether the symptoms are severe enough to warrant diagnosis or treatment. Results generated in this rough and ready way should be considered as no more than an upper limit on the rate of disorder. They should never be taken at face value as a true reflection of the real rates of mental disorder in the community.

 

But the exaggerated rates are always reported without proper caveat and are accepted at face value as if they are an accurate reflection of the real prevalence of psychiatric disorder. This is ludicrous, especially since the methods used have become much looser with time. Psychiatric epidemiology is a young field that emerged only 30 years ago with the development of diagnostic criteria. The first large, systematic study was (the Epidemiologic Catchment Area) found fairly high rates of disorder, but in the next generation these bubbled further (sometimes almost doubling) due to the use of even less rigorous methodology. The reported diagnostic inflation did not reflect any real change in the mental health of America—just greater carelessness in the application of the tools used to measure it. We are not suffering from an epidemic of psychiatric disorder—instead we are being misinformed by the results of misleading science.

 

Were one to believe NIMH, almost 25% of the general population of the US has a mental disorder in any given year. ....Prospective studies suggest even higher rates of lifetime disorder. But all this is based on the fantasy that the lay interviewers can make accurate diagnoses using simple symptom counts. They can't.

 

So my advice is never to take seriously the rates reported by any epidemiological study. Be aware that they are always overestimates including people who may have the required symptom pattern but do not necessarily have sufficient severity, duration, distress, or impairment to be considered mentally ill. The dirty little secret of epidemiological research is that it can report only upper limits, not true rates—which would almost certainly be much lower especially for the milder conditions at the populous boundary with normality. Epidemiological research is necessary and valuable, but its limitations must be acknowledged and its results taken with more than a grain of salt.

 

 

http://www.psychiatrictimes.com/blog/couchincrisis/content/article/10168/1950964

[cinephile]

We are not suffering from an epidemic of psychiatric disorder—instead we are being misinformed by the results of misleading science.

Bravo, Mr. Frances. Now was that so hard?

 

How can an entire field of scientific endeavor go so far astray? The reason for the error comes down to simple dollar and cent cost considerations. Epidemiologic studies sampling the general population require very large n's with the performance of many thousands of interviews. It would be too expensive to employ clinicians in so extensive an endeavor—so the studies rely on the cheap labor provided by lay interviewers who have no clinical experience and no discretion in judging whether symptoms are clinically meaningful.

I'll respond to this question the way I do whenever I hear that there's no money for public school education, there's no money for research into non-pharma health interventions, etc etc:

 

WHY ARE WE ALWAYS BROKE WHEN IT COMES TO DOING THE RIGHT THING?

[brighteningup]

Because there is no obvious PROFIT in it.

 

IMHO some of society needs to wake up and realise that some things are good for overall human wellbeing and this is good for all of us and they are worth paying for (and yes this might mean higher taxes, or more richer people being more generous with giving their money to good causes).

 

But these are often things that will make all of society a bit better off , not just a few people quite a lot richer. This is research where there are no patents to be had, nothing to sell, no profit to be made, just findings that can lead to people being happier in the long term.

 

Until we accept this, much research that could make a lot of peoples lives a bit better but has no money in it, won't get done.

[Altostrata]

It's true profit is behind this, but I believe they don't realize the harm that doing on the cheap causes. Also, the exaggerations serve the purposes of the pro-drug lobby and those who have made an industry of mental health, including the very well-meaning group that works for removing the stigma from mental illness.

[alexjuice]

Yes, there was an article in The Atlantic a while back about studies and the validity of their conclusions.

 

Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science.

The article, it's really a feature on Ioannidis is very interesting. I forget if this in the article or not but Ioannidis has been a big publicist for the 'exercise beats antidepressants' bit... I'll post a quote or two and HERE's THE LINK.

 

[ioannidis is] what’s known as a meta-researcher, and he’s become one of the world’s foremost experts on the credibility of medical research. He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences. Given this exposure, and the fact that his work broadly targets everyone else’s work in medicine, as well as everything that physicians do and all the health advice we get, Ioannidis may be one of the most influential scientists alive. Yet for all his influence, he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem.

“I realized even our gold-standard research had a lot of problems,” he says. Baffled, he started looking for the specific ways in which studies were going wrong. And before long he discovered that the range of errors being committed was astonishing: from what questions researchers posed, to how they set up the studies, to which patients they recruited for the studies, to which measurements they took, to how they analyzed the data, to how they presented their results, to how particular studies came to be published in medical journals.

 

This array suggested a bigger, underlying dysfunction, and Ioannidis thought he knew what it was. “The studies were biased,” he says. “Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there.” Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them.

It's an interesting piece if anyone wants to peruse.

 

Alex