Mailbag: Anti-Psychiatry Misinformation About Clinical Significance
It is time for another entry in the mailbag series where I answer feedback email from readers and others. If you want to send me a question, comment or any other kind of feedback, please do so using the contact info on the about page.
Anti-psychiatry is a form of pseudoscience that is based on at least three false core beliefs: the denial of the existence or severity of metal illness, the rejection of mainstream treatments for mental illness (including medication and therapy) and the demonization of psychiatrists. There are many different kinds of anti-psychiatry activists. This includes some religious extremists who deny the intimate connections between the mind and the brain, some new age believers who wrongly think that it is just a matter of positive thinking, some alternative medicine proponents who falsely claim that it is due to eating too much acidic foods and so on.
In particular, anti-psychiatry activists spread misinformation and hate about psychiatric medications in much the same way that anti-vaccine and anti-GMO activists fearmonger about vaccines and genetically modified foods. Many anti-psychiatry researchers make obvious statistical errors (by wrongly calculating standardized effect sizes) and create smokescreens about the clinical significance of antidepressants by selecting outdated and arbitrary cutoffs, when clinical significance should be based on the totality of evidence and the scientific context.
In How Anti-Psychiatry Researchers Attack Antidepressants With Faulty Statistics, Debunking Denialism exposed some of these deceptive methods when studying the efficacy of antidepressants. A recent study appeared to concluded that antidepressants did not have any beneficial effect that was clinically significant. In reality, the study relied on an outdated standard (removed around eight years ago) for clinical significance previously held by the National Institute for Health and Care Excellence. It is also arbitrary and black-and-white that ignores the fact that clinical significance is a spectrum and the scientific context. Furthermore, the standard is so high that it would reject a large chunk of all treatments in psychiatry (including psychotherapy) and general medicine.
Essentially, the anti-psychiatry researchers attempted to dismiss the clinical significance of antidepressants by trying to nuke all of medicine out of desperation. My criticism of this study showed up on a couple of anti-psychiatry websites. As a result, an anti-psychiatry activist by the name of Stephen decided to send in the following comment that can be useful to discuss in some detail:
First, Stephen uses the logical fallacy called argumentum ad hominem circumstantial. This involves judging the truth of a statement based on the circumstances of the individual making the claim. In reality, a claim stands and falls with the arguments and evidence for or against the claim, not the circumstances of the individual making the claim. Labeling the authors as anti-psychiatry is reasonable because they repeat deceptive tropes and methods common among anti-psychiatry activists. In particular, they make extreme attacks against psychiatric medication by relying on classic anti-psychiatry statistical tricks. It is thus not at all “absurd” to call them anti-psychiatry. If it quacks like a duck, walks like a duck and looks like a duck, it is probably a duck no matter how much you insist that it is really a painting of a tractor.
There is no general agreement that a HRSD different of three points (equivalent to d = 0.5) is “barely noticeable”. This is because the studies that the authors referenced to back up their claims typically traces back to other anti-psychiatry activists who rely on the subjective assessment of clinicians. However, the entire point with using rating scales to begin with was to get away from too much subjectivity on the part of the clinician! This is about how large improvements have to be to make a clinically significant improvement for the patient, not about the opinions of clinicians. The proposed level of “minimal improvement” would be seven points or a standardized effect size d = 0.875. This corresponds to a large improvement and would reject almost all effective treatments in medicine (see previous post for references). It is therefore extremely untenable. In their efforts to attack antidepressants, they end up nuking most of modern medicine.
There are many other reasons for why the claim is wrong. This is because the HRSD is an ordinal scale and heterogeneous. Because the variable measured by HRSD is measured on the ordinal scale (like rating a movie from 1 to 10, but goes higher). This means that a constant difference of, let’s say, three points can mean very different things depending on what score a person has. A decrease in three points for someone with a low score can be less important, but a three-point decline for someone with a very high score can be exceedingly important. Because the HRSD cover many different areas, from food and sleep to suicidal ideation. A difference of three points on items relating to food and sleep can be less important than a three-point decline on items related to suicidal ideation. Therefore, it is not valid to claim that an arbitrary effect size cut-off is a valid way of estimating clinical significance.
Stephen did not actually engage the objections made against the study, but merely repeated the claims he likely read in post on an anti-psychiatry website. Therefore, it is not accurate to say that the refutation of using arbitrary effect size cut-offs to evaluate clinical significance is biased. Quite the contrary.
Follow Debunking Denialism on Facebook or Twitter for new updates.
