Lies, damned lies and statistics of antidepressant effectiveness

The BMJ has published a head-to-head about whether antidepressants are overprescribed, with Des Spence saying Yes and Ian Reid saying No. Reid quotes the study by Fountoulakis & Möller (2011) that provided a re-analysis and re-interpretation of the Kirsch data, which I have mentioned previously (eg. see post). Reid concludes, "Sadly, demonstrations of methodological flaws and selective reporting suggest that the conclusions [of Kirsch] were 'unjustified.'"

What Reid doesn't quote is the response by Kirsch et al (2012) which shows that the original calculations were in fact correct. The discrepancy comes from using different statistical techniques, the effect of which is that the analysis by Fountoulakis & Möller treats individual studies as though they are equivalently powered. This is contrary to the standard meta-analytic technique of weighting studies with a large sample size more than the ones with a small sample size.

Let's not get too hung up about the statistics! What is significant is that Reid uses a discrepancy like this to try and undermine Kirsch's conclusion. The fact is that the effect size in antidepressant trials is much smaller than is commonly assumed. Not everyone responds to antidepressants even in the clinical trials. It is possible that the small effect size could be explained by expectancy effects introduced through unblinding (eg. see the article by Jo Moncrieff and myself).