I have been struggling recently to find ways to rehash my post on scientific authority without causing NPG any further distress. This evening, on the train journey home, I think I finally found a way because I read one of the most remarkable scientific papers I have ever come across.
The paper, by Keating et al., brings us directly back to the ongoing saga of the British Chiropractic Association’s libel suit against Simon Singh. It was published in 2005 in the open access journal Chiropractic & Osteopathy and at least three of the six authors have affiliations with Chiropractic colleges. To all six authors I would like to say:
Gentlemen, I salute you.
The title of the paper, Subluxations: Dogma or Science?, is quite telling. Inside the authors fearlessly examine the status of the subluxation syndrome, a founding concept in chiropractic since the introduction of this method of treatment in the 1890s. I have struggled a little to find a definition of subluxations. As quoted in the paper, the Association of Chiropractic Colleges states:
A subluxation is a complex of functional and/or structural and/or pathological articular changes that compromise neural integrity and may influence organ system function and general health.
You see what I mean? If I understand correctly, the idea is that problems with the spine can propagate to the rest of the body via neural pathways. But I am not the only one wrestling with this concept. The whole substance of the paper is a discussion of how the idea of subluxations has come to pervade chiropractic without any experimental verification. It’s a pretty rigorous treatment. The summary is worth quoting in full (I am taking full advantage of the open access!):
Hypothetical constructs involve tentative assertions about physical reality. They serve as essential tools in the development of science, and permit the empirical testing of the non-obvious. However, when the speculative nature of an hypothesis or hypothetical construct is not made obvious, an otherwise acceptable proposition becomes a dogmatic claim. Such is the history of subluxation in chiropractic.
This brief review of the role of subluxation dogma in clinical practice, in marketing, in the legal and political arenas, as a basis for professional identity, and in the rhetoric of leading chiropractic organizations and agencies, is not a statement about subluxation’s validity or lack thereof. Only focused clinical research will enable us to determine whether the traditional chiropractic lesion merits clinicians’ attention. We don’t know whether subluxation is meaningful or not.
The dogma of subluxation is perhaps the greatest single barrier to professional development for chiropractors. It skews the practice of the art in directions that bring ridicule from the scientific community and uncertainty among the public. Failure to challenge subluxation dogma perpetuates a marketing tradition that inevitably prompts charges of quackery. Subluxation dogma leads to legal and political strategies that may amount to a house of cards and warp the profession’s sense of self and of mission. Commitment to this dogma undermines the motivation for scientific investigation of subluxation as hypothesis, and so perpetuates the cycle.
The simple expedient of amending dogmatic assertions to note their tentative, hypothetical character could do much to improve the image of the profession, to re-orient it to the challenge of testing its cherished hypotheses and to establishing the cultural authority of chiropractors in our unique realm of health care. The task of reorienting the profession to a credible science and art belongs to all who understand the scourge of dogma, and who seek a brighter future for the chiropractic profession and its patients.
If that has whet your appetite, I can recommend the rest of the paper. It was certainly a revelation to me who, in common with many others, I suspect, knew very little about chiropractic until a short while ago. The authors’ candour in squaring up to a fundamental problem within their own field is refreshingly scientific.
What a contrast this makes to the rather feeble pronouncements that have come from the BCA in recent weeks in response to the mounting campaign in support of Simon Singh. Their main complaint is that the libel case “has been re-characterised by his supporters as a freedom of speech issue”. Writing in New Scientist today, Richard Brown (chiropractor and vice-president of the BCA) avowed “To stifle scientific debate would clearly be wrong.”
I think they are terribly confused about the necessary connection between free speech and scientific debate.
If Keating and colleagues are correct, the chiropractic profession has spent the past century not engaging in scientific debate. An earlier press release from the BCA mentioned “27 different publicly-available research papers” that Singh should have consulted before writing his contentious article but I haven’t been able to find out which papers they are referring to. Since they are already in the public domain, why can they not be listed, so that a proper scientific debate can commence?
In science we know — from hard-won experience — that the culture of vigorous and ruthless examination is our best prospect for learning about the workings of the world, including the processes of health and disease. It’s how we figure out what authorative statements can be made about the nature of reality. The process isn’t always pretty or efficient. Brilliant ideas are conceived and nurtured but they’re not always right and when the error is found, the idea is eventually discarded and left to die. No-one likes to be proved wrong – we are all human, after all. But as scientists we accept that only through the detection of error can progress be made. Hypotheses are made to be falsified. Science flourishes because it has the freedom to err.
That is why the stifling of scientific criticism through the law courts is so lamentable. The BCA claims it is only defending its reputation but the implications of its actions have spread far wider than that.
Let me leave you with what was for me the most interesting paragraph from the Keating paper:
Chiropractors’ insistence upon defining the profession in terms of a hypothetical (and largely untested) construct is foolish at best: subluxation may or may not be a meaningful notion. This commitment also augurs against the conduct of clinical research to confirm or refute the utility of the subluxation construct, firstly because the presumption of validity undermines the motivation to investigate, and secondly because such research has the potential of undermining this proposed identity (i.e., subluxation doctor). The erosion of reimbursement for chiropractic services is also a possibility if subluxation research fails to measure up to expectations.
I have no further comment, but would welcome your comments and criticism!
Keating, J., Charlton, K., Grod, J., Perle, S., Sikorski, D., & Winterstein, J. (2005). Subluxation: dogma or science? Chiropractic & Osteopathy, 13 (1) DOI: 10.1186/1746-1340-13-17
I think you (and the paper by Keating et al.) have hit on the very essence of the issue. Science only operates by intentionally designing hypotheses so that they can be disproved. By endorsing subluxation chiropractic — like intelligent design or crystallography — is little more than pseudoscience. You must subject your ideas to rigorous testing procedures and open debate or you forfeit the authority that comes with the modern scientific process. You can’t have one without the other. Many have tried to utilize the language of science without the method and they have failed. Insofar as the BCA rejects the scientific method they will also fail.
Thanks Eric, but I’m hoping you mis-typed when you lined up crystallography alongside intelligent design!
snerk
Whoops, that’s not what I meant at all. Let’s try crystal healing or pyramidology.
I’ll let you off this time, especially since I’ve made the link between crystallography and crystal healing once before myself…
I have to confess that I found it difficult to understand that paper, but then I find if difficult to understand many papers that are not in my constantly diminishing area of expertise. Sometimes this is because the ideas are difficult, but I do have a theory that some disciplines dress up simple concepts with complicated words that actually mean very little. I suppose the argument could be that these words are a short-hand for specific ideas that are frequently used by practitioners – but there is another, more cynical explanation. This is that they are there just to keep the non-practitioners out. However, whatever the reason, the result seems to be that because few people (if any) know what these words really mean they are accepted with a sort of faith, and the discipline starts to become religion. As such it cannot be examined in a scientific way because there is no science there. It is, in fact, a matter of faith – and to be honest I think the majority of the world’s population is quite happy with that. I think this acceptance is a big obstacle to scientific investigation.
@Clare – I do have a theory that some disciplines dress up simple concepts with complicated words that actually mean very little.
Yes – but this tendency is rife in evidence-based medicine as well as the complementary variants!
there is another, more cynical explanation. This is that they are there just to keep the non-practitioners out.
I think also that your cynicism may not be mis-placed. From reading the paper I got a strong sense of how much chiropractors seem to want to identify themselves as separate from the mainstream (though often at the same time seeking endorsement from GPs and the NHS). I came across a chiropractor web-site this week that proclaim in block capitals: Only a chiropractor can tell you if you need chiropractic treatment!
The impression I got from the Keating paper is that certain branches of the profession believe they have some kind of special or sacred knowledge that sets them apart. Hence the word ‘dogma’ in the title – which of course is more commonly associated with religion. In fact the assertions about subluxation given for various different chiropractic associations in Table 1 of the paper struck me as a type of creed. If this it the mindset, it is no surprise that there is a reluctance to engage fully with the scientific method. That is what makes Keating et al.’s stance all the more admirable. However, I couldn’t find anything more recent in PubMed from these authors indicating that they had initiated their own experimental probe of subluxation.
I did come across this paper from Keating in 2001: Unsubstantiated claims in patient brochures from the largest state, provincial, and national chiropractic associations and research agencies. This seems very interesting in the light of the Singh case…
The idea that many alternative therapies are effectively religions, founded on faith in a belief system that is not subject to testing, is a popular one among those of us that have debated the Alternative Medicine protagonists.
Homeopathy is perhaps the classic example. Beyond just the regular mystic hogwash (the Vitalist philosophy which underlies many Alternative Therapies) it has a founding father, a sacred book, a priestly caste (the homeopaths), and transfiguration of base substances (water) into medicine / curative artifact (the remedy) through a special ritual. It also has regular schisms where the practitioners argue bitterly about whose faction is “truer” to the unchanging wisdom of the founder.
At various times some of the Alternative Therapies have overlapped closely with religions. Indeed, the founder of Chiropractic, DD Palmer, and his son BJ, both thought hard about turning Chiropractic into an overt religion in the early part of the 20th century.
I’m beginning to see this more and more, Austin. This is an excerpt from the abstract of a 1998 review of chiropractic (full text freely available):
The italics are mine. I guess there is a culture of ‘debate’ within the profession, but it may be more about dogma than science…
I wonder if, when they are having these (apparently very heated) debates within the profession, they regularly threaten to fill libel suits against each others too…
“Internally, chiropractic has a dramatic legacy of strife and factionalism”
Indeed. When the true faith is how you make a living, you are defending the faith and your livelihood. A powerful pair of drivers.
Apart from the analogies with religions, the bitter disputes and schisms within branches of Alternative Medicine also remind me irrestibly of this classic.
Another characteristic of Alternative Therapies and their practitioners, no matter how bonkers, is that they take themselves incredibly seriously. One of their successes in making inroads into conventional medicine in recent years has been via insisting that other people have to “respect” them by taking them, and their claims, just as seriously as they take themselves. And if you don’t, they get tremendously offended. I have offended quite a few since, like David Colquhoun, I tend towards the idea that ridicule is often the most appropriate response to complete rubbish. To repeat an apt phrase:
“Laughter is the best detergent of nonsense” [A saying of AV Hill’s, quoted by his protege Bernard Katz – for famous 1930s context see here or here.
@Cristian – that’s an interesting thought but I imagine most legalistic ire is reserved for outsiders.
@Austin – glad to see my mangling of the italics didn’t divert you from the point. Isn’t is amazing how much great good sense there is in the classic you link to, Monty Python’s Life of Brian?
More seriously, ridicule is fun and perhaps the only way to stay sane if one engages in debate with alternative medicine practitioners, but are there no alternative ways (no pun intended) of trying to reach adherents of these therapies…?
I guess there’s no remedy for the diversity of human nature. 😉
BTW, the “Chiropcalypse” has now popped up over at Nature’s The Great Beyond blog
More seriously, ridicule is fun and perhaps the only way to stay sane if one engages in debate with alternative medicine practitioners, but are there no alternative ways (no pun intended) of trying to reach adherents of these therapies…?
Now that is a really interesting question, Stephen. I think the clarifying question is “adherents of what degree of adhesiveness?”
I think there is little one could do to reach full-on True Believers, although I suppose if one goes on explaining the evidence then once in a while you may reach somebody. But many of the folk who believe this way postively revel in feeling “opposed and oppositional”, as that is part of their self-validation.
Most people blogging these kind of areas think that you simply have to keep trying to put the real story out there as a way of reaching the genuinely misinformed, rather than the “Militantly Misinformed and Proud to Be”.
An interesting question, which we were alluding to over on Henry’s MMR compulsory vaccination thread when all the steam went out of it, was what ways one could put the evidence over that “score” better than the narratives of the Alt.Medicine lot. This is the dreaded “framing” argument. There are some ideas e.g. here.
One thing I have tried to plug on those rare occasions I meet one of the scientific Great and Good is that there ought to be, somewhere, the scientific equivalent of the “rapid rebuttal” team from the early New Labour days. When a Crap Science Story gets out into the wild (aka the news), it usually has days and days of getting re-hashed in the media before anyone points out authoritatively that it is rubbish.
I used to argue that if the Royal Society, say, had an office that would put out a statement saying “This is overblown nonsense”, and would offer to link journos with eminent people who would tell them why, that that might perhaps help. I guess outfits like Sense About Science and the Science Media Centre are trying to do some part of this, but I have to say it doesn’t seem to be working that well generally – though SaS are doing a bang-up job with the Singh vs. BCA campaign. But then that has been drawn-out, offering time to get organised.
Of course, any time any such organisation can be “framed” by the Alt.Reality lot as “a paid lobby” – which happens any time they have any sort of underpinning funding from industry, for instance – that can end up being more of a hinderance than a help. It really needs to be done by a body which is unambiguously identified with “disinterested scientific expertise”, and cannot be smeared easily as “lobbyists”.
Re. your first line about “ridicule is… perhaps the only way to stay sane if one engages in debate with [the CAM people]”, I think that is right – it is certainly how I feel. Though most bloggers mix exposition and mickey-taking in varying proportions.
Of course, one nay be just preaching to the choir, but Ben Goldacre uses two arguments which I have a lot of time for, namely:
(i) you have to equip people with a basic critical “tool kit” somehow, and if schools are missing people then maybe blogs are another way to try;
(ii) if you give online nerds the info and tools to be able to explain and debunk, they will get out there into the wider world (or into their various little communities) and get debunking
– the latter being a kind of “viral anti-anti-science” idea.
Anyway, enough for now. I notice this is getting so protracted that I may have to get myself a NN blog to put this sort of stuff on, rather than clogging up the comments…
Austin – you are welcome to clog my blog with your comments anytime.
it usually has days and days of getting re-hashed in the media before anyone points out authoritatively that it is rubbish
But, as I think you would say yourself (framing argument?), who is to determine whether someone is an authority? One person’s independent expert is another’s paid lackey. If we are leaning on someone’s word as an ‘expert’, we have to take things on trust. But trust has to be earned. I think this may be picking up on themes that emerged on Henry’s piece about MMR (and the PLOS Biology article that you link to), but it probably does little good for scientist to swagger about with a ‘we know best’ attitude.
A degree of humility or modesty about the limitations of science (and evidence-based medicine) is perhaps one way to try to earn that trust. But I guess factional interests, real or percieved, will always be an obstacle for some people.
I second your recommendation of Ben Goldacre’s approach – power to the people! And, having just read it, I can thoroughly recommend his book.
As a footnote to re-iterate the point that convincing some people is really not an option, this comment from the Times’ Daniel Finkelstein, though on a different subject, is apposite.
Thanks Austin – a good article, if a little depressing.
I see tonight that the BCA has, albeit rather belatedly, released the list of the “available research about the
effectiveness and safety of chiropractic treatment”. It can be downloaded here.
Well done to them for doing so! The analysis and debate can now begin in earnest. It’s a good idea to follow #singhbca on Twitter if you want to keep up.
Well things have really begun to kick off. The BCA said in their statement that it “welcomes full, frank and open scientific debate.” That’s good, because the analysis of their evidence is coming thick and fast.
I have been struggling to keep up myself, but for those who would like to read some of the emerging commentary on the BCA listed evidence to back their claims that chiropractic can help various childhood conditions, check out these bloggers, who have been doing sterling work.
This list has been pinched in its entirely from Martin Robbins, The Lay Scientist, who has made his own trenchant analysis revealing possible evidence of quote mining by the BCA. I’ve read — and would recommend — the one’s in boldface (and Martin’s piece!).
*Jack of Kent – General commentary and legal background.
Prof. Colquhuon – Detailed look at the nine colic papers.
*
Ministry of Truth – General review focusing on three of the colic papers.
Andy – Comment on the BCA statement.
Evidence Matters – Review of the paediatric asthma papers.
*Gimpy – Review of the ear infection papers.
Phil Plait – An overview of the BCA statement and aftermath.
*
HolfordWatch – What would constitute good evidence?
*Apgaylard – A more detailed look at the bed-wetting papers.
JDC – General comment on the BCA statement.
*
Think Logic – General comment on the BCA statement.
The blogosphere – it’s a thing of beauty.
The Keep Libel Laws Out of Science campaign is still rolling on (12000 signatures reached by last Friday).
Latest news is that Nature has donated a full page ad to run in tomorrow’s edition.
Kudos to Nature for taking a stand, both with its earlier editorial and now with this.
Yes I heard about the ad. Well done to Nature for publicising this campaign!
Hooray! When do we march on Parliament?
@Richard – I got this in an email from Sense About Science last night:
I’m not sure I understand their strategy. In any case, like you, I’ve already written to my MP on this issue. I’ve not yet had a reply. Did you have any luck?
That strategy does seem odd. I had an automated reply but nothing personal yet.
Interesting paper on scientific reliability (mentioned on Twitter by @bengoldacre) has been published in PLos ONE: Large-Scale Assessment of the Effect of Popularity on the Reliability of Research
Haven’t had time to digest fully yet. Here’s the opening:
Conclusion seems to be that ‘popular’ areas of science are more likely to lead to erroneous publications. But you knew that already.
What is impressive is how it exemplifies the self-critical nature of science. Real science.