It is timely to communicate a little about our experiences of planning and (at last) implementing RCTs over the last 15-20 years.
It’s become increasingly apparent how detached from reality RCT success rate results are and there are many reasons why we should perhaps be prepared to recalibrate our expectations in advance of the results of the Dublin RCT and, in a couple of years, the London RCT.
What is a good success rate for Allen Carr’s Easyway to Stop Smoking in a Randomised Controlled Trial (RCT)?
Based on the 3-month money back guarantee offered at our stop smoking centres the success rate is over 90%. Independent scientific studies published in peer reviewed journals indicate that even after 12 months the success rate is over 50% – a result which is unprecedented in the field of smoking cessation.
So, what would a good success rate be in an RCT that compares the effectiveness of Allen Carr’s Easyway to Stop Smoking method to that of another method such as a national stop smoking service that uses nicotine products?
Would it surprise you to know that a 7.5% success rate could be good? Or that a 10% success rate could be great? Or that a 15% success rate might be truly historic in the field of smoking cessation?
How can that be?
The important result in an RCT isn’t really how high the success rate is, but how much better the success rate is compared to the other method in the trial.
There are lots of reasons why the headline success rate in an RCT is certain to be much lower than the success rate of Allen Carr’s Easyway to Stop Smoking method in the real world.
For example, under the rules of the RCT, a smoker who signs up for the study, is allocated to a treatment, but then doesn’t bother to turn up and receive the treatment is classed as having failed to stop smoking with the treatment. Bizarre but true.
Furthermore, in the real world when a smoker attends one of our seminars and successfully stops smoking they don’t need to come back, they just get on with enjoying their smoke-free life. However, in the RCT they need to attend a follow up test to prove that they are a non-smoker and if they forget or simply can’t be bothered, they are deemed a smoker.
Another key difference is that in the real world a smoker attends our seminars as a result of their own initiative. They probably heard about Allen Carr’s Easyway to Stop Smoking method from a friend or family member who was successful with the method. They make their appointment as a result of their own thought processes. They are committed enough to pay their own fee to attend, in their own time, either taking a day off work or during their weekend. Furthermore, they are motivated to attend back-up seminars to qualify for our money back guarantee. Their profile and motivation are therefore very different from someone who is attracted to take part in an RCT, who has perhaps never heard of Allen Carr’s Easyway before, who has the cost of the seminar paid for them, and who in some cases is even paid to take part in the study and may even be attending primarily to receive the payment for doing so.
Attending follow up tests simply to prove that you quit smoking takes commitment, time and motivation which is why in studies it is very normal for a high number of people to drop out and be mistakenly recorded as smokers even though they may have actually stopped smoking.
There are many more aspects of an RCT which make it incomparable with real world results. For example, someone who quits smoking but didn’t turn up for the carbon monoxide test in advance of the study to prove that they were a smoker at the outset, is counted as a smoker. And so on.
If Usain Bolt were to challenge your grandfather to a 100 metres race, there is no doubt who would win. The complications and restrictive protocols involved in conducting the study are a bit like loading Usain and your grandfather with heavy back-packs. There’s no doubt that Usain Bolt would still win easily, but he couldn’t possibly achieve anything like his personal best time or break a world record. It’s the same with an RCT.