A facility tells you it has an 87% success rate. Here is how to find out what that means, which takes about five minutes and almost always ends the conversation.
Nobody is checking
There is no regulator auditing success rates in American addiction treatment. No standard definition, no required methodology, no penalty for choosing a flattering number. A facility can define success however it likes and print it.
This is not a conspiracy. It is an absence — and the absence is filled by marketing departments, as absences are.
The five questions
1. Success at what?
Completion of the programme? Abstinence at discharge? At thirty days? At a year? Reduced use? Improved quality of life?
These are all real outcomes and none of them is the same as another. A facility with an 87% completion rate has told you it is pleasant enough to stay in for a month.
The field argues about this seriously among itself. Falk and colleagues (2010) published an entire paper evaluating whether “percentage of subjects with no heavy drinking days” works as a sound efficacy endpoint. When researchers need papers to settle what success means, an unqualified 87% means nothing at all.
2. Measured when?
At discharge is close to meaningless — almost everyone is abstinent on the day they leave a residential facility. Thirty days is barely better.
Twelve months is the field’s usual standard, and it is the minimum worth taking seriously.
3. Out of whom?
This is the one that does the most damage.
Does the denominator include people who left early? Who were asked to leave? Who never completed the assessment? A “success rate” calculated only on graduates has excluded exactly the group most likely to have done badly.
4. Who did you fail to reach?
Related, and quieter. People who are doing badly are systematically harder to contact — the phone changes, they move, they do not call back.
A follow-up that reaches 40% of people and reports on those is describing the reachable, not the treated. This is precisely why credible trials state their follow-up rate: Project MATCH interviewed over 90% of living participants at one year and says so.
Ask what proportion they managed to follow up, and how the unreachable are counted. If unreachable people are excluded rather than counted as unknown, the number is worthless.
5. Compared to what?
People entering treatment usually improve, because they arrive at their worst. Improvement against their own worst week is not evidence that a programme works.
Project MATCH found little difference in outcome between three different well-designed treatments, while all three produced significant sustained improvement. Any of the three could advertise “significant improvement” honestly. None could claim superiority.
What a credible answer sounds like
Nobody will say 87%. A serious answer sounds more like:
“We follow up at six and twelve months. We reach about two-thirds of people. Of those, roughly a third report continuous abstinence at twelve months, and about half report substantially reduced use. We count the people we cannot reach separately and we do not assume they are doing well.”
That is a less impressive number and a far more trustworthy organisation. It is also roughly what the outcome literature would predict, which is the point.
Why the honest numbers look low
Because the frame is wrong, not because treatment fails.
McLellan and colleagues (2000) made the argument that has shaped the field since: substance dependence behaves like other chronic conditions, and judging it by whether someone is symptom-free after one acute episode misunderstands the condition.
Nobody asks a diabetes clinic for its success rate. We ask whether the condition is being managed, by whom, and for how long. Applied here, the useful questions stop being what percentage of people you cure and become what happens in month nine, who is responsible, and how fast does anyone respond when something slips.
Those have answers. And a facility that can answer them is telling you something real, which no percentage on a homepage ever will.
What to do with all this
Ask the five questions. Note whether they are answered or deflected. A programme that says we don’t publish a success rate because we don’t think we could produce an honest one has just told you more than any competitor quoting 87%.
Working through this on a family’s behalf is a large part of what placement and case management involves — not because the brochures are lying, but because they are all optimised, and comparing optimised documents is not comparing programmes.
References
- Falk D, Wang XQ, Liu L, et al. (2010). Percentage of Subjects With No Heavy Drinking Days: Evaluation as an Efficacy Endpoint. Alcoholism: Clinical and Experimental Research. 10.1111/j.1530-0277.2010.01290.x
- Project MATCH Research Group (1997). Matching Alcoholism Treatments to Client Heterogeneity. Journal of Studies on Alcohol, 58, 7-29. 10.15288/jsa.1997.58.7
- McLellan AT, Lewis DC, O’Brien CP, Kleber HD (2000). Drug Dependence, a Chronic Medical Illness. JAMA, 284, 1689-1695. 10.1001/jama.284.13.1689
Read more about Rehab Placement.
