I noticed on twitter that Mark Willenbring referred to SBIRT as "dead in the water" and found the following recent studies:
- http://onlinelibrary.wiley.com/doi/10.1111/add.12600/abstract
- https://jama.jamanetwork.com/article.aspx?articleid=1892249
- https://jama.jamanetwork.com/article.aspx?articleid=1892250
What are your thoughts?
Oooh, I hope this generates discussion. Why do we clump three intreventions together and call it an EBP? We know people ought to be screened for unhealthy behavior. We know that many people who do not have a pathological level of other problem behaviors can respond to simple advice about what is healthy. We know that counseling works for people whose health behaviors result in disease symptoms. So why the controversy? It seems to me the heterogeneity of both the treatment populations and the actual responses that are called SBIRT probably is what causes the variation in outcomes.
Why do we clump three intreventions together and call it an EBP?
My thoughts exactly.
I've always been puzzled by the assumption that brief interventions would reduce substance use among heavy users.
However, the second study looked at treatment admissions as a secondary measure. That seems to be a better outcome measure with that population and I want to know more about reasons admissions did not increase.
SBIRT is not at all dead in the water. The evidence for alcohol SBIRT remains very strong.
In most respects, I found the Roy-Byrne/JAMA study was very strong methodologically. Indeed, it found that brief motivational interventions for drug use were ineffective at reducing the frequency of drug use and the severity of life impacts at 3, 6, 9 and 12 months after the intervention.
Although the headline is that brief intervention doesn't work for drug abuse, a thorough read of the paper reveals limitations, and to the authors' credit, these are appropriately emphasized in the discussion section. Most relevant is that the sample was not at all generalizable to most Americans. Less than 10% of the sample was employed. Almost two-thirds were disabled. There were high rates of poverty and mental illness. About 30% of the subjects had been homeless for at least one night in the 90 days before study registration.
So, it should not be surprising that the one-session intervention most of these very complicated and disadvantaged subjects received (fewer than half received a follow-up phone call) was insufficient. This still leaves open the possibility that:
- more intensive interventions would be helpful for this complicated and disadvantaged population, and
- brief interventions would be effective for drug use by non-drug dependent individuals with much less complicated lives. I wish someone would conduct a similar randomized controlled trial on people receiving care at typical commercial primary care clinics. My guess is that we'd see different results, but we won't know until such research is done.
Rich, are you really saying, above that -- OK, it doesnt work for those with poverty and symptoms of depression and anxiety or those who had one night on the street in the past three months but it will be great for the other people who use drugs, as long as they are not dependent, dont have complicated lives, and live outside of major cities and have commercial insurance? If so, SBIRT is pretty unimpressive---wouldnt really be a robust solution to the problem. Of course we would need a study in the population you are talking about, as you point out. Meaning, we dont have that evidence for efficacy yet either.
By the way, in a motivational intervention, which is I am sure the type you would favor in this circumstance, one does not force a booster on anyone---we had 98% follow-up but it is true that in the MOTIV group only around half had a second session---but you cant force it (in fact Bill Miller found that among those who are made to have a change plan, motivational interviewing did not work for drugs---and he, despite being its developer, was not afraid to report it and speak out about it. This is complex stuff. "treatment works" and "sbirt works" ignores complexity. And the other group in our study tested exactly what samhsa promotes. So if it isnt a 1 time session, what exactly does work? And where is that evidence? (for alcohol the best evidence as you know is for 3-4 contacts---but what about for people who dont come----who are exactly the people who need the help?).
Rich (Saitz) - The authors themselves cautioned about generalizing their findings to other populations. Really, consider the psychosocial complexity of the population studied. In medical terms, concluding from this study that brief drug interventions don't work would be like concluding that diuretics don't work for congestive heart failure because of a negative RCT for one dose of a diuretic in patients with pulmonary edema. - Rich (Brown) (Are we an embarrassment of Riches?)
Being one of the authors, I understand what I wrote...But there is a big difference here...not at all like diuretics. Diuretics are proven to work, and there may be subsets in whom they dont work.
In this case, there has yet to be robust evidence for efficacy of drug SBIRT. And it isnt because of studies havent been done. It is because studies have been done and are negative or have very minimal effects (see my other post for detail). So when you have 2 large high quality studies showing lack of efficacy (in exactly the populations one would need SBIRT for), and no sign of efficacy in anyone else....the burden of proof is to demonstrate it works in someone somewhere. This proof is lacking. Also, I dont expect SAMHSA to be saying soon, that SBIRT is great except for in FQHCs, Medicaid populations, drug users who also have mental health symptoms and might be unemployed etc. If that turns out to be the case---only works in suburban employed educated nonminority populations....then....it would be hard to imagine a less useful approach for drug use. We are not just talking about "a negative RCT". We are talking about the totality of the literature of RCTs on drug SBI, including these two high quality studies, and a large multisite ER RCT that also found zero efficacy. So one can continue to say that the negative studies are not definitive but isnt it more logical to demand proof of efficacy? And then when studies are done to test efficacy, to accept the conclusions?? Or, why bother with studies. We can just practice belief based medicine. Personally I think we would never tolerate this for other medical conditions but somehow it is ok to practice what is believed and hoped for substance related conditions and not require real science. I think our patients deserve better...
One other thing---your analogy is off. Patients with heart failure and pulmonary edema are severely affected. The equivalent would be people with severe substance use disorder. In our study, 82% did not have scores consistent with dependence, They were using drugs around 3 times per week. (and diuretics work just fine in the inner city, with people with anxiety and depressive symptoms, and low SES).
So if SBIRT doesnt work in people with risky use in primary care (these are folks who self select by an ability to keep appointments to see a PCP in advance--so actually pretty good in terms of being organized and caring about their health) then who eactly do we think it will work for?
The onus on all of us is to prove that SBIRT works, not to prove that it does not work. Obviously patients and the general public want an intervention that has efficacy, and there should be high level of proof for a preventive intervention like SBIRT, or for an intervention on which we are relying in national programs to reduce the burdens and harms of drugs, a leading cause of preventable death. If it has no efficacy the next logical step would be to find something that does, not to continue doing something inefficacious, right? And we want to know whether there is efficacy, right? I certainly hope so for the sake of our patients.
What randomized controlled trial evidence is there that SBIRT works for drugs? Very little. There is one study 2005 Drug and Alcohol Dependence that showed 5-9% decreases in heroin and cocaine use in urgent care patients; treatment utilization was unaffected. This was impressive and high quality methods. There is another study done by the WHO that did not use biological outcomes (which is a big problem because in studies when people are aware that the researchers want them to use less drugs that is what they report) and found no benefit in the US part of the sample and in the rest of the world some benefit. What was the benefit? A 2-3 point difference on a drug severity scale that ranges from 0-336. Not impressive. There is one study of prescription drug SBIRT in a hospital in Germany that is difficult to interpret. There is a small (n=59) study in adolescents in Brazil.
What about unpublished studies? There is a multisite very large (~1800) person emergency department trial. It found no efficacy and will be published this year. A study out of UCLA presented at CPDD found some small effects on frequency of drug use on those with non-dependent but higher rather than lower problem scores. Yes that is confusing. But it is consistent with D'Onofrio ER studies of alcohol SBI -- she found no efficacy of SBIRT for alcohol when she included risky drinkers but not dependent ones. But in another study she found efficacy but only when she excluded the low end of risky and the high end too. She found "the zone" or slice of patients in whom it might work.
What about nonrandomized studies? There are many, but frankly I do not believe them and they shouldnt be the standard for a serious health condition. The most common is Madras et al observation 2009 Drug and Alcohol Dependence, that after SBIRT, people decreased their use. For example, ~85% decreases in heroin and cocaine use. It is simply not believable that is due to SBIRT. Yet why does no one critique studies that show positive effects of SBIRT? and only critique the studies that find no effects? Why do we assume this works? Why do we discount negative studies, and just choose to go with the ones that show what we believe even if they are not well done?
To the current JAMA studies (one of which I authored). I wanted to test efficacy and then report the result regardless of what it was. I suspected SBIRT would work. As Rich Brown said, the methods were strong. 1 in 5 of the participants had ASSIST scores 27+ and 4 in 5 were therefore risky users not likely dependent. We found no efficacy. No effect on drug use frequency, frequency of use >1 a day, drug related problems, quality of life, ER visits, hospitalizations, sex and drug risk behaviors. The results held regardless of whether you mainly used marijuana, cocaine, or opioids, or whether you had ASSIST 27+ or not or whether you were ready to change or not or whether you also reported drinking. There was also no effect on entry into treatment for those for whom it was relevant. No randomized trials yet show this, and anyone with a lot of experience with SBIRT knows that it rarely gets people who werent already wanting treatment to get into treatment.
We tested exactly the intervention that has been the most common disseminated and encouraged by SAMHSA--a one time brief negotiated interview by a health educator. The trainers in our study were the developers of that intervention, the Bernsteins. Because we thought maybe a more intensive intervention might be better, we also did a psychologist delivered motivational interview with a booster (we did not tie people down and force them to have one but we encouraged it and offered it, consistent with MI style). This is more than is done in most SBIRT programs. Neither worked.
It is of course true that both our study and that by Roy-Byrne were in urban settings with low income minority populations. But, seriously, are we saying that SBIRT might work in nonminority employed populations? Why? Are we really going to say therefore that SBIRT is great and works everywhere......except Boston, Seattle, NY, Washington, major urban centers, Federally qualified health centers (Roy Byrne study)? If so, then SBIRT wouldnt seem very valuable.
The bottom line is that drug SBIRT has very little to no evidence from high quality studies to support its efficacy. And if we are going to rely on it we need such evidence. If the evidence is otherwise then we better find out what works and do that.
By the way, the evidence for alcohol SBI in primary care is solid for nondependent risky drinkers but none for those with dependence. And the latest review -- Jonas et al, Ann Intern Med, found that the best evidence is for reducing self report of drinking, but there was insufficient and inconsistent evidence re reducing any "hard" outcome like hospitalization or ER use or injury or health problems that people care about. And setting matters---the context of where this is done. Which is probably why we find consistent studies in primary care, but inconsistent efficacy results for alcohol SBI in general hospitals, emergency rooms and trauma centers (unless one selectively cites individual studies that say what you want to hear). Go back and look at the original 1999 Gentilello study Ann Surg that is often cited as evidence for efficacy. The main result was statistically NOT signifcant (and 4 other studies after that were negative) and the drinking result was based on only half the sample self report because they lost everyone else (and in these studies the people you lose tend to be the ones with bad outcomes...). The point is it is important to read the actual literature and not take on faith and belief statements from government agencies such as "SBIRT is evidence based." That is a platitude and not specific enough to have meaning. What exactly has efficacy? What is the evidence? See above.
Now. Does that mean we should not be identifying and addressing unhealthy drug and alcohol use in general health settings? ABSOLUTELY NOT. But it does mean that if we screen and identify, and then briefly counsel, a la SBIRT, then we should NOT expect to see reductions in use and improvements in substance related outcomes. To get those we need better interventions. (and, by the way, why do we in the substance use field hold ourselves to such low expectations??? Drug use is complex. It will not be fixed with a simple solution. Not other medical condition is expected to get benefit from one brief hit. One identifies high blood pressure in one visit and it often takes many visits to get the patient to buy in to the fact that this is a condition that needs treatment and many more to get the right treatment on board). We do need to know, just like we need to know if someone has allergies or is taking complementary medications or over the counter meds, if people are drinking and using drugs--to properly diagnose any symptoms, to appropriately give treatment for any conditions, to get to know the patient in a longitudinal relationship, and to be able to inform the patient about their health risks. That is all not SBIRT.
Why would anyone choose to ignore well done studies that tell us drug SBI has no detectable efficacy? Maybe people are emotionally invested in this? Something else? I do not know (FYI I have no financial investment/conflict of interest in whether SBIRT works or not--in fact I wish it did--but wishing does not make it so). NIDA and NIAAA titled their editorial "back to the drawing board." They get it and are moving on. Frankly, we choose to ignore these results to the detriment of the health of the public and our patients. We can and should do better. Again the onus is on us to prove things work, not to just believe on something and then critique or ignore any data that conflict with our belief (some might call that religion, not science and healthcare).
I spoke with ONDCP. They, and NIDA are not at all upset by the results of these studies. It isnt about knee-jerk reactions rejecting results we dont like or are disappointed by. They know that we do studies to better inform practice. When studies tell us something, we alter and improve our practices. So it is time to do something to improve practice and to find things that might work and test them.
I may not be fully grasping all of this but it seems to me the discussion centers on the results from the SBI part of SBIRT and doesn't address the RT part, which is a bit wimpy. Giving people a list of programs/services is not quite what I would consider an effective referral to treatment. The lack of positive outcomes from an incomplete SBIRT intervention somehow isn't very surprising to me. In practice, it might take several attempts at SBIRT to finally have a successful referral (i.e., one in which the client connects with the treatment service to which s/he is being referred). I don't think that necessarily means only the last SBIRT intervention "worked" and previous SBIRT interventions failed. Maybe the studies could take into consideration the differences between subjects who follow through with various referrals to treatment vs. those who don't?
The BI in SBI in both studies included RT if it was appropriate, Not just a list of services, this was part of the skilled counseling that reviewed options and connected people as best one can in a brief intervention, Just like "SBIRT". More importantly, at least in our study, only 18% (less than 1 in 5) had dependence. 4 of 5 were risky users. RT and T are not appropriate for them. One could follow through with the few who needed treatment, who then got it. But the numbers would be small and not particularly relevant since they would be self selected.
The real issue that is SBIRT populations are not people with severe substance use disorders. In fact in primary care, they are mildly affected. That is who BI should work for. And it doesnt.
This is such a great discussion. Exactly what I was hoping for. Providers frequently don't have access to the original literature, so go by the blanket statement of a practice being evidence based and then getting frustrated when it doesn't work in their setting with their patients. So, is it safe to say for SBIRT, that the evidence is that physicians should screen, should do brief counseling, but should not expect the brief counseling to have any greater effect than letting their patients know what is a healthy, but not expecting people with a real problem to change behavior based on that any more than they expect people like my diabetic grandmother to stop eating candy because she knows she shouldn't? But it probably works to reduce risky drinking (bingeing? heavy drinking? what definition of risky are we using here?) which might be comparable to telling my sister she needs eat less candy to lose ten pounds or risk getting the diabetes that runs in her family?
PS We know ONE THING like SBIRT cannot solve all drug and alcohol issues. We would never say we have one answer to, say, heart disease. The details matter....
Hi Kim, It is frustrating to paint "evidence based" with a broad brush when we know health is complex. I think you are right about what the randomized trial evidence says--among patients identified by screening (and that is key), we an expect small individual decreases in drinking among those without dependence, when the BI is done repeatedly, and in a longitudinal care context like primary care. Beyond that, the evidence is weak or nonexistent, but it does make sense to identify and address for a bunch of other reasons, like managing other conditions, and to inform people of risks so they can decide. I wish it did more...but wishing doesnt make it so...
As for defining risky...for alcohol most would use NIAAA cutoffs (on avg more than 14 a week for men, 7 for women, and in any day more than 4 for men, 3 for women). In truth the exact cutoffs dont matter so much. You could also define by a positive screening test (AUDIT-C, single item, AUDIT etc). And without moderate to severe DSM disorder. So they could have a few consequences. See NEJM 2005 publication (free online full text) on Unhealthy Alcohol Use. As for drugs, it is a bit more complex. For a while it was easier to define risky drug use if one included legal risks--we could just say ANY use. But with many things legal...And it is harder to define specific amounts as being associated with harms because there is less study and we dont have good ways of knowing the quantities and concentrations people are using. That said, many people would still say any drug use has some risks. So it would be any drug use but not dependence....similar conceptually. No doubt there are infrequent small amount users of cocaine, MJ, other drugs who are at very low risk....but it may not be worth worrying about the distinctions too much though if you get into a session with someone with low level MJ use it will definitely become the focus so we need more research on such things....
Now, here's my BELIEF but not based on evidence. I do think repeated consistent messages in the context of a trusting longitudinal relationship will make an impact on some. So eventually your sister and grandmother may change. But. That is not SBIRT. That is something else...
Great discussions in deed. Good to see a consolidation on publications to date on the SBIRT guidelines. I am glad to see the discussion come back to drivers in Patient Centered Medical Home. Repeated, consistent messaging, in a trusting, longitudinal relationship with pts. over time may arrest a spiraling condition for some. I disagree that this is not at the center of an SBIRT guideline. In a community treatment program, I saw 3 pts. in the past 6 months, accompanied by their neighbor/friend as an intervener, asking for help to get the pts. in treatment. In all 3, they were 36-38 yr. old women with advanced liver disease and high volume daily drinking for many years. The neighbors got it. They noticed health problems likely caused or exacerbated by alcohol use, they spoke to the pts., regularly, and convinced them to get help. All 3 pts. had great insurance, jobs, community profiles and multiple in pt. hospitalizations, ER visits and long standing care coordination from their Primary Care providers. What all 3 pts. did not get....was asked about the relationship of their substance use to their spiraling health conditions. They did not get an counsel on the risks of their current drinking. They did not get a referral to treatment. They got these things when a neighbor said enough...I will help you. All 3 pts. died within 6 months. It seems a system for screening, providing some counsel, not firing the pt. but staying in the clinical relationship and referral to treatment are not just a WISH but medically necessary. In the case of these 3 pts. some brief counsel/teaching about the withdrawal they all experienced in the ER or IP setting followed by a referral to detox would not be a BELIEF, but medically necessary. Of course these pts. all had their right to folly, and in deed they did. There is a line where too much is abdicated based on a persons right to folly. I wonder if systematically screening for substance or other health risks, then teaching the pt. about the risk and alternatives, followed by a referral to speciality care when needed, has greatest utility in assisting the medical establishment to ask the questions, get over the stigma and bring substance risk competency into the practice.
Im sure you didn't have this issue in the RCT; but in the settings I coach SBIRT most commonly what we are discussing is that these are pts. noone wants to talk to, with conditions none wants to talk about, nor have any idea what to do with when they encounter it. It is still the crux of addressing substance risk competently in a medical setting.
The health field is chalked full of GOMER(GET OUT OF MY Emergency ROOM ) stories in every medical setting. An epidemic of accidental opioid overdoses in the 18-26 age range. 6 cases in one 8 hr. shift at a Portland, ME hospital 2 weeks ago. 2 of the 6 cases were from inter-nasal use. Too bad screening and brief intervention doesn't work. Not a single case was referred to treatment or detox. In fact, (the sigh of relief) if you work the ER shift that night.... is that opioid overdoses are "medical" not "Psych" so none of the staff in the Psych ER actually get to talk to the overdosed pt. They are Treated and streeted. I think the science and outcomes Dr. Saitz discusses here are relevant and important but the Dialogue is far to thin, polarizing and lacking in empathy or solution for those in need of identification and brief intervention and TREATMENT. That is of course Isomorphic. Story after story after story of pts. whose spiraling medical condition called addiction were chased away, ignored, GOMER'ed, had medically necessary care delayed. These seem like important, Real World variables; and they include the social aspects of working with substance use disorders. . More of that please.
This is interesting discussion and I'm glad someone referred me to it.
A few thoughts... I guess from my perspective, much of the SBIRT work that Rich Saitz describes seems somewhat misguided. Why even use all these resources on trying to influence people to change their relatively minor cannabis use, or drinking which minimally or moderately exceeds the low NIH/CDC standards for drinking? To me, these recommendations are on par with population-wide recommendations on intake of calories, fat, sodium, carbohydrates, etc. They seem like aspirational standards - long-term health guidelines that seemingly have little impact on what the majority of the population actually eats on a daily basis. My own physician asked me about my drinking habits recently as part of my wellness checkup, then before I even answered he commented that the bar is set so low by CDC that it probably includes half of the people who drink in America (don't worry if that's incorrect, the point is the perspective even of this community physician was that the standards are not helpful to him to try to sort out problem drinking from commonplace drinking).
Rich says that people with these relatively minor use habits are precisely the people that SBIRT should work in, but why? The logic for this seems to be (not Rich's but generally) that if severity of use is low, then minimal interventions should be sufficient, while if severity is high, more complex and intensive treatment may be needed to foster change. But that logic seems to ignore the fact that in low severity use, people often have virtually no motivation to change, as there are no obvious problems with current behavior and no substantial benefits to changing, and advice from the outside is as likely to harden their current perceptions and choices as it is to alter them, even if they play along in the moment. With cannabis, it seems even more likely to harden current perceptions, given that virtually all cannabis users believe (mostly rightly I think) that it is a relatively harmless activity that has received incredibly overblown public messaging for decades, and there is little evidence of harm that holds up to much scrutiny. I know Rich does good motivational interviewing work, and so the focus in MOTIV and other MI-related approaches involves eliciting participants' perceptions and thoughts for the future as much as providing professional assessments and advice, but I suspect that even elicited motivation to change likely fades quickly once the person is out of the momentary social influence situation and they return to their baseline perception that while some change might be good in some abstract way, they have bigger fish to fry, and the issue returns to one of relative irrelevance for them as they focus on things that are more important to them.
Just some thoughts off the top...
I think I have a different concept of the role of SBIRT in practice (in fact it is actually more like the "repeated consistent messages in the context of a trusting longitudinal relationship" in the example of intervening with the sister and grandmother"!) I discourage providers from thinking one SBIRT intervention is likely to have an effect when addressing a chronic condition and if it does, it's probably because of a confluence of factors (such as degree of dependence, motivation, confidence, access to treatment tools, etc., that put the client at a tipping point), not the single SBIRT intervention they did. The USPHS clinical practice guideline for treating tobacco use and dependence uses what they call the 5Rs as strategies for effectively doing a brief intervention with patients not willing to quit/make a quit attempt. The first 4 "Rs" are techniques based on MI concepts (although very simplified), but the last one is "Repetition" - emphasizing the need to do one of those 4 "Rs" at the next visit and the next visit and the next... until the patient does express willingness and the provider can now provide assistance in developing an individualized plan. Even though the provider should still inform the patient about and refer them to the state quitline as part of an "R" intervention (in case they decide later they want info/help), the expectation of a behavior change (e.g., quitting smoking) should be low (until the patient has progressed in their readiness to change - you all know what I'm talking about). So for me, studies that assess efficacy of a single (or even a second, third) SBIRT intervention, especially if not followed by appropriate involvement in actual treatment, aren't all that helpful back in the practice setting. And findings that they don't "work" fuel the primary care providers' perceptions that 1) they "can't do anything" to help until the patient is highly motivated to change, and 2) once they do provide "help" (usually telling the patient what they need to do), they're done, and 3) if the patient doesn't go off and change - then see #1 (although this is not how most primary care providers address non-addiction chronic diseases). Just a few more musings on my part!
Interesting reframing. I've often thought it's silly to expect much from a single, brief intervention. That made me ambivalent about SBIRT. Maybe the problem was SBIRT, but, rather, our expectations and thinking around it?
So, SBIRT could get all of the helpers in a health care system working from the same script and the repetition could could have en effect over time. It'd be interesting to look at what happens to patients in a primary care practice or health system over time. My doctor has a BMI chart on the wall right across from the chair I'm seated in while waiting for the doc. It moved me a little each time. Combine it with cholesterol lab results and it wore me down.
Yes. Thanks for these thoughts, jschwarts. It is the scripting and repetition that holds utility in elevating an index of suspicion and comfort with involving substance use risk routinely in the differential.
In my opinion, the problem with the two drug studies is the "R-T". Handing a substance use disorder client a list of resources is not synonymous with referring to treatment. Appropriate referrals should involve a start date and time, if not a transfer, to the appropraite level of care. This, of course takes fiscal and human resources. A list of resources is erronously perceived as a referral.
A new study in the American Journal of Health Promotion (Baxter, Siyan, et al. "The relationship between return on investment and quality of study methodology in workplace health promotion programs." American Journal of Health Promotion 28.6 (2014): 347-363.) found that the closer you look, the lower the return on investment for wellness programs. High quality studies calculated benefits nearly 90% lower than low quality studies. The 18 studies that qualified as "high quality" had an average ROI of 0.26 (in other words, for a dollar spent returns $0.26). From the 27 low quality studies, the ROI was 2.32. The randomized control trials found that for every dollar spent, only $0.78 was gained in benefit. High ROIs for wellness rpograms do not appear to be realistic. Studies that conclude otherwise are likely to be poorly designed.
Rich Saitz has been conducting well designed research of screening and brief intervention for drug use. I am not surprised that Rich's well designed SBI study found small effects that are inconsistent with the uncontrolled studies done with the state SBIRT programs, such as the Madras et al report. Findings of small effects in well controlled studies ought to be expected. This is the way that science and medicine should advance. For potentially life threatening conditions, it is essential that practitioners and researchers keep building from best science towards ever more effective treatments. Screening and brief interventions may be effective for some people with substance use problems some of the time. That's not good enough, and should never be good enough. We need researchers like Rich, and Craig Fields, Larry Gentilello, Jen Mertens, Connie Weisner, Don Zatzick and many others to keep testing our practice-based hypotheses.
Eric
Below is the abstract of the AJHP article:
Objective. To determine the relationship between return on investment (ROI) and quality of study methodology in workplace health promotion programs.
Data Source. Data were obtained through a systematic literature search of National Health Service Economic Evaluation Database (NHS EED), Database of Abstracts of Reviews of Effects (DARE), Health Technology Database (HTA), Cost Effectiveness Analysis (CEA) Registry, EconLit, PubMed, Embase, Wiley, and Scopus.
Study Inclusion and Exclusion Criteria. Included were articles written in English or German reporting cost(s) and benefit(s) and single or multicomponent health promotion programs on working adults. Return-to-work and workplace injury prevention studies were excluded.
Data Extraction. Methodological quality was graded using British Medical Journal Economic Evaluation Working Party checklist. Economic outcomes were presented as ROI.
Data Synthesis. ROI was calculated as ROI = (benefits – costs of program)/costs of program. Results were weighted by study size and combined using meta-analysis techniques. Sensitivity analysis was performed using two additional methodological quality checklists. The influences of quality score and important study characteristics on ROI were explored.
Results. Fifty-one studies (61 intervention arms) published between 1984 and 2012 included 261,901 participants and 122,242 controls from nine industry types across 12 countries. Methodological quality scores were highly correlated between checklists (r = .84–.93). Methodological quality improved over time. Overall weighted ROI [mean ± standard deviation (confidence interval)] was 1.38 ± 1.97 (1.38–1.39), which indicated a 138% return on investment. When accounting for methodological quality, an inverse relationship to ROI was found. High-quality studies (n = 18) had a smaller mean ROI, 0.26 ± 1.74 (.23–.30), compared to moderate (n = 16) 0.90 ± 1.25 (.90–.91) and low-quality (n = 27) 2.32 ± 2.14 (2.30–2.33) studies. Randomized control trials (RCTs) (n = 12) exhibited negative ROI, −0.22 ± 2.41(−.27 to −.16). Financial returns become increasingly positive across quasi-experimental, nonexperimental, and modeled studies: 1.12 ± 2.16 (1.11–1.14), 1.61 ± 0.91 (1.56–1.65), and 2.05 ± 0.88 (2.04–2.06), respectively.
Conclusion. Overall, mean weighted ROI in workplace health promotion demonstrated a positive ROI. Higher methodological quality studies provided evidence of smaller financial returns. Methodological quality and study design are important determinants.
New study published on effectiveness of SBIRT: http://www.bu.edu/aodhealth/issues/issue_july14/friedmann_williams.html
A couple of responses...
1. The SBI trials do not simply hand people a list of programs (except in control groups). In intervention groups they match the referral intensity to patient readiness. One must remember that it can be counter-productive (i.e. harmful) to make great efforts in linkage to care for those who are not ready for that step yet.
2. Many of the comments above provide thoughtful and reasonable hypotheses (i.e. about repeated messages). But they are hypotheses that need testing. Hopefully they will be borne out but for life-threatening conditions, as Eric says, I think we want more than hypotheses and hope.
3. Most of the comments above are about people with substance use disorders. Let's recall that the scientific basis of SBIRT is actually of SBI as a preventive service. For people with use that risks consequences, not for people with disorders already. Chris points out that maybe we shouldn't expect people with elevated but relatively low risk to have motivation to change. But the alcohol SBI literature has as its most robust findings precisely that--efficacy of BI after S for people with low level risky use (excluding very heavy use and disorders). And Mark Willenbring has pointed out many times that we don't wait for people to have a heart attack to do something about it. We manage risk--we lower high blood pressure, cholesterol, and improve physical activity and nutrition. Motivational interventions are great for that--helping people recognize their own personal risks. Yes they are also great for people who already have consequences - that is easier for patient and practitioner alike to recognize and to change. But that isn't SBIRT. And lets remember that despite the severe examples seen in the ER and addiction treatment, the majority of health impact is in people who do not have a substance use disorder but use substances nonetheless (and in primary care, the majority of people, as in the RCT's being discussed, had risky (not problem or disorder) use. This is often difficult for people mired in the day to day treatment of severe cases to see.
So should we discuss people who have SUDs and need help? Of course. Can motivational interventions work for them? Of course. Should they be offered treatment with warm handoffs? Of course. Does that have anything to do with SBIRT? No. Why the focus on SBIRT? I don't know. Chris W suggests SBIRT shouldn't work for low risk drug use. He is right, as proven now by at least 3 randomized trials. But MI should work for it as we say it does for many risky health behaviors including risky alcohol use, but it just doesn't. And others complain that SBIRT (1-2 counseling sessions with referral gauged to level of readiness) isn't enough for people with SUDs. RCT's prove them correct too. Yes other things are needed. That is precisely the point of the results of the trials. And the trials were done because (haven't you noticed??!!) that there is a huge bandwagon of what is now almost an industry of promoting "SBIRT."
Seems to me we should abandon what is proven to not work, and test other approaches, and promote things that do work. Why that is controversial remains beyond my comprehension.
By the way, I like the possible benefits of "SBIRT programs" of making addiction more mainstream, and the possibility that over time with messages culture of medicine and patients may change, and individuals may change with repeated messages. But if that is why we are paying for SBIRT, lets be honest and say that, rather than claiming that 1-2 BI in those ID by screening will lead to reduced use and consequences, which is simply counter to the best evidence available.
PS Chris--just because many or most people in the US use alcohol in a risky way does not mean they should not have their risk reduced. Many Americans are obese. That doesn't mean it isn't harmful and unhealthy. Ditto for high cholesterol and high blood pressure (each about 1/3 of the US adult population). I (and the CDC, and NIAAA and Australian and UK and European health agencies and the WHO) see no reason to treat alcohol risks differently just because it is common. And by the way, it is 28% of the US adult population who drink unhealthy amounts. That includes the approximately 7% who have a disorder. The rest don't have a disorder. (see also 'prevention paradox'). Focusing on the few with a problem is a mistake if we want to improve the health of the public and prevent people from developing more severe disorders. It is also true that about half of current drinkers drink too much (levels that increase their health risks including the risk of death from it). But that is no reason to not do something about the 3rd leading cause of preventable death in the US (alcohol)(deaths that occur in young people) and note that it is ALCOHOL (not other drugs) and it is ALCOHOL (not alcohol use disorder only).
OK. Back to work.