The popular perception is that They Said What tries to catch vendors doing something wrong. Nope – they generally self-immolate and we just take screenshots. Or as we say, in this industry, “you don’t have to challenge the data to invalidate it. You merely have to read the data. It will invalidate itself.”
Yet no matter how screamingly obvious the data, a journalist, essayist, or blogger loses credibility if they always say the same thing. That’s why we try very hard to catch vendors doing something right.
For us to do that, vendors have to give us the opportunity – by actually doing something right. To avoid being judgmental, we like to see them independently validated for, in as many words, “doing something right.” Validated not just by anyone, but by the Care Innovations Validation Institute. Care Innovations, a wholly owned subsidiary of Intel Corporation, launched the Validation Institute in 2014 to provide companies with 3rd party validation of their outcome claims. (Disclosure: while I am neither a Validation Institute employee nor an advisor, my book Why Nobody Believes the Numbers provides the methodological basis for some of their validations, and they sometimes retain me as an outside expert/validator.)
The Validation Institute is the Gold Standard of validation. Everything is vetted carefully and has never been challenged. Validation can be done two ways: on the basis of valid contractual representations or on the basis of actual outcomes. The vast majority of validated organizations have the former, because their outcomes to date are insufficient for the latter. An example of contractual language validation would be Quizzify’s savings guarantee. (Disclosure: I founded Quizzify. The validation for Quizzify was obviously conducted by another of the Validation Institute’s team of 3rd party, independent validators).
While many companies guarantee or show savings, it turns out the language used in that guarantee or demonstration of savings determines whether savings can quite literally happen on their own due to faulty study design or whether they truly reflect underlying improvements. Quizzify, for example, couldn’t get outcomes validation because it hasn’t been around long enough to apply these valid contractual representations/guarantees to its own outcomes.
By contrast, the four organizations below are among the few whose validation is specifically outcomes-based – meaning these companies took the next step and what they say they did, is what they actually did.
Even so, if you read the validation language carefully, you’ll see it never exceeds what the outcomes data allows.
Alphabetically, we look at each of these four in some detail, describing the outcomes that were achieved. Just to reiterate, these four companies are among the very few in population health that can lay claim to outcomes improvement, measured validly. Why? Because any company that could get Validation Institute validation, would. (Quizzify sought it as soon as we had language that could be validated…and are very pleased with the attention it has brought us and the doors it has opened.)
Evolent Health: Focused on the Most “Impactable” Patients
Evolent is the first value-based care company with a complex care management program to show savings. Typically, companies compare the “pre” cost to the “post” cost, but anytime you target a chronic group that is high-need, high-cost, the “post” will always look better compared to the “pre.” Statisticians dryly call this regression to the mean and many vendors claim credit for the decline in cost when it had nothing to do with their interventions.
Instead, Evolent showed savings the hard way – by actually achieving them. They measured how much the cost of high-risk chronic patients declined on their own and then only took credit for the additional reduction. Suppose you have a magic potion to flip 100 coins from heads to tails. If only 50 flip, your potion is worthless; the probability of landing on tails is already 50-50. If 60 coins flip, the Validation Institute would give you credit for 10. Evolent showed that their program lowered utilization and costs beyond the reductions that would have happened anyway.
In a field known for the time lapses between a patient’s need for care management and its delivery, Evolent’s more advanced predictive modeling (covering more datasets than a carrier would typically use) expeditiously determine those at highest risk of having an “impactable” event. Further, whereas most such programs focus on just checking off boxes, Evolent intervenes across the spectrum of clinical, behavioral, social, nutritional and environmental domains.
Having reviewed many of these programs, I’ve been shocked by how long it takes them to find and enroll patients, how little they do for the patient, and how little they know about what they’re doing. Evolent is the opposite. This is their business, not a sideline – they take it seriously and it shows.
Healthways Well-Being: It’s Not Just about the Cholesterol
For the uninitiated, the philosophy of well-being is to address gaps not just in employee physical health but – as importantly – in their emotional, financial, occupational and social health. In many cases, those latter issues are the root cause of high healthcare spending and low productivity. Addressing those issues should help a given population – from the healthy to the sick – perform noticeably better while possibly spending less on healthcare.
Before you even heard of “well-being,” Healthways was measuring it, more than a decade ago. Since 2008, Healthways has partnered with Gallup to definitively measure well-being via the Gallup-Healthways Well-Being Index, the most proven, seasoned and comprehensive measure of well-being in populations in the world. Quite literally, if there is any component of this industry which has penetrated the public consciousness in a positive way, it’s the Gallup-Healthways Well-Being Index, whose publication often reaches the lay media.
Healthways can use surveys, down to the community level, to benchmark similar surveys for companies, departments and employees, so that organizations can focus their improvement efforts where they are needed most. The Validation Institute has confirmed Healthways’ findings that in fact performance (holding constant as many other variables as possible) correlates far more closely with indicators of well-being than with biometrics alone. This data collection, insight and benchmarking allows targeted interventions to complement or replace conventional wellness…and get closer to the root cause of underperformance.
Rarely is the root cause of poor health “I-don’t-care-itis,” as one wellness vendor calls it. Often it’s a different personal issue. Sometimes the root cause is department-specific. This data can be used to identify managerial or process flow issues far beyond the scope of – and far more powerfully than — conventional wellness.
Quantum Health: the Story Tells Itself
While I make more general comments about the other vendors on this list, I don’t need to for Quantum Health. They were the first and are still the only company validated for total savings across an entire organization.
Instead I will share a story that shows how their incentives for members to call in – combined with their non-siloed approach to those calls – create a confluence of time and place that change behaviors and likely outcomes.
Once, when I visited them, an employee of a new customer called, asking if diabetic shoes were a covered benefit. In most, if not all, carriers, the employee answering that query would be evaluated based on accuracy of the answer, number of rings, politeness and how many calls they handled that hour. So the person would say “yes” or “no” and then get off the phone. At Quantum Health, the agent answered the query but was prompted by the supporting software (and by training) to recognize that question as a red flag. Here was an employee whose diabetes was already so advanced he was asking about shoes…and yet he was nowhere in the diabetes registry. A typical carrier wouldn’t find out about this person until after the inpatient claim for his inevitable crash was filed, warehoused, prioritized and queued for telephonic outreach. And then, assuming the carrier had the correct phone number, and this patient answered the call and was receptive, rehabilitation could begin.
And yet there he was – right on the phone – asking for help. So the agent probed a little further and then transferred him to a nurse in the same pod, who engaged him right away, almost certainly avoiding or forestalling a future high-cost medical event.
US Preventive Medicine: Finding the Formula
The editor of the American Journal of Health Promotion, Michael O’Donnell, famously admitted that up to 95% of wellness programs fail. U.S. Preventive Medicine is squarely in his other 5%. As quite literally the purest wellness program validated by the Validation Institute, USPM has – alone in the wellness industry – found the formula for a significant and sustained reduction of wellness-sensitive medical events (hospitalizations and ER visits).
The Validation Institute analysis showed that USPM generated a sustained average 41% reduction of hospitalizations and ER visits across several chronic conditions (Diabetes, Asthma, Coronary Artery Disease, Hypertension, Chronic Obstructive Pulmonary Disease and Congestive Heart Failure) over a four-year timeframe, significantly outperforming the averages as tallied by the Healthcare Cost and Utilization Project (HCUP). USPM provides a unique data-driven, high-tech and high-touch combination of conventional and unconventional interventions to enhance engagement and translate that engagement into actual behavior change.
The Validation Institute has publicly urged all wellness vendors to collect real data, apply their value event rate-based template (the only methodology that They Said What and HERO agree on, as also described in Health Affairs), to see if they can match USPM’s performance…and so far, none have come close.
Michael O’Donnell might have been optimistic in his assessment—the failure rate seems much higher. But it’s not 100% — USPM is the exception that proves wellness can indeed be done successfully…if all the components fit together.