By Al and Vik

Harvard Professor Katherine Baicker is arguably the most acclaimed health policy researcher at arguably the most acclaimed (and not even arguably, the best-endowed) school of public health in the country. Her seminal account of the effect of Medicaid coverage on utilization and health status is a classic. As luck would have it, in 2008 Oregon used a lottery to ration available Medicaid slots. A lottery controls for motivation and as such eliminates participant-non-participant bias, since everyone who enters the lottery wants to participate. That meant only one major variable was in play, which was enrollment in Medicaid or not.

Wellness ROI claims appear lost in space

Chance favors the well-prepared, and Professor Baicker jumped on this research windfall. She found that providing Medicaid–and thereby facilitating access to basic preventive medical care–for the previously uninsured did not improve physical health status, but did increase diagnoses and utilization. Because of the soundness of the methodology, the conclusion were unassailable – more access to medical care does not improve outcomes or optimize utilization, which is a proxy for spending. (We ourselves reached a similar conclusion based on a similar analysis on North Carolina Medicaid’s medical home model.)

Yet Professor Baicker herself used exactly the opposite methodology to reach the exact opposite conclusion for workplace wellness.  And that’s where the identity crisis begins.

She and two colleagues published a meta-analysis in 2010 of participant-vs-non-participant workplace wellness programs. Somehow—despite her affinity for Oregon’s lottery control—she found this opposite methodology to be acceptable.  She concluded that workplace wellness generated a very specific two significant-digit 3.27-to-1 ROI from health care claims reduction alone, with another 2.37-to-1 from absenteeism reduction. The title of the article–now celebrating its fifth anniversary as the only work by a well-credentialed author in a prestigious journal ever published in support of wellness ROI—was equally unambiguous:  Workplace Wellness Can Generate Savings.

This article wasn’t just an academic exercise. It gave the Obama administration academic cover for what has proven to be the most unpopular, dishonest, and even hazardous component of the Affordable Care Act:  allowing employers to financially and clinically punish employees with coercive directives to lose weight, get unnecessary checkups, and answer intrusive, distasteful, and counterproductive questions about (for example) checking their testicles.

Professor Baicker did not question her too-good-to-be-true conclusion. Yet the Law of Diminishing Returns clearly contradicts her finding. Compelling privately insured people to get more healthcare is very unlikely to improve health status and reduce healthcare expense if provision of basic insurance to a medically needy population doesn’t noticeably improve health status while increasing healthcare expense.

Instead, she reveled in the limelight, receiving 307 citations, vs. 18 and 9 for two other Health Affairs articles on wellness that didn’t support more spending on vendors and consultants. (Even 307 citations aren’t enough to satisfy one of the leaders of the wellness movement, Larry Chapman, who says this study should be cited much more frequently since it’s basically the one that supports the entire industry.) However, at some point in 2013, overwhelming evidence totally invalidated her findings. At that point – like Dee Edington and Al Lewis, both of whom had previously reversed positions when the data didn’t support their previous positions—she could have acknowledged that her initial findings had been wrong and moved on.

Instead, she neither defended her position nor clearly refuted it, choosing instead a yin and yang middle ground that shifted with every interview. The metamorphosis from Queen of Significant Digits into the Queen of Significant Doubt started in July 2013, when she announced on NPR’s Marketplace that “it’s too early to tell” if wellness saves money, and that employers need to “experiment” with these programs to “see what happens to participants’ weight and blood pressure.” Right there, she invalidated herself. First, by then she certainly knew that a participants-vs-non-participants methodology was invalid since the key “smoking gun” slide in our Health Affairs posting was already widely circulated and her own opposing Oregon methodology was being widely praised. Second, even if she is right, the financial payoff for the modest “weight and blood pressure” improvements that the best programs might generate is 10-20 years in the future — and even then only if the improvements are sustained.

But then came another personality change.

In February 2014, she blamed readers for focusing on her attention-grabbing headline, the certainty of her two significant digits, and the gist of the conclusion…while ignoring the fine print, such as a caution about publication bias. Publication bias? You think? Start with the standard publication bias that negative articles rarely get published because they don’t get cited and hence reduce the all-important “impact factor” – recall the difference in Health Affairs citations in her own wellness article vs. the others.

Add to that a publication bias specific to those journals:  most of the articles comprising her meta-analysis were published in third-tier journals. Among them, these journals have exactly once published an article critical of wellness (twice if you include a book review by the esteemed Norton Hadler, whom a third-tier journal is thrilled to publish regardless of what he says, and three times if you include publication of an article by a graduate student at the University of Tasmania that accidentally undercut the true believers’ own storyline, that they are now having to explain away).

Weeks later, a totally different personality emerged:  she told the editor of Insurance Thought Leadership that she no longer focused on wellness and consequently has no opinions to share.  Leaving aside the irony that the wellness true believers continue to cite as gospel someone who says she has no interest in what they are citing her for, this spin further invalidates her next comment, delivered in December 2014 – when suddenly, as a result of yet another personality change, she has opinions again. She told All Things Considered: “It could be that when the full set of evidence comes in, [wellness] will have huge returns on investment.”

Oops. First, she has just admitted she doesn’t follow wellness, so why speculate on future studies she has no knowledge of in a field she’s not involved in? Second, there is a rule of thumb in epidemiology:  the bigger the impact, the smaller the sample size needed to discern it. An example would be smoking and lung cancer, a previously very rare disease whose cause was discernable from a handful of cases. A sample of only hundreds of veterans was needed to prove that very high blood pressure causes strokes, and studies of exercise almost always show either a physical or emotional benefit, even in small groups of people with significant disease. On the other hand, there have been probably close to a half-billion employee-years of wellness with nothing to show for themselves except results going the other way and a bunch of self-invalidating vendor lies.

Available now at Amazon.com

So we are going to make a radical proposal to the true believers:  you can continue to cite Katherine Baicker but must also note that she herself no longer supports the study you are citing — until and unless she says she does. In exchange for this disclosure, when do you cite her, we will acknowledge that you are telling the truth for a change.