Home » Posts tagged 'Katherine Baicker'
Tag Archives: Katherine Baicker
We’ll go walking out
While others shout of war’s disaster
Oh, we won’t give in
Let’s go living in the past
A delegation from the Validation Institute attending the World Health Care Congress time-traveled over to the US Chamber of Commerce Monday to watch Ron Goetzel and a few acolytes try to re-create the shiny new wellness object from the 1980s. Just like in the 1980s, the theme was, and these are the exact words of one of the speakers: “Why aren’t more employers seeing the value in wellness?”
Um, because there isn’t any? 7 years after I first proved, and guaranteed, that none of this “pry, poke and prod” nonsense could possibly add up, the evidence has finally caught up with the conclusion.
Of course, there is value in wellness done for employees, rather than to employees. And far be it from me to discourage anyone from being screened according to guidelines. But that’s not what the US Chamber is all about. The US Chamber is about pressuring the EEOC into restoring the punitive “voluntary” wellness penalties that a federal judge put the kibosh on in December 2017, effective this year.
The irony is, the Chamber is its own worst enemy. If the EEOC ever got hold of a recording of that session, it would constitute an irrefutable argument for the EEOC not to change the rules. The speakers other than Ron were all about improving employee well-being through autonomy, independence, responsibility for projects from beginning to end, and creating a culture of health. All those are worthy goals. Goals which, more than coincidentally, have nothing to do with screening the stuffing out of the workforce and penalizing employees 30% for not losing weight. (Shame on Oprah Winfrey!) Quite the contrary, any organization trying to achieve those goals would never dream of manhandling their employees like that.
Ron Goetzel: Katherine Baicker is living in the past
If a fortune-teller had predicted the following as recently as three weeks ago, I would have demanded a refund: Ron Goetzel dissed Katherine Baicker., Specifically, he accused Professors Baicker and Song, along with BJ’s and their vendor, of “doing a program out of the 1980s,” saying that their outstanding study could safely be ignored. (These are the same 1980s in which wellness, according to its promoters, couldn’t fail.)
What horrible 1980s elements did the BJ’s program have? Coaching, screening, learning “modules” and HRAs. Of course, no one has included those things in a wellness program for at least 30 years. (not)
Curiously, Ron didn’t have any issue at all with Baicker’s original 2010 “Workplace Wellness Can Generate Savings” article, which was chock full of programs literally from the 1980s. Only 2 of the programs analyzed in that study were conducted in the 21st century.
If that fortune-teller had also predicted that Ron would start telling the truth someday, I would have demanded double my money back. But no fortune-teller would ever predict that. Fortune-telling industry malpractice insurers will no longer cover lawsuits arising out of that prediction.
And for good reason. It’s generally a safe bet, given Mr. Goetzel’s tenuous grasp on integrity, that he’s going to misrepresent something. Yesterday, as is his wont, he deliberately misrepresented the position of The Incidental Economist (TIE) on the subject of wellness. He quoted TIE as saying that wellness “usually” doesn’t save money. (As a sidebar, I’d note that it is a bit of a head-scratcher that even the misrepresented TIE position makes Goetzel’s position look bad.)
Perhaps I need new glasses, but I can’t seem to find the word “usually” in the most recent TIE article on wellness savings:
There’s another reason Ron doesn’t quote TIE, which is they completely dissed him and his Koop Award friends in that article. Totally worth a read.
What does a Goetzel program look like?
So how do you succeed? How do you avoid programs “from the 1980s”? Maybe you avoid that by using a vendor where Ron has a “strong working relationship,” like Vitality. And just to be on the safe side, maybe you report only the results from Vitality’s own program for Vitality’s own employees. That way you can’t fail…
…and yet fail they did…
Even with the investigator bias and self-selection and “home court advantage,” Vitality employees got 2% fatter and their eating habits got worse after being coached.
Although maybe that counts as a success in the wellness industry, because employees weren’t too badly harmed, as they are in a typical Koop-Award winning program. Congratulations to Mr. Goetzel!
Part 2 picks up where Part 1 left off, as coincidence would have it.
Soeren Mattke (as mentioned in the last installment) and I were quite relentless in trying, quixotically, to get Professor Baicker to explain her results. Its popularity could have landed her many profitable speaking and consulting gigs, but she evinced no interest in cashing in, or even in defending her position. Indeed, the four times she spoke publicly on the topic, she didn’t do herself, or her legions of sycophants in the wellness industry, any favors. In each interview, she distanced herself more and more from her previous conclusion. Here are her four takeaways from her own study “proving” wellness has precisely a 3.27-to-1 ROI:
- It’s too early to tell (um, after 30 years of workplace wellness?)
- She has no interest in wellness anymore
- People aren’t reading her paper right (Shame on us readers! We’re only reading the headline, the data, the findings and the conclusion, apparently)
- “There are few studies with reliable data on the costs and the benefits” (um, then how were you able to reach a conclusion with two significant digits?)
Individually or in total, these comments sounded an awful lot like retractions, but she (and her co-author and instigator, David Cutler) claimed those comments didn’t constitute retractions. Whatever they were, she wasn’t exactly doubling down on this 3.27-to-1 conclusion.
The industry at this time — 2013 and 2014 — came under a lot of criticism for basically being somewhere between hilariously worthless and a fraud. (Here is some comprehensive documentation of the fraudulent and harmful activity in the wellness industry. Wellness is a classic example of the observation that every cause starts out as a movement, becomes a business, and degenerates into a racket, the poster company for the last being Interactive Health.)
Here’s where it gets interesting. Due to this relentless criticism, instigated and sometimes written by me (most of which is catalogued here) with credit to Soeren Mattke and Jon Robison as well (with Pulitzer Prize-winning LA Times columnist Michael Hiltzik piling on multiple times), Ron Goetzel and his colleagues urged Professor Baicker — who had previously claimed to have no more interest in wellness — to get a grant from Ron’s friends at the Robert Wood Johnson Foundation and do an actual controlled study of workplace wellness to set the record straight. As a strategic gambit, this showed a level of professional judgment worthy of Gary Hart.*
Obviously if you are perpetrating a fraud and you know it, you don’t urge people to investigate it. Mr. Goetzel at this point (or at least shortly thereafter) knew that his industry was a fraud, going so far as to erase some accurate data and disown inconveniently accurate conclusions he had forgotten to suppress.
He even forged a letter from the governor of Nebraska to cover his tracks. (To be completely accurate, he did not forge the signature on the letter. He merely doctored the wording in a letter that was already signed to say something much different than the letter said when it was actually signed. Maybe that’s not technically forgery. Who knows?)
And yet that’s exactly what Mr. Goetzel did. Maybe he thought she was as corrupt as he is, but she has an excellent reputation in this industry. My suspicion is that she accepted his offer for exactly the opposite reason that he made it — to clear her name after her ill-considered 2010 meta-analysis.
Way back in 2016, I predicted how this gambit would turn out — that Katherine Baicker was not going to mail it in this time but rather do a high-quality study of the type she is known for.
Her now-famous outcome was preordained for four reasons:
- In this industry, as Michael O’Donnell pointed out in the trade journal where he formally served as Prevaricator-in-Chief, the higher quality the study, the lower the ROI, to the point where RCTs invariably show negative ROIs. (Owing to the embarrassing accuracy of this unintended finding, Mr. O’Donnell walked it back in a subsequent editorial.)
- Kate Baicker was not going to sully her legacy by being known as the Typhoid Mary of wacky wellness programs, just to please her sycophants.
- She picked the most clueless vendor imaginable to carry their water. This selection couldn’t have been an accident.
- No wellness study since Johnson & Johnson (which was probably also wrong–just not obviously from the data presented) had shown even a remotely positive outcome. This list includes PepsiCo, Barnes Hospital, Nebraska, Vitality, Newtopia, Health Fitness Corp, Health Fitness Corp again, Boise School District/Wellsteps, McKesson, Connecticut, and the University of Illinois. All were epic fails, according to their own data. Not to mention that the wellness trade association itself published a case study showing wellness loses money and then ran away from it.
Where does the wellness industry go from here? They’re hoping for Tiger Woods, wiser souls are channeling Gary Hart, and probably what we’ll get instead is Harold Stassen. (Look it up.)
*For those of you who aren’t old enough to remember, here’s what happened. Dogged by rumors of philandering, the front-running Democratic presidential candidate urged reporters to follow him around. It took about a day for reporters to discover he was indeed philandering. It only took another few weeks for him to withdraw from the presidential race.
There is a lot more to this study than meets the eye.
Some tourist attractions feature an “A” tour for newbies and then a “behind-the-scenes” tour for those of us who truly need lives. For instance, I confess to having taken Disney’s Magic Kingdom underground tour, exploring, among other things, the tunnels through which employees travel so as not to be seen out of costume in the wrong “Land.”
Likewise, there have been many reviews of the recent wellness study conducted by the National Bureau of Economic Research (NBER), the first-ever randomized control study of a wellness program. This, however, is the first review to go beyond the “A” tour of the headlines.
By way of background, the headline is that the mainstream wellness program the investigators examined at the University of Illinois did not noticeably move the needle on employee health. They didn’t address return-on-investment (ROI), because there obviously was none. Achieving a positive ROI would require moving the health risk needle—not just by a little, but rather by enough to significantly improve the health of many employees. Then, since wellness-related events, such as heart attacks, would not otherwise have befallen these employees immediately, this improvement would have to be sustained over several years before there was a statistical chance of some events being avoided.
Finally, the magnitude of this improvement would have to be great enough to violate the rules of arithmetic, because it is not mathematically possible to avoid enough medical events to break even on wellness. For instance, it actually costs about $1 million to avoid a heart attack through a screening program.
This finding, therefore, represents an existential threat to conventional wellness programs.
It all boils down to: why would an associate professor (Damon Jones) publicly humiliate his own dean (Katherine Baicker — yes, the very same Katherine Baicker who always seems to be on the wrong side of every wellness debate) …unless he is absolutely sure he is right?
She can’t fire him now because that would get picked up by the lay media. Perhaps she should have paid him $130,000 not to disclose the results.
OK, this time I’m not the one causing the kerfluffle in the wellness industry, though I will confess to being a force multiplier.
Not since 2014, when the very unstable morons at the Incidental Economist made fun of the very stable geniuses who give out the Koop Award and also unequivocally concluded wellness loses money — combined with continued fallout from the Penn State debacle and the Nebraska scandal — has the wellness industry had such a bad year. And it’s only February.
Let’s review what’s happened so far in 2018. First, a federal judge ruled that voluntary wellness programs need to be — get ready — voluntary. The EEOC’s responded with the legalese equivalent of: “Fine, be that way.”
Next, WillisTowersWatson did something that might get them in hot water with the very stable wellness industry leaders: they were honest. They published a study revealing that employees hate wellness even more — way more — than they hate waiting for the cable guy to show up.
Finally, the very unstable National Bureau of Economic Research conducted a controlled study finding basically no impact whatsoever of a wellness program. More importantly, they specifically invalidated the “pre-post” methodology. Even more importantly, they specifically invalidated 78% of the studies used in Kate Baicker’s “Harvard Study” meta-analysis.
Here is an interesting piece of trivia. The lead researcher is an assistant professor at the Harris School of Public Policy. Why is this interesting trivia? Because Katherine Baicker — the Typhoid Mary of Wellness, whose THC-infused 3.27-to-1 ROI is the basis for essentially every subsequent genius wellness outcomes claim — is now the dean of that very same Harris School. I’m just guessing here, but I’d say it’s gotta be a trifle embarrassing when your own subordinate publicly disproves your own study. I mean, it’s one thing for me, RAND, Bloomberg, and anyone else with five minutes, internet access and a calculator to do it, but…your very subordinate?
On the other hand, the researcher, Damon Jones, just demonstrated not just amazing competence, but amazing integrity as well. In other words, he has no future in wellness.
The Wellness Empire Strikes Back
How does the wellness industry respond to these smoking guns threatening their entire revenue stream? Apparently, there is little cause for concern on their planet.
Let’s start with America’s Health Insurance Plans (AHIP), the health insurance industry lobbying group. Here is AHIP’s oxymoronic Wellness Smartbrief (January 26), on the NBER research. Yes, it summarizes the same wellness-emasculating study as the one above, though you could never guess it from the headline:
Continuing, AHIP said:
Offering incentives for completing wellness activities might be more cost-effective than offering incentives for wellness screening, a recent study of a comprehensive program found.
Perhaps AHIP has been infiltrated by Russian trolls, because here’s what the NBER article actually said about “completing wellness activities”:
We…do not find any effect of treatment on the number of visits to campus gym facilities or on the probability of participating in a popular annual community running event, two health behaviors that are relatively simple for a motivated employee to change over the course of one year.
Wellness programs might attract mostly employees who are already fitness-conscious, but the potential to attract healthy employees whose medical spending is already low could nonetheless be a boon to employers, the researchers found.
And on the subject of “the potential to attract healthy employees” being a “boon to employers,” the authors actually said:
We further find that selection into wellness programs is associated with both lower average spending and healthier behaviors prior to the beginning of the study. Thus, one motivation for a firm to adopt a wellness program is its potential to screen for workers with low medical spending. Considering only health care costs, reducing the share of non-participating (high-spending) employees by just 4.5 percentage points would suffice to cover the costs of our wellness program intervention.
In other words, you can apply some workplace eugenics to your company by using wellness to weed out obese employees, employees with chronic or congenital diseases, and so on. Good for you!
Soon, if AHIP and others have their way, there will be no need for guesswork in eugenics: employer wellness programs will be able to screen these employees out based on their actual DNA.
AHIP’s take on AARP v. EEOC
And now, AHIP’s take on this landmark case, their ace reporters scooping everyone with this February 2 headline on the December 20th court ruling:
Here are more typical headlines on that court ruling, headlines that came out the same month that the court ruling came out. Perhaps AHIP used the interim six weeks to focus-group various verbs until they settled on…tweak???
AHIP: It’s not just the headlines
One prominent healthcare executive recently attended an AHIP conference and reports:
I just returned from one of the dumbest meetings I’ve ever attended in Washington. Report of a new “study” by AHIP. Turns out people don’t mind health costs all that much, they just want more benefits. And everything is hunky-dory with their health plans, people like them so much. They love wellness benefits and crave more. Prescription drug prices have been nicely controlled thanks to the competitive marketplace (no, I am not making this up or exaggerating for drama). For every $1 employers spend on benefits workers get $4 in value. Priorities for SHRM rep: Fitbits for all employees, solving the outrage that only 20% of her employees got an annual physical. 85 cents of every dollar spent on health care goes to chronic disease.
Over these same two hours, I’d estimate about a thousand employees were misinformed, harmed or harassed by wellness vendors, roughly equal numbers of employees got useless annual checkups, employers spent about $200-million on healthcare and 40 people died in hospitals from preventable errors. But I’m being such a Debbie Downer! I’m going home to read Why Nobody Believes the Numbers to remove myself from this alternative universe.
Enter the Health Enhancement Research Organization (HERO)
HERO’s Prevaricator-in-Chief, Paul Terry, is demonstrating his usual leadership abilities in this crisis, of course. After all, HERO is the wellness industry trade association and these three items — the NBER invalidating their product, employees hating their product, and a federal judge forbidding them to force employees to use their product — represent existential threats to his “pry, poke and prod” members.
Teddy Roosevelt said, “complaining about a problem without posing a solution is called whining.” It’s a quote that also reminds me why I’ve not thought of angry bloggers who target health promotion [vendors] as bullies. Though they relish trolling for bad apples, their scolding is toothless, more the stuff of chronic whiners.
I suspect he is talking about me here as the “chronic whiner” who is “scolding” them. Or perhaps he is referring to the “angry bloggers” at the Los Angeles Times, the New York Times, Slate, or STATNews, since those “toothless” publications seem to be scolding wellness vendors more than I ever have. For instance, I’ve never called wellness vendors’ offering a “scam” or a “sham.” I simply quote these very stable wellness geniuses verbatim, as above or below, or last week.
Being quoted verbatim, not angry bloggers, is their worst nightmare. (One thing I would concede, though, is that “Paul Terry and the Angry Bloggers” would be a great name for a rock band.)
Yep, looks like the implosion of his industry all my fault. Otherwise, I’m not quite sure who is the “angry blogger” he is referring to, other than to note that Mr. Terry himself seems to blog a tad angrily himself, both above, and here…
Why I choose to ignore the blogger critics: We’re fortunate to work in a profession with a scant number of vociferous critics. My take is that there is one thing these few angry loners [Editor’s note: the complete “scant list” of the 220 “few angry loners” who have been “vociferous critics” can be found here] want more desperately than attention: that’s to be taken seriously. What they fail to comprehend is that as they’ve gotten ever more farfetched and vitriolic in search of the former, they’ve cinched their inability to attain the latter.
Baiting people with misinformation and offensive insults (but just a tad under highly offensive) is a pesky ploy that trolls hope will eventually land a bite that confers credibility where there is none. Even reading such drivel is a form of taking the bait; responding is swallowing it whole. Some say dishonesty should not go unchallenged and I respect their view; nevertheless, I’m convinced responding to bloggers who show disdain for our field is an utter waste of time. I’ve rarely been persuaded to respond to bloggers, and each time I did it affirmed my worry that, more than a waste, it’s counter-productive.
…and especially here, a seemingly incongruous decision to “act out” by someone who claims to be “choosing to ignore the blogger critics.”
Having read years of my “drivel” alongside Mr. Terry’s posting explaining why you shouldn’t “swallow this bait,” perhaps readers might opine here: which of us, exactly, is the “chronic whiner”?
Coincidentally, when I run live health-and-wellness trivia contests, the first of our 3 rules is: No Whining. Seems to me that he would have just violated it. Indeed the only rule HERO hasn’t violated so far is #3 below. Not that I want to put ideas in their head.
Not to be confused with those immortal words often attributed to the great philosopher Yogi Berra, a big joke among economists is: “An economist is someone who upon learning how something works in practice, wonders how it will work in theory.”
That joke morphed into reality last month — though it was a controlled study testing a theory, rather than the theory itself. The “theory” would be that inactive unmotivated non-participants can be used as a control for active motivated participants. Ironically, this study design has never been proposed as legitimate, even in theory. Wellness “researchers” like it because it always show savings, even when nothing happens. For example, even when there was no program for the participants to participate in.
Obviously if this were a legitimate design, the FDA would approve it for clinical trials, saving a ton of time and money vs. having to do controlled trials.
To wit, the National Bureau of Economic Research (NBER) just published a study showing that the participants-vs-non-participants (“par-vs.-nonpar”) study design, used extensively by the very stable geniuses in the wellness industry to do their alt-research to fabricate their alt-findings, is completely invalid.
Highlights of NBER study for the wellness industry: bad news first
First the bad news. Fasten your seat belts and be shocked, shocked to learn that the researchers could identify:
- No noticeable change in health behaviors due to the wellness program
- No noticeable change in health outcomes due to the wellness program
- Clear self-selection bias among participants opting into the wellness program
Lest anyone think we are taking this out of context, here are their exact words:
We do not find any significant effects of treatment on total medical expenditures, employee productivity, health behaviors, or self-reported health measures in the first year following random assignment. We further investigate the effect of our intervention on medical expenditures in greater detail, but fail to find significant effects on different quantiles of the spending distribution or on any major subcategory of medical expenditures (pharmaceutical, office, or hospital). We also do not find any effect of treatment on the number of visits to campus gym facilities or on the probability of participating in a popular annual community running event, two health behaviors that are relatively simple for a motivated employee to change over the course of one year.
This of course merely confirms what we observe in outcomes reports published by wellness vendors, including the two most recent proud recipients of the Deplorables Awards. Actually, in the case of Wellsteps there was indeed a noticeable change in health outcomes among program participants — they got worse.
Also, the authors — no doubt anticipating the objection from the very stable geniuses at the Health Enhancement Research Organization — specifically note that nothing in Year One’s results presage any step-function improvement in Year Two. So the specious “Wait ’til next year” argument is off the table.
You might be thinking, “Another nail in the wellness industry coffin.” True, except that there almost isn’t room for any more nails. Soon the coffin will have enough nails to create its own gravitational field.
Next, the good news
Here are the two pieces of good news for the wellness industry. One finding was:
Our 95% confidence intervals rule out 78 percent of previous estimates on medical spending and absenteeism.
That means that it is possible that 22% of previous estimates may conceivably not be completely invalid. This is not to say that 22% are valid, just that they aren’t automatically invalid. That is great news for the wellness industry, where clearing the bar for not being automatically invalid is cause for celebration. As Dave Chase says, the bar for wellness is so low a snake could jump over it.
However, the 78%-totally-invalid figure specifically invalidates Katherine Baicker, author of the so-called “Harvard Study.” Depending on whether you are a wellness vendor or an oppressed employee, she is either the Johnny Appleseed or the Typhoid Mary of wellness. Her famous 3.27-to-1 ROI was tallied entirely from par-vs.-nonpar studies, exactly the methodology that the NBER just invalidated, citing exactly the studies she cited. (The three examples in my study referenced above were also part of her meta-analysis.)
So perhaps she might now make a formal statement regarding par-vs-non-par as a study design? Either a retraction or a defense. Just something that clarifies her previous statements, which seem to be neither retractions nor defenses but rather more like excuses:
- It’s too early to tell (um, after 30 years of workplace wellness?)
- She has no interest in wellness any more
- People aren’t reading her paper right (we’re only reading the headline, the data, the findings and the conclusion, apparently)
- “There are few studies with reliable data on the costs and the benefits” (um, then how were you able to reach a conclusion with two significant digits?)
The irony is that Kate Baicker has otherwise done outstanding research. Her study on Oregon Medicaid is a classic. In Oregon at the time, Medicaid eligibility was determined by lottery amongst applicants. That meant that — quite the opposite of wellness control groups — the control group of people not picked in the lottery had equal motivation to seek insurance coverage as people who were picked. After following both groups going forward, her finding was that obtaining insurance to access basic medical care did not change outcomes. (Having insurance did bring peace of mind, though.)
And yet somehow in “Workplace Wellness Can Generate Savings,” she was quite comfortable reaching a conclusion that was completely inconsistent with her Oregon finding, not to mention the Law of Diminishing Returns: throwing additional unrequested, generally unwanted, and largely misdirected medical interventions and advice at employees who already have insurance — and recall that most insured Americans are drowning in medical care — could dramatically improve their outcomes enough to calculate not just a massive ROI, but an ROI precise to two significant digits.
What she will hopefully learn through the NBER study is something that I learned 11 years ago: when the data proves you wrong, fess up. Then people like me have to find someone else to blog about. Fortunately, in wellness, that is not a heavy lift.
The second piece of good news for the wellness industry
To quote the study:
…wellness incentives may shift costs onto unhealthy or lower-income employees if these groups are less likely to participate in wellness programs. Furthermore, wellness programs may act as a screening device by encouraging employees who benefit most from these programs to join or remain at the firm…
To be clear, this calculation does not imply that adoption of workplace wellness programs is socially beneficial. But, it does provide a profit-maximizing rationale for firms to adopt wellness programs, even in the absence of any direct effects on health, productivity, or medical spending. [emphasis theirs]
In other words, employers can use wellness programs to subtly discriminate against unhealthy — read, older and poorer — workers. Many of the very stable geniuses in the wellness industry will be happy to hear this. The exceptionally stable genius who will be most thrilled to hear this is Michael O’Donnell. Mr. O’Donnell is the former Prevaricator-in-Chief of the wellness industry trade publication and a current member of the Koop Award cabal. These are excerpts from one of his editorials:
First, he says prospective new hires should be subjected to an intrusive physical exam [editor’s note: notwithstanding the fact that this is totally illegal], and hired only if they are in good shape. OK, not every single prospective new hire needs to be in good shape — only those applying for “blue collar jobs or jobs that require excessive walking, standing, or even sitting.” Hence he would waive the physical exam requirement for mattress-tester, prostitute, or Koop Award Committee member, because those jobs require only excessive lying.
Second, he would fine people for not meeting “outcomes standards.” In an accompanying document, he defines those “outcomes standards.” He specifies fining people who have high BMIs, blood pressure, glucose, or cholesterol.
In other words, Mr. O’Donnell wants to charge for insurance by the pound, as that accompanying document says. Actually, by BMI, which of course is of dubious value as a measure of weight, let alone health.
Here is his actuarial formula:
Although having read his very stable arithmetic elsewhere in this same document, I’d worry about the accreditation status of any actuarial school, or for that matter any school of any kind within the 50 states, that would accept him:
Thirty-one states have no laws that prohibit employers from using smoking status as the reason for not hiring… In the remaining 29 states…smoking status cannot be used as the reason for not hiring.
I’m not waiting around for a retraction from this genius either.
Here’s a similar apology, for Professor Baicker’s infamous “Workplace Wellness Can Generate Savings” meta-analysis claiming the 3.27-to-1 ROI from wellness that RAND also eviscerated:
I had been quite adamant in the previous post that this meta-analysis was likely just a gold-plated package of garbage case studies. I compared it to packaging subprime home loans into AAA-rated collateralized mortgage obligations (CMO).
That was before I looked at the individual studies comprising this meta-analysis. I realize now that comparing the Harvard Study to the CMO scam was unfair. So I owe an apology to Bear Stearns, Lehman Brothers, and Countrywide.
Next, I apologize for pointing out that Ron Goetzel, as recently as last week, is still quoting this very same thoroughly discredited 7-year-old study, as well as many other outdated analyses. For example, he insists on continuing to quote the New York Times economists’ September 2014 analysis that wellness programs “generally” don’t work, even though they subsequently made their conclusion much clearer: “We’ve said it before, many times and in many ways: workplace wellness programs don’t save money.” They then specifically criticized Mr. Goetzel’s own methodology (“industry studies based on study designs that cannot produce valid causal estimates”).
I apologize for thinking that this deliberately selective misinterpretation of these economists’ previous conclusions makes him sound deceitful. And I apologize for being sure that the people who forwarded me this slide would agree with that assessment. And I apologize for once again making phony apologies.
The real apology
Now that I’ve checked off the usual Wellsteps-and-Goetzel-integrity boxes, it’s time to step out of character and seriously apologize. Here’s what I did that I really do need to apologize for: not cutting wellness professionals enough slack on giving them enough time to learn what took me several years to learn about wellness losing money.
I was once guilty myself of believing this 3-to-1 ROI nonsense, specifically about disease management (DM). Why DM in particular? I had some ego wrapped up in it because I am actually credited with inventing DM. Really. Just google on “invented disease management.” Whether or not I did (and plenty of others could share the credit), I was not just drinking the DM Kool-Aid. I was mixing it up and selling it to others.
Then, 10-12 years ago, a few people told me none of the DM savings numbers added up. I didn’t believe them. I thought it was sour grapes because they missed the boat.
True, I had enrolled a few of my own extended family members and friends into DM programs. Vendors were more than happy to offer me their best nurses, VIP treatment, you name it. Yet no one I referred thought these free programs were even worth a second free phone call.
Nonetheless, I was sure that somewhere there existed a whole lot of employees who did benefit from DM. Yes, in my fantasy world tons of people really appreciated these unsolicited calls from their health plan offering to help. After all, who among us doesn’t trust an unsolicited caller from their health plan offering to help?
In my worldview, the lucky recipients of these calls would respond: “You’re right. I should be taking my pills. Hey, thanks. I never would have thought of that on my own. And I was just about to have a heart attack, so you saved a ton of money.”
Yes, I realize this made no sense. Yet I never questioned my own findings. Basically, I ignored the warning signs about DM’s sketchy economics for years. When I finally had the epiphany, it got quite the headline:
Once I questioned my own figures, the rest because immediately obvious to me — one after another after another, sets of numbers in this field simply did not add up. Wellness was a far worse offender than DM, which does appear to roughly break even or better. No surprise about wellness. Screening costs about 10 times as much as DM and whereas people who qualify for DM are already well down the slope to infirmity, screens are performed mostly on healthy employees, who can’t generate any savings. (We of course support screenings according to guidelines, for the health of employees rather than an ROI. But most vendors ignore guidelines and screen the stuffing out of employees.)
Still, it had taken me years to have the initial epiphany…and yet now I was quite curt and dismissive with other people who didn’t immediately get it, and were defensive in support of their lifelong assumption. I was basically saying to others in the field: “You know all those savings claims? Total malarkey. Prying, poking and prodding doesn’t save any money.” I’d expect everyone else to get this right away. When they didn’t, I was not gracious with them in many cases.
Over time, a large number of folks have come around. They did it at their own speed, same as I did. Take WELCOA, for example. They were among the worst (and God bless ’em, funniest) offenders…and yet now you won’t find an organization more committed to getting wellness right, helping employees, and being honest than WELCOA. (In their case, a night-and-day change of leadership helped.) I’m not just saying this–I’m walking the walk. Quizzify is joining WELCOA’s Premier Provider Network for 2017, joining vendors I have a lot of respect for, like It Starts with Me, Populytics, and SelfHelpWorks. The first two, also like Quizzify, are validated by the Validation Institute.
Making good on the apology
A good apology comes with an offer to make it up. So if you feel like I dissed you prematurely, while you were learning wellness economics on your own, you can have one of:
- a free pdf of any of my books — Surviving Workplace Wellness, Cracking Health Costs, or Why Nobody Believes the Numbers
- half-price on a hardcopy of those books (order direct, not through Amazon, of course)
- a free analysis of any outcomes report
- a free month of Quizzify with no commitment (there is a minor asterisk on this one, please inquire)
In the workplace wellness epidemic, Harvard School of Public Health Professor Katherine Baicker is Patient Zero. However, she may soon come up with the cure for the disease she spread. This would be like when the inventor of the PSA test said it shouldn’t be used.
By way of background, her original dalliance with wellness in Health Affairs–timed and designed to get the Business Roundtable’s “Safeway Amendment” allowing a 30% to 50% clawback of insurance premiums into the Affordable Care Act–introduced the infamous “3.27-to-1 ROI” into wellness vendor vernacular. That was almost seven years ago, but wellness vendors continue to cite this figure as gospel even though RAND demolished it and Professor Baicker herself walked it back four times.
For good reason, as it turns out — this Pulse posting says it all. The 10-to-30-year-old studies comprising her meta-analysis are even more amusing in retrospect. Lots of overcreening, overreliance on now-discredited BMIs, and of course recommendations for low-fat diets–even for employees with metabolic syndrome. Most published in third-tier journals, by the Wellness Ignorati and their friends.
And almost every one of them compared active motivated participants to non-participants. 6 years ago we suspected that design was invalid. Today we know for certain, thanks to Health Fitness Corporation and Aetna. Two more elegant studies to prove that point could not be imagined. No “investigator bias” or “publication bias” there, since both were hoping for the opposite finding.
However, mindful of her reputation, Professor Baicker recently announced (in as many words) that she is going to redeem herself.
Her protestation that she is no longer interested in wellness appears to have taken a backseat to her desire to remove this one blot–granted, a blot of Lady MacBeth proportions–on an otherwise excellent curriculum vitae.
She is going to do a study for BJ’s Wholesale Club, in conjunction with an outfit called Wellness Workdays. Wellness Workdays is a classic wellness vendor. That is to say, they won’t be winning a Nobel Prize anytime soon, or even a spelling bee. Let’s start by examining their analytic and clinical prowess.
To start with, their “White Paper” doesn’t just quote the infamous 3.27-to-1. They’ve upped the ante to 6.00 to 1, maintaining the two significant digits while almost doubling the savings. How? They’ve added the 3.27-to-1 for healthcare savings to the 2.73-to-1 for absenteeism reduction from that same 2010 study. Those two separate conclusions were reached from almost totally different studies. Anyone can tell that from reading the original. Anyone, that is, except Wellness Workdays.
Their analytic qualifications are matched only by their clinical qualifications. One member of their medical advisory board is Chief of Allergy and Clinical Immunology at the Indian River Medical Center in Vero Beach, Florida. While this expertise is not exactly central to the mission of the pry,poke, and prod industry, in all fairness it should be noted that the Indian River Medical Center runs one of the best allergy programs in all of Vero Beach.
Another is an OB-GYN in Colorado. Perhaps this advisor will develop a protocol for employees who want to be screened and induced at the same time. A third consults to orthopedists at “Lennox Hill Hospital,” a role that probably doesn’t require too much heavy lifting, because there is no hospital by that name.
This guy is also an expert on steroids and other performance-enhancing products, and has “published rseveral esearch studies.”
So they can’t spell, can’t proofread, can’t understand study design, and can’t cobble together a qualified advisory board. In other words, to paraphrase the immortal words of those great philosophers Gilbert & Sullivan, they are the very model of a modern clueless wellness vendor.
Katherine Baicker’s proposed study design
We are confident she is going to get it right this time and “discover” that wellness loses money. By selecting a vendor of the caliber of Wellness Workdays, she isn’t leaving anything to chance.
Having learned her lesson — and I’ve been pretty conscientious about forwarding helpful study design materials to guide her — she will certainly tally wellness-sensitive medical events across the entire population. More importantly, here’s what she is not going to do with this study:
- “Match” volunteer participants to — you guessed it — non-participants. This nonsense that keeps the wellness industry afloat, and Professor Baicker no doubt sees right through it;
- List the “unobservable differences” between participants and non-participants as a “limitation.” It’s not a limitation. It’s a baldfaced lie. It is now known that this face-invalid study design is truly invalid. She wouldn’t lie, right? That would damage the reputation of the entire Harvard School of Public Health in what will certainly be a high-profile study;
- Show a high ROI even though the change in health risk factors is trivial. This of course is the stock-in-trade of the Koop Award committee, but a real academic researcher would know better;
- Compare the costs to “trend” and say “costs were projected to rise by [this amount] but they only rose by [this lower amount], and therefore a huge pile of money was saved”;
- Attribute all cost differences to the program, whereas a real researcher would look only at utilization differences that could actually be attributed to urging employees to eat more broccoli. (By contrast, one of the studies in her previous meta-analysis credited the wellness program with a reduction in cat scratch fever.)
How do we know she isn’t going to make up phony outcomes again? She “tipped her hand” with the Oregon Medicaid lottery study. A terrific natural experiment, her study emphasized the value of a “lottery control,” meaning every subject had the same intent-to-treat. Exactly the opposite of the wellness participants-vs-non-participants study design.
Further, who wants to be known as the Typhoid Mary of workplace wellness? The Oregon study was an excellent one, so naturally it showed exactly the opposite of what wellness studies show. Specifically, facilitating access to care doesn’t reduce the cost of care or improve physical health status. And that was for people — newly minted Medicaid recipients — who didn’t have any insurance to begin with. Wellness, of course, takes employees who already have plenty of access to care and drown them in even more, unwanted and largely unneeded, screenings and checkups.
As for BJ’s Wholesale Club, I suspect they got suckered into this. Who volunteers to become the next Pepsico, a case study of how wellness programs fail? BJ’s obviously isn’t studying their own competitors: Target has one of the best programs in the country — precisely because they are far too smart to use a vendor like Wellness Workdays.
In any case, we look forward to her research study. Or perhaps, since this is in conjunction with Wellness Workdays, to rseveral esearch studies.
If you listen to the Centers for Disease Control and Prevention (CDC), you would think chronic disease is the main health problem we face, and workplace wellness is the main weapon we have to face it with. I know what you’re thinking (at least for the former): isn’t it?
Nope. The country’s main health problem — at least among those addressable by the CDC as opposed to by Congress — is something else altogether, essentially the opposite of what the Wellness Ignorati say it is. But before we reveal the answer, let’s review the CDC’s chronic disease talking points, which naturally are hilarious, as most talking points in support of wellness tend to be.
First, in the screenshot below, they quote the “arresting” statistic that “7 out of 10 deaths are due to chronic disease.” Um, that is called civilization, folks. Countries where 7 out of 10 deaths are due to causes other than chronic disease would love to have this arresting statistic. In case anyone doesn’t believe that the CDC — or indeed, that any human being other than a wellness vendor — could possibly be so stupid as to think civilization is a problem that needs solving, here is the screenshot, and here is the link.
Second, they recently bumped the “75% of costs are due to chronic disease” urban legend in the first line of the screenshot to a mind-boggling 86%. Surely even the dumbest CDC employee can’t believe this. Surely they can back-of-the-envelope an estimate that birth events, preventive care, and trauma alone account for much more than 14% of spending. Birth events by themselves account for about 16% of all hospital discharges.
Meanwhile, wellness vendors are now flogging those “7 of 10 deaths” and “86% of the nation’s healthcare cost” statistics to lobby Congress for wellness subsidies. Congress had wisely stopped funding one of the CDC’s many wellness boondoggles (Work@Health). That didn’t sit well with the industry, so they are starting a lobbying campaign. Fortunately, if their lobbying prowess is anything like their wellness prowess, the budget deficit is not likely to increase anytime soon. The letter reads:
Here is the real problem
This would all be very amusing, as the CDC and wellness vendors converge on these two statistics like monarch butterflies of innumeracy, except that our health is stake. And that (finally) brings me to the title of this posting.
The Maginot Line, as you might recall, proved about as worthless combating the Nazis as the CDC’s wellness obsession is today in combating the real healthcare problem: a massive explosion in blood-borne infections, or septicemia. While the CDC, wellness vendors, and of course the Health Enhancement Research Organization are all atwitter about diabetes and heart attacks (which none of these people can prevent and whose admissions in combination have been in check in all subpopulations for many years), consider ICD-9 038.9, Septicemia. There were 928,000 inpatient cases in 2013, the last year available.
It’s not just that it’s huge, almost twice as costly as the next most costly ICD9. It’s also exploding:
- has increased almost sevenfold;
- is now the by far the largest single diagnosis code;
- twice as costly as the second-largest…
- …and its growth is accelerating?
More importantly, why doesn’t anyone at the CDC seem to care about pathogens? This is what they are supposed to do–identify pathogens and prevent, contain or eradicate them.
Literally anyone (almost 1 in 300 people annually) could get a cut or injury or infection in the hospital, get septicemia, and, 13% of the time, die. Yet the CDC is blissfully unaware of this. If you’ve heard this “blissfully unaware” song before, the CDC’s Wellness Watchdogs also completely missed the workplace opioid epidemic. That happened right under their noses. The drugs were legal, prescriptions were filled, and PBMs paid for them.
Where was the CDC when this was happening? The same place the wellness industry was: nowhere. Most health risk assessments queried about illegal drug use and alcohol, but abuse of legal opioids? Off the table.
We can’t let the CDC overlook this epidemic too, due to their singularly misguided wellness obsession. We need to embarrass them into action–please send this note around to as many people as possible.
And if you’re wondering how the CDC (with the very notable exception of NIOSH!) has dumbed down so fast, so was I. These were, after all, the people who rid the US of malaria and rid the world of smallpox. So I did a little search on their site.
The first thing I noticed was that their workplace wellness information is “science-based.” That was the giveaway. In wellness, the phrase “science-based” means “not science-based.” To use one example, Wellsteps’ claim that their ROI model is “based on every ROI study ever published.” This translates as: “We made the whole thing up.”
Additionally, the references the CDC relied upon should look familiar. Besides being comprised of the usual serial liars, serial cheaters, and serial idiots, the list of references ends with Katherine Baicker, truly the Typhoid Mary of the workplace wellness epidemic–and hence one of the people most responsible for advising the CDC to create the Maginot Line that failed to prevent or event identify the opioid and bacteria epidemics that have taken millions more lives than workplace wellness has ever saved.
By the way, while you were reading this and the links, 6 to 12 more people (depending on how fast you read) just contracted a hospital-acquired infection, with probably 1 or 2 people dying from it.
To put that in perspective, the comparable statistics for wellness would be that 6 to 12 vendors just lied to their prospects, with 1 or 2 prospects believing them.
“You know that laundry detergent that gets bloodstains out of t-shirts? I’d say if you’re routinely coming home with bloodstains in your t-shirts, laundry probably isn’t your biggest problem.”
Recognize that line? That was Jerry Seinfeld’s signature joke, the one that more than any other propelled his early fame. Having used it for a decade, he retired it shortly after Seinfeld went on the air.
Now it’s time to retire the wellness industry’s signature joke: the so-called Harvard Study and its 3.27-to-1 ROI. Among other things, as we’ll see, a large chunk of the data in that study also predates Seinfeld, a show which recently celebrated its 25th anniversary. You know the old line: “How can you tell a lawyer is lying?” Answer: “His lips move.” Well, the way you can tell a wellness vendor is lying is that they still cite that study, knowing it to be completely invalid. (If they actually think that 3.27-to-1 is valid, in a way that’s even worse.)
The article, which today wouldn’t pass peer review by the normally cautious Health Affairs, was rushed into publication during the healthcare reform debate as a favor to David Cutler, a healthcare advisor to President Obama and an author on the study. Along with the results from the Safeway wellness program — which turned out not to exist (and I’m referring not just to the results but literally to the program itself) — this meta-analysis provided cover for the 30%-to-50% clawback provision (for non-smokers and smokers respectively) in what became ironically known as the Safeway Amendment to the Affordable Care Act.
In turn, this clawback was needed to secure the support of the Business Roundtable (BRT) for the Affordable Care Act. The BRT was and is obsessed with this provision. Not because they are especially concerned about employee health — the guy who ran their wellness lobbying also ran a casino company, exposing thousands of employees to second-hand smoke. Rather, because this clawback provision creates billions in forfeitures that go back into the coffers of its member organizations. Bravo Wellness accidentally spilled that secret. The worse the program, the more an organization can save via forfeitures. For instance, Wellsteps saved millions for the Boise School District even though healthcare spending/person jumped–simply by creating a program so unappealing that employees preferred to obtain insurance through their spouses. (Either that or they outright lied–you make the call.)
How Prof. Baicker Invalidated Her Own Study
The title was definitive and the conclusion was amazingly precise. The title was: “Workplace Wellness Can Generate Savings.” Not “might” or “possibly could on a good day” but “can.” And the conclusion itself left little doubt. The ROI wasn’t “approximately 3-to-1” or even “approximately 3.3-to-1” but rather a statistically precise 3.27-to-1. You don’t make such a definitive pronouncement without being ready to defend it.
Or maybe you do. Here’s how Professor Baicker’s story changed on various occasions once it became clear that her result was a major outlier, and that wellness loses money:
- It’s too early to tell whether these programs pay off, and employers should experiment on their employees;
- She’s not interested in wellness any more;
- People aren’t reading the paper right. They aren’t paying enough attention to the “nuances.” Shame on the readers!
- “There are few studies with reliable data on the costs and the benefits”
This may be unique in healthcare history: writing a paper and then almost immediately retracting its key finding, claiming you have no interest in the topic even as your fans deify you, and then blaming readers for believing the conclusion.
But the biggest head-scratcher is the 4th item. Here is someone who just published a conclusion with two-significant-digit precision now saying (accurately, as we’ll see) that, oh, by the way, the studies she analyzed are mostly garbage. You know the old saying: “In wellness, you don’t have to challenge the data to invalidate it. You merely have to read the data. It will invalidate itself.” She was kind enough to save us the trouble.
When RAND’s Soeren Mattke saw this stuff along with her refusal to defend it, he went ballistic. If you know RAND’s wellness uberguru Soeren Mattke, you know he is a very even-tempered, even-handed guy. it takes a lot to annoy him…and yet she did. Here is his smackdown of her work.
Studies Comprising the Meta-Analysis
If you’ve been following the Baicker saga for a while, none of this is news. Here’s what is news: thanks to a long plane ride with both an internet connection and an assortment of in-flight films I had already seen, I was able to dig into the actual studies. My conclusion: combining these third-rate studies in third-rate journals into a Health Affairs meta-analysis was like combining a hodegpodge of individual subprime mortgages written for first-time homebuyers with 5% downpayments and no credit history into an AAA-rated collateralized mortgage obligation.
In terms of timely relevance, of these 22 studies:
- Only 1 is less than 10 years old (that was Highmark, and they fired their vendor, ShapeUp)
- Only 2 are from the 21st century
- 5 predate Seinfeld
Keep in mind too that the average time between when a study begins and publication is about four years, meaning much of this data was collected before some TheySaidWhat? readers were born. Wellness-sensitive medical events (in both exposed and non-exposed populations of all ages, for reasons having nothing to do with wellness) have fallen by about 70% since most of this data was collected, meaning even if these studies were valid at the time (they weren’t), they have no relevance today. Likewise, dietary recommendations are now the opposite, people can no longer smoke in their offices etc. There is nothing else in healthcare where data on interventions from 20 and 30 years ago is considered relevant.
In terms of publication bias, consider the journals:
- 3 no longer exist
- 11 are wellness promotional journals, like the Journal of Occupational and Environmental Medicine and the American Journal of Health Promotion, both of which have thoroughly discredited themselves as legitimate journals and neither of which has ever published a case study showing negative returns, despite the editor of the latter saying that “90% to 95% of programs lose money” and “RCTs show a negative ROI“
Piling investigator bias on top of author bias, 14 of these studies were authored or co-authored by members of the Koop Award Committee, a committee that can’t spot obviously fabricated outcomes even after I point them out, people whose entire livelihoods depend on making up savings figures, and who endorse admitted liars.
The Actual Data
Every study that used a comparison group employed some variation of the demonstrably invalid participants-vs-non-participants methodology. Not one study was plausibility-tested. Almost every one of them showed savings far in excess of what could be saved if all wellness-sensitive medical events were wiped out.
Quite literally no one thinks savings can be achieved within 18 months, and yet the majority of studies with a comparison group were 18 months or less. All showed massive savings. One study lasted only 6 months, but showed roughly 20% savings nonetheless.
You might ask, how could all that money be saved in such a short period of time? For that answer, you’ll have to wait a couple of days. For Part 2, we will dig into a few of the studies themselves to see where the magic happens…and we do mean “magic”, since nothing else can explain those preternatural results.
Looks like a lot of you employees should be getting your employers to refund your penalties for not losing weight…or retroactively award you your incentives…read on. One way or the other, it’s time to end corporate fat-shaming.
Here is yet another in the unending stream of reasons to be certain all those Koop-award-winning savings claims and just about every other announced wellness savings figure are fabricated: they are all based partially or totally on Body Mass Index (BMI) reductions among active motivated participants — but BMI turns out to be a worthless indicator of health status. (We’ve already pointed out that the whole concept of measuring anything on “active motivated participants” is garbage anyway, as Aetna recently proved by accidentally telling the truth.)
How worthless? A study due out in the International Journal of Obesity says 75-million Americans are misclassified, meaning their BMI doesn’t match their true underlying health status. (As an aside, this scoop comes from this morning’s STAT News, the new must-read healthcare daily.) Many people with high BMIs are healthy, while many people with low BMIs are high-risk. Shocking! Who knew?
Naturally (cue the smirk on our faces), we did. As recently as in last month’s expose of The Vitality Group‘s squirrely outcomes claims, we pointed out that BMI is a 200-year-old construct based on faulty reasoning to begin with — and noted it has been challenged on multiple bases for years. We were also the first to observe that BMIs don’t correlate with a company’s financial success. And our series: “The Belly of the Beast” chronicled one vendor’s misunderstanding of BMIs as well, though I understand they are improving quite a bit now and we wish them the best and look forward to telling you about their improvements.
The implications of this new research are staggering:
- Companies need to refund penalties, and also award incentives retroactively, to people who were unfairly denied their money because wellness vendors don’t know how to measure outcomes;
- The “subject matter experts” who wrote the HERO Report need to retract basically their whole ball of wax, since it obsesses with BMI — and they need to apologize to me for inaccurately calling my claims “inaccurate” when they are specifically and relentlessly urging readers to do the wrong thing;
- Wellness vendors need to learn a thing or two about wellness, for a change.
- And guess who’s 3.27-to-1 ROI was based on studies obsessing with BMI? Kate Baicker, that’s who. Despite multiple hedges and walk-backs, she has yet to issue a formal retraction of her puff-piece on wellness economics. This gives her a good excuse.
Of course, in wellness, “the implications are staggering” means: “business as usual,” and they won’t do a thing to address these new findings.
The EEOC can’t ignore this. How can they give employers more leeway — as they now intend to do — to fine employees based on a variable they now know to be wrong?
Quizzify 1, Wellness Vendors 0
Those of you who use Quizzify don’t have to fret, by the way. The correct answers to the Quizzify question: “What is a BMI?” already include:
- “A crude measure comparing your height and weight,”
- “Can be very misleading if you in otherwise good condition,” and
- “Is good to know but not to obsess with.”
(For those of you keeping score at home, Quizzify’s incorrect answer is: “It stands for ‘Bowel Movement Intensity,’ and indicates how much effort you required in order to stay regular.”)
Even so, we will be adjusting the answers going forward to take into account this new research, starting next week. We expect the wellness ignorati will follow suit in a few years — once they stop advocating lowfat diets, PSA tests, and annual mammograms. The good news is that they generally no longer recommend bloodletting.