Home » Posts tagged 'Koop Committee' (Page 2)
Tag Archives: Koop Committee
Wellsteps claims to have dramatically reduced the total cost of the Boise School District’s health spending. Their Koop Committee colleagues gave them an award for it, as they typically do for their fellow board members and sponsors. (Yes, Wellsteps’ CEO, Steve Aldana is on the committee that grants the award, but, in accordance with Koop Committee tradition and Wellsteps ethical standards, there is no mention of this possible conflict of interest in the announcement from Wellsteps. This is not a violation of the Wellness Industry Code of Ethics, because there is none.)
Unfortunately for Wellsteps, three completely distinct observations from Wellsteps’ own data invalidate their analysis separately. In combination, these observations create a level of impossibility demonstrating that whoever invented the English language was unfamiliar with the wellness industry, or they would have come up with a word meaning: “Impossible doesn’t begin to describe it.”
First, as we saw in the last posting, employee health deteriorated over the course of the program, measured both subjectively and objectively.
There is a concept, covered in Health Services Research 101, called “causation.” Wellness vendors love taking credit for everything that happens during their program. Hence we can conclude that Wellsteps’ program caused employee health to decline.
Therefore, no reduction in costs due to a healthier employee population can be attributed to Wellsteps’ program.
Second, also covered in Health Services Research 101, is the concept of “fifth-grade arithmetic.” Potentially Preventable Hospitalizations (PPH), as described in the official Health Enhancement Research Organization (HERO) outcomes measurement guidelines (a report on which Wellsteps’ CEO claims to have collaborated), account for a very small percentage of all spending. Specifically, as described in the HERO report and reproduced below, these events account for 2.62 hospitalizations per 1000. Boise’s 3284 employees would therefore suffer about 9 PPH’s. If each PPH cost Boise the HERO-assumed cost/admission of $22,500, that’s about $202,000. Not enough to cover even the out-of-pocket fees for the Wellsteps program, assuming the Wellsteps did a perfect job. And as we’ve learned in the past, a beam of light leaving “perfect” wouldn’t reach Wellsteps for several seconds.
Still, we’ll never know because despite their endorsement of the HERO report, Wellsteps decided not to disclose the rates of Boise’s PPH’s, likely to obscure the fact that they didn’t reduce it.
And as the HERO guidebook says, any reduction in PPH’s is offset by more spending elsewhere. In Wellsteps’ case, “more spending elsewhere” is annual biometric screenings. Curiously, Wellsteps admitted annual screenings are a stupid idea four whole days before they announced their Koop Award for doing exactly the opposite.
Yet Wellsteps reported annual savings ultimately exceeding $5-million, or about a third of Boise’s total spending. This would be equivalent to wiping out every hospitalization unconnected with childbirth plus every ER visit. Adding yet another layer of impossibility to this narrative, at the end of this, we show the rank order of the top 25 reasons people visited the ER or went to the hospital, which basically have nothing to do with corporate wellness programs. Hence the cost of these visits and admissions couldn’t be dented, let along wiped out, by massive overscreening or even by appropriate screening.
Compounding this impossible outcome (and wellness is one of the few industries in which there are degrees of impossibility, since a typical wellness vendor makes at least a dozen impossible claims before breakfast) is the surprising health of the Boise population to begin with. People rated their health as 7.98 out of 10, and only 2.5% reporting smoking (vs. 20% for Idaho as a whole) and only 20% reported drinking (vs. 70% for the US as a whole). These ridiculously low levels did not strike Wellsteps as suspicious, so we will assume they are accurate.
Further, most Boise employees had fairly normal blood pressure, glucose and cholesterol — at least before they got sucked into the program.
So, with impossibly smoking and drinking, excellent reported health status, and largely normal biometrics, how is there room to improve health enough to save money, especially when health status is declining and the population is being overdiagnosed?
Speaking of misunderstanding the concept of arithmetic, third and most important is the data Wellsteps suppressed between their initial report and their Koop Award application. Normally Koop Award Committee Chairman Ron Goetzel suppresses the invalidating data after the award is announced, as with Health Fitness Corp/Eastman Chemical, and then the state of Nebraska (technically actually incriminating, not just invalidating). That’s because in the past I haven’t predicted who would win the award. Rather I just pointed out all the obviously incorrect data after the fact. By predicting Wellsteps would win a Koop Award and pointing out exactly why their data was sufficiently fictitious to merit it, I gave Wellsteps itself the opportunity to suppress their own data, so Ron wouldn’t have to do it for them.
Contrast below the trend they reported for total spending for Boise against the claims cost per person for Boise. The former goes up. The latter goes down even though the number of employees stays the same. Obviously, this is an impossible coexistence, as mentioned both in the first installment of this series and in every elementary school in the world. This time, we are going to transpose the bar graph, which separated participants from non-participants, onto the line graph, which included both cohorts, for the baseline and the first two years of the program. The assumption, as Wellsteps states, is that their participation rate is 80%, largely because of the massive $870/year incentive, which in classic wellness fashion is not included in the savings calculation. The number of employees seems to bounce around a bit between reports, but we’ll go with 3284 for this valuation.
By way of review from the first installment, here is total spending:
Also by way of review, here it total spending, participants vs. non-participants, going exactly the other way:
Now let’s overlay the second set of figures onto the first. In addition to trending in opposite directions during the wellness program years, the total spending on these dueling slides doesn’t even coincide in the baseline year. Since the whole point of the exercise is to look at the trend subsequent to the baseline, we will add about $3-million to each year for the bar graph figures, so that the 2011 starting points coincide. That let’s us focus on the difference-of-differences in the program years. Starting at the baseline, costs increased about $1.7-million by 2013, putting the 2013 red endpoint almost exactly in line with their prediction.
In all fairness, let’s continue the analysis, by looking at the Wellsteps performance in 2014:
As you can see, costs did fall below the “prediction” in 2014, though still way above “Wellsteps Begins” and “Actual.” The only problem? More than 100% of the entire decline from the previous year was due to non-participants’ costs plummeting, while participant costs increased.
So where do we stand? Wellsteps, by their own admission, overscreened and overdiagnosed this population. And their own admission, the population was quite healthy to begin with. By their own admission, the health of the Boise employees deteriorated. And, by their own admission, the only successful annual performance is attributable to a large improvement in non-participants.
To paraphrase the immortal words of the great philosopher Samuel Goldwyn, the title of this post, Wellsteps has raised the Koop Award standard for outcomes invalidity to a new low.
Top hospital discharge codes:
Top ER visit codes:
Yikes! Wellsteps just announced–and the Koop Award Committee just agreed–that 20% of Boise’s teachers have a drinking problem. Boise parents, before you start worrying, the good news is that it may not specifically be the teachers with the drinking problems. It’s technically 20% of all school employees, so you can relax. Maybe the problem is more prevalent in non-teaching employees, such as, oh, I don’t know, school bus drivers.
Is Wellsteps right or wrong about the drinking problem in Boise? Read on and then you can decide for yourself. Having read their alcohol analysis ourselves, we’re doubling down on our original conclusion that Wellsteps deserves a Koop Award like Vladimir Putin deserves a Nobel Peace Prize. Wellsteps is way off base, meaning Boise’s schoolchildren are safe.
Our original conclusion was reached simply by looking at the “smoking gun” slide in our previous posting, which they had tried to suppress (but we took a screenshot). We’ll expand on that slide another day. For now, just ignore that particular smoking gun and instead consider the rest of the smoking arsenal they just wrote up in their self-congratulatory press release.
Speaking of smoking, today in addition to reviewing the Wellsteps analysis of Boise’s drinking, we’ll also include their smoking write-up.
Alcohol Consumption in the Boise School District
There was a saying in the old Soviet Union: “We pretend to work. They pretend to pay us.” For employees completing the Wellsteps’ Health Risk Assessment (HRA), the corollary would be: “We pretend to tell the truth. They pretend to believe us.” Of 3284 employees, 671 admitted to drinking. That means 80% of the Boise School District employees are teetotalers, as compared to 30% of the country as a whole. Or else — a shocker — 50% of the HRA respondents are lying.
The Koop Committee did not question this result.
But wait. There’s more.
Now let’s consider the 20% of respondents who admitted to drinking at all. How much do they claim to drink? A paltry 1.31 drinks a day, or about 10 ounces of alcohol a week. By contrast, the top 20% of drinkers in the US not employed by the Boise School District (“Boise”) consume roughly 43 ounces of alcohol a week. Let’s overlay the Boise School District consumption (in blue) against US consumption, on the following chart. It shows 10 ounces for Boise vs. 15 ounces for the second-to-top decile for the US as a whole, and vs. a whopping 73 ounces for the top decile.
The Koop Committee did not question this result.
But wait. There’s even more.
How much do you have to drink in order for Wellsteps to accuse you of having a drinking problem? Answer: any amount. According to Wellsteps, drinking alcohol at all counts as a “worst health behavior.” Shame on us for not realizing the evils of a glass of wine!
This screenshot from their writeup captures both the average respondent’s self-reported consumption of alcohol, at the very top, with the “worst health behaviors” in alcohol consumption at the very bottom. Note that the two rows of figures are identical. In other words, everyone who drinks has a “high level of…alcohol use” no matter how much they drink. The average consumption (meaning at least half are even lower) to count as a “worst behavior” and “high level of alcohol use” is the aforementioned 10 ounces a week.
The Koop Committee did not question this result.
Squirrelly HRA findings aren’t confined to drinkers. Of the 3284 school district employees, apparently only 77 smoke (see chart above), placing the Boise School District’s alleged smoking rate at roughly 2.5%, or about 80% lower than the national or Idaho average. And those 2.5% are very light smokers. Whereas the average Idaho smoker burns through 16 cigarettes a day, Boise School District smokers abstain from smoking 10 days a month. So just like alcohol, hardly anyone smokes…and even the smoking employees hardly smoke.
With this amazingly healthy population, it’s a wonder Wellsteps could make any improvements at all. But as we will see later this week, they made enough improvements to save massive sums of money, defying all odds and all rules of arithmetic, which are strictly enforced.
The Koop Committee did not question this result.
In the workplace wellness epidemic, Harvard School of Public Health Professor Katherine Baicker is Patient Zero. However, she may soon come up with the cure for the disease she spread. This would be like when the inventor of the PSA test said it shouldn’t be used.
By way of background, her original dalliance with wellness in Health Affairs–timed and designed to get the Business Roundtable’s “Safeway Amendment” allowing a 30% to 50% clawback of insurance premiums into the Affordable Care Act–introduced the infamous “3.27-to-1 ROI” into wellness vendor vernacular. That was almost seven years ago, but wellness vendors continue to cite this figure as gospel even though RAND demolished it and Professor Baicker herself walked it back four times.
For good reason, as it turns out — this Pulse posting says it all. The 10-to-30-year-old studies comprising her meta-analysis are even more amusing in retrospect. Lots of overcreening, overreliance on now-discredited BMIs, and of course recommendations for low-fat diets–even for employees with metabolic syndrome. Most published in third-tier journals, by the Wellness Ignorati and their friends.
And almost every one of them compared active motivated participants to non-participants. 6 years ago we suspected that design was invalid. Today we know for certain, thanks to Health Fitness Corporation and Aetna. Two more elegant studies to prove that point could not be imagined. No “investigator bias” or “publication bias” there, since both were hoping for the opposite finding.
However, mindful of her reputation, Professor Baicker recently announced (in as many words) that she is going to redeem herself.
Her protestation that she is no longer interested in wellness appears to have taken a backseat to her desire to remove this one blot–granted, a blot of Lady MacBeth proportions–on an otherwise excellent curriculum vitae.
She is going to do a study for BJ’s Wholesale Club, in conjunction with an outfit called Wellness Workdays. Wellness Workdays is a classic wellness vendor. That is to say, they won’t be winning a Nobel Prize anytime soon, or even a spelling bee. Let’s start by examining their analytic and clinical prowess.
To start with, their “White Paper” doesn’t just quote the infamous 3.27-to-1. They’ve upped the ante to 6.00 to 1, maintaining the two significant digits while almost doubling the savings. How? They’ve added the 3.27-to-1 for healthcare savings to the 2.73-to-1 for absenteeism reduction from that same 2010 study. Those two separate conclusions were reached from almost totally different studies. Anyone can tell that from reading the original. Anyone, that is, except Wellness Workdays.
Their analytic qualifications are matched only by their clinical qualifications. One member of their medical advisory board is Chief of Allergy and Clinical Immunology at the Indian River Medical Center in Vero Beach, Florida. While this expertise is not exactly central to the mission of the pry,poke, and prod industry, in all fairness it should be noted that the Indian River Medical Center runs one of the best allergy programs in all of Vero Beach.
Another is an OB-GYN in Colorado. Perhaps this advisor will develop a protocol for employees who want to be screened and induced at the same time. A third consults to orthopedists at “Lennox Hill Hospital,” a role that probably doesn’t require too much heavy lifting, because there is no hospital by that name.
This guy is also an expert on steroids and other performance-enhancing products, and has “published rseveral esearch studies.”
So they can’t spell, can’t proofread, can’t understand study design, and can’t cobble together a qualified advisory board. In other words, to paraphrase the immortal words of those great philosophers Gilbert & Sullivan, they are the very model of a modern clueless wellness vendor.
Katherine Baicker’s proposed study design
We are confident she is going to get it right this time and “discover” that wellness loses money. By selecting a vendor of the caliber of Wellness Workdays, she isn’t leaving anything to chance.
Having learned her lesson — and I’ve been pretty conscientious about forwarding helpful study design materials to guide her — she will certainly tally wellness-sensitive medical events across the entire population. More importantly, here’s what she is not going to do with this study:
- “Match” volunteer participants to — you guessed it — non-participants. This nonsense that keeps the wellness industry afloat, and Professor Baicker no doubt sees right through it;
- List the “unobservable differences” between participants and non-participants as a “limitation.” It’s not a limitation. It’s a baldfaced lie. It is now known that this face-invalid study design is truly invalid. She wouldn’t lie, right? That would damage the reputation of the entire Harvard School of Public Health in what will certainly be a high-profile study;
- Show a high ROI even though the change in health risk factors is trivial. This of course is the stock-in-trade of the Koop Award committee, but a real academic researcher would know better;
- Compare the costs to “trend” and say “costs were projected to rise by [this amount] but they only rose by [this lower amount], and therefore a huge pile of money was saved”;
- Attribute all cost differences to the program, whereas a real researcher would look only at utilization differences that could actually be attributed to urging employees to eat more broccoli. (By contrast, one of the studies in her previous meta-analysis credited the wellness program with a reduction in cat scratch fever.)
How do we know she isn’t going to make up phony outcomes again? She “tipped her hand” with the Oregon Medicaid lottery study. A terrific natural experiment, her study emphasized the value of a “lottery control,” meaning every subject had the same intent-to-treat. Exactly the opposite of the wellness participants-vs-non-participants study design.
Further, who wants to be known as the Typhoid Mary of workplace wellness? The Oregon study was an excellent one, so naturally it showed exactly the opposite of what wellness studies show. Specifically, facilitating access to care doesn’t reduce the cost of care or improve physical health status. And that was for people — newly minted Medicaid recipients — who didn’t have any insurance to begin with. Wellness, of course, takes employees who already have plenty of access to care and drown them in even more, unwanted and largely unneeded, screenings and checkups.
As for BJ’s Wholesale Club, I suspect they got suckered into this. Who volunteers to become the next Pepsico, a case study of how wellness programs fail? BJ’s obviously isn’t studying their own competitors: Target has one of the best programs in the country — precisely because they are far too smart to use a vendor like Wellness Workdays.
In any case, we look forward to her research study. Or perhaps, since this is in conjunction with Wellness Workdays, to rseveral esearch studies.
No new postings for a few days, for personal family reasons. Yes, I know it’s not always about me but my daughter’s getting married Saturday. Your reward for scrolling through this pukey proud-father’s-pictures-of-daughter stuff is that you can then be regaled with one of the wellness industry’s greatest hits.
And now, as promised, a gem from my Special Reserve Collection: A blogger talking about how bad wellness is, how his colleagues roll their eyes when he mentions it, how it’s “low-value care,” etc.
You might say, “So what else is new?”
New? New? You want new? Here’s what’s new: This blogger’s organization (Altarum Institute) is represented on Mr. Goetzel’s Koop Award Committee. Or they were, but no longer. The problem that apparently Mr. Goetzel hadn’t considered is, when you invite an organization into your little cabal known for its integrity, there is always the chance that they will display integrity.
So needless to say, the two organizations had a parting of the ways, because while integrity Altarum’s hallmark, it’s one of the Koop Award Committee’s five worst nightmares. In case you’re keeping score at home, the other four are:
The wellness industry is about nothing if not irony. Ironically, wellness vendors and consultants don’t understand irony, so they keep doing and saying things they think are being taken seriously. Ironically, they are being taken seriously, but only by students of irony.
For example, these wellness people don’t understand that it is ironic that employees can be forced to submit to “voluntary” wellness programs, or face fines of thousands of dollars. They say this unabashedly. Whereas when we make an ironic comment, such as: “Wellness vendors make employees happy whether they like it or not,” we do it deliberately.
The May issue of Managed Care displays a cornucopia of unintended irony, in a debate between myself and Harris Allen, of Navistar fame, on the effectiveness of wellness programs in preventing diabetes.
Speaking of Navistar, Mr. Allen was already famous for irony before this debate. He showed Navistar how to claim a wellness ROI of 400-to-1, later reduced to 40-to-1, before jumping again to 400-to-1. That by itself — adding/removing extra zeros in your ROI but claiming it’s real the whole time — is ironic, but that’s not even the ironic part. The irony is that he was concocting these figures even as Navistar itself was making up $4-billion of phony shareholder equity, perhaps including these wellness savings. A lot of the perps (excluding Harris) are ending up in jail over this caper. Ironically, despite his pride in his work on wellness for Navistar, he didn’t cite their results in his counterpoint.
Not being Navistar shareholders ourselves, we found this whole escapade highly amusing, so it is recounted in This Is Your Brain on Wellness, our humor column.
Back to the debate irony. The irony is that, in his attempt to justify wellness, he cited two examples that lead to the opposite conclusion. First, he cited US Preventive Medicine (USPM). USPM did indeed achieve an excellent result, and it is validated by and displayed by the Validation Institute. On that everyone can agree. I myself just wrote a column praising their performance. The thesis of the column: “See, not every wellness vendor fails.”
He cites that exact same company and exact same validation to conclude: “See, wellness vendors can succeed.” Yeah, one wellness company has succeeded while the staggering number of failures — companies that couldn’t get validation or didn’t even bother to apply — is in the thousands, a statistic I noted just yesterday.
Using the same logic as Mr. Allen, one might profile Powerball winners and say: “See? Powerball works.”
The other irony is that he cited the Koop Award-winning companies as examples of successes in preventing diabetes, when — according to their own applications — they basically failed. Ironically, I also cited that very same award in my argument. Specifically, McKesson won an award for preventing diabetes even though its employees’ glucose and BMIs increased. Mr. Goetzel’s and his Koop Award committee cronies never been much for fact-checking, even when the facts are right on the application itself:
The final irony is that Mr. Harris ends his argument with a call for “evidence-based” wellness programs. Ironically, the “evidence” is overwhelming…in the other direction: wellness programs have not avoided a single wellness-sensitive medical admission, according to US government figures. The green line below represents the wellness-exposed population while the red line represents the rest of the country. There is no separation, meaning that the wellness-exposed population has achieved zilch.
Actually, there is slight separation –but ironically it goes the other way. You’d statistically be better off not being exposed to wellness.
This graph is part of my proof of the ineffectiveness of wellness vendors, and allows me to offer a million-dollar reward to anyone who can show wellness doesn’t lose money.
Where did the government get the data for this graph? It was compiled by Truven Health, the division of IBM that — you guessed it, ironically — employs Mr. Goetzel.
No program epitomized conventional “pry, poke and prod” wellness more than Nebraska’s state employee wellness program. And by that of course I mean no wellness vendor has ever lied about outcomes more blatantly or won more awards than Nebraska’s state employee wellness program vendor, Health Fitness Corporation. (Blatantly lying about outcomes and winning Koop Awards, in the immortal words of the great philosopher Frank Sinatra, go together like a horse and carriage.) Their big mistake was admitting it. (See the timeline link.)
Not to mention the cover-up of the lies, that Ron Goetzel and his Koop Committee friends botched so badly that the state’s HR team and procurement department could no longer do the Sergeant Schultz thing. I guess now, finally, Mr. Goetzel will stop referring to this program as a “best practice.”
Now, the program is officially dead. It was close. On October 1, we thought we had lost:
But then last week, following a number of behind-the-scenes conversations and finally a bit of googling by the state:
In other words:
“You know that laundry detergent that gets bloodstains out of t-shirts? I’d say if you’re routinely coming home with bloodstains in your t-shirts, laundry probably isn’t your biggest problem.”
Recognize that line? That was Jerry Seinfeld’s signature joke, the one that more than any other propelled his early fame. Having used it for a decade, he retired it shortly after Seinfeld went on the air.
Now it’s time to retire the wellness industry’s signature joke: the so-called Harvard Study and its 3.27-to-1 ROI. Among other things, as we’ll see, a large chunk of the data in that study also predates Seinfeld, a show which recently celebrated its 25th anniversary. You know the old line: “How can you tell a lawyer is lying?” Answer: “His lips move.” Well, the way you can tell a wellness vendor is lying is that they still cite that study, knowing it to be completely invalid. (If they actually think that 3.27-to-1 is valid, in a way that’s even worse.)
The article, which today wouldn’t pass peer review by the normally cautious Health Affairs, was rushed into publication during the healthcare reform debate as a favor to David Cutler, a healthcare advisor to President Obama and an author on the study. Along with the results from the Safeway wellness program — which turned out not to exist (and I’m referring not just to the results but literally to the program itself) — this meta-analysis provided cover for the 30%-to-50% clawback provision (for non-smokers and smokers respectively) in what became ironically known as the Safeway Amendment to the Affordable Care Act.
In turn, this clawback was needed to secure the support of the Business Roundtable (BRT) for the Affordable Care Act. The BRT was and is obsessed with this provision. Not because they are especially concerned about employee health — the guy who ran their wellness lobbying also ran a casino company, exposing thousands of employees to second-hand smoke. Rather, because this clawback provision creates billions in forfeitures that go back into the coffers of its member organizations. Bravo Wellness accidentally spilled that secret. The worse the program, the more an organization can save via forfeitures. For instance, Wellsteps saved millions for the Boise School District even though healthcare spending/person jumped–simply by creating a program so unappealing that employees preferred to obtain insurance through their spouses. (Either that or they outright lied–you make the call.)
How Prof. Baicker Invalidated Her Own Study
The title was definitive and the conclusion was amazingly precise. The title was: “Workplace Wellness Can Generate Savings.” Not “might” or “possibly could on a good day” but “can.” And the conclusion itself left little doubt. The ROI wasn’t “approximately 3-to-1” or even “approximately 3.3-to-1” but rather a statistically precise 3.27-to-1. You don’t make such a definitive pronouncement without being ready to defend it.
Or maybe you do. Here’s how Professor Baicker’s story changed on various occasions once it became clear that her result was a major outlier, and that wellness loses money:
- It’s too early to tell whether these programs pay off, and employers should experiment on their employees;
- She’s not interested in wellness any more;
- People aren’t reading the paper right. They aren’t paying enough attention to the “nuances.” Shame on the readers!
- “There are few studies with reliable data on the costs and the benefits”
This may be unique in healthcare history: writing a paper and then almost immediately retracting its key finding, claiming you have no interest in the topic even as your fans deify you, and then blaming readers for believing the conclusion.
But the biggest head-scratcher is the 4th item. Here is someone who just published a conclusion with two-significant-digit precision now saying (accurately, as we’ll see) that, oh, by the way, the studies she analyzed are mostly garbage. You know the old saying: “In wellness, you don’t have to challenge the data to invalidate it. You merely have to read the data. It will invalidate itself.” She was kind enough to save us the trouble.
When RAND’s Soeren Mattke saw this stuff along with her refusal to defend it, he went ballistic. If you know RAND’s wellness uberguru Soeren Mattke, you know he is a very even-tempered, even-handed guy. it takes a lot to annoy him…and yet she did. Here is his smackdown of her work.
Studies Comprising the Meta-Analysis
If you’ve been following the Baicker saga for a while, none of this is news. Here’s what is news: thanks to a long plane ride with both an internet connection and an assortment of in-flight films I had already seen, I was able to dig into the actual studies. My conclusion: combining these third-rate studies in third-rate journals into a Health Affairs meta-analysis was like combining a hodegpodge of individual subprime mortgages written for first-time homebuyers with 5% downpayments and no credit history into an AAA-rated collateralized mortgage obligation.
In terms of timely relevance, of these 22 studies:
- Only 1 is less than 10 years old (that was Highmark, and they fired their vendor, ShapeUp)
- Only 2 are from the 21st century
- 5 predate Seinfeld
Keep in mind too that the average time between when a study begins and publication is about four years, meaning much of this data was collected before some TheySaidWhat? readers were born. Wellness-sensitive medical events (in both exposed and non-exposed populations of all ages, for reasons having nothing to do with wellness) have fallen by about 70% since most of this data was collected, meaning even if these studies were valid at the time (they weren’t), they have no relevance today. Likewise, dietary recommendations are now the opposite, people can no longer smoke in their offices etc. There is nothing else in healthcare where data on interventions from 20 and 30 years ago is considered relevant.
In terms of publication bias, consider the journals:
- 3 no longer exist
- 11 are wellness promotional journals, like the Journal of Occupational and Environmental Medicine and the American Journal of Health Promotion, both of which have thoroughly discredited themselves as legitimate journals and neither of which has ever published a case study showing negative returns, despite the editor of the latter saying that “90% to 95% of programs lose money” and “RCTs show a negative ROI“
Piling investigator bias on top of author bias, 14 of these studies were authored or co-authored by members of the Koop Award Committee, a committee that can’t spot obviously fabricated outcomes even after I point them out, people whose entire livelihoods depend on making up savings figures, and who endorse admitted liars.
The Actual Data
Every study that used a comparison group employed some variation of the demonstrably invalid participants-vs-non-participants methodology. Not one study was plausibility-tested. Almost every one of them showed savings far in excess of what could be saved if all wellness-sensitive medical events were wiped out.
Quite literally no one thinks savings can be achieved within 18 months, and yet the majority of studies with a comparison group were 18 months or less. All showed massive savings. One study lasted only 6 months, but showed roughly 20% savings nonetheless.
You might ask, how could all that money be saved in such a short period of time? For that answer, you’ll have to wait a couple of days. For Part 2, we will dig into a few of the studies themselves to see where the magic happens…and we do mean “magic”, since nothing else can explain those preternatural results.
The pundits said it would never happen. The oddsmakers had it at 100-to-1 against. Nate Silver predicted it could never come to pass. And yet, LEWIS AND GOETZEL WILL BE DEBATING WELLNESS ECONOMICS ON NOVEMBER 2 in WASHINGTON DC
It’s at the Population Health Alliance Annual Forum. It’s not officially announced yet but save the date. More details to follow.
PS Feel free to add your own choice to the poll. We’ll award a free pdf of Surviving Workplace Wellness to the most creative proposed answer
When you are in the “countererrorism” business like we are, it’s important to have a zero tolerance for errors. Occasionally one slips through. In that case the important thing to do is to admit it, rather than fire the Attorney General and the Special Prosecutor and have your secretary erase the tape.
Vik and I wrote a posting for The Health Care Blog, the upshot of which was that the Affordable Care Act should no longer require insurers to cover adult checkups. Free checkups are ubiquitous in self-administered plans. On balance, our posting shows what grownups in health services research already know: they are worthless. Not completely satisfied with their innate worthlessness with a full subsidy, many employers — guided by benefits consultants — attach additional money to them: you are either fined for not getting one or else receive a bonus for getting one. Our proposed solution was/is quite simple: employers that attach bonuses or fines to physicial exams need to disclose that checkups are a waste of time and money. That simple disclosure requirement would end forced checkups.
It turns out, however, that adult checkups are not required by ACA. It was a complete benefits consulting urban legend and we fell for it. So we were wrong.
Here are the lessons from this.
First, if someone proves us wrong, we admit it. See? Admitting error is a concept that is lost on the wellness and benefits consulting industries. For example, after we pointed out that Mercer’s client British Petroleum got completely snookered by Mercer Staywell, the response of Mercer to BP wasn’t: “We apologize to for letting your vendor snooker you.” It was: “Let’s nominate this program for a Koop Award,” which naturally they won because both Mercer and Staywell are represented on the board of the group that gives out the award.
Second, the particular someone who proved us wrong, Chris Glason, did not “bully” us. He merely asked a tough question that invalidated a (minor) premise of our argument. However, when we do something quite similar, the people who are wrong (or lying) say we are “bullying” them. But all we did was ask 11 questions to clarify what someone already said — and offer him $1000 to answer the questions. Trust us: if Chris Glason had offered us $1000 to look it up and get back to him, we would have. (Instead we were rather dismissive, to put it mildly, for which we also publicly apologize.)
Third, our mistake was to assume that benefits consultants actually know something about, well, benefits consulting. We know they know nothing about wellness—Mercer and Milliman have both basically self-immolated by participating in the aforementioned Koop Award Committee and getting snookered four times by dishonest vendors. On two occasions the Committee was forced to backtrack as a result of our exposes, though they never admitted they got snookered. We kind of assumed that since benefits consultants don’t know anything about wellness, the only way they could stay in business was to actually know the first thing about benefits….and we listened to them.
Sidebar: a few benefits consultants are highly competent. We recommend the ones whom the Validation Institute (which is not connected with us but which we have a lot of respect for) has certified. (Don’t strain your eyes–no one from Mercer or Milliman appears on their listing.)
Fourth and most importantly, the answer doesn’t change: End “pry, poke, prod and punish” programs — especially the “prodding” part, now that even benefits consultants can see that prodding someone to go to the doctor when they aren’t sick is a complete waste of time and money.
“I made a mistake. I listened to the experts.”
— John F. Kennedy
By Al and Vik
Oh, the twists and turns as Ron “The Pretzel” Goetzel tries to wriggle out of all his ethical stumbles.
This time around, we thought we had nailed both him and his cabal handing out the ironically named C. Everett Koop Award to themselves and their friends based on made-up outcomes. Specifically, this time they gave their sponsor (Health Fitness Corporation, or HFC) an award based on data that was obviously made up, that no non-sponsor could have gotten away with submitting. This was the third such instance we’ve uncovered of a pattern of giving awards to sponsors for submitting invalid data while making sure that the award announcement contains no reference to the sponsorship. (There are probably others; we’ve only examined 3, which might explain why we’ve only found 3.)
How obviously was the data made up? Well, take a looksee at this slide, comparing participants to non-participants. This is the classic wellness ignorati ruse: pretending that non-motivated inactive non-participants can be used as a valid control for comparison to active, motivated participants. The wellness ignorati would have us believe that any healthcare spending “separation” between the two groups can be attributed to wellness programs, not to inherent differences in motivation between the two groups. Unfortunately for the ignorati, their own slide invalidates their own argument: in 2005, the label “Baseline Year” shows there was no program to participate in, and yet – as their own slide shows – participants (in blue) significantly underspent non-participants (in red) nonetheless. In Surviving Workplace Wellness, we call this “Wellness Meets Superman,” because the only way this could happen is for the earth to spin backwards.
Given that the 2005 baseline label was in plain view, we just assumed that HFC did not indeed have a program in place for this customer (Eastman Chemical) in 2005, which is why they called 2005 a “Baseline Year” instead of a “Treatment Year.” Not actually having a program would logically explain why they said that didn’t have a program, and why they used that display or variations of it like the one below for 4 years with the exact same label. Presumably if they had had a program in 2005, someone at HFC would have noticed during those 4 years and relabeled it accordingly.
Originally we thought the Koop Award Committee let this invalidating mistake slide because HFC — and for that matter, Eastman Chemical — sponsor the awards they somehow usually win. But while trying to throw a bone to HFC, the Koop Award luminaries overlooked the profound implication that the year 2005 separation of would-be participants and non-participants self-invalidated essentially the entire wellness industry, meaning that is is an admission of guilt that the industry-standard methodology is made up.
Goetzel the Pretzel to the rescue. He painstakingly explains away this prima facie invalidation. Apparently the year 2005 was “unfortunately mislabeled.” Note the pretzelesque use of the passive voice, like “the ballgame was rained out,” seemingly attributing this mislabeling to an act of either God or Kim-Jung-Un. He is claiming that instead of noticing this invalidator and letting this analysis slide by with a wink-and-a-nod to their sponsor, none of the alleged analytical luminaries on the Koop Committee noticed that the most important slide in the winning application was mislabeled — even though this slide is in plain view. We didn’t need Edward Snowden to hack into their system to blow up their scam. They once again proved our mantra that “in wellness you don’t need to challenge the data to invalidate it. You merely need to read the data. It will invalidate itself.”
We call this the “Dumb and Dumber” defense. Given two choices, Goetzel the Pretzel would much prefer claiming sheer stupidity on the part of himself, his fellow Koop Award committee members like Staywell’s David Anderson and Wellsteps’ Steve Aldana, and his sponsor HFC, rather than admit the industry’s methodology is a scam and that they’ve been lying to us all these years to protect their incomes.
Still, the Dumb-and-Dumber defense is a tough sell. You don’t need Sherlock Holmes, Hercule Poirot or even Inspector Clouseau to detect a few holes in the Pretzel’s twisted logic:
- How could no one – no member of the Koop Award Committee or employee of Health Fitness Corporation (which used this as its “money slide” for years) – have noticed this until we pointed it out for the third time (the first two times not being as visible to the public)?
- In early 2012, this slide was reproduced–with the permission of Health Fitness Corporation–right on p. 85 of Why Nobody Believes the Numbers, with the entire explanation of its hilarious impossibility. We know Mr. Goetzel read this book, because he copied material out of it before the publisher, John Wiley & Sons, made him stop. So we are curious as to why it has taken until now for him to notice this “unfortunate mislabeling.” Hmm…would the fact that it was just exposed to the world in Health Affairs have anything to do with this sudden epiphany? We’re just sayin’…
- If indeed it was just an “unfortunate mislabeling,” how come HFC has now expunged all references to this previously highlighted slide from their website, rather than simply change the label?
As regards the third point, we would recommend that next time Mr. Goetzel invokes the Dumb-and-Dumber defense, he coordinate his spin with his sponsor.
But let’s not overlook the biggest point: the entire Koop Committee – including “numbers guys” like Milliman’s Bruce Pyenson and Mercer’s Dan Gold — is apparently incapable of reading a simple outcomes slide, as they’ve proven over and over.
So, as a goodwill gesture, we will offer a 50% discount to all Koop Committee members for the Critical Outcomes Report Analysis course and certification. This course will help these committee members learn how to avoid the embarrassing mistakes they consistently otherwise make and (assuming they institute conflict-of-interest rules as well to require disclosure of sponsorships in award announcements) perhaps increase the odds that worthy candidates win their awards for a change.