Home » Articles posted by whynobodybelievesthenumbers (Page 46)
Author Archives: whynobodybelievesthenumbers
Should the Wellness Vendor Oath Be: “First, Do Harm” ?
The Quizzify oath: First, do help.
When Thomas Edison said: “We don’t know a millionth of 1% about anything,” he wasn’t talking about the wellness industry, because wellness vendors aren’t that knowledgeable. And much of what they “know” is harmful.
Smoking and exercise aside, taking wellness vendors’ advice 10 years ago — during the time wellness was somehow allegedly racking up its famously fictitious 3.27-to-1 ROI by making employees healthier– would have been a very bad idea. PSA tests, annual mammograms for younger women, colonoscopies at 5-year intervals, and EKGs were perfect examples of must-to-avoid screens, even if it meant leaving incentives on the table.
And yet even though most wellness vendors (Star Wellness, Bravo Wellness Total Wellness, HealthFair Services and Aetna being notable exceptions) won’t harm employees as much as they did 10 years ago, a lot of mythology still causes a lot of harm today, albeit more subtly.
Myth: “We need to ‘do wellness’ because 75% of our healthcare cost is due to preventable chronic disease.” (Ron Goetzel, in our recent debate, boosted this figure to 80% for reasons unknown.)
Fact: Have ya looked at your high utilizers and other expenses? We-can-prevent-75%-of-cost-due-to-chronic-disease is the biggest urban legend in healthcare. We’ve done multiple articles on it — there are too many fallacies to squeeze in here. Though it’s just arithmetic, this is the most harmful fallacy of all, because by causing employers to obsess with overprevention, it spins off all the other fallacies below.
Myth: “Reducing our employees’ BMIs will save money.”
Fact: The actual science is far more nuanced. Some people have high BMIs because they are healthy. And belly fat — even at “normal” weights — is riskier than all but the highest BMIs. Further, attaching money to weight loss between weigh-ins creates a binge/crash-diet cycle that is decidedly unhealthy.
Myth: “Corporate weight loss programs save money.”
Fact: No corporate weight loss program has ever saved money. They don’t reduce BMIs, BMIs are the wrong measure (see above), and the link between reducing BMIs and saving money is nonexistent.
Myth: “Screening our employees will be good for their health.”
Fact: Annual screenings are a bad idea for the majority of employees. The head of Optum’s wellness operations, Seth Serxner, just acknowledged this inconvenient truth last week. (He somehow shifted the blame to employers, for stupidly spending too much money on Optum and other vendors. That’s a topic for another post.) The US Preventive Services Task Force has a schedule of screenings that essentially no wellness vendor follows. Because so few biometric screens are recommended for working-age adults by the card-carrying grownups who comprise the USPSTF, following USPSTF guidelines would bankrupt the industry.
Myth: “Screening guidelines balance costs and benefits so at worst we’ll break even.”
Fact: Screening guidelines balance harms and benefits, not costs and benefits. The subtlety of the distinction would be lost on most wellness vendors, but it is important. (1) Unless screens are provided free, an employer will lose money even on a screening program done according to guidelines; (2) you are not doing your employees any favors by providing screening “greater than” guidelines, like the Health Fitness Corporation/Nebraska program did. You are simply raising the likelihood of harm.
Myth: “Annual checkups will keep our employees healthy.”
Fact: For wellness vendors, the annual checkup has almost mystical power. Bravo’s CEO Jim Pshock loudly credits checkups with preventing cancer. Wellness vendor bloviating aside, the science is quite settled: employees are more likely to be harmed than benefited by annual checkups.
Myth: “Our employees need to eat healthier.”
Fact: OK, there is a, uh, grain of truth here. Many people have bad diets–fried food, sugar etc. But beyond eating less fried food and sugar, the science remains unsettled. Salt, saturated fat, complex carbohydrates…all in the realm of not completely settled. What is true and remarkably overlooked is the epidemiological rule of thumb that if an impact is major, it shows up in small samples. 86 cases were needed to link lung cancer to smoking. And a famous study of 523 veterans proved very high blood pressure causes strokes. Yet after tons of controlled and observational studies — even comparing countries to one another — we still haven’t found “the answer.” That means “the answer,” whatever it is, won’t matter much in the workplace. So you’re wasting your time trying to get employees to “eat right.”
We could keep going — antioxidants are more likely to cause cancer than prevent it. Sitting is not the new smoking. And drinking eight glasses of water a day is good for you only in that you’ll get more exercise going to and from the restroom.
The biggest myth of all? Wellness vendors actually do anything of value, other than make up savings figures to show your CFO so you look good. Or as my colleague Vik Khanna says: “Love your employees. Fire your wellness vendor.”
Share this:
Hey, How Come Wellness Needs an ROI But Real Healthcare Doesn’t?
Now that the myth that there is any ROI in wellness is thoroughly both debunked and also even acknowledged by the wellness industry, vendors often fall back on the “argument” that nothing else in healthcare needs an ROI. Why should workplace wellness be singled out? The editor of the American Journal of Health Promotion, Michael O’Donnell, even asked: “Who cares about an ROI anyway?”
The answer to Michael’s question? Everyone should care. And everyone should insist on an ROI from wellness, for three distinct reasons. Each reason is sufficient on its own. So even if there were a fallacy in two of the reasons (and there isn’t), the remaining reason would still be definitive.
First, consider an employee with appendicitis. You don’t calculate an ROI. You call an ambulance. But suppose a vendor says to you: “If an employee’s appendix bursts, the cost could be $100,000. So we propose taking out everyone’s appendix preventively.”
You’d ask: “What’s the rate of burst appendixes and how much do appendectomies cost?” While that’s an extreme example (and we didn’t mean to give these people any ideas), this is basically the calculation you should make when vendors propose screens. Here’s how to do the calculation. You’ll be shocked at how much it costs to avoid even one event by screening everyone.
Second, wellness is the only thing in healthcare that employees are forced to do, subject to a financial forfeiture of penalties or lost incentives. Other activities which people are penalized for not doing include: wearing helmets/life jackets/seat belts and getting kids vaccinated. In each case, the clinical evidence/science overwhelms considerations of personal choice. (Even then, in some states personal choice still rules.)
By contrast, the only thing that’s overwhelming about wellness evidence/science is how overwhelmingly the evidence eviscerates wellness, which of course is what this site is all about. Unfortunately, wellness vendors don’t understand evidence — or for that matter healthcare itself. Many vendors have no knowledge of basics like clinical guidelines, or even arithmetic. One wonders how they can do their jobs as wellness vendors without understanding healthcare. And that brings us to the third reason that wellness (unlike healthcare) needs an ROI, which is…
…Wellness isn’t healthcare. Quite literally every other provider of physical healthcare–from heart surgeons to acupuncturists–needs to train, pass a test, get a license, take continuing education, and be subject to review by an oversight board. Not wellness vendors. You can become a wellness vendor for $67,000. “Up to” eight days of classroom and on-the-job-training are also included in that $67,000. (To put that in perspective, Four Seasons housekeepers get ten days of training.) The vendor that offers these franchises, Star Wellness, brags about how no healthcare background is needed to be a wellness vendor. A background in “sales or municipal administration” is perfectly sufficient.
So if you’re wondering why wellness vendors know so little about wellness, there’s your answer: they aren’t required to know anything about wellness. Knowing just a little exceeds the minimum requirement.
Quizzify does have an ROI…and it’s guaranteed.
To conclude, here is our advice to workplace wellness vendors trying to justify what popular healthcare blogger Paul Levy calls the “wellness tax“: shut up and just be happy you still get to collect it, and that the authorities haven’t shut you down. (A doctor doing all this overscreening and billing for it would have been shut down.)
Don’t try to justify your hyperdiagnostic jihad on the basis of ROI or any other purpose other than enriching your bank accounts. Every time you try, you provide yet another reason why whatever college gave you a degree in anything other than advanced idiocy should lose its accreditation.
Share this:
Part 3 of the Proof Wellness Doesn’t Work.
This posting should be read in conjunction with Part 2 of the proof that wellness doesn’t work, which in turn links to Part 1 that shows mathematically wellness can’t work.
During the Great Debate, Ron Goetzel’s side admitted that wellness admissions haven’t fallen, but added: “Yes, wellness admissions haven’t fallen. However, without workplace wellness, wellness-sensitive medical admissions (WSMEs) would have risen. We kept WSMEs from rising.”
We explained how this wasn’t the case, and promised that we would post the analysis, so that he wouldn’t have to take our word for it. So here it is.
Since $11.3-billion was spent on these admissions in the private-pay population according to the wellness industry’s own HERO Report, these events would have had to rise by at least 60% in order to make the claim that the $7-billion wellness vendor industry broke even by avoiding them. If indeed workplace wellness prevented this huge increase in the privately insured population, one would expect that these very same events would have risen by something similar to 60% in the non-privately insured population–meaning the combined Medicare, Medicaid, and uninsured.
As researchers might say, the privately insured population was “exposed” and the remaining US population was “non-exposed.” As wellness spending snowballed, the separation between those two populations’ WSME trendlines should have increased significantly.
Instead, we find these populations WSME-as-percent-of-total-admissions also flat-lined, just like the private-pay population. Ironically, to the extent there is a difference, the population without access to workplace wellness trended slightly more favorably in WSME admission rates than the population with access to workplace wellness.
The graph below compares WSMEs — or what the HERO Guidebook refers to as Potentially Preventable Hospitalizations (PPH) — over this century, using the national Healthcare Cost and Utilization Project database.
Likewise, the idea that wellness events have indeed fallen on an absolute basis in the employed population due to wellness is also a fallacy. These events have fallen even faster in the population not privately insured.
Share this:
And the Envelope Please. The Goetzel-Lewis Debate winner was:
Lewis.
It wasn’t even close. I offered a proof that wellness has not avoided a noticeable number of hospital admissions this century. Ron Goetzel accepted that the data was accurate and declined to claim the million-dollar reward for finding a mistake in it.
Here is the first post-debate coverage. Highly favorable. Quoted Ron as saying employees like wellness. Try telling that to an employee…
The exchange I will be dining out on for years will be when Ron admitted I am the best peer reviewed in the industry. After he accused me of not being qualified to do my job, the following transpired:
“Ron, will you admit I am the best peer reviewer in the industry?”
“No”
“Then who is?”
Silence, followed by laughter from the audience.
Ron’s biggest points are:
- Wellness works if you do it right
- Almost nobody does it right. Maybe Procter & Gamble did it right a quarter-century ago (yes, he cited a 25-year-old study) but unless you are Ron’s client or a few others, you’re doing it wrong
- You can only expect a 1-to-1 return, and he cited an example to that effect
Ron’s other points were (in no particular order):
(1) He had nothing whatsoever to do with the Penn State program even though he was one of four people in the conference room holding the press conference titled: Penn State Takes Offensive in Health Plan Controversy.
(2) Four people stood up to say how bad their program was. In each case, Ron’s answer was that it was their own fault or it was a lousy program. In one case when someone complained about the typically cliched advice given to them by a wellness coach, he said: “You should listen to them and then maybe you will change your behavior.” He and Michael O’Donnell agree that most programs (95% in Michael O’Donnell’s estimation) are done badly.
(3) I was castigated because Quizzify has a 100% guarantee of savings. In the wacky world of wellness, companies that don’t guarantee anything because they can’t achieve savings are better than companies that guarantee savings because they can achieve them.
(4) Companies do not screen according to guidelines, but that is their fault. Wellness vendors would be perfectly happy to cut their fees by 70% and just screen the people who the USPSTF says to screen.
(5) He said none of my stuff is peer-reviewed so I am not qualified to do peer review. That’s why I’ve been blacklisted. Never mind that my most recent peer-reviewed article is trending #1 for the year in a major journal. Never mind that all of their stuff is full of obvious rookie mistakes, and never mind that legitimate journals like Health Affairs ask me to peer-review. And never mind that Ron himself admitted I have found many mistakes in his stuff.
I also named all the household names (in healthcare, at least) who had reviewed our stuff–Stuart Altman, Tom Scully, Regina Herzlinger, Jim Prochaska, Bob Galvin, Leah Binder, Norton Hadler. Oh, yes–and Quizzify is the only company in population health whose content is reviewed and approved by doctors at Harvard Medical School. (Formal announcement forthcoming.) And then finally I observed that we weren’t standing up there today because of his peer-reviewed stuff. We were standing up there today because of my non-peer-reviewed whistleblowing.
(6) They will still be quoting Katherine Baicker’s “Harvard Study” with the 3.27-to-1 ROI for years to come. They claim the authors haven’t backed off it one iota, regardless of what Baicker has said. And since there are no other studies to cite other than their own (which accidentally found wellness loses money), they’re stuck with this one.
(7) RAND, the NY Times Incidental Economist, and the rest of the world are all wrong because they oppose wellness. His view is: “Everyone’s out of step but Ronnie.”
(8) Even though it appears that every employee in a “pry,poke, and prod” program thinks it’s a joke or worse (and that’s why the bribes and fines have to keep rising), employees really do love wellness and want more of it, according to Ron.
(9) He accused me of spreading lies and misformation but didn’t actually name any.
(10) He admitted to doctoring original applications in the Koop Award file after my exposes, but didn’t apologize either for doctoring them or for writing in Health Affairs that they weren’t doctored, but rather that the “original application is online and available for review.”
(11) He is still defending the lies told in the Nebraska program about saving the lives of cancer victims, and still thinks Health Fitness Corporation deserves their Koop award, even though only 161 state employees improved a risk factor and the state also lied about their savings.
(12) He says 80% of costs are due to preventable diseases like cancer. I have had cancer and it was not preventable. It is frustrating to see someone tell people that my disease was preventable, when it wasn’t. He is referring to the old urban legend, thoroughly debunked on this site, that 75% of cost is preventable…but, what the heck, why not add 5%? 75% or 80%, we had already shot this fallacy down anyway.
(13) Even thought the HERO Guidelines which he co-authored and which represent “countless hours of collaboration” say wellness loses money, he doesn’t believe that section. He didn’t write it or (I guess) read it during the “two years” of this guidebook’s development. And maybe the person who conducted the webinar defending it was a different Ron Z. Goetzel.
My points–none of which were rebutted except where noted–were:
(1) I proved (using the proofs on this site) that wellness loses money and that if he thought it made money he would claim his million-dollar reward. He accepted the proof (no choice — the database that the proof was based on is maintained by another part of his company) and didn’t rebut it. One of his cronies in the audience, Seth Serxner, tried to rebut the proof but we were fully prepared for that and already had the data. We apologize to Seth for making him look bad. So the proof that wellness has lost money for 14 years is established.
(2) I proved that his entire participants-vs-non-participants methodology is made up. This was not rebutted. So all the savings in all the Koop Awards are toast.
(3) There was no rebuttal to my observation that most programs don’t work according to their own data. The rebuttal was that their own data was only one case study.
(4) I showed that basically everybody who does not make their living off wellness is opposed to it, like the left wing media and the right wing media. (Wellness supporters are running out of wings.) See the In the News section.
(5) I also made the unrebutted point that his own allies on his own committee have strayed off the reservation too — Debra Lerner and Altarum and of course Michael “95% of programs don’t work and RCTs have negative ROIs” O’Donnell.
Conclusion: nothing will change. He and his cronies — even though they have now admitted my proof, and have all passed on claiming the $1-million award — will still attempt to shove programs down employees’ throats — even though most of them (in their own words) fail and (in their own words) it is “very hard” to do a successful program.
And as long as they continue to do try to protect their revenue streams on the backs of corporations and employees, we will continue to protect them from the “pry, poke, prod and punish” jihad that is increasingly part of our corporate culture.
A special shout-out to Professor Matthew Woessner of Penn State, who made the trip down to DC, and was able to ask very pointed questions about Ron’s/Highmark’s program, the worst in wellness history. Ron disavowed any part of the “awful” Penn State program (he must have gotten lost and wandered into that conference room mentioned above) A question was also raised to the point of, that’s only one university. So the others are all fine. Matthew then got up and said Ohio State, which was the only other one he was closely familiar with, also hated their program. Quite a timely put-down and along with the questions from actual wellness program participants, highlighted the disregard in which most employees hold these programs.
Share this:
American Journal of Health Promotion Admits Massive Wellness Losses
RAND’s Soeren Mattke said it best:
The industry went in with promises of 3-to-1 and 6-to-1 ROIs based on health care savings alone. Then research came out that said that’s not true. They said, “Fine, we are cost-neutral.” Now research says: “Maybe not even cost-neutral.” So they say: “It’s really about productivity, which we can’t really measure, but it’s an enormous return.”
In other words, whenever you invalidate one metric, they come up with another one. We then have to shoot that one down, and the cycle repeats. It’s invalidity-meets-Whack-A-Mole. After the healthcare spending ROI fiction imploded, Michael O’Donnell, editor of the wellness industry trade journal, asked dismissively: “Who cares about ROI anyway?”
Since ROI wasn’t working, they then tried value-on-investment (VOI), which turned out to show even greater losses than a straight ROI calculation.
Continuing that tradition, Michael O’Donnell of the American Journal of Health Promotion presents: Return on Allocated Resources, or ROAR. ROAR counts everything, including productivity. By counting everything, ROAR shows far greater losses than VOI.
Michael says that a 1% increase in productivity is worth $1933:
However, a much greater 3.75% (90 minutes of a 40-hour workweek) reduction in productivity only costs $2184:
How did he accomplish this sleight-of-hand, where a 1% increase in productivity practically offsets a 3.75% decrease? Simple: by putting both thumbs and every other appendage on the scale. He accounts for lost work time at an employee’s hourly rate. So far so good. However, he then applies a magic multiplier to the hourly rate to calculate increases in productivity based on hypothetically enhanced corporate revenues due to the productivity increase. So if payroll is 30% of revenues, and productivity climbs 1%, then revenues would also automatically climb 1%. That means in dollar terms revenues climb more than three times faster than productivity.
Had he used the same revenue multiplier for the certain 3.75% productivity decrease due to wellness-induced lost work time that he used for his speculative 1% productivity increase, his time-off-for-wellness scheme would cost a whopping $7143/employee/year.
And wellness vendors wonder why line managers are so reluctant to allow employees to work out on company time.
So while per-employee losses from wellness based purely on added healthcare spending and program expense are “only” in the three figures, the net reduction in productivity from a (speculative) 1% increase less a (certain) 3.75% decrease due to lost work time amount to a mind-boggling $5210/year.
And that is probably an understatement. The 3.75% lost work time due to wellness doesn’t include the time employees spend changing clothes after their workouts, lying on HRAs, standing in line to be screened and “coached,” complaining to HR that they haven’t received their incentive checks yet, and hanging out at the water cooler dissing the program.
If you’re keeping score at home, this is the third time Michael O’Donnell has strayed off message. Just like some people are convinced that Donald Trump is a closet Democrat trying to torpedo the GOP, you would be excused for thinking that Michael O’Donnell is a member of our Welligentsia group, trying to sow chaos amongst the Wellness Ignorati.
He isn’t, but nonetheless I count him among our greatest assets. First, he admitted that up to 95% of wellness programs don’t work. Then he admitted that studies done using randomized control trials lose money. And now this one, detailing — using his own math — by far the greatest losses that a wellness metric has ever shown.
Quizzify’s guarantee numbers add up (source: Intel-GE Validation Institute)
Ron Goetzel is probably tearing his hair out over his crony’s unforced errors on the eve of our debate. Or, in the immortal words of the great philosopher Warren G. Harding: “I can handle my enemies. It’s my friends who have me pacing the floor at night.”
Share this:
NY Times 1, Goetzel 0 (“Incidental Economist” obliterates wellness)
If you want a preview of Monday’s big wellness debate, look no further than a 3-way exchange (us, New York Times, Ron Goetzel) from late 2014.
First, our November 25th Health Affairs blog did for the wellness industry what Upton Sinclair did for meatpacking. Despite its Thanksgiving-week publication date, it became the #1 most-read for November and #12 for all of 2014, in Health Affairs. (This was the article generating the famous Los Angeles Times moniker for wellness: Scam.)
Next, New York Times “Incidental Economists” Austin Frakt and Aaron Carroll had a field day with their followup column. It should be read in its entirety. It’s basically a cut-and-pasted version of our own, with some hilariously scathing color commentary (and we have always said the greatest value wellness vendors deliver is humor).
Their lead line says it all:
“We’ve said it before, many times and in many ways: workplace wellness programs don’t save money.”
Finally, let’s look how Ron spun their elegant smackdown, using his best Goetzel “The Pretzel” twisting and turning of their words:
The recent Health Affairs Blog post by Al Lewis, Vik Khanna, and Shana Montrose titled, “Workplace Wellness Produces No Savings” has triggered much interest and media attention. It highlights the controversy surrounding the value of workplace health promotion programs that 22 authors addressed in an article published in the September 2014 issue of the Journal of Occupational and Environmental Medicine titled, “Do Workplace Health Promotion (Wellness) Programs Work?” That article also inspired several follow-up discussions and media reports, including one published by New York Times columnists Austin Frakt and Aaron Carroll who answered the above question with: “usually not.”
Four observations about that one paragraph presage Ron’s strategy for Monday.
First, How do you translate Frakt and Carroll saying: “We’ve said it before, many times and in many ways: workplace wellness programs don’t save money” into: “usually not”? Simple, you reference the December “follow-up discussion” by these NYT columnists…but then link to a previous discussion, in September. (Try the link — really.)
Second, our article didn’t “highlight the controversy surrounding” the value of wellness. Our article “highlighted” that pry-poke-and-prod wellness has no value. There is no controversy. We have a proof — now backed with a $1-million reward, in case anyone doesn’t understand the meaning of proof.
Third, Ron writes: “22 authors addressed this” in September 2014. Ron needs to understand that math is not a popularity contest. It doesn’t matter how many wellness vendors want to protect their revenue stream, because the rules of math are strictly enforced. This is a typical Goetzel tactic, sort of like Claude Rains saying: “Owing to the seriousness of this crime we’ve rounded up twice the usual number of suspects.” In any event, two of the non-wellness-vendor authors were horrified to learn that this article had endorsed the Nebraska fraud as a “best practice.”
Finally, he appears to conflate the Journal of Occupational and Environmental Medicine with Health Affairs. The latter is the most respected journal in health policy (impact factor: 4.966), which is why our essay got picked up so broadly. The former is largely a wellness industry mouthpiece with an impact factor of 1.630. Sort of like the old tag line for Hustler: “The Magazine Nobody Quotes.”
No “controversy” around Quizzify’s 100% savings guarantee
What he will try to do is sow doubt, the classic last resort when the facts all go the other way. You saw it from the tobacco industry, and more recently from climate change-deniers. He will bring up all his articles and all his authors and say they are all “peer-reviewed” and say that there is “evidence” on both sides. He’ll use words like “controversy” and “discussion” when in reality it’s settled science (and more importantly, settled math) that prying, poking and prodding employees is nothing more than the Wellness Vendor Full Employment Act, and it’s time to repeal it.
Share this:
Aetna Accidentally Invalidates the Wellness Industry’s “Savings” Model
If engineers learn more from one bridge that falls down than from 100 that stay up, this new Aetna-Newtopia study is the Tacoma Narrows of wellness industry study design. No article anywhere — including our most recent in Harvard Business Review — has more effectively eviscerated the fiction that wellness saves money than Aetna just did in a self-financed self-immolation published in the Journal of Occupational and Environmental Medicine. Hopefully the people who give out Koop Awards to their customers and clients will read this article, and finally learn that massive reductions in the cost of participants associated with trivial improvements in risk are due to self-selection by participants, not wellness programs. And certainly not wellness programs centered around DNA collection.
Aetna studied Aetna employees who, by Aetna’s own admission, didn’t have anything wrong with them, other than being at risk for developing metabolic syndrome, defined as “a cluster of conditions that increase your risk for heart attack, stroke and diabetes.”
In other words, taking the wellness industry’s obsession with hyperdiagnosis to its extreme, the subjects’ “diagnosis” was being at risk for being at risk. Not only did they not have diabetes or heart disease, but they didn’t even have a syndrome that put them at risk for developing diabetes or heart disease. You and I should be so healthy.
As this table shows, after one year, the changes in health indicators between the control and study groups were trivial (like a difference in waist measurement under 3/10 of an inch), and only triglycerides was barely statistically significant (p=0.05). Additionally, the control group actually outperformed the study group in 3 of the 6 measured variables, as would be predicted by random chance. Bottom line: nothing happened.
And yet, Aetna reported savings of $1464/participant in the first year. This savings figure is more than 20 times higher than what Aetna’s co-authored HERO Report says gets spent in total on wellness-sensitive medical events. It’s also far higher than Katherine Baicker’s thoroughly discredited 3.27-to-1 ROI, that she has basically retracted, published six years ago–that, yes, in keeping with wellness industry tradition, their article cited. (Only now, because the study is now six years old, Aetna feels compelled to insist that it is “recent.”)
How did they achieve such a high savings figure in a legitimate RCT? Simple. That savings was not the result of the legitimate RCT. Having gone through the trouble of setting up an RCT, they then proceeded to largely ignore that study design, since as their own table above shows, nothing happened.
Spending was a bit lower for the invited group, but obviously there couldn’t have been attribution to the program. A responsible and unbiased researcher might have said: “While there is a slight positive variance between the spending on the control group and the spending on the invitee group that wouldn’t begin to cover the cost of our DNA testing, we can’t attribute that variance to this program anyway. The subjects were healthy to begin with, there was no change in clinical indicators, and we didn’t measure wellness-sensitive medical events even though we know from our own HERO report both that those represent only a tiny fraction of total spending, and that those are the only thing that a wellness program can influence.”
Instead, they coaxed about 14% of the invitees to give up their DNA, and measured savings on them. More than coincidentally, that decidedly uninspiring 14% participation rate was about the same as the Aetna-Newtopia debacle at their Jackson Labs reference site-from-hell. Basically, employees don’t want their DNA collected, and DNA turns out to be quite controversial as a tool to predict heart disease down the road, let alone during the next 12 months. Further, Newtopia admits they store employee DNA, lots of people have access to it, and they could lose it.
The DNA also seems to have had precious little to do with the actual wellness program itself–and for good reason given the links above. This seems like a classic wellness intervention of exactly the type that has never been shown to work, with the DNA being only an entertaining sidebar. The subjects themselves exhibited no interest in hearing about their DNA-based predictions.
This is the first time a study has compared the result of an RCT to the result of a participants-only subset of the same population. The result: a mathematically and clinically impossible savings figure on the subset of active participants, and an admission of no separation in actual health status between the control and invitee groups by the end of the program period.
So Aetna — in this one article — accidentally proved what we’ve been saying for years about the fundamental bias in wellness study design that creates the illusion of savings:
Participants will always massively outperform non-participants, period — even when the program doesn’t change health status or even when there was no program for the “participants” to participate in.
Quizzify’s savings model is a 100% guarantee.
Share this:
Red Meat 1, World Health Organization 0
Number of deaths attributable to eating processed meat, according to the World Health Organization (WHO): 34,000
Number of people struck by lightning annually: 240,000
And yet the WHO generated a huge headline by saying that eating red and processed meat could increase your risk of colon cancer by 18%.
Let us assume that they are right. (And we will let the trade associations debate them on the scientific merits of that 34,000 figure.) Even if they are right, this is a perfect example of confusing an increase in relative risk of one disease with absolute risk of dying. To use the lightning example, you probably have a 1-in-a-billion chance of being struck by lightning if a thunderclap is audible but the sky above is clear. Some states close public pools when that happens. If the sky above is clear but you can see lightning in the distance, your odds of getting struck may jump to 1-in-100,000,000. That’s a 10-times relative increase, but only a 9-in-a-billion absolute increase. So these states inconvenience parents and fidgety kids for basically no reason other than misunderstanding relative and absolute risk.
To make matters worse, the WHO conflates the risk of smoking and asbestos with red meat. Both the former cause perhaps something like an 18% increase in age-adjusted death rates in total, not an 18% increase in one form of cancer. The difference? Probably about a thousand times in total, unvarnished, absolute risk.
Yes, I know it’s not always about me, but this is exactly what Quizzify teaches. Newscasters who had taken the Quizzify quiz (and relative-vs-absolute risk is in the advanced level…but they are newscasters so they should get to that level) would have led with the headline: “WHO Demonstrates No Understanding of Health” instead of “You Could Die from Eating Red Meat”.
The absolute risk of getting bad information from Quizzify is very low.
That’s it for now. Funny thing, I used to be a quasi-vegetarian because I was concerned about the impact of red meat and processed meats on my colon cancer risk. But anyone who understands health research would read this the same way I do: it’s OK to live on the edge. Don’t deprive yourself of red meat for this reason. I myself am headed out for a burger. Not just any burger but a bacon burger. In the immortal words of the great philosopher Sammy Davis Jr., I’ve got a lot of living to do.
Share this:
Deja Vu All Over Again, Again: Koop Award Channels Sergeant Schultz
In keeping yet again with the Koop Award tradition of bestowing awards upon themselves, the 2015 award went to one of Ron Goetzel’s own clients, McKesson. Curiously, not one but two other Koop Award sponsors (Alere and Vitality) are also listed as McKesson wellness vendors. This new record for undisclosed Koop sponsor self-awarding would make Nero proud.
And in keeping yet again with the Koop Award committee’s tradition of not noticing invalidating mistakes, none of what you are about to read — not one single invalid, mathematically impossible or self-contradictory datapoint that were all self-evident — was noticed by any of the Koop Award Committee members. They saw nothing. Our previous Koop Committee posting, entitled Koop Award Committee Meets Sergeant Schultz, seems presciently titled indeed.
Highlights of Risk Reduction and Wellness Activities
McKesson employees attended 160,000 Weight Watchers meetings. McKesson claims their employees collectively lost 24,000 pounds. That’s about 2.5 ounces lost per meeting for all attendees combined.
So if 10 people attended the average meeting, literally the amount of weight each lost (0.25 ounces) could be accounted for by walking to and from the meeting room, plus maybe a sneeze or two. Here’s a surprise: Weight Watchers has been a total failure in the corporate weight control market. Truven and McKesson must have missed this memo.
Oh, and did we mention that this weight-loss figure is self-reported, doesn’t count dropouts, and apparently contradicts McKesson’s own screening results — in which employees actually gained weight? (See below)
As the chart below shows, McKesson showed roughly a 2% decline across all risk factors, in the half of the company willing to be measured. The other half of the employee population declined to participate despite massive incentives. This is typical because employees hate wellness so much that even the biggest bribes can’t generate participation. So that 2% risk reduction amongst the 50% of repeat participants means – optimistically assuming the half who dropped out or didn’t participate stayed the same risk-wise – a 1% reduction in risk factors for McKesson as a whole.
Some other curiosities about this chart below that no one noticed. First, only 1% of its employees have an “elevated” risk of being a problem drinker, not bad for a company that used to be in the liquor distribution business. By contrast, the rest of the country’s alcoholism rate is 4.7% (women) or 9.4% (men) — and that’s an actual diagnosis of alcohol use disorder, not simply “elevated” risk.
Second, despite 160,000 Weight Watchers meetings and the 0.25 ounces each participant lost, the average employee who bothered to show up twice to be weighed-in actually gained weight. So which is it? Did they lose weight or gain weight?
Answer: they gained weight. Participants always outperform non-participants, and when you recombine the two groups, you always show that performance by participants is a misleading indicator of actual overall performance by everyone in total.
Third, despite McKesson’s claims of smoking cessation, note that a large chunk of employees refused cotinine (nicotine) testing. So 27% of employees self-reported tobacco use but only 9% of those willing to be tested were positive for tobacco use. 27% of employees admitting tobacco use might be a record for a company winning a wellness award. It’s about 9 points higher than the national average.
Finally, McKesson — reading datapoints off this very chart — claims a 9% reduction in risk, using the simple expedient of ignoring the people whose risk factors increased. This fallacy is the classic way to overstate savings. We call it “Lake Wobegon meets wellness” because everyone improves in this calculation. Still, we’ve never seen evaluators simply ignore the people getting worse when the data was right in front of them like this.
The Koop Committee reviewers somehow ignored or overlooked all of these things, and yet still they wonder why they are known as the Wellness Ignorati. Ah, well, as Tom Friedman wrote: “We wouldn’t be human if we didn’t ignore facts that dash our hopes upon the cliffs of reality.”
This 1% risk reduction across the population generated more than $13,000,000 in gross savings. McKesson’s graph below shows it spends about $4000 for each of its 45,000 eligible adults—or roughly $180,000,000 overall (excluding dependents). That means a 1% risk reduction creates a 7% spending reduction.
Using that logic, a 14% risk reduction would save 14 times as much–or $180,000,000, enough to wipe out all of McKesson’s healthcare spending, using the McKesson/Goetzel math.
The Dog that Didn’t Bark in the Nighttime
What makes this even more of a head-scratcher is that most people and organizations on this award committee – and a large number of the vendors and consultants involved in McKesson – also produced the HERO Outcomes Guidelines Report, which is quite insistent that the wellness-sensitive medical event rate (“potentially preventable hospitalizations” as they call them) be tallied as the only indicator of wellness program success, since no other costs decline and probably actually increase:
And yet no one noticed this event rate wasn’t disclosed — despite the insistence of everyone on the committee in this HERO report that these events are the only element of cost reduced by wellness.
Your employees will know NOTHING if you don’t Quizzify your workplace!
Had this rate been disclosed, Sergeant Schultz himself – or even a Koop Award Committee member — might actually have noticed that the decline (if any) in these events would account for roughly 1% of McKesson’s claimed savings from wellness, about $12/person/year. The other 99%? Those are what we call the “wishful thinking multiplier,” which is the basis for all savings reported by the wellness industry.
Share this:
Part 2 of the Proof: Even If Wellness Could Save Money, It Doesn’t
Recently we promised a Part 2 to our original proof that wellness savings are mathematically impossible. Commenters said: “How can you have a Part 2 to a proof? You just proved it.”
Read on.
The previous proof showed wellness can’t save money, even if programs were perfect. This installment proves that even if wellness could save money, it hasn’t. Meaning even if wellness were free, it couldn’t pay for itself. So this proof is independent of the previous proof. For wellness to save money, the wellness true believers would have to find fallacies in both proofs. Either is sufficient to make our case…but we have both.
Quite literally, forcing employees to “do wellness” or lose money has avoided basically zero wellness-sensitive medical events in the 13 years ending 2013 (2014 data isn’t in yet), according to the federal government. If the name “federal government” sounds familiar, it’s because it’s the very same federal government that has passed a law encouraging vendors to pitch “pry, poke and prod” programs to you despite their complete lack of evidence basis, lack of effectiveness, and potential for harm.
Here is the way our analysis was done. We used the government database called the Healthcare Cost and Utilization Project, or HCUP. That database tracks all hospitalizations due to all causes, by population. So it is possible to focus on just the commercially insured population, which they call “privately insured.”
The privately insured population is 100% sensitive, meaning everyone whose workplace “offers” wellness is in that database. The database isn’t specific, meaning plenty of people in it do not have access to wellness. Nonetheless, the dramatic increase over the 13 years in the number of people whose employers push wellness should produce an equally dramatic decrease in wellness-sensitive medical events. While wellness was rare at the start of this analysis in 2001, today most large companies, nonprofits, and governments have wellness. In total, one can project from the Kaiser Family Foundation data that about 75-million people (or roughly half of all privately insured people) are subject to what Jon Robison has termed wellness-or-else.
Keep in mind that all hospitalizations have been declining over this 13-year period, due to shifts to outpatient, better usual care, etc. So the question is not whether WSMEs have been declining, but whether they have been declining faster than the rates of all other hospitalizations in combination due to the large and increasing “dose” of wellness” being applied to them.
Instead, as you can see, these WSME admissions have trended essentially flat over the period, as a percentage of all admissions. In other words, there is no difference between the decline in admissions for WSMEs – despite $7-billion/year being spent on vendors to prevent them – and the declines in every other category of hospitalization. 13 years ago about 6.9% of events were wellness-sensitive. Now it’s about 7.0%. (This is 2013. 2014 is also in, for our customers for whom we track WSMEs, and shows no change.)
This is based on ICD9s 401-405, 410, 430-438, and 250 — strokes, hypertensive events, heart attacks, and diabetes events.
To make the point visual, the “dose” of wellness probably quintupled, in total, over this period, so the directional expectation of the chart would be:
Prima facie, the debate is over, again, just like it was over after our last proof.
Needless to say, the true believers aren’t about to give up their revenue stream just because we’ve double-proved they’re fabricating savings. They will make two arguments against this proof of their own ineffectiveness. First, they’ll argue that wellness reduces all events and other costs equally, so really we should credit wellness for the total cost reduction, not the reduction in just wellness-sensitive admissions. This might seem like a pollyannish view of wellness, but wellness true believers attribute everything that’s good to wellness. True believer Bruce Sherman has even argued that wellness actually reduces industrial waste, so to a wellness true believer, eating more spinach makes every employee a Popeye.
Unfortunately for Bruce and others, the wellness industry’s own HERO report says wellness can only reduce WSMEs. Other costs go up, it says:
Second, one could argue that there isn’t enough penetration of wellness yet to bend this trend, since the HCUP privately insured population includes tons of people without access to wellness, and even many people with wellness access refuse to participate.
Unfortunately, that argument self-immolates. Vendor fees are $7 billion. All these WSME ICD9s combined (using the HERO-estimated admission cost of $22,500) amount to about $11.3 billion. That $11.3 billion includes the half of privately insured people who don’t have access to wellness. Already, when you cut that figure in half to account for those employees with employers who’ve decided not to “do wellness” to them, the $7 billion size of the wellness industry exceeds the size of avoidable events ($5.7 billion). This is consistent with our first proof, which showed the same thing, but on an individual company level. Now—assuming participation is 50%, you need to cut the WSME hospitalization total in half once again. You’re down to $2.85 billion in potentially avoidable events — that companies are spending $7 billion on vendors to avoid.
So, no matter how you look at it, “pry, poke and prod” programs have been singularly ineffective in reducing WSMEs. And if the HERO Guide is right that these are the only admissions wellness can avoid (while other costs increase, as they admit), wellness does not and cannot save money.
Instead, wellness-or-else is basically a pile of, um, industrial waste.
Anyone still want to try to claim the million-dollar reward for showing pry, poke and prod programs aren’t a total waste of resources? I didn’t think so.
The thing most likely to save money is knowledge.
Note: This graphical analysis is copyright 2015 to Quizzify. However, any disinterested researcher or journalist may request a copy of the backup material from us.
They Said What? 