If engineers learn more from one bridge that falls down than from 100 that stay up, this new Aetna-Newtopia study is the Tacoma Narrows of wellness industry study design. No article anywhere — including our most recent in Harvard Business Review — has more effectively eviscerated the fiction that wellness saves money than Aetna just did in a self-financed self-immolation published in the Journal of Occupational and Environmental Medicine.  Hopefully the people who give out Koop Awards to their customers and clients will read this article, and finally learn that massive reductions in the cost of participants associated with trivial improvements in risk are due to self-selection by participants, not wellness programs.  And certainly not wellness programs centered around DNA collection.

Aetna studied Aetna employees who, by Aetna’s own admission, didn’t have anything wrong with them, other than being at risk for developing metabolic syndrome, defined as “a cluster of conditions that increase your risk for heart attack, stroke and diabetes.”

In other words, taking the wellness industry’s obsession with hyperdiagnosis to its extreme, the subjects’ “diagnosis”  was being at risk for being at risk.  Not only did they not have diabetes or heart disease, but they didn’t even have a syndrome that put them at risk for developing diabetes or heart disease.  You and I should be so healthy.

As this table shows, after one year, the changes in health indicators between the control and study groups were trivial (like a difference in waist measurement under 3/10 of an inch), and only triglycerides was barely statistically significant (p=0.05).  Additionally, the control group actually outperformed the study group in 3 of the 6 measured variables, as would be predicted by random chance.  Bottom line: nothing happened.

And yet, Aetna reported savings of $1464/participant in the first year. This savings figure is more than 20 times higher than what Aetna’s co-authored HERO Report says gets spent in total on wellness-sensitive medical events.  It’s also far higher than Katherine Baicker’s thoroughly discredited 3.27-to-1 ROI, that she has basically retracted, published six years ago–that, yes, in keeping with wellness industry tradition, their article cited.  (Only now, because the study is now six years old, Aetna feels compelled to insist that it is “recent.”)

How did they achieve such a high savings figure in a legitimate RCT?  Simple. That savings was not the result of the legitimate RCT.  Having gone through the trouble of setting up an RCT, they then proceeded to largely ignore that study design, since as their own table above shows, nothing happened.

Spending was a bit lower for the invited group, but obviously there couldn’t have been attribution to the program. A responsible and unbiased researcher might have said: “While there is a slight positive variance between the spending on the control group and the spending on the invitee group that wouldn’t begin to cover the cost of our DNA testing, we can’t attribute that variance to this program anyway.  The subjects were healthy to begin with, there was no change in clinical indicators, and we didn’t measure wellness-sensitive medical events even though we know from our own HERO report both that those represent only a tiny fraction of total spending, and that those are the only thing that a wellness program can influence.”

Instead, they coaxed about 14% of the invitees to give up their DNA, and measured savings on them.  More than coincidentally, that decidedly uninspiring 14% participation rate was about the same as the Aetna-Newtopia debacle at their Jackson Labs reference site-from-hell.  Basically, employees don’t want their DNA collected, and DNA turns out to be quite controversial as a tool to predict heart disease down the road, let alone during the next 12 months.  Further, Newtopia admits they store employee DNA, lots of people have access to it, and they could lose it.

The DNA also seems to have had precious little to do with the actual wellness program itself–and for good reason given the links above.  This seems like a classic wellness intervention of exactly the type that has never been shown to work, with the DNA being only an entertaining sidebar. The subjects themselves exhibited no interest in hearing about their DNA-based predictions.

This is the first time a study has compared the result of an RCT to the result of a participants-only subset of the same population.  The result: a mathematically and clinically impossible savings figure on the subset of active participants, and an admission of no separation in actual health status between the control and invitee groups by the end of the program period.

So Aetna — in this one article — accidentally proved what we’ve been saying for years about the fundamental bias in wellness study design that creates the illusion of savings:

Participants will always massively outperform non-participants, period — even when the program doesn’t change health status or even when there was no program for the “participants” to participate in. 

Quizzify’s savings model is a 100% guarantee.