Like sausage-making, Trump University, or the sun, the “Harvard Study’s” 3.27-to-1 ROI should not be examined too closely.

And yet, throwing caution to the wind and making sure my stomach was empty, I delved (as promised) into the individual studies comprising this meta-analysis to understand what it was about their design and findings that allowed them to be repackaged into the wellness industry’s raison d’etre.

I had been quite adamant in the previous post that this whole gold-plated meta-analysis was likely based on garbage case studies. I compared it to packaging zero-doc home loans into AAA-rated collateralized mortgage obligations (CMOs). I realize now that was a premature and unfair comparison. So I owe an apology–to Bear Stearns, Lehman Brothers, and Countrywide.

What Katherine Baicker did was way more mathematically unforgivable because it was already obvious that the results were wrong when she wrote her article.  For the CMO packagers, the invalidity didn’t become completely obvious for at least a few weeks after they started foisting these CMOs onto their customers.

Only four of Professor Baicker’s studies have abstracts or full text available at no cost. Let’s consider these four case studies one by one.  I’ll be gosh-darned if I am going to pay for the privilege of reading the others. The authors should be paying me for remedial lessons in arithmetic.

This study, in a now-defunct journal, found that members of a high-risk cohort showed a greater decline in claims than members of lower-risk cohorts.  To use a highly technical health services research term, and please don’t be put off by my jargon: duh.

Suppose risks are “heads” and you have three sets of 100 pennies. The first has 100 heads, the second 75 heads, the third 50 heads.  Flip all three groups of pennies. You’ll reduce heads by about 50 in the first group, 25 in the second, and 0 in the third. Biostatisticians call those declines “regression to the mean.” Dee Edington calls them the “natural flow of risk.” Apparently Professor Baicker calls them “pennies from heaven.”

Stanford researchers chimed in with a study that self-invalidates three different ways:

1. They also used a low-risk group as a control for a high-risk group — and were equally thrilled when the high-risk group showed a greater decline in risk factors than the low-risk group
2. They achieved a whopping $1000 per participant in savings (in today’s dollars) — in a six-month period, no less. Let your imagination run wild to imagine the dramatic risk reduction needed to achieve an immediate $1000 in savings.
3. Actually, you can let your imagination run tame, because the high-risk group’s self-reported risk scores improved only 2% vs. the low-risk control.

And at the risk of sounding like Yogi Berra, even if this result were possible, it would be impossible. The improved scores were achieved in part by eating more carbs and less fat. This study was done in the era when fat of all types was demonized, and Kellogg’s Frosted Flakes earned a “heart-healthy” label from the American Heart Association.

Awful stuff, those fats and oils. (Not!)

What raised my eyebrows for this next study is that wellness produced the greatest savings in the following categories: “neoplasms, digestive disorders, and blood and blood-forming organs.” Let’s examine each category in turn.

If anything, “neoplasm” expense would rise in that short a period, since in those days women over 35 were given annual mammograms despite the overwhelming likelihood of false positives.

Digestive disorders?  Even wellness vendors don’t claim to reduce cases of ulcerative colitis or Crohn’s Disease.  Colon cancer would increase because in those days people got screened every five years.  And we know from the Nebraska program that those screenings catch a boatload of benign polyps — or as gastroenterologists call them, thousand-dollar bills.  (They can charge much more when they remove polyps than when they can’t find any to remove.)

How, you might ask, does a wellness program reduce diseases of “blood and blood-forming organs”?  A better question: what is a disease of “blood and blood-forming organs”? Most wellness vendors probably couldn’t even name one. Here’s the list.  The highly favorable results in this wellness program suggest that eating more Frosted Flakes and less olive oil prevented sideropenic dysphagia, lymphangioma, and of course von Willibrand’s Disease.  And for you Ted Nugent fans, there’s cat scratch fever.

No meta-analysis showing massive savings is complete without Ron Goetzel. His study of Procter & Gamble showed a 29% reduction in spending for participants vs. non-participants in wellness.  Those annual mammograms and lowfat diets must really have paid off for them.  Either that or…could be it that participants-vs-non-participants is now proven to be an invalid study design?  Maybe Ron didn’t know that at the time — that was years before he tampered with some data in order to maintain the fiction that this study design isn’t obvious nonsense.

This massive savings — worth billions in the stock price given P&G’s reasonably high P/E multiple — was so critical to P&G (my client now), that no one there has any memory of it.  And no wonder — even the world wide web is younger than this study.

So what next?  We are asking Katherine Baicker to set things right.  It’s Pottery Barn Rules–she broke it, she fixes it.  We would like her to do a study showing the mathematical and clinical impossibility of savings from “pry, poke and prod” programs.   To paraphrase the immortal words of the great philosopher Charlie Brown, this is her chance to be the hero instead of the goat.