In the immortal words of the great philosopher Britney Spears: “Oops, I did it again.”
For the fourth time in as many decades, I’ve been caught! This is not to say that I’ve only made 4 miscues in the most recent 4 decades. Just that I’ve only been caught 4 times in these 4 decades. Not including the time I caught myself actually thinking disease management (DM) saves money. Until then, basically everything I said was a lie, however unintentional, because in my naivete I thought DM worked. Silly me.
Jeff Hogan joins the few, the proud, who have called me out for saying things that aren’t exactly accurate. I’m putting this blog on top of the previous one to make it easier to track my cumulative miscues, in case you’re keeping score at home.
In this case, he referenced a study in Health Affairs showing that bundled payments reduced the cost of surgeries by 11%. I saw that abstract and immediately assumed that, like many other bundled payments, the reason the cost per procedure declined is that the number of procedures increased, by surgeons adding more “easier” and hence less costly procedures. This would reduce the cost/procedure but total costs would increase due to more procedures.
I couldn’t link through to that study (nor can you, most likely) from that abstract, to test that hypothesis. But since it is Health Affairs I just assumed that their peer review for that article is as sloppy as it is for wellness articles. Ron Goetzel published a nonsensical article there, which, among other things, concluded that only about 5.5% of the cohort smoked because only 5.5% of the cohort admitted they smoked, on a risk assessment. Since the US smoking rate is more like 18%, the correct conclusion would have been that two-thirds of smokers lie on risk assessments. I would have caught that in peer review but Health Affairs allows authors to pick their own toadies as peer reviewers.
So, without actually reading the Health Affairs study, I assumed they applied the same lofty standard of peer review to this article as to Goetzel’s:
They don’t appear to have tracked the number of cases. A classic thing hospitals and doctors do when they get paid per case is to perform surgeries on people who may not have needed them. These people will have lower-than-average costs and complication rates…but be reimbursed the same.Or, in the immortal words of the great philosopher Claude Rains, “Owing to the seriousness of this crime I’ve instructed my men to round up twice the usual number of suspects.
Jeff wrote back:
Al: Did you read the same Health Affairs article that I did? The citations and case tracking is quite detailed in the report and appendix. Not only did they carefully examine the number of cases but they used some very intensive methodologies for doing disruption analysis.
He helpfully attached a pdf. It turned out they had indeed tracked the number of referrals not going to surgery, and almost a third did not, in fact, go to surgery. This factoid never made it into the abstract, but was buried in the article.
So kudos to Jeff and if he sticks around another 8 years, 9 months and 22 days, at my current pace, I’ll be due for another mistake.
Guilty as charged. Someone called me out on yet another mistake buried in my 500,000 words published to date.
Yep, the number of members in the most exclusive club in healthcare outcomes analysis just rose by 33%, as Tom Milam of TrueLifeCare joins Corey Colman and Keith McNeil in justifiably calling me out.*
To put this track record in perspective, Ron Goetzel has been caught 14 times. You might say, well, 14 isn’t that much different from 3 in absolute terms. (In percentage, it is, but we’ll let that slide.)
Except that I needed an entire decade to rack up 3, while Ron needed only 45 minutes to tally 14. Over the decade, his number would be more like approximately eleventy zillion. It depends how you count the ones where he doctored numbers that were phony to begin with and then doctored them back again to the original phony numbers, after insisting that the doctored numbers were real. If you’ve lost track on all the doctorings that I just now published a companion blog post on it.
So what was the mistake?
[SPOILER ALERT: The rest of this post is boring.]
It’s kind of anticlimactic, and quite obscure. By way of background, I routinely analyze wellness-senstive medical event (WSME) rate trends for large employers and health plans. It’s not rocket science, but it’s totally valid. Indeed, it’s the only population-based observational analysis that is valid. (RCTs are not observational. But you knew that.). It was even embraced by Ron Goetzel’s very own outfit: the Health Enhancement Research Organization — before they realized that valid measures are the wellness industry’s kryptonite.
The WSME tally is also the only observational methodology accepted by the Validation Institute for employers and health plans.
Here’s what the national WSME rate looks like. (I think there was a reporting or transcribing error by one of the reporting states in 2005-2006, to the extent anyone noticed the inflection in the graph, or cares.) This graph of WSMEs shows that, over the decade+ period of the greatest growth of workplace wellness, that there was no improvement in event rates relative to the US population that would not have had access to workplace wellness — Medicare, Medicaid and the uninsured. Obviously their raw rates were higher. This is a difference-of-differences analysis.
Quite the contrary, it appears that if anything the employer-insured cohort trended worse than the control.
Tallying this rate requires our data extraction algorithm to collect ER and IP events primary-coded both to the disease in question, or else are common complications of the disease in question. We pick common complications based on two factors:
- How likely is someone with the disease in question to get the complication?
- How likely is it that the complication in question occurs in someone with the disease?
Remember, we only tally primary codes because we want to simplify the analysis enough that we can be 100% sure of comparability between any given payor and other payors comprising the benchmark. So we look for an “80-20 rule” in what we include in the primary code data extraction.
Our diabetes rate includes quite a number of complications that fit that description., one of which is cellulitis. Diabetics are much more likely than non-diabetics to get cellulitis in their extremities — feet in particular — because they often can’t feel a cut. (Also the skin on their feet can be thinner than it should be.) Likewise, cellulitis of extremities is much more likely to be diagnosed in diabetics than non-diabetics.
If you can’t feel it, you won’t treat it. And therefore your odds of cellulitis in your foot are high. However, cellulitis in non-extremities would correlate much more loosely with diabetes, since diabetics can still feel and see skin issues elsewhere on their bodies. Therefore, not all cellulitis codes, by a longshot, are included in our analysis.
While we included cellulitis of the foot (and leg, also common enough), we somehow — despite having done these analyses 20 to 30 times a year for 15 years — omitted cellulitis of the toes. Sort of like the Matisse painting hanging upside down in the Museum of Modern Art for 47 days, no one else noticed either. Yet even the most intellectually challenged members of the wellness industry’s self-anointed awards committee understand the anatomical fact that, technically speaking, the toe is part of the foot.
Le Bateau, Henri Matisse
Honestly, when all is said and done, this won’t change anyone’s results much, and all the changes will be in the same direction vs. history (which is also going to be recoded) and vs. the benchmark/average, likewise recoded. This is especially true in the working-age population, which comprises most of our analysis. Nonetheless, kudos to Tom Milam for becoming the third member of this most exclusive club.
*Your chances of joining this club are quite remote, statisically speaking. They are even more remote if you didn’t notice the arithmetic error just now. n increase in membership from 2 to 3 is a 50% increase, not a 33% increase. And the painting is still upside down…