A shocking statistic just appeared on Linkedin:

Does this concern you? “A small share of doctors, 11 percent, accounted for about 25 percent of the complications. Hundreds of surgeons across the country had rates double and triple the national average. Every day, surgeons with the highest complication rates in our analysis are performing operations in hospitals nationwide.

Darn right it concerns me. It concerns me that someone who claims to be an expert (and I won’t mention a name, because it’s the holiday season, and because he is quite good at other things, and because he ordered my book so I don’t want to embarrass him) thinks this paragraph makes any sense whatsoever.  Sure, it sounds dramatic to a lay reader, but a true expert would have parsed it just a bit…

First, it is possible that those 11% of surgeons also accounted for about 25% of surgeries–and that’s why they accounted for about 25% of complications.

Second, “11% accounting for 25%” actually proves the opposite point than the one this guy is trying to make. If there were truly “bad apples” there would be an 80-20 rule. 11% accounting for 25% is about as far from an 80-20 rule as you can get.  In sharp contrast, in healthcare spending generally, 5% of the population accounted for 50% of spending last year.  And a tiny percentage of doctors account for half of all oxycontin.  Truly bad apples. 11% of doctors probably account for 90% of oxy prescriptions, I would guess.

Third, he said “hundreds of surgeons…had complication rates double and triple the national average.” Newsflash: it’s called an “average” because many people will be above it and many below it.

Indeed, with maybe 135,000 surgeons in the country, it would almost be mathematically impossible for only “hundreds to be double or triple the national average,” given that many surgeons, particularly those with low volumes of patients, would be at zero complications.

This whole thread is a perfect example of why people should read Why Nobody Believes the Numbers: Separating Fact from Fiction in Population Health Management. It teaches how to read this type of statistic using a critical eye, rather than accept stuff at face value that makes no sense whatsoever.

Note: none of this is to say there aren’t bad surgeons out there. Certainly, there are. But there are better ways to make the point than using mathematical fallacies about averages and 80-20 rules. And sometimes surgeons are really competent in the OR because they perform high volumes of surgeries…but maybe many of those surgeries shouldn’t have been performed in the first place and that’s why the surgeon is performing so many. As Peter Drucker says: “Nothing is more wasteful than doing something efficiently that need not have been done at all.”

That, of course, is what Quizzify teaches…how to avoid medical interventions that should not be done at all.

Two postscripts. First, he ends with:

Every day, surgeons with the highest complication rates in our analysis are performing operations in hospitals nationwide.

Well, where the heck else would they be performing operations besides hospitals (and surgicenters)?

Second, although he is a very capable guy in other ways, biostatistics are not his long suit. For instance, he once wrote:

“For millenia [sic], employers had no way of knowing that their employees were walking around without a clue that they had a debilitating or terminal condition…percolating in their bodies.”

I must confess I learned a lot from this exposition.  I had not realized that employers’ concerns about employee chronic disease had their roots in ancient history.  But there it is, right in the opening words:  these concerns date back “millenia,” when employers failed to get their employees tested for “percolating” conditions before throwing them to the lions.