They Said What?

Home » 2018 » October

Monthly Archives: October 2018

False positives for dummies (and smarties)

Wellness vendors love to brag about the number of sick employees they find through their screenings. Screening employees against their will, overdiagnosing them, then bragging about how many of them are sick is called “hyperdiagnosis.” Most of those “newly discovered conditions,” to use Interactive Health’s phrasing, are false positives.

No surprise here, given they perform about 40 tests. Doctors, as it turns out, don’t understand how to interpret lab results. If doctors — with four years of pre-med math and science requirements, four years of medical school, and likely four more years of residency – can’t do this, it is a very safe bet that unlicensed, unregulated, unsupervised wellness vendors would demonstrate even less ability to distinguish false positives from true ones. We’re talking about outfits that can’t even distinguish the words “cessation” and “recession.”

And speaking of not being able to distinguish things, if you haven’t already done so, take the Interactive Health IQ Test.


Here is another quiz: what is wrong with this list of vaccinations from Star Wellness:


Yet another quiz, from Cerner:

This one might be a little harder, since Cerner was so oblivious that they put this screenshot right on their brochure. This person’s blood pressure is 110/85, which Cerner says is “in the moderate risk category of pre-hypertension.” They overlooked the ever-so-slight detail that the “pulse pressure” of this unfortunate employee is only 25, a level typically found only in ICU patients with DNR orders.  More likely, Cerner’s employee simply didn’t take the reading correctly, and no one at Cerner knew enough about blood pressure to realize the error before the brochure went to press.

…And yet we are trusting this industry — whose ethical and competence lapses consume 50 pages in the leading law-medicine journal — to interpret our lab results?

These examples illustrate two realms in which wellness vendors can mess up. The first is in the actual testing, as with Cerner. The second in the interpretation, as with Interactive Health. Star Wellness is in a league of its own, having outstupided virtually every other vendor in wellness, which is stiff competition indeed.


Lab test science for dummies and smarties

Scientific error is largely but not entirely based on the innate inaccuracy of finger-pricks. Wellness vendors like to tout these fairly painless blood draws because they don’t scare employees away, hence creating more revenue potential. However, finger pricks are somewhat unreliable. First, there isn’t a lot of blood to begin with. Measuring multiple blood values requires diluting the specimens so that each reagent has enough to work with. And it’s Labs 101 that every manipulation of the blood – dilution being a prime example — increases the potential for error.

Another “manipulation” is the very fact of pushing the blood through the skin. This can contaminate the blood, and also causes something called hemolysis, the rupture of red blood cells.  Hemolysis can seriously skew results for many blood values. And yet it’s extremely doubtful that your vendor has figured out how to mitigate hemolysis, largely because they’ve never heard of it. Test this hypothesis simply by asking a vendor how they mitigate hemolysis in a finger-stick test.

Don’t be surprised if you get a non-answer. Remember, wellness vendors are not trained in screening–or in anything else for that matter. One of the attractions of becoming a wellness vendor is no one is tasked with educating, licensing, regulating or supervising you in any way.  Becoming a wellness vendor is the healthcare industry equivalent of applying for a zero doc loan, except with a lower chance of being turned down.

Venipuncture is more accurate than finger-sticks, but even so, the level of training that wellness vendors receive – as little as five days, provided you have a background in “municipality administration, finance or sales” – does not inspire confidence that they will get this right.  Many variables are involved, in collection, handling, storage, transport and interpretation of the specimens.  Each sample is like a little lab experiment, one that’s been repeated many times and has a standard protocol, but is not a pure algorithm. Technique, time of day, employee activity and diet before the test, elapsed time between collection and analysis, and even ambient temperature during transport influence the outcome.


Lab test arithmetic for Interactive Health and smarties

Nonetheless, let’s give the vendors the benefit of the doubt and assume their tests are 90% accurate, because in all fairness, most finger-stick tests don’t purport to measure values that specifically involve red blood cells. Glucose is one such accurate-enough test – assuming the employee has really fasted.

Yet accurate or not, glucose is one of the least cost-effective blood values to measure. An employee in the lower part of the diabetic range now (meaning most of them) wouldn’t crash for years no matter what you do. Sure, you can find them, treat them, and get them to check their glucose daily, but it’s doubtful you’ll change anything other than how much money you spend on preventing a complication or hospital admission likely not to occur until after retirement. (Annually, only about 1 in 1000 <65 insured people end up with the hospital with a primary diagnosis of diabetes.) It also turns out that for the vast majority of diabetics (those who are not insulin-dependent), daily glucose testing is a waste of time and money.

And do you really want employees to obsess with keeping their blood sugar down? Does it increase productivity if employees have low blood sugar?

But at least the science of the glucose test itself is accurate enough, assuming the employee has fasted and assuming you retest several times before coming to a conclusion. Let’s look at some predictive tests (including mammograms) whose accuracy is more like the mainstream 90% and see what happens when you apply lab test math in those circumstances. Heart attacks are the best example. If you, as an employer, could predict and prevent a heart attack, that would be cost-effective…or would it?

Maybe 2 in every 1000 employees will have a heart attack next year. Of those, at least 1, if not more, fit one of the following descriptions:

  • already have a known CAD diagnosis, diabetes, or metabolic syndrome (and hence screening won’t yield any new insights) or
  • are among the many people whose heart attacks are completely unpredictable from a standard blood test.

Optimistically, then, you’re looking at a 1-in-1000 chance of actually finding someone this year who would otherwise not be found and would have a heart attack unless it is predicted and prevented. It doesn’t help your accuracy that the Genetic Information Nondiscrimination Act does not allow inquiries about family history, which is a very major risk factor.

Let’s further assume that — highly optimistically and despite not being able to inquire about family history — the cardiac tests that a wellness vendor runs on an employee can predict a heart attack with 90% accuracy. Anyone who could predict heart attacks with 90% accuracy would win a Nobel Prize, so this is a very generous assumption.

Watch what happens when that 90%-accurate test is performed on a population of 1000 employees. First, the good news: there is indeed a 90% chance that the employee who would infarct can be found. That’s 1 positive. Next, the bad news. A 90%-accurate test is also 10% inaccurate, which means about 99 of the other 999 employees will have a phantom heart attack “predicted.” We can send this Excel spreadsheet on request:

100 employees in total will test positive. 99 of those will be false. And yet all 100 will be sent to the doctor, and likely given a barrage of further tests and possible stents. Lab test arithmetic explains why it costs about $1-million for an employer to prevent a heart attack assuming it can be prevented at all.


What does this mean for employers and employees?

Simple: testing decisions should be left to employees and their doctors. Employers should not play doctor with their employees.

Overtesting creates overdiagnosis, which in turn requires overspending on overtreatment. Example: A bank sent its top executives to a hospital for testing. The 16 executives tested were informed of a total of 18 new diagnosis, in addition to diagnoses they already had. The bank spokesman admitted “it’s still too early to see savings,” while the hospital spokesman candidly admitted that this executive screening program “offers another source of income” to the hospital.

Next is one of the wackier ideas to come down the pike, the celebrity-fueled myth that employees should get tested for ovarian cancer. This test doesn’t approach any baseline level of accuracy, so the USPSTF rates it a D.

Even the wellness vendor that advertises it the most, Star Wellness, admits it is “notoriously difficult to detect in its early stages.”  Insurance won’t pay for the test, and as Star says, it is “not readily available from doctors.” Perhaps there might be a reason for that?

It also means that you shouldn’t encourage mammograms for the <50 population, following USPSTF guidelines. That decision can be between them and their doctors. Mammograms in younger women are notorious for generating false positives.  In one well-regarded study, of 2100 women screened for 11 years starting at age 45, 690 received a false-positive along the way, and 75 underwent an unnecessary biopsy.

On the other hand, you don’t want to discourage mammograms, on the off-chance that an employee really does have cancer. Staying out of a conversation in which you have no expertise is the best idea.

Another good rule of thumb in these situations is, look at what Interactive Health does…and then do the opposite. This is their result from testing the generally healthy population of young employees, testing them for 40 things, 39 of which (except blood pressure) are not recommended for young employees due to the preponderance of false positives. So their rate of positives is not much lower than their 45% rate overall.

They are, of course, fully aware that most of their positives are false. I know this for two reasons.

  1. Here is a list (scroll down) of the top 25 hospitalizations for people insured by employers, in order. Obviously, if 38% or 45% or whatever huge percent really had these “newly discovered conditions” and the employer didn’t discover them, they’d crash, right?  Do you see anything on here that could have been “discovered” and prevented by a wellness vendor?
  2. They also follow my postings, where I have made this quite clear, Exhibit A being this one. And as Confucius said: “If you don’t correct a mistake after it is pointed out, you are creating a lie.”In that case, Interactive Health’s “mistakes” could make a White House correspondent blush.


Update October 30

Yes, Interactive Health is that stupid.

 

Wellness News Roundup: New EEOC rulemaking delay, obesity myths, and celebrity-fueled overscreening

Folks, if you want the news, you’re going to have to subscribe to it.” From now on, TSW is only going to carry summaries, likely a few days late now that fall frisbee season is taking up my weekends. Here’s what you missed Friday:

  1. In a completely unanticipated* move, the EEOC is pushing out its proposed rulemaking date once again, this time to June 2019. That means new rules likely won’t be formally in place until 2020.
  2. Of course what’s another week of news without yet another expert observing that wellness vendors’ obsession with weight loss is harming employees?
  3. Speaking of yet anothers, yet another celebrity is urging mass screening for yet another USPSTF D-rated screen, this time ovarian cancer. Early-stage ovarian cancer is pretty darn undetectable — except of course by wellness vendors.  Total Wellness says its test is “possibly an indicator of cancer cells,” a ringing endorsement indeed.  Another vendor pushing these screens, Star Wellness, says these tests are “not readily available from your doctor,” as though that’s a selling point.  (It reminds me of a kid I knew in high school who used to brag about not brushing his teeth.)  Maybe doctors know something that Total Wellness and Star Wellness, which thinks Vitamin B12 is a vaccine, don’t.

Read the full summary, with links, here.

Got news? Clue us in. (How do you think we found these nuggets?)

 

*Except by me

 

Vivify Springs Back to Life

In politics today, one strategy dictates: never apologize, never admit error, never correct an earlier misstatement.  If that strategy works, the CEO of Vivify would carry all 50 states, not to mention the District of Columbia. Vivify just updated their website, without correcting any of the numbers that I pointed out in 2015 simply didn’t add up. Maybe they didn’t know their arithmetic was wrong, even though presumably sometime in the last three years at least one of their employees earned their GED.

Not only did they keep the original “math” on the new website, they also kept the original grammar and spelling. Consequently, just like they updated their website by keeping the original intact, I am “updating” my own posting below simply by keeping the original intact.

With one addition: in 2015 I failed to congratulate them for reducing the total annual cost of care for people with heart failure to $1231. To put this feat in perspective, that’s about 80% less than the average annual cost of care for people whose hearts haven’t failed.


The population health industry never ceases to delight us with its creativity.  Vendors come up with ways of demonstrating their incompetence that are so creative we are compelled to use screenshots to back up our observations.  Otherwise no one would believe us.

Consider Vivify.  They reported on a study of in-home post-discharge telemonitoring led by a:

vivifyprincipalinvestigatorlessname

Not being able to spell the name of his own occupation is the good news.  The bad news is, the “principle investigator” also can’t write, can’t count, and – most importantly for someone who claims to be a “principle investigator” — can’t investigate.  (Those shortcomings aside, this is a very impressive study.  For instance, the font is among the most legible we’ve ever seen.)


The Writing

There is some redundancy in the writing, but, giving Vivify the benefit of the doubt, perhaps the extra verbiage reflects the principle investigator’s concern that someone might miss the nuances or subtleties in his exposition.  Examples:

  • Vivify’s home monitoring system is “simple and easy”;
  • The patient receives a “weight scale”;
  • They had an “ROI of $2.44 return for every dollar invested”, and…
  • “With appropriate connectivity, patients could engage in real-time interactive videoconferencing.”

Needless to say, these product attributes are very intriguing, so intriguing that you may want to learn more about the company. They are only too happy to oblige, making sure we catch yet another nuance:

vivify about us


 

The Arithmetic

The study claims the average patient’s cost declined $11,706, for a 2.44-to-1 ROI.  Doing the math, that means Vivify’s post-discharge in-home self-care telemonitoring costs roughly…let me just get out my calculator here…$4797/patient.   At that price, why rely on self-monitoring?  Why not just move a nurse in?

vivifyROIandsavings


 

The Principle Investigation

In general, Vivify targets patients with “specific chronic illnesses,” including pneumonia.  (Vivify, I don’t know how to break this to you gently, but: pneumonia isn’t a chronic illness, specific or otherwise.  No one ever says: “I was diagnosed with pneumonia a few years ago, but my doctor says we’re staying on top of it.”)

vivify penumonia

However, for this investigation, only CHF was targeted: specifically, a cohort of 44 recently discharged CHF patients with an average age of 66.  This raises the question: How did the principle investigator scrounge up a cohort of 44 discharged CHF patients with an average age of only 66?  More than half of CHF discharges are over 75.  It’s statistically impossible to randomly select 44 CHF discharges with an average age of 66.   And – isn’t this a lucky coincidence – the study claimed a large (65%!) reduction in readmission rates but readmission rates are already much lower for younger patients.   Once again, not a word of explanation.

Because Vivify’s misunderstanding of basic arithmetic and study design boggled even our minds (and our minds are not easily boggled, because we mostly blog about wellness), we decided to give them a chance to explain directly that we might have missed something. Further, because these explanations would have taken them 15 minutes if indeed we were missing something obvious, we offered them $1000 to answer them, money they decided to leave on the table.  (Anyone have questions for me? Send me $1000 and I will happily spend 15 minutes answering them.)

This email to Vivify is available upon request. [Or at least it was in 2015. I’d have to hunt for it now.]

We don’t even know what the 65% reduction is compared to.  Usually – and call us sticklers for details here – when someone claims a 65% reduction in something vs. something else, they offer some clues as to what the “something else” is.  Are they saying 35% were readmitted?  Or 66-year-olds are readmitted 65% less than 75-year-olds? Or that they scrounged up yet another group of 66-year-olds with a CHF hospitalization and compared the two groups but forgot to mention this other group?


Savings Claims

My freshman roommate was like the bad seed in the old Richie Rich comics.  Among other things, he would have a snifter of cognac before bed, whereas I had never tasted cognac and thought a “snifter” was for storing tobacco.  We didn’t get along and at one point I accused him of being decadent.

“Decadent, Al?  Let me tell you about decadent.  I spent last summer at a summer camp – everyone was there, Caroline Kennedy, everyone – where we played tennis on the Riviera for a month and then went skiing in the Alps.”  I had to admit that was indeed decadent.

“Al,” he replied.  “I haven’t even gotten to the decadent part yet.”

Likewise, we haven’t even gotten to the clueless part yet:  the savings claim.  Remember that $11,706 savings claim above?  Well, read that passage again–it turns out that represents a “90% decrease in the cost of care.”  Apparently, the patients cost $12,937 when they were in the hospital, but after they went home, they only cost $1231.  (We have no idea how that squares with the other finding, that the Vivify system itself costs $4797, based on the “ROI of $2.44 return for every dollar invested.”.)

The irony is that other vendors in this space really do save money and really do measure validly.  It’s one thing to make up outcomes in wellness. That’s a core part of the industry value proposition. But, unlike wellness vendors, tele-monitoring vendors other than Vivify typically know the basics: what they are doing, how to measure outcomes, how to save money–and how to spell “principal investigator.”

Wellness News Roundup: Shocking New Diabetes Findings, Provant’s Bankruptcy, & Too Many EKGs

We’re only gonna carry Quizzify’s News Roundups a few more weeks. If you want to see it every week, you need to follow us.


Diabetic Employees Need Not Test Glucose Daily

A study showed that non-insulin-dependent diabetics (meaning about 80% of them) do not benefit from daily blood glucose checks. These checks have no impact on HbA1c’s or quality of life. The specific wording: People who tested their blood sugar levels daily had slightly lower blood sugar levels in the middle of the study than those who didn’t test. But this difference didn’t last until the end of the study. Among people who checked their blood sugar levels, there were no differences in blood sugar levels or quality of life between those who did and didn’t receive text messages.

 

Conclusion: the inconvenience, discomfort and expense of daily monitoring – estimated at $1630/diabetic employee – is not merited. This result matches a study by the Ohio Public Employee Retirement System (available through Freedom of Information Act request) showing that making drugs, supplies, and strips free to diabetics had no impact on admissions or ER visits for diabetes, over a two-year period.


Prominent Wellness Vendor Goes Bankrupt
Provant Health’s sale of assets to Quest Diagnostics through Chapter 11 bankruptcy proceedings has been completed.  Nonetheless, Provant plans to continue its campaign of misinformation — for instance, insisting that employees drink eight glasses of water a day.

Should Employees Get Routine EKGs?

The new Apple Watch provides electrocardiograms (EKGs). This will be helpful for employees with atrial fibrillation but experts warn of a wave of false positives among everyone else. The US Preventive Services Task Force (USPSTF) gives routine EKG testing a “D” for just that reason. This recommendation covers asymptomatic, low-risk adults.

The USPSTF didn’t extend this recommendation to teenagers because of course no pediatrician would ever perform routine EKGs on children, and obviously no high school would ever be stupid enough to encourage all its students to get EKGs.



Missed last week’s news roundup, and maybe the ones before that? Here they are.


So much winning! Provant Health goes bankrupt.

Even by wellness industry standards, Provant Health’s business practices earned it our first multi-part series. It required nine parts to do them justice…

And now they are bankrupt too. Their remaining assets are being purchased by Quest Diagnostics, a reference lab company no doubt hoping to run additional lab tests on Provant’s dwindling employee base.

We knew this was coming because they had already started cutting back on spending, by dropping their internet connection, and hence didn’t realize that employees are not supposed to drink 8 glasses of water a day.

 

Or perhaps the kiss of death was being endorsed by Wellsteps,


 

Here is a real-life example of what an employer can expect to achieve by spending a million bucks on these very stable geniuses:

BMI and glucose got worse. Somehow, blood pressure stayed the same — 120 over 75. Yes, despite the average employee being almost prediabetic, and sporting a 40-inch waist for males (and close to that for females), pretty much everyone in this company has ideal blood pressure.  One explanation for this would be that Provant, whatever their shortcomings in weight control, are the world’s leading experts in all things cardio. (And that’s even before they started — when everyone in this company already had ideal blood pressure.)

That expertise might also explain how they were able to keep the cardiac ratio so low.  An alternative explanation might be that there is no such thing as a “cardiac ratio.” (Google it.)  I’ve heard of wellness vendors making up data, but this is a first–making up an entire metric to present its made-up data.  Maybe the reason Quest Diagnostics purchased them was to develop a new lab test for cardiac ratios.


The good news is that, despite these underwhelming results, Provant was able to beat its targets, which is easy enough if you move the goalposts to the 50 yard line:

So much winning! Provant “wins” if the average employee is overweight but not quite obese yet, has glucose only 74 points higher than the threshold to be considered diabetic, and is only borderline hypertensive.  No target is set for the cardiac ratio, of course, because it doesn’t exist.

In that sense the cardiac ratio is not unlike Provant itself.

 

 

 

 

 

 

 

Wellness News Roundup: Too much healthcare, too many tests…and too much prevention

Too much healthcare can be hazardous to your health, as three findings released last week have shown.


  1. America’s Epidemic of Overtreatment

Health Affairs reports an epidemic of overtreatment. Author Shannon Brownlee proposes that patients “start asking uncomfortable questions,” to determine if treatments are appropriate. The problem is that employees don’t know what uncomfortable questions to ask, unless one counts: “Do you mean to say I’m not eating enough broccoli?” The questions don’t have to be uncomfortable. They get more comfortable, the more one learns about healthcare. Seems simple enough, and yet 99% of employers still don’t offer employee healthcare literacy education.


2. America’s Epidemic of Innumeracy

It would help if doctors knew the first thing about interpreting lab test results before recommending these treatments, but apparently they don’t. The Washington Post reports that professionals reading lab test results don’t understand false positive arithmetic.  As a result, your employees may be getting diagnosed and treated for conditions they don’t have.  Wellness vendors don’t understand it either. As compared to real medical professionals, the difference is that the they take great pride in their ignorance, and brag about how many false positives they find. I’m talking to you, Interactive Health.

No, they don’t. We’ll post on false positive arithmetic next week.


3. America’s Epidemic of Overprevention

As reported on They Said What Thursday, it looks like too much spending on prevention can backfire.  Specifically, spending more on employee healthcare, and sending more employees to the doctor, does not deliver better outcomes, fewer ER visits, or even more HbA1c checks for diabetics. Quite the contrary, the correlation, though weak, goes in the “wrong” direction.


Bottom line: looks like Quizzify had it right all along: just because it’s healthcare doesn’t mean it’s good for you.

 

Comprehensive new report shows workplace wellness efforts firing blanks

The Commonwealth Fund recently released a comprehensive report comparing employer healthcare spending with various outcomes measures. The result — and you’re way ahead of me here, I’m sure — is bad news for the wellness industry. It shows basically no correlation between employer spending and outcomes. Examples:

Let’s see how all those diabetes programs are doing…

And of course, the holy grail: how are employers doing in preventing hospitalizations?

  • Spending more doesn’t get better results. It’s in how wisely the money is spent. Iowa scores way ahead in preventing hospitalizations while spending less than average, perhaps due to Bill Appelgate’s Iowa Chronic Care Consortium. Nebraska, right next door, shows the opposite, spending somewhat more and preventing somewhat less. As you recall, Nebraska put their very stable genius wellness vendors on the case and ended up on the front page of the Omaha World Herald. Not to mention ongoing color commentary from They Said What, featuring — you guessed it — Ron Goetzel, trying to explain how Nebraska’s state program got an award for lying about saving the lives of cancer patients who didn’t have cancer in the first place. Not to mention the rest of program simply threw money away: of 20,000 state employees, only 180 reduced a risk factor.

Curiously, the loser states in these analyses are not the usual loser states in health status. Mississippi scores high. Not because the outcomes are great. They are around the average. However, very few employers in Mississippi spend anything on prevention or workplace wellness to achieve those outcomes. Other states spend tons of money on vendors and doctor visits…with basically the same results as Mississippi.


The displays don’t lend themselves to cut-and-pasting, since they are interactive (by state) and for some reason the axes are rather shy when you try to copy them. Nonetheless, worth noting is the slightly negative correlation between spending and quality in the employer population.   (Quality on the left axis, spending on the bottom.)  After a point, more healthcare only makes things worse. Or, as Quizzify says, “Just because it’s healthcare doesn’t mean it’s good for you.”  Exhibit A being Interactive Health, of course. They are the most expensive wellness vendor, do the most tests, and send the most employees (45%!) to the doctor with “newly discovered conditions.” And yet they haven’t actually accomplished anything, other than getting exposed in the Wall Street Journal.

The conclusion?  Like everything else in life, there can be too much prevention.  Annual screenings plus annual doctor visits for asymptomatic and younger employees is simply way too much…and as this graph shows, too much healthcare can be hazardous to your health.

Too much healthcare and not enough healthcare education is a recipe for lots of excess costs.

 

 

%d bloggers like this: