They Said What?

Home » Uncategorized » False positives for dummies (and smarties)

False positives for dummies (and smarties)

Do you know whether heartburn pills are safe for long-term use?

Quizzify knows. Click to learn more.

Wellness vendors love to brag about the number of sick employees they find through their screenings. Screening employees against their will, overdiagnosing them, then bragging about how many of them are sick is called “hyperdiagnosis.” Most of those “newly discovered conditions,” to use Interactive Health’s phrasing, are false positives.

No surprise here, given they perform about 40 tests. Doctors, as it turns out, don’t understand how to interpret lab results. If doctors — with four years of pre-med math and science requirements, four years of medical school, and likely four more years of residency – can’t do this, it is a very safe bet that unlicensed, unregulated, unsupervised wellness vendors would demonstrate even less ability to distinguish false positives from true ones. We’re talking about outfits that can’t even distinguish the words “cessation” and “recession.”

And speaking of not being able to distinguish things, if you haven’t already done so, take the Interactive Health IQ Test.


Here is another quiz: what is wrong with this list of vaccinations from Star Wellness:


Yet another quiz, from Cerner:

This one might be a little harder, since Cerner was so oblivious that they put this screenshot right on their brochure. This person’s blood pressure is 110/85, which Cerner says is “in the moderate risk category of pre-hypertension.” They overlooked the ever-so-slight detail that the “pulse pressure” of this unfortunate employee is only 25, a level typically found only in ICU patients with DNR orders.  More likely, Cerner’s employee simply didn’t take the reading correctly, and no one at Cerner knew enough about blood pressure to realize the error before the brochure went to press.

…And yet we are trusting this industry — whose ethical and competence lapses consume 50 pages in the leading law-medicine journal — to interpret our lab results?

These examples illustrate two realms in which wellness vendors can mess up. The first is in the actual testing, as with Cerner. The second in the interpretation, as with Interactive Health. Star Wellness is in a league of its own, having outstupided virtually every other vendor in wellness, which is stiff competition indeed.


Lab test science for dummies and smarties

Scientific error is largely but not entirely based on the innate inaccuracy of finger-pricks. Wellness vendors like to tout these fairly painless blood draws because they don’t scare employees away, hence creating more revenue potential. However, finger pricks are somewhat unreliable. First, there isn’t a lot of blood to begin with. Measuring multiple blood values requires diluting the specimens so that each reagent has enough to work with. And it’s Labs 101 that every manipulation of the blood – dilution being a prime example — increases the potential for error.

Another “manipulation” is the very fact of pushing the blood through the skin. This can contaminate the blood, and also causes something called hemolysis, the rupture of red blood cells.  Hemolysis can seriously skew results for many blood values. And yet it’s extremely doubtful that your vendor has figured out how to mitigate hemolysis, largely because they’ve never heard of it. Test this hypothesis simply by asking a vendor how they mitigate hemolysis in a finger-stick test.

Don’t be surprised if you get a non-answer. Remember, wellness vendors are not trained in screening–or in anything else for that matter. One of the attractions of becoming a wellness vendor is no one is tasked with educating, licensing, regulating or supervising you in any way.  Becoming a wellness vendor is the healthcare industry equivalent of applying for a zero doc loan, except with a lower chance of being turned down.

Venipuncture is more accurate than finger-sticks, but even so, the level of training that wellness vendors receive – as little as five days, provided you have a background in “municipality administration, finance or sales” – does not inspire confidence that they will get this right.  Many variables are involved, in collection, handling, storage, transport and interpretation of the specimens.  Each sample is like a little lab experiment, one that’s been repeated many times and has a standard protocol, but is not a pure algorithm. Technique, time of day, employee activity and diet before the test, elapsed time between collection and analysis, and even ambient temperature during transport influence the outcome.


Lab test arithmetic for Interactive Health and smarties

Nonetheless, let’s give the vendors the benefit of the doubt and assume their tests are 90% accurate, because in all fairness, most finger-stick tests don’t purport to measure values that specifically involve red blood cells. Glucose is one such accurate-enough test – assuming the employee has really fasted.

Yet accurate or not, glucose is one of the least cost-effective blood values to measure. An employee in the lower part of the diabetic range now (meaning most of them) wouldn’t crash for years no matter what you do. Sure, you can find them, treat them, and get them to check their glucose daily, but it’s doubtful you’ll change anything other than how much money you spend on preventing a complication or hospital admission likely not to occur until after retirement. (Annually, only about 1 in 1000 <65 insured people end up with the hospital with a primary diagnosis of diabetes.) It also turns out that for the vast majority of diabetics (those who are not insulin-dependent), daily glucose testing is a waste of time and money.

And do you really want employees to obsess with keeping their blood sugar down? Does it increase productivity if employees have low blood sugar?

But at least the science of the glucose test itself is accurate enough, assuming the employee has fasted and assuming you retest several times before coming to a conclusion. Let’s look at some predictive tests (including mammograms) whose accuracy is more like the mainstream 90% and see what happens when you apply lab test math in those circumstances. Heart attacks are the best example. If you, as an employer, could predict and prevent a heart attack, that would be cost-effective…or would it?

Maybe 2 in every 1000 employees will have a heart attack next year. Of those, at least 1, if not more, fit one of the following descriptions:

  • already have a known CAD diagnosis, diabetes, or metabolic syndrome (and hence screening won’t yield any new insights) or
  • are among the many people whose heart attacks are completely unpredictable from a standard blood test.

Optimistically, then, you’re looking at a 1-in-1000 chance of actually finding someone this year who would otherwise not be found and would have a heart attack unless it is predicted and prevented. It doesn’t help your accuracy that the Genetic Information Nondiscrimination Act does not allow inquiries about family history, which is a very major risk factor.

Let’s further assume that — highly optimistically and despite not being able to inquire about family history — the cardiac tests that a wellness vendor runs on an employee can predict a heart attack with 90% accuracy. Anyone who could predict heart attacks with 90% accuracy would win a Nobel Prize, so this is a very generous assumption.

Watch what happens when that 90%-accurate test is performed on a population of 1000 employees. First, the good news: there is indeed a 90% chance that the employee who would infarct can be found. That’s 1 positive. Next, the bad news. A 90%-accurate test is also 10% inaccurate, which means about 99 of the other 999 employees will have a phantom heart attack “predicted.” We can send this Excel spreadsheet on request:

100 employees in total will test positive. 99 of those will be false. And yet all 100 will be sent to the doctor, and likely given a barrage of further tests and possible stents. Lab test arithmetic explains why it costs about $1-million for an employer to prevent a heart attack assuming it can be prevented at all.


What does this mean for employers and employees?

Simple: testing decisions should be left to employees and their doctors. Employers should not play doctor with their employees.

Overtesting creates overdiagnosis, which in turn requires overspending on overtreatment. Example: A bank sent its top executives to a hospital for testing. The 16 executives tested were informed of a total of 18 new diagnosis, in addition to diagnoses they already had. The bank spokesman admitted “it’s still too early to see savings,” while the hospital spokesman candidly admitted that this executive screening program “offers another source of income” to the hospital.

Next is one of the wackier ideas to come down the pike, the celebrity-fueled myth that employees should get tested for ovarian cancer. This test doesn’t approach any baseline level of accuracy, so the USPSTF rates it a D.

Even the wellness vendor that advertises it the most, Star Wellness, admits it is “notoriously difficult to detect in its early stages.”  Insurance won’t pay for the test, and as Star says, it is “not readily available from doctors.” Perhaps there might be a reason for that?

It also means that you shouldn’t encourage mammograms for the <50 population, following USPSTF guidelines. That decision can be between them and their doctors. Mammograms in younger women are notorious for generating false positives.  In one well-regarded study, of 2100 women screened for 11 years starting at age 45, 690 received a false-positive along the way, and 75 underwent an unnecessary biopsy.

On the other hand, you don’t want to discourage mammograms, on the off-chance that an employee really does have cancer. Staying out of a conversation in which you have no expertise is the best idea.

Another good rule of thumb in these situations is, look at what Interactive Health does…and then do the opposite. This is their result from testing the generally healthy population of young employees, testing them for 40 things, 39 of which (except blood pressure) are not recommended for young employees due to the preponderance of false positives. So their rate of positives is not much lower than their 45% rate overall.

They are, of course, fully aware that most of their positives are false. I know this for two reasons.

  1. Here is a list (scroll down) of the top 25 hospitalizations for people insured by employers, in order. Obviously, if 38% or 45% or whatever huge percent really had these “newly discovered conditions” and the employer didn’t discover them, they’d crash, right?  Do you see anything on here that could have been “discovered” and prevented by a wellness vendor?
  2. They also follow my postings, where I have made this quite clear, Exhibit A being this one. And as Confucius said: “If you don’t correct a mistake after it is pointed out, you are creating a lie.”In that case, Interactive Health’s “mistakes” could make a White House correspondent blush.


Update October 30

Yes, Interactive Health is that stupid.

 


In the immortal words of the great philosopher Pat Benatar, hit me with your best shot.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Recent Comments

whynobodybelievesthe… on Wellness News Roundup: Are Dia…
whynobodybelievesthe… on Wellness News Roundup: Are Dia…
williammcpeck on Wellness News Roundup: Are Dia…
whynobodybelievesthe… on A vendor’s guide to snoo…
whynobodybelievesthe… on Ron Goetzel Spins Gold into St…
%d bloggers like this: