They Said What?

Home » HERO » Measuring Wellness-Sensitive Medical Events: The Grand Finale of the HERO Analysis (part 2 of 3)

Measuring Wellness-Sensitive Medical Events: The Grand Finale of the HERO Analysis (part 2 of 3)

Do you know whether heartburn pills are safe for long-term use?

Quizzify knows. Click to learn more.

London fireworksThe first post in this three-part conclusion to our analysis of the HERO report is here.

Part Two: How to Do WSME plausibility-testing

This is the installment where we actually offer useful guidance.

All the tools and guidance are available from our site (for a fee in some cases — but no fee for employers requesting this information for their own populations) but here is the simple process (minus a few asterisks):

  • Make a list of the key ICD-9 codes. Since you would do smoking cessation with or without a wellness program, we recommend taking out COPD and chronic bronchitis.  Likewise, if you’re trying to save money, take out asthma, where the “number needed to treat” and the cost of medications means spending about $10 to save $1.00 in ER and inpatient expense.
  • Count the number of events in the codes. We use an Excel sheet to tally them, as seen here.  We combine ER and IP for tallying event rates for various reasons, but separate them out when calculating savings. Be sure not to double-count ER visits leading to admits, professional fees billed separately etc.

outcomes measurement screen shot

  • Apply your covered population (total covered person-months times 12) to create a rate per 1000.
  • Do it for several years going back to discern a trend and – if you don’t use our database – get together with some of your counterparts at other similar organizations to create a benchmark.

You should end up with a result that looks like this one though likely not as steep, from Providence Health Plan’s award from the Validation Institute. 

providence VI writeup

Where the HERO write-up is wrong is suggesting that this event-rate methodology be used to check other methodologies. Sure, but if this methodology shows no noticeable event reduction and another methodology “shows savings,” which one is wrong?  So why use the other methodologies at all if indeed you’re just going to have to keep paying your consultants to redo them to match this one?

Calculating Savings and ROI

Both savings and ROI are remarkably easy to calculate. For savings, just take a “standard cost” for each event and multiply it by the number of events that you believe to have been “avoided.”

For ROI just divide that savings figure by the cost of the program (inclusive of all the costs to run it including all vendor fees and incentives).

Cautions for WSMEs

First, because there are so few WSMEs to tally (as noted in the government reports analyzed in the first installment), you need a largish population to identify enough of them to discern a consistent trend.  Otherwise there will be a lot of “bouncing.” Two ways that smaller populations can smooth the bouncing are:

  1. Look at the total number of events rather than try to track individual events, and
  2. Combine multiple years in both baseline and intervention periods. So if you started in 2013, compare 2013+2014 to 2011+2012, rather than, or in addition to, drawing a graph like Providence’s above.

Second, even if you get a decline, you still need to check that decline against improvements in compliance or risk, to make sure that the decline in WSMEs was not due to good fortune but rather to a significant improvement in workforce health. (Also, consider workforce “disruptions.” Did your company merge with another?  Offer early retirement packages or buy-outs, etc?) This is a very important check-and-balance because the wellness industry is notorious for attributing tremendous declines in cost to small improvements in risk factors or compliance. (Those programs usually win Koop Awards.)

Third, you need to compare to national averages. (We can do this for you but you can also puzzle this out from government data, if you have the time or expertise.) As a group, the national average benchmark cardiometablic WSME rate tends to be stable except for cardiac events, which have been falling dramatically for years. In the <65 population, cardiac events are no longer common. Heart attacks themselves are 1-in-800 shots in the <65 population. (That low rate raises the much more basic question of why anyone would do “pry, poke, prod and punish” wellness in the first place, because HERO now agrees with us that wellness loses money.)

Hence, if you read Installment #4 of this series, you’ll see that HERO is claiming the WSME methodology shows savings in cardiac events…but they forgot to adjust for the benchmark decline. That adjustment wipes out all their savings.

Fourth, don’t double-count. You can only measure WSMEs against the combined spending for wellness and disease management. Anyone who tells you that you can measure each intervention separately is wrong. If your event rates fall, was it due to a small intervention on a large number of people (wellness) or a focused intervention on a small number of people (disease management)?

Fifth, your consultants will try to talk you out of a WSME analysis, which itself should be reason enough to do it. It will show exactly what the HERO report and government data show – that WSMEs typically represent a trivial amount of your spending, that they don’t decline much except in cardiac (which comes down anyway), and that the savings doesn’t remotely approach the cost of the program.

Plus, this analysis doesn’t cost much — another drawback from the consulting firm’s viewpoint.

Two classic vendor sleight-of-hand tricks

The vendor (and probably your consultants, especially if they “selected” the vendor, a la Staywell and Mercer) will try to wriggle out of this valid methodology two different ways. First, they will say this analysis should only be done on participants. That, of course, is exactly the wrong answer. Participants always show savings. The whole point of a plausibility test is to see if the event rate decline among participants was simply self-selection and regression to the mean, or whether it was real. If the latter, the real improvement amongst participants will drive the entire WSME rate down.

Second, they will want to “adjust” for the increase in prevalence. Do not allow this (unless you significantly changed your workforce demographics). “Prevalence” doesn’t mean anything. If you look hard enough, you can “find” people with disease. For instance, the Koop Award-winning Nebraska state program vendor “found” 40% of state employees were at high risk. Only 3% of them subsequently went on drugs, and only 3% reduced their risk factors…and yet somehow a huge amount of money was saved.

And the database to which you compare your rates doesn’t “adjust” for rising prevalence either, so the comparison is valid. The rate is the rate, period.

What happens when you use this methodology

You may show some very modest savings, like the textbook example on Page 23 of the HERO guidelines shows – 99 cents PMPM.

hero page 23 99 cents

HERO admits this doesn’t cover the $1.50 of vendor fees noted on Page 15, let alone all the other costs of wellness.

HERO list of costs

Quizzify Q in B and W

Quizzify helps employees master facts, the best  tool for reducing unnecessary care


And you can forget about covering the average $693 in incentives with your 99-cents-a-month savings. Even Quizzify, the only population health vendor to guarantee savings, doesn’t claim it can cover incentives too.

There simply aren’t enough WSMEs, and they aren’t sufficiently reducible to generate meaningful savings. Your costs aren’t concentrated in “unhealthy” employees who should be eating more broccoli. Your costs are all over the map. The mantra of our universally acclaimed award-winning outcomes measurement trade bestseller Why Nobody Believes the Numbers:

Everything in life has an 80-20 rule.  In healthcare, the 80-20 rule is that 80% of the time, there is no 80-20 rule.

That’s why focusing on one area, even one that’s all the rage, won’t move the needle. And that’s why Quizzify’s “Jeopardy®-meets-health benefit education-meets-Comedy Central” philosophy is to educate employees and change employee behavior on everything, not just broccoli.


In our third and final installment, we’ll give you the chance to download gratis a very user-friendly tool that you can use to do your own analysis of the ROI of your own wellness program.

In the immortal words of the great philosopher Dizzy Dean, don’t fail to miss it.

In the immortal words of the great philosopher Pat Benatar, hit me with your best shot.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: