They Said What?

Home » Disease Management

Category Archives: Disease Management

Virta Health Becomes First Diabetes Vendor to Save Money!

It’s not news that diabetes vendors don’t really do anything. The larger the vendor, the less they do. One even earns an “F” from the Better Business Bureau.

Here’s what is news: Virta saves money!  Really.  I’ve been over their numbers up, down and sideways because I was so skeptical. And who can blame me?  Generally, to paraphrase the immortal words of the great philosopher the Queen of Hearts, I can invalidate six impossible diabetes vendor claims before breakfast.*

So when the Validation Institute contacted me to say that Virta wanted me on a validation call, I said: “No, they don’t.”  They dutifully reported back to the Virta folks, who said: “No, really.”

Which VI dutifully reported back to me. I replied: “Fine.”

“Fine” is one of those words whose meaning depends on the intonation. It could mean better-than-good, like in numismatics. Whereas in a marital argument, “Fine” means: “I know I’m right, but I just don’t feel like getting into it.”

My intonation meant: “Sure, if Virta wants to have their validation request eviscerated, I’ll join the call.”

However, I am pleased to report that “fine” in this case really did mean “better-than-good,” as in: “Virta has the most valid and impressive outcomes in the diabetes field.”

Most vendor claims incorporate some or all of the following fallacies: regression to the mean, participation bias, or “savings vs. trend.” Virta had none of those.

Quite the opposite. Virta did two studies, both using methodologies that met the highest level of VI validity. As a result, in addition to the $50,000 Credibility Guarantee offered by the VI in support of their results, I am offering a $100,000 personal guarantee in support of the following two statements:

  1. Virta Health has proven it can deliver more cost savings (measured PMPM) than any other digital health point solution commercially available today, using a valid measurement methodology. This holds true regardless of condition category, including but not limited to: diabetes management, diabetes prevention, weight loss, mental health, musculoskeletal disorders, heart health, substance abuse, women’s health, fertility care, and cancer care.
  2. No diabetes solution—other than Virta Health—has demonstrated positive net financial savings in-year, using a valid methodology.

Whoever can disapprove either statement to the judges (described as selected below) will receive $100,000.

Terms and conditions are listed at the bottom of this blog.

While it’s a layup to bet the farm to challenge vendor claims  due to their inherentlly sketchy nature, my standards for putting my own money at risk in support of a vendor’s claims are extremely high. Quizzify2Go (ER visit cost reduction) and Sera Prognostics (prediction/prevention of premature birth) are the only entities I’d risk my hard-earned dollars on, and neither is disease-focused. Virta is and will likely remain the lead dog in savings amongst all disease-focused vendors.

Here are the Virta studies that gave me the confidence to offer this challenge with my own money at stake:.

  1. Parallel Assignment: Indiana University

A parallel study is one where would-be participants are randomly assigned to control or the study group. The randomized control trial (RCT) is one such methodology.  In the case of drugs, the control group gets a placebo, so they don’t know whether they are getting the drug or not. This is called a “blinded” study. In many drug studies, even the investigators don’t know. This is called “double-blinded.”

Neither is possible in population health because you would know whether you are in a wellness/diabetes program or not. So studies must be unblinded.

Even unblinded RCTs are rarely undertaken in population health because (in addition to employers not hitherto having access to claims data) such studies need Institutional Review Board approval as an investigation before proceeding, as ERISA plans are otherwise required to offer the same benefits to every employee. (One easy way around this is to offer the intervention to all comers, but promote actively to some worksites but not others.)

Virta minimized that threat at Indiana University, because the parallel assignment took place in different sites, to minimize the chance that (though the consent included the possibility of being in the control group) one diabetic employee might demand the intervention that the others are getting, once they see how helpful it is. Take a look at some of the results for those completing two years with Virta as compared to the parallel control, with both arms experiencing a similar lost-to-followup:

  • HbA1c was reduced by 0.9 points on average
  • Weight loss averaged more than 10%
  • Prescription medications were cut in half, including an 81% reduction in daily dose of insulin
  • More than $3000 savings in prescription drug cost reduction between years 1 and 2
  1. Wait-List (Lottery) Control

Another valid design, also used by Virta, is a Wait-List, or Lottery, Control.

The most famous natural experiment in population health using this control is the Oregon Medicaid study. Medicaid was expanded there to a higher income level, but slots were limited. People who wanted coverage had to enter a lottery. Medicaid eligibility was assessed only after names were drawn – so only for the lottery winners who completed the Medicaid application forms.  That’s one of the reasons it was so important to assess effects of insurance by comparing the entire control group to the entire treatment group, rather than the subset of the treatment group deemed eligible or actually enrolling.

The researchers still assessed the effect of insurance coverage itself (not just winning the lottery) by using instrumental variables estimation, but relying on only the variation of lottery selection to identify those causal effects. (The two-year finding was that being covered by Medicaid as opposed to being uninsured didn’t appreciably change physical health status, but did quite dramatically reduce both depression and financial strain.)

In Virta’s case, the Veterans Health Administration (VHA) signed on as a client, but with a limited budget that could not accommodate all who qualified and wanted to participate. Therefore, those who were wait-listed became the natural control group. The VHA, whatever its other controversies, excels at data collection amongst veterans who stay within its system, and was able to compare the results of the actual participants to the would-be participants.

Virta’s approach delivered significant reductions in HbA1c (-0.69 points) and reduced diabetes medications fill by 34.5%. BMI, blood pressure, and even the number of outpatient visits were all reduced. Read more at the full study.

Terms and Conditions of Challenge

Selection of Judges

There will be five judges, selected as follows:

  • Each side gets to appoint one, drawn from Brian Klepper’s listserve with almost 1000 people on it, from all walks of healthcare.
  • Two others are appointed objectively. That will be whichever health services researchers/health economists are the most influential at the time the reward is claimed. “Most influential” will be measured by a formula: the highest ratio of Twitter followers/Twitter following, with a minimum of 15,000 followers.
  • Those four judges will agree on the fifth.

Using the criteria below, judging will be based on validity of the measurement. Measurements deemed invalid, such as those described on the Validation Institute site, is a disqualifying factor, i.e., any challenge by a vendor that is not validated by the Validation Institute.

If the challenging party/vendor is deemed by the judges to have an equally valid metric as Client, the decision is made on the impact of the program in drug use reduction.

Written submissions

Each side submits up to 2,000 words and five graphs, supported by as many as 20 links; the material linked must pre-date this posting to discourage either side from creating linked material specifically for this contest.

Publicly available materials from the lay media or blogs may be used, as well as from any of the 10 academic journals with the highest “impact factors,” such as Health Affairs, published within the last five years.

Each party may separately cite previous invalidating mistakes made by the other party that might speak to the credibility of the other party. (There is no limit on those.)

Oral arguments

The judges may rule solely on the basis of the written submissions. If not, the parties will convene online for a 2.5-hour virtual presentation featuring 10-minute opening statements, in which as many as 10 slides are allowed. Time limits are:

  • 30-minute cross-examinations with follow-up questions and no limitations on subject matter;
  • 60 minutes in which judges control the agenda and may ask questions of either party based on either the oral or the written submissions;
  • Five-minute closing statements.

Entry process

The entry process is:

  1. Challenger and Service Provider deposit into escrow the amount each is at risk for ($100k for the Challenger, and $100k to the Service Provider). Each party forwards $10,000 to the judges as well, as an estimate of their combined fees and/or contributions to their designated nonprofits.
  2. If the Challenger or Service Provider pulls out after publicly announcing an application, the fee is three times the amount deposited.
  3. The escrow is distributed to the winner and the judges’ fees paid by the winner are returned by the judges to the winner, while the judges keep the losers’ fees. (This challenger fee goes to the judges.)


The competition is open to any wellness, diabetes or disease management vendor outcomes claim made before April 15, 2023. This date may be updated by Service Provider from time to time.

*Alice laughed: “There’s no use trying,” she said. “One can’t believe impossible things.”

“I daresay you haven’t had much practice,” said the Queen. “When I was younger, I always did it for half an hour a day. Why, sometimes I’ve believed as many as six impossible things before breakfast.”

Congratulations to RAND’s Soeren Mattke on PepsiCo study award

8758572616_64ec78d961_bWe are proud (but also insanely jealous) of our friend Soeren Mattke, whose PepsiCo article  was named the #2 most-read for the year 2014 in Health Affairs.  We, as our avid albeit narrow fan base may recall, ranked only #12–and even then that was just for blog posts, not articles in print.

Yes, we know it’s not always about Ron “The Pretzel” Goetzel and his twisted interpretations, but he seems to have come up with what appears to be exactly the opposite interpretation of what the PepsiCo study said.  Don’t take our word for it — we’ve cut-and-pasted both what the study says about PepsiCo’s results and what he says about the study.

Here is what the article says about the financial impact of health promotion at Pepsico:  ROIs well below 1-to-1, meaning a net financial loser, for health promotion. (DM, though, was a winner.)

mattke ROI graph pepsico

As low as these ROIs are, several major elements of cost were not available for the calculation — probably enough extra cost to literally make the financial returns so meager that even if the program had been free, PepsiCo would have lost money.

mattke ROI omitting consultant fees etc Pepsico

Clear enough?  Negative returns from health promotion at PepsiCo, even without tallying many elements of cost.  Nonetheless, Mr. Goetzel pretzelized that finding in his recent wellness apologia.  Listed under “examples of health promotion programs that work” as a program that is a “best practice” is:  PepsiCo.  It stands proudly beside the transcendant programs at Eastman Chemical/Health Fitness and the State of Nebraska.

quote from goetzel article on pepsico

We look forward to a clarification from Mr. Goetzel about how a program that lost a great deal of money on health promotion can be an “example of a health promotion program that work(s),” which we will duly print…but don’t be sitting by your computer screens awaiting it.

Pharos Innovations Produces Wellness Savings On Day One

Pharos Innovations

Short Summary of Intervention as described by company:

“Today, Health Systems, Physician Groups and Accountable Care Organizations are utilizing Pharos programs to:

  • Reduce health care costs and increase care quality
  • Increase care coordinator case loads and population penetration
  • Increase care plan and treatment compliance and improve clinical outcomes
  • Drive reduced readmissions and increased gain share bonus participation”

Materials Being Reviewed

Evaluation of Tel-Assurance Heart Failure Module

Summary of key figures and outcomes:

Pharos All-cause Inpatient Admission

79% reduction in admissions, and an 85% reduction in total costs ($4458 per patient per month falling almost immediately to $652).

Questions for Pharos Innovations:

Your savings happened immediately after the program began. No other disease management program claims that its savings are immediate and yet many programs have interventions similar to yours. What did you do differently to make you so successful?

ANS: Refused to answer

You write that to be included in the analysis during this 18-month study period, a member need only have participated for 15 days. How were you able to achieve such dramatic results over such a long period with only 15 days’ required participation?

ANS: Refused to answer

The most dramatic decline in admissions – about 90% — happened the first month (February) of the program. Are you saying that you were able to find all these members’ contact information, schedule the phone calls to the members and their caregivers to convince them to join the program, schedule initial followup calls to start trying to manage the members, make the scheduled phone calls, collect the information, get members to visit their doctors, and adjust lifestyles and medications — all by February 1 for a program starting January 1?

ANS: Refused to answer

Your “unchanged” matched cohort seems to have declined by 25% over the course of your intervention. How are you defining “unchanged”?

ANS: Refused to answer

Why did Wellpoint ask you to take their name off this study?

ANS: Refused to answer

Can you get someone at Wellpoint to endorse this program in the space below?

ANS: No one from Wellness endorsed the program in this space

If admissions declined 79% but total costs declined 85%, wouldn’t the use of physicians, labs, drugs, home care and all other services have to decline by much more than 85% in order to have the average decline in costs be 85%?   Very conservatively assuming that admissions account for only half of all costs for CHF patients, wouldn’t all other costs need to decline to about $200/month, which is much lower than a typical commercially insured person spends and far lower than a Medicare member spends?

ANS: Refused to answer

Wouldn’t such a low non-admissions spending figure mean that most patients would no longer be taking most meds or insulin, seeing doctors regularly, getting tested, participating in therapy, etc.?

ANS: Refused to answer

What did the New England Journal of Medicine get wrong when they tested your intervention and found no impact at all, which is much different from an 85% cost savings?

ANS: Refused to answer

Pharos isn’t just validated, but rather it is claimed to be strongly validated. Can you distinguish being “strongly validated” from garden-variety validation?

ANS: Refused to answer

Who did that “strong validation” and can they explain their rationale below?

ANS: Refused to answer

Why, if you can’t answer these questions that have been asked for several years now and Wellpoint has withdrawn its name, is this study still on your website?

ANS: Refused to answer

Mercer Says “Choice of Trend” Drives Savings Estimates


Short Summary of Intervention as described by company:

Mercer Health AdvantageSM – Mercer Health Advantage (MHA) allows self-funded employers to enroll their employees in new medical plans starting January 1, 2013. These programs are designed to save employers 5% or more of medical plan cost with the same plan design they have in place today. The savings come from select networks with providers chosen for their quality and cost effectiveness. Employers also gain access to dedicated MHA clinical care management with ongoing oversight and audit by a team of Mercer clinicians. Mercer plans to offer MHA to smaller self-funded employers in 2014.   Self-insured clients with Aetna need 1,000 employees, Anthem-1,500 employees or greater on WGS or NASCO claims platform and UHC must have 3,000 employees in the National Accounts segment.

Materials Being Reviewed

All publicly available Mercer outcomes reports and related materials, plus Mercer Health Advantage

Related materials:

  1. Georgia Medicaid
  2. North Carolina Outcomes Excerpts below
  3. Staywell and British Petroleum

Summary of key figures and outcomes:

Comparison of actual vs. predicted spending per North Carolina Medicaid member per month in medical home, by category of service

Predicted vs. actual by age grouping for North Carolina Medicaid enrollees in medical home

predicted versus actual by age for NCpredicted versus actual by age for NC Questions for Mercer

I: Mercer Health Advantage

Since most employers spend less than 5% of their total budget on disease management-sensitive events, how is it possible to save 5% through a disease management program even by eliminating every event with no increase in preventive expenses?

ANS: Refused to answer

If the state of Georgia were able to save 19% through APS disease management, which according to your own reconciliation APS is able to do, shouldn’t you be advising clients to use APS or another vendor instead of yourselves?

ANS: Refused to answer

If you are being retained to help a client find the best disease management solution, wouldn’t offering your own such solution create a conflict of interest?

ANS: Refused to answer

How have you determined the quality and cost-effectiveness of physicians that you “choose” for this network?

ANS: Refused to answer

II: Mercer North Carolina Patient-Centered Medical Home Analysis

The first North Carolina chart shows savings in every category. How is it possible to save money in all categories? Doesn’t some component of spending have to go up to make everything else come down? Or, as the outcomes measurement textbook says: “If you insulate your house, you’ll save money on heat, but not on insulation.”

ANS: Refused to answer

Is it possible that the reason savings appeared in all categories is that you simply chose to project a high trend, so that you could show more savings against that trend, or as you’ve said in the past: ““We can conclude…that the choice of trend has a large impact on estimates of financial savings”?

ANS: Refused to answer

If medical homes save money through more primary care reducing the need for specialist visits, why combine both categories when reporting savings?

ANS: Refused to answer

Inpatient spending fell by more than 50%, which implies that non-birth-event admissions would have fallen by more than 70%. How does this reconcile with the official government admissions data, which shows no change in admissions?

ANS: Refused to answer

There was no noticeable decline in North Carolina in the official government list of primary care-sensitive admissions during the period you analyzed. How do you reconcile that data with your own data showing massive admissions reduction?

ANS: Refused to answer

The second North Carolina chart shows that per-member per-month expenses in children under 1 year of age declined more than 50%. Since there is essentially no common chronic disease in this age group, where did the savings come from?

ANS: Refused to answer

The largest expenditure in this age group is in neonates. How does your data reconcile with the government data showing no change in neonatal admissions?

ANS: Refused to answer

How were you able to show such massive savings for this age group in your medical home analysis when this age group wasn’t eligible for the medical home?

ANS: Refused to answer

III: Mercer Georgia Analysis

Assuming that disease management-sensitive medical events account for roughly 8% of spending in a Medicaid population, how is it possible to save 19% through a disease management program?

ANS: Refused to answer

How do you reconcile your conclusion that the APS disease management program saved 19%, when an FBI investigation found that APS had largely failed to perform its disease management services?

ANS: Refused to answer

IV: Staywell and British Petroleum

Did you caution British Petroleum that the savings you validated for them was at least 100 times the savings that Staywell itself claims is possible?

ANS: Refused to answer

Did you question Staywell about how they were able to outperform their benchmark by 100-fold?

ANS: Refused to answer

Why didn’t you or Staywell provide your viewpoint when requested to, following the observations on The Health Care Blog that these savings were mathematically impossible?

ANS: Refused to answer

V: Mercer Qualifications to Do Outcomes Analysis

It appears that no one at Mercer has ever achieved Advanced or even Standard Certification in Critical Outcomes Report Analysis, either through DMPC or one for the Validation Institute that is specifically geared to benefits consultants.   Has any Mercer consultant taken either course and failed, or has no one at Mercer ever taken either course?

ANS: Refused to answer

Assuming the mistakes highlighted above are innocent miscalculations and not purposeful deceptions – and with senior consultant hourly billing rates well in excess of $500/hour – don’t you think it would be a good idea to become qualified in analyzing outcomes reports and reconciliation methodologies that you are being paid to analyze?

ANS: Refused to answer

Is there another course in outcomes analysis that we are unaware of that your consultants have taken, and if so, how did they still make all the mistakes above?

ANS: Refused to answer

%d bloggers like this: