Home » Uncategorized (Page 2)
Category Archives: Uncategorized
Are Accolade customers violating the Consolidated Appropriations Act?
Addendum June 2024: Looks like Aon did it again, this time for Lyra. Aon is clearly the go-to actuarial consulting firm for any vendor that needs savings to be “shown,” rather than earned.
Executive Summary
The Consolidated Appropriations Act includes civil penalties for private-sector employer fiduciaries who misspend their employee health funds. That would include spending significant sums on vendors whose own data, in plain view, demonstrates they lose money.
As the graph above shows, Accolade claims to save 8.3% before fees on one study group of their clients and 7% on a second study group of their clients, according to a “validated and rigorous” study available right here.
How do they do this? According to their grammatically challenged website:
If that’s the case, one would expect people who need support to trend much more favorably using the Accolade support resources than people who don’t.
However, the reverse turns out to be true: in neither Accolade group “studied” by Aon did what we will term the “support-sensitive” cohorts (high-acuity) outperform the no/low-acuity cohort – people who would mostly have no reason to seek support. One would expect the opposite. Compare the expectation on the left to the reality in the two sets of bars on the right, showing no net savings (even a negative variance) for the support-sensitive cohorts vs. the cohorts not needing support, in both groups studied:
Accolade is fully aware that this is the true outcome, having declined my offer of a million dollars to show that my analysis is wrong. This puts all their client fiduciaries using this particular service (they also offer benefits guidance to employees and a few other things, where savings are not claimed) in the awkward position of being personally liable for funds misspent on Accolade.
So how did Accolade generate “validated healthcare cost savings” for people who don’t need their services and likely never contacted them?
It turns out that their consultants, Aon, made rookie mistakes in concocting these savings. Once those mistakes are corrected, the opposite result appears, as represented by the 2 sets of bars on the right above.
They violated rules of fifth-grade arithmetic and basic biostatistics.
Arithmetic: Aon misunderstood how to average 5 unequal numbers
Cohort #1: They save money by “supporting” people who have pretty much no need for support
When averaging the 5 categories of morbidities, Aon “forgot” that since most people have nothing or very little wrong with them and only a few people are really sick, the number of people in each of those categories should be weighted differently when calculating an average. Aon averaged the very unequal cohorts the same, instead of weight-averaging them.
Let’s use an extreme example to illustrate this mistake: Suppose 19 of 20 kids in a class score 100% on a quiz, while 1 scores 0%. The average score is obviously 95%. Applying the “methodology” that Aon used for this report, where the single kid scoring 0% counts as much as the 19 scoring 100%, Aon would say the average score in that hypothetical is 50%, not 95%. Aon would then conclude the average student flunked, when in reality 95% of these students got A’s.
The entire answer reverses due to Aon’s rookie mistake.
What Aon did for Accolade was quite similar: they equal-weighted the phantom “savings” from the 18,062 people with virtually nothing wrong with them (and hence would have no reason to call Accolade for “support,” and wouldn’t generate savings anyway because they weren’t spending much) with the 3831 support-sensitive people who had 2 or more comorbidities:
Let’s graph this up and watch what happens. First, a simple graph of those numbers. For some reason if you have 3 comorbidities, their magic doesn’t work, but I’ll leave that for health services researchers to dissect:
Collapsing those three bars on the right into one weight-average yellow bar for the support-sensitive members with multiple comorbidities yields:
Next, let’s collapse the two blue bars into one weight-average blue bar for people who (possibly with a few exceptions) would have no need for Accolade support:
Here is the difference:
- Aon-miscalculated incorrect savings: 8.3%
- Actual correctly calculated variance in trend between the 3831 multiply comorbid support-sensitive people with 2 or more things wrong with them and the 24,849 healthier employees with nothing or almost nothing wrong with them: 0.1%
Aon therefore overstated savings-before-fees by 830%. The overstatement of savings-after-fees is not calculable since putting fees into the equation causes significant losses…and hence likely CAA liability for their customers.
Cohort #2: They save money by “supporting” people who have absolutely no need for support
In the second study group, the weight-average of all 4 morbid cohorts shows that they actually lost money for their customers even before fees, as compared to people with zero health issues for Accolade to “support.” Here is the graph, with the raw analysis provided at the end:
In this study group, as one would expect, the more morbidities a person has, the better job Accolade does at supporting them. And yet somehow they managed to save 7% by allegedly supporting the 64% of the population who mostly didn’t have anything that needed supporting.
Further, of the people who did have something wrong with them, the vast majority had only one chronic condition. That means when you weight-average all the morbidity cohorts and compare the support-sensitive people to the people who (likely with an exception here or there) don’t need Accolade’s help, you get:
To summarize:
- Aon-calculated phantom “savings” for the 66,104 people with nothing wrong with them, who spend very little money on healthcare and who most likely would never need or contact them for care: 7.0%
- Aon-calculated savings for the people whom Accolade might have talked to: 6.6%
- Actual corrected negative impact of possibly having a reason to contact Accolade: minus-0.4%.
They lost money even before fees are subtracted, according to their own data:
There is only one way to save money vs. trend on people whom you don’t interact with and who have nothing wrong with them, and that’s to inflate the trend. Inflating the trend is a staple of the vendor industry.
Biostatistics: Aon used an obviously invalid control group
Just like weight-averaging is Actuarial Science 101, knowing how to set a trend is Biostatistics 101. You get the same result biostatistically as actuarily. Let’s look at it biostatistically.
The baseline trend should not be one that Aon alleges would have happened anyway based on some cherrypicked random unnamed companies. Rather, it should be the trend of the 0 (or 0 and 1) morbidity categories in the companies that signed up with Accolade. There is virtually nothing that care navigation/support can do for people who mostly (Study Group 1) or totally (Study Group 2) don’t need care navigation/support because they aren’t spending money on high-cost care to begin with.
But everything else is the same if the same companies are used as the controls, making “same companies” the correct control group. You are holding everything constant except the possibility of benefiting from care navigation. Isolating those who would qualify for care navigation allows the reader to see the impact of care navigation when everything else is held constant.
The central tenet of a “control group” – which Aon would know if they’ve read either of my award-winning books or taken a basic course (mine or anyone else’s) in study design – is to be “identical in all respects” except the respect you are evaluating. Using cherrypicked anonymous random companies as a control is exactly the opposite. Here is an easily understood table that explains how to do basic case-control analysis, using the “identical in all respects” standard:
The percent increase in costs in the zero (or 0-1, in Accolade’s first cohort) morbidity group would therefore be the obvious proxy for the likely percent increase in costs of the comorbid categories if they did not have access to Accolade’s support.
Using the same-company control group methodology yields the following:
| Same Company | ||
| Control Group Savings | Support-sensitive Group Savings | |
| First Cohort | 8.3% | 8.4% |
| Second Cohort | 7.0% | 6.6% |
You get the same result when you fix this biostatistical mistake as when you fix the weight-average arithmetic mistake. Even before fees are subtracted:
- Cohort #1: There is no statistically significant cost savings on support-sensitive group;
- Cohort #2: The support-sensitive group trended 40 basis points worse.
Despite my requests, Accolade has not corrected this on their website. If you’d like to ask them, the email to request the correction (or to suggest taking me up on my million-dollar bet) is Steve.barnes@accolade.com or steven.barnes@accolade.com.
If you are an advisor and you have private-sector clients using Accolade, you might want to let them know this looks like a prima facie violation of CAA.
Also, I am not an expert in SEC regulations, but I think they would need to disclose that their front-page go-to “validated” and “rigorous” savings claim analysis is incorrect because the 5 categories of morbidity are averaged instead of weight-averaged.*
*Source: any fifth-grade math teacher.
8 Things to Know about Splenda [SPOILER ALERT: Contains grossness]
No one has ever claimed that artificial sweeteners are a health food. The argument usually runs: “Well, they aren’t as bad as sugar.”
Or are they? We aren’t going to “take sides” here generally, but rather just make 8 observations – 4 pro and 4 con – regarding the study just released that recommends not consuming Splenda, the brand name of sucralose. Let’s start with the arguments against Splenda.
1. The study claims some very specific harms and risks
To quote a cogent summary of the study in MedicalNewsToday:
The results showed signs of genotoxicity. The researchers also found that sucralose caused leaky gut or damage to the gut lining. In addition, they observed the genetic activity of the gut cells and discovered that sucralose caused an increase in gene activity linked to oxidative stress, inflammation, and carcinogenicity.
The linkage to leaky gut syndrome – in which digestive matter well on its way to the usual exit venue leaks into your perineum – was the one we found to be, to use a technical clinical term, the grossest.
2. The quantities involved were realistic
This isn’t one of those studies where they gave rats 10,000 times what a human would consume and found an elevated risk of whatever, like toenail fungus or uromysitisis.

The European Food Safety Authority has a threshold of toxicological concern for all genotoxic substances of 0.15 micrograms per person per day…The trace amounts of sucralose-6-acetate in a single, daily sucralose-sweetened drink exceed that threshold. And that’s not even accounting for the amount of sucralose-6-acetate produced as metabolites after people consume sucralose.
The other six can be found here.
Please put comments here: https://www.linkedin.com/posts/al-lewis-%F0%9F%87%BA%F0%9F%87%A6-57963_8-things-to-know-about-splenda-a-study-just-activity-7074713333819416577-A6sH?utm_source=share&utm_medium=member_desktop
Review of “A Cure for the Common Company,” by Dr. Richard Safeer
Here’s something you don’t see every day: a book slamming wellness written by someone in charge of wellness at their organization.
In this case, it’s Dr. Richard Safeer, who is the Chief Medical Director, Employee Health and Well-Being for Johns Hopkins Medicine and its 60,000+ (formerly) soda-loving employees.
Eschewing and verily even dissing (“it’s a band-aid”) the old “pry, poke and prod” wellness model breathlessly promoted by fellow Hopkins employee Ron Goetzel before his own heart attack slowed him down, Dr. Safeer writes of his continuing and generally successful quest to create not just a culture of health at Johns Hopkins, but actually a healthy culture. The latter includes the former but goes much farther. You can have the healthiest employees in the country but if your culture is unhealthy, that won’t do you any good. And a poisonous culture will also impact the actual health of even the healthiest employees. (That happened to me three times in my earlier days.)
I would strongly recommend this book to anyone who wants to know what it’s like in the trenches, trying to nudge a (very) large organization into healthier habits. It’s not remotely as easy as all those smiling faces on wellness vendor websites would have you believe. As the person in charge of wellness, you have to do your job almost totally by the strength of your ideas and persuasion, since less than 0.1% of those employees report to you, and in any event you can’t force people to be healthy. (Although there is a saying: “Wellness programs will make employees happy whether they like it or not.”)
As with most of these books although more so due to his first-hand experience and observations, the strength is in the storytelling. Starting with his first week on the job, when he noticed a big red fire truck outside the ER. Only it wasn’t a fire truck. It was a soda truck, doing quite the robust business. (He also noticed that on campus, soda cost less than water. Reversing that was an early success.) And, later, there was a fundraiser for the American Diabetes Association – in the form of a bake sale.
It’s not just about the broccoli: It’s the “building blocks.”
Those are examples of nudging Hopkins towards healthier eating, building awareness of the perils of added sugar among people who should know better already. But two-thirds of the book is about the far more important task of creating a healthy culture through building blocks. This is a dramatically different approach than the typical flavor-of-the-month “challenges,” like who can crash-diet the fastest or drink the most water.
Quite the opposite, there is some behavior change science that when applied intentionally and methodically – and slowly enough to avoid pushback while building consensus – will be far more impactful in the long run. Pursuing wellbeing in the workplace is less about what an individual is doing for themselves (“challenges”) and more about what the organization, leaders and co-workers are doing together so that everyone feels supported and everyone benefits. Many of the stories and lessons in these chapters are about cultures, “sub-cultures,” peer cultures and “culture-killers.”
Some of the stories in these chapters are very relatable, at least for me, particularly the last. Three times I’ve fired people who were “culture killers,” as Dr. Safeer calls them – and three times the output from the remaining staff increased immediately. One of those people was so poisonous that when I prepared to fire him over the phone, I bought one of those recording apps and drove to Rhode Island, since I thought he might threaten me over the phone, and recording calls is illegal in Massachusetts. It took half a day to make a five-minute call. (On the other hand, the surf was up, so the other half-day was well-spent.)
His reaction was the opposite – that he was expecting the call. And that brings me to my own observation about culture and hiring and firing. HIring is like a civil trial–the weight of the evidence. Firing is like a criminal trial–beyond a reasonable doubt. We often wait too long to fire people who are disrupting the culture.
The overall message of A Cure for the Common Company: If we were each able to improve our own habits and maintain a positive outlook on our own, we would have done that by now. Yet it’s 2023 and our workplaces are still taking the same approach of telling the employee this is your problem because you weigh too much etc. A diametically different approach is needed, particular in an era when employees have so many choices of where to work…and that’s what’s laid out in these pages.
Healthcare Heroes of 2022
Yes, I know. You read TheySaidWhat for the same reason you rubberneck. You simply can’t look away. You were hoping this week we would be publishing the annual Deplorables Awards.
In 2022, we are not bestowing any Deplorables Awards because at this point most of the vendors who would otherwise qualify have crawled back into their holes. Nonetheless, anxious readers need not be walk away empty-handed. There are enough Deplorables Award winners from 2021 and previous years (separate links) that I think we’re gonna need a bigger basket. Yes, there are that many vendors striving to outstupid their competitors to win the Race to the Bottom in the quest to give cluelessness a bad name.
Healthcare Heroes of 2022
Last year, albeit with a less catchy moniker (“Healthcare Heroes”), we also started recognizing people and organizations that were doing the opposite, and helping to reduce healthcare waste, corruption and misinformation. Last year’s Healthcare Heroes can be accessed here, and would all requalify this year.
One addition to the vendors list would be Virta. I originally thought that all diabetes vendors were like Livongo, with their sleight-of-hand savings claims, and Better Business Bureau and Amazon ratings and reviews (“Obviously developed by people who are clueless about diabetes”) that could make Dr. Duntsch blush.
Consequently, I started out being totally skeptical of Virta’s claims…but now am quite literally putting my money where my mouth is supporting those very same claims.
This year, we are going to highlight winners in two categories we overlooked last year – government and unions – that have done the most to reduce healthcare waste and corruption.
From the government sector
We often say that we can’t rely on the government to tackle healthcare waste…until, apparently, we can. But that’s only because of the concerted efforts of a few committed people on both sides of the aisle, people who not just believe that the role of government is to curb corruption, but also do it.
First on that list would be Virginia McMillin, a longtime Senate HELP Committee staffer, when Sen. Alexander (R-TN) ran it. I first met Virginia in 2015. The circumstance was that Rep. Foxx (R-NC) was doing the bidding of the American Benefits Council to railroad the Preserving Employee Wellness Programs Act through her House committee. This bill would have given employers the right to, among other things, collect employee and family DNA in the name of wellness and fine people who didn’t measure up. I asked Tennessee resident and uber-concerned citizen Sally Pace to invite her to the World Healthcare Congress, where we were jointly able to convince her to make sure that bill – corrupt even by the standards of healthcare – died in the HELP Committee.
Ms. McMullin is recognized today because she was clearly the lead dog in the Consolidated Appropriations Act too. The full backstory is here, courtesy of Dave Chase. The highlights:
It wouldn’t have happened if Virginia Heppner McMillin and her Senate staff colleagues hadn’t done the hard work to understand the deep conflicts-of-interest and dereliction of fiduciary duty in employer #healthplans.
Upon reading the explosive Marshall Allen ProPublica expose about how benefits brokers had up to 17 undisclosed revenue streams, Virginia dug in to understand how this was possible and the devastating ramifications. In short, the perverse incentives are a key enabler of why healthcare became the #1 driver of inflation, debt, poverty and bankruptcy for over two decades.
It’s not only difficult to understand the #healthbenefits industry, they had to craft legislative language that would get a hearing, be durable against attacks during Senate hearings and survive legal challenges.
Through their success, every non-government employer in the U.S. is now legally required to report on direct and indirect forms of compensation. Failing fiduciary duties is a personal legal liability for CFOs, CEOs and board members not covered by Directors & Officers liability policies which gets their attention.
On the local politics side, kudos to Julie Menin, the New York City Councilperson (from the Yorkville district where, as coincidence would have it, I cast my first vote at Julia Richmond High School), who recently introduced legislation to find as much as $2 billion in savings by auditing exactly how much city workers are paying for their health care at various hospitals, and making recommendations on ways to lower the prices. All told, roughly 10% of New York City’s entire budget goes to employee healthcare costs.
Extra kudos for this Profile in Courage because the largest employer in her district is also the most rapacious hospital, New York-Presbyterian’s Weill-Cornell Medical Center. This is what elected representatives are supposed to do – stand up to the monied interests on behalf of their constituents. And yet very few do.
Rep. Foxx’s wellness scam was just the opposite: probably not one single person in her district was asking for a bill to let their employer collect their DNA. But her financiers were, and that’s what counts.
The Union Runner-Up…
As they generally enjoy much more generous benefits than others, union members have traditionally not considered wages plus benefits to be a zero-sum game. Especially in the private sector, the connection between the two has been so attenuated that the unions would be 100% correct in that conclusion. (There is also the tax issue. Wages are taxed. Health benefits aren’t.)
The corollary would be that unions would naturally distrust the idea that if they make concessions in benefits that are generally painless but save money, they could get much more back in wages. Management is largely to blame, because most proposals they put forward require that the union give up a significant benefit, as opposed to one such as the aforementioned Livongo, to get higher wages.
But when was the last time you saw management propose a reduction healthcare costs by adding a free benefit that demonstrably saves money?
One easy example: why not cover silver diamine fluoride for cavities at 100%, or even just cover it at all? Quizzify has posted at length about this. Everyone wins except the dentist. My wife’s plan doesn’t cover it at all…and yet the $39.92 I paid for my own cavity was half my 50% responsibiltiy for a drill-and-fill.
Or, as we covered last year, why not give pregnant employees free access to PreTRM, which (assuming the results are followed with a high-risk maternity program) achieves almost Nobel Prizeworthy increases in gestational age.
No, it’s always about us vs. them in healthcare negotiations.
Fortunately, there are two cases where unions and management are often on the same side of the table in health benefit strategies, and hence earn the awards for 2022.
The runner-up is New Jersey. The state runs and finances the health plan, with a commission comprised partially of representatives of several unions. The unions have put their most knowledgeable representatives on this Commission, representatives who are generally much better versed in healthcare financing than most benefits consultants, with none of the latter’s conflicts of interest. (The fact that certain vendors even exist is testament to the benefits consulting industry’s conflict of interest. Some choose instead to “partner” with vendors, so that they can collect money from both parties to a transaction.)
The magnitude of New Jersey’s spend is visible enough that the unions and management are generally aligned in demanding that the carrier and the hospital cartel (ER costs in New Jersey are #1 in the nation!) come to heel. I expect that 2023 will see them in the Winner’s Circle in this column.
…And the Envelope Please
But for 2022, the winner is the 32BJ Health Fund. By federal law, this Welfare Plan is run by Trustees with equal representation from management and labor. Along with the trustees, the directors who run this fund are very knowledgeable and conscientious. Most importantly, they realize that a dollar saved is a dollar earned, meaning every dollar they save gets back to the members.
There are two reasons for this realization.
First, savings specifically go back to the rank-and-file. They don’t line the pockets of management. And not just on a small scale. The 32BJ Health Fund was able to (among other things) return a $3000 bonus to all eligible members by (also among other things) removing the aforementioned New York-Presbyterian Hospital system from their network. (Groups that remove hospitals can avoiding overcharging with Quizzify’s ER Sticker Shock Prevent Consent, to pay a very reasonable 200%-of-Medicare price for all non-electives, even out of network.)
Second, because 32BJ consists largely of people at the lower end of the pay scale, benefits are an outsized portion of total compensation and hence get far more attention than, for example, at the United Auto Workers, which has shown no particular interest in looking for win-win opportunities to reduce cost. Nor have the auto companies proposed any.
Whether you are union or management looking for solutions that perhaps your consultants have overlooked, or if you want to nominate a vendor for a Deplorables Award, you can contact me directly through this blog.
Please put comments on Linkedin instead of here. I don’t moderate these posts for comments.
Webinar: “The Answer” on the colonoscopy controversy (Nov. 10)
Register here to attend (or view the recording of) this November 10 webinar (1:00 PM EST).
No doubt you’ve seen the latest on colonoscopies, from the New England Journal of Medicine. The bottom line, so to speak, is that they don’t do anything…or do they?
It was a large, multiyear randomized control trial, which would normally be considered the Gold Standard. And it concluded that the rate of colon cancer cases was not much lower in the invited group than in the control group, while the all-cause death rates were almost identical. It was good enough for the leading medical journal in the world, so shouldn’t it be good enough for us?
However, the subset of the invited population that actually got the colonoscopies did enjoy a much more noticeable reduction in cases and colon cancer death rates. (All-cause death rates barely budged.)
But as we’ve learned in wellness, comparing participants to a reference or “passive control” group is not valid, because participants always outperform non-participants.
And we can credit Ron Goetzel’s brilliant analysis for that. He proved that participants will outperform non-participants even when there isn’t a program to participate in, as noted below, where the groups were separated in 2004 but the program didn’t start until 2006…at which point the participants had already dramatically outperformed the reference group.
And yet, this time the rebuttalists may have a point. Unlike in wellness, where there is clearly a state-of-mind difference between people who want to participate in a program and people who don’t, it’s hard to imagine that same amount of difference in a decision to get a colonoscopy. Some people just don’t want them, even if they are otherwise conscientious about patrolling their health.
Needless to say, we have deconstructed the pro and con arguments – and found the optimal solution for employers, a solution that we are pretty darn sure (and will poll the audience to confirm) you have overlooked…and yet are likely to implement once you see the webinar.
Solving the colonoscopy controversy
No doubt you’ve seen the latest on colonoscopies, from the New England Journal of Medicine. The bottom line, so to speak, is that they don’t do anything…or do they?
It was a large, multiyear randomized control trial, which would normally be considered the Gold Standard. And it concluded that the rate of colon cancer cases was not much lower in the invited group than in the control group, while the all-cause death rates were almost identical. It was good enough for the leading medical journal in the world, so shouldn’t it be good enough for us?
However, the subset of the invited population that actually got the colonoscopies did enjoy a much more noticeable reduction in cases and all-cause death rates.
But as we’ve learned in wellness, comparing participants to a reference or “passive control” group is not valid, because participants always outperform non-participants.
And we can credit Ron Goetzel’s brilliant analysis for that. He proved that participants will even outperform non-participants when there isn’t a program to participate in, as noted below, where the groups were separated in 2004 but the program didn’t start until 2006…at which point the participants had already dramatically outperformed the reference group.
And yet, this time the rebuttalists may have a point. Unlike in wellness, where there is clearly a state-of-mind difference between people who want to participate in a program and people who don’t, it’s hard to imagine that same amount of difference in a decision to get a colonoscopy. Some people just don’t want them, even if they are otherwise conscientious about patrolling their health.
Needless to say, Quizzify has deconstructed the pro and con arguments. You can read about that here in Part One of our series on colon screening.
Or you can cut right to the chase and see our elegant solution here in Part Two. You’ll see in Part Two that our solution has, in the immortal words of the great philosopher Zero Mostel, something familiar, something peculiar, something for everyone. It’s…
Outcomes Measurement for Dummies…and Smarties
There is an old joke: “How can you tell if a vendor is lying about ROI?”
Answer: “They’re claiming an ROI.”
That’s not entirely accurate. Some vendors really do achieve savings. This is particularly true with vendors that just reduce the cost of something you need to buy anyway, like Quizzify for ER visits, where in most states 50% reductions in ER Bills are routine for employees using Quizzify2Go. Or Diathrive for diabetes supplies. Or any number of vendors for drugs, my personal choice being Drexi/AMPS, home of the $2.76 90-pill Ambien supply.
But, for behavior-change vendors, you need to be the judge to translate vendorspeak into English. Examples:
- “We retained independent actuaries to validate our savings” translates as “We paid off some actuaries to fabricate our savings.”
- “We reduced the risk level for many of the highest-risk employees” omits “…but they would have come down anyway due to regression to the mean.”
- “We achieved tremendous savings vs. trend” needs the asterisk: “because how we choose the trend determines the savings.” (This one, by the way, was a real quote from a well-known consultant.)
- “We compared participants to matched non-participants, and found that…” Hard stop. Invalid.
Fortunately, in the next 3 weeks, you’ll have four opportunities to learn how to distinguish valid from invalid measurement.
On August 31 at 2 PM EDT, I’ll be joining Virta Health on a webinar, kicking off the discussion with this very topic: Outcomes Measurement for Dummies…and Smarties. And you can learn why Virta is so valid that I am putting up $100,000 as a “Challenge” for anyone who thinks another vendor’s outcomes are better. Sign up here.
If you really want to dig deep, join Health Benefits Nation in Orlando, hosted by The Validation Institute. This September 14-16 conference in Orlando covers the gamut, but specifically, there is a two hour session on valid measurement starting at 1 PM on the 14th.
Besides digging deep, the Validation Institute is my forum for, uh, naming appellations and kicking posteriors. And of course, I invite people to sue me if they don’t like what I have to say about them.
We will also have case studies where you need to spot the lies, rather than have me tee the lies up for you. Because that’s what happens in the real world. Lies told by “independent actuaries” aren’t going to invalidate themselves.
Actually, they usually do, but only if you look hard enough…and that’s what this session is all about.
One more vendor is about to join the pantheon of validity, Medencentive. I’ve been over their numbers up, down and sideways, as have many others. They are a health literacy vendor. Not like Quizzify (though they are a Quizzify customer) because they go deep on literacy with people who have specific conditions already, whereas Quizzify is educating on all the points where employees and healthcare meet, as the four examples on the homepage show. They will be hosting a webinar on September 13 at noon EDT. You can sign up here.
Finally, if anyone is anywhere near Houston on the 21st of September and is an early riser, register for The Healthcare Digital Dilemma, where I and several of the leading lights of the field — Josh Berlin, David Carmouche and others — to discuss this “dilemma.” You want to do more digital health…and yet you know most of it doesn’t work. We’ll be discussing how to find the solutions that do.
Webinar: Find out why I am betting $100,000 that Virta saves money
If you were one of the almost 8000 people to read the post Virta Health becomes the first diabetes vendor to save money, you may be wondering what exactly they do that no other diabetes vendor is able to do, to get me to put $100,000 at risk for the first time for a vendor other than Quizzify.
Part of the reason is that Quizzify and Virta are the peanut butter-and-chocolate of diabetes.
Virta reverses diabetes, while Quizzify helps pre-diabetics and diabetics avoid hidden sugars. That’s no easy task because 74% of processed foods contain them. This even includes some products that claim to contain “NO ADDED SUGAR,” but in fact are bursting with added sugars. So that I want to encourage our clients to adopt Virta, and vice-versa.
The other part of the reason is that their savings are measured validly. While plenty of vendors claim to save money and some vendors measure validly, very few are in the intersection set.
Virta and I and the Validation Institute are doing a webinar on valid measurement generally, and you can attend that webinar both to learn about valid measurement and to learn about Virta’s own results using valid measurement.
Please join us for that webinar at 2 PM EDT, August 31st.
Let’s Name Names for the $250,000 reward: Virta Health is better than…
This is the April 2025 update of the original 2022 reward offer. The reward offer is now open until April 15, 2026. In addition to being open tovendors covered in the seminal report highlighted below (and we’re naming names too), it is now also open to employers using those vendors as well as benefits consultants who have recommended them.
Further and most importantly, the reward is now raised to $250,000, having had no takers at $100,000 or even $200,000. The entry fee is still 10%.
It’s not news that diabetes vendors don’t really do anything. The larger the vendor, the less they do. One even earns an “F” from the Better Business Bureau. We’ll name some names below.
Here’s what is news: Virta saves money! Really. I’ve been over their numbers up, down and sideways because I was so skeptical. And who can blame me? Generally, to paraphrase the immortal words of the great philosopher the Queen of Hearts, I can invalidate six impossible diabetes vendor claims before breakfast.*
So in 2022 when the Validation Institute contacted me to say that Virta wanted me on a validation call, I said: “No, they don’t.” They dutifully reported back to the Virta folks, who said: “No, really.”
Which VI dutifully reported back to me. I replied: “Fine.”
“Fine” is one of those words whose meaning depends on the intonation. It could mean better-than-good, like in numismatics. Whereas in a marital argument, “Fine” means: “I know I’m right, but I just don’t feel like getting into it.”
My intonation meant: “Sure, if Virta wants to have their validation request eviscerated, I’ll join the call.”
However, I am pleased to report that “fine” in this case really did mean “better-than-good,” as in: “Virta has the most valid and impressive outcomes in the diabetes member management field.”
Most vendor claims incorporate some or all of the following fallacies: regression to the mean, participation bias, or “savings vs. trend.” Virta had none of those in the two studies originally submitted. (Note that one study was so successful that the control aarm was dropped, and now has a whopping 5 years of data on the study arm.)
Quite the opposite. Virta did two studies, both using methodologies that met the highest level of VI validity. As a result, in addition to the $50,000 Credibility Guarantee offered by the VI in support of their results, I am offering a $100,000 personal guarantee (note: they pay for this – terms on request) in support of the following two statements:
- Virta Health has proven it can deliver more cost savings (measured PMPM) than any other digital health point solution commercially available today, using a valid measurement methodology. This holds true regardless of condition category, including but not limited to: diabetes management, diabetes prevention, weight loss, mental health, heart health, substance abuse, women’s health, fertility care, and cancer care.
- No diabetes solution featured in the Peterson Health Technology Institute (PHTI) report that only Virta Health has demonstrated clear and significant patient benefits and the potential for positive net financial savings, using a valid methodology.
- Heres the naming-names part: This means the following vendors didn’t: Omada, Livongo, Vida, Gluuko, Perry, DarioHealth, Verify.
Anyone successfully disproving either statement to the judges (described as selected below) will receive $250,000.
Terms and conditions are listed at the bottom of this blog.
While it’s a layup to bet the farm to challenge vendor claims due to their inherently sketchy nature, my standards for putting my own money at risk in support of a vendor’s claims are extremely high. Quizzify2Go (ER visit cost reduction) and Sera Prognostics (prediction/prevention of premature birth) are the only entities I’d risk my hard-earned dollars on, and neither is disease-focused. Virta is and will likely remain the lead dog in savings amongst all disease-focused vendors.
Here are the Virta studies that gave me the confidence to offer this challenge with my own money at stake:.
- Parallel Assignment: Indiana University
A parallel study is one where would-be participants are randomly assigned to control or the study group. The randomized control trial (RCT) is one such methodology. In the case of drugs, the control group gets a placebo, so they don’t know whether they are getting the drug or not. This is called a “blinded” study. In many drug studies, even the investigators don’t know. This is called “double-blinded.”
Neither is possible in population health because you would know whether you are in a wellness/diabetes program or not. So studies must be unblinded.
Even unblinded RCTs are rarely undertaken in population health because (in addition to employers not hitherto having access to claims data) such studies need Institutional Review Board approval as an investigation before proceeding, as ERISA plans are otherwise required to offer the same benefits to every employee. (One easy way around this is to offer the intervention to all comers, but promote actively to some worksites but not others.)
Virta minimized that threat at Indiana University, because the parallel assignment took place in different sites, to minimize the chance that (though the consent included the possibility of being in the control group) one diabetic employee might demand the intervention that the others are getting, once they see how helpful it is. Take a look at some of the results for those completing two years with Virta as compared to the parallel control, with both arms experiencing a similar lost-to-followup:
- HbA1c was reduced by 0.9 points on average
- Weight loss averaged more than 10%
- Prescription medications were cut in half, including an 81% reduction in daily dose of insulin
- More than $3000 savings in prescription drug cost reduction between years 1 and 2
- Wait-List (Lottery) Control
Another valid design, also used by Virta, is a Wait-List, or Lottery, Control.
The most famous natural experiment in population health using this control is the Oregon Medicaid study. Medicaid was expanded there to a higher income level, but slots were limited. People who wanted coverage had to enter a lottery. Medicaid eligibility was assessed only after names were drawn – so only for the lottery winners who completed the Medicaid application forms. That’s one of the reasons it was so important to assess effects of insurance by comparing the entire control group to the entire treatment group, rather than the subset of the treatment group deemed eligible or actually enrolling.
The researchers still assessed the effect of insurance coverage itself (not just winning the lottery) by using instrumental variables estimation, but relying on only the variation of lottery selection to identify those causal effects. (The two-year finding was that being covered by Medicaid as opposed to being uninsured didn’t appreciably change physical health status, but did quite dramatically reduce both depression and financial strain.)
In Virta’s case, the Veterans Health Administration (VHA) signed on as a client, but with a limited budget that could not accommodate all who qualified and wanted to participate. Therefore, those who were wait-listed became the natural control group. The VHA, whatever its other controversies, excels at data collection amongst veterans who stay within its system, and was able to compare the results of the actual participants to the would-be participants.
Virta’s approach delivered significant reductions in HbA1c (-0.69 points) and reduced diabetes medications fill by 34.5%. BMI, blood pressure, and even the number of outpatient visits were all reduced. Read more at the full study.
Terms and Conditions of Challenge
Selection of Judges
There will be five judges, selected as follows:
- Each side gets to appoint one, drawn from Brian Klepper’s listserve with almost 1000 people on it, from all walks of healthcare.
- Two others are appointed objectively. That will be whichever health services researchers/health economists are the most influential at the time the reward is claimed. “Most influential” will be measured by a formula: the highest ratio of Twitter followers/Twitter following, with a minimum of 15,000 followers.
- Those four judges will agree on the fifth.
Using the criteria below, judging will be based on validity of the measurement. Measurements deemed invalid, such as those described on the Validation Institute site, is a disqualifying factor, i.e., any challenge by a vendor that is not validated by the Validation Institute.
If the challenging party/vendor is deemed by the judges to have an equally valid metric as Client, the decision is made on the impact of the program in drug use reduction.
Written submissions
Each side submits up to 2,000 words and five graphs, supported by as many as 20 links; the material linked must pre-date this posting to discourage either side from creating linked material specifically for this contest.
Publicly available materials from the lay media or blogs may be used, as well as from any of the 10 academic journals with the highest “impact factors,” such as Health Affairs, published within the last five years.
Each party may separately cite previous invalidating mistakes made by the other party that might speak to the credibility of the other party. (There is no limit on those.)
Oral arguments
The judges may rule solely on the basis of the written submissions. If not, the parties will convene online for a 2.5-hour virtual presentation featuring 10-minute opening statements, in which as many as 10 slides are allowed. Time limits are:
- 30-minute cross-examinations with follow-up questions and no limitations on subject matter;
- 60 minutes in which judges control the agenda and may ask questions of either party based on either the oral or the written submissions;
- Five-minute closing statements.
Entry process
The entry process is:
- Challenger’s entry fee and Service Provider’s full exposure deposited into escrow ($25k for the Challenger, and $250k to the Service Provider). Each party forwards $10,000 to the judges as well, as an estimate of their combined fees and/or contributions to their designated nonprofits.
- If the Challenger or Service Provider pulls out after publicly announcing an application, the fee is three times the amount deposited.
- The escrow is distributed to the winner and the judges’ fees paid by the winner are returned by the judges to the winner, while the judges keep the losers’ fees. (This challenger fee goes to the judges.)
Other
The competition is open to any wellness, diabetes or disease management vendor outcomes claim by any vendor, consultant or employer related to a vendor in the PHTI report made before April 15, 2026. This date may be updated by Service Provider from time to time.
*Alice laughed: “There’s no use trying,” she said. “One can’t believe impossible things.”
“I daresay you haven’t had much practice,” said the Queen. “When I was younger, I always did it for half an hour a day. Why, sometimes I’ve believed as many as six impossible things before breakfast.”
Bringing your dental benefit into the 21st Century
Time to bring your dental benefit into the 21st Century. You may not look twice at it because it’s a small part of your spend, but it’s a big part of your employees‘ spend.
Further, unlike most of the stuff we cover in Quizzify, dental issues don’t go away on their own. Quite the opposite, they get worse in an exponential manner. An ignored 50% covered tooth issue can become a much larger 80%-covered medical issue.
Fortunately, there is plenty you can do about this at pretty much no cost. Read our article in Employee Benefit News, and then join the Linkedin conversation here (rather than comment below).






























