They Said What?

Home » Uncategorized » A lesson in valid measurement: Vida vs. Virta

A lesson in valid measurement: Vida vs. Virta

The Peterson Health Technology Institute (PHTI) recently published an exhaustive study demonstrating what we’ve been saying (and guaranteeing) all along: Virta is unique among diabetes vendors in that it actually has an impact, while digital diabetes vendors don’t.  (Diathrive, a diabetes supply company, also saves money but supplies are a different diabetes category.)

PHTI did a great job as far as the report went.  And Livongo and Omada have accepted their conclusions. By “accepting their conclusions,” I mean declining my bet. Sure, they did some perfunctory whining about not looking at the data the right way yada yada yada in one of those “he said-she said” articles, but at this point everyone (at least everyone with a connection to the internet) knows they lose money.

That brings us to Vida.  Until now Vida has flown under the radar screen, but the sound-alike name gives them the opportunity to be confused with Virta.  That strategy was coupled with a brilliant two-part strategy regarding the PHTI report:

  1. Don’t submit any studies to PHTI
  2. Complain that PHTI didn’t look at their studies.

To the first point, PHTI lists cooperating vendors who submitted articles. (Virta submitted 12 to Vida’s 0.)  To the second point, that aforementioned “he said-she said” article notes:

Vida says its diabetes programs are clinically proven to [reduce Hb A1c], as reflected in its body of peer-reviewed publications and the satisfaction of the “tens of thousands of members” who have found success with its diabetes program.

Not to be too semantic here on the latter point, but of course the people “who have found success” with your program will be satisfied.  The question is what percentage have “found success.”  In their case, according to this article, the percentage who dropped out or were lost to followup or didn’t complete their Hb a1c tests was…hmmm…I can’t seem to find it.  Am I missing something here?

Results: Participants with HbA1c ≥ 8.0% at baseline (n=1023) demonstrated a decrease in HbA1c of -1.37 points between baseline (mean: 9.84, SD: 1.64) and follow-up (mean: 8.47, SD: 1.77, p<0.001) . Additionally, we observed a decrease in HbA1c of -1.94 points between baseline (mean: 10.77, SD: 1.48) and follow-up (mean: 8.83, SD: 1.94, p<0.001) among participants with HbA1c ≥ 9.0% (n = 618) .

Bookmark this paragraph because we will be coming back to it.  Turns out one could teach an entire class based on this paragraph alone, comparing Vida to Virta. And Part 2 will do exactly that.


Peer Review

Here is the Validation Institute’s summary of so-called “peer-reviewed publications,”  in Part 5 of their 9-part series on validity:

Vendors have realized that prospects consider the phrase “peer-reviewed” to settle all debates about legitimacy. Part Five will take you inside the thriving peer-reviewed journal industry to show you how peer reviews are bought and sold.

Often, vendors will brag about being peer-reviewed. Most prospects of vendors will then assume that the data was carefully vetted and reviewed by independent highly qualified third parties before seeing the light of publication because, after all, no journal would ever publish an article that was obviously flawed, right?

Well, certainly not for free.

Indeed, probably 95% of journals have turned article submission into a profit center. The euphemism for this business model is “open access.” Open access means that instead of the subscriber paying to read these journals, the author pays to publish in them.  In other words, vendors are placing ads. Livongo at least had the good sense to at least pretend their journal was real, by buying some space in something called the Journal of Medical Economics, which sounds pretty legit, right?  Not open-access, right? Um…

Well, Vida didn’t even bother to pretend it wasn’t open-access when they placed an ad (technically called “sponsored content”) in JMIR Formative Research. It actually includes the word “open” in the logo…

…and lest there be any doubt about where their vig comes from, JMIR even publishes their price list, which they call  an “article processing fee.

Also, have you ever heard the phrase “investigator bias“? Well, here is the list of authors and funders. Notice a pattern?

Authors:

G. Zimmermann: Employee; Vida Health. A. Venkatesan: Employee; Vida Health. K. Rawlings: Employee; Vida Health. R. S. Frank: Employee; Vida Health. C. Edwards: Employee; Vida Health.

Funder:

Vida Health


Like a real journalist, I reached out to Vida to ask for comments. Here’s what they wrote back:

  1. We aren’t submitting any comments
  2. But then we will complain that you didn’t look at our comments

Haha, good one, Al. Actually they didn’t say the second but wouldn’t it be funny if they do?


36 Comments

  1. Matthew Holt says:

    I think i win the bet, or at least my version of it. If Diathrive saves money on supplies, it then has a positive ROI. Livongo got so big so quickly because it supplied the strips for its diabetes patients and charged basically the same amount for its services as they were spending on strips. In that case the cost of Livongo’s management services was $0 so their ROI was infiinite.

    I want the $100k in cash, I’ll meet you at the rock outside Vegas at 6am (same one as in the Hangover)

    Like

  2. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  3. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  4. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  5. […] kompaniyalari ularning ko’pchiligi buni isbotladilar. ular da’vo qilgan pulni tejamang. U buni odatda juda kulgili qiladiu hech qachon to’lamaydigan (va hech qachon g’alaba qozona olmaydi) 100 ming dollarlik […]

    Like

  6. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  7. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  8. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  9. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  10. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  11. […] a number of digital well being firms, proving that lots of them don’t save the cash they declare. He does this usually in a very funny way, together with a lot of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  12. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  13. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  14. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  15. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  16. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  17. […] of digital well being corporations, proving that a lot of them don’t save the cash they declare. He does this usually in a very funny way, together with numerous $100ok bets which he by no means pays out on (and by no means wins […]

    Like

  18. […] of digital well being corporations, proving that a lot of them don’t save the cash they declare. He does this usually in a very funny way, together with plenty of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  19. […] of digital well being corporations, proving that lots of them don’t save the cash they declare. He does this often in a really humorous means, together with numerous $100k bets which he by no means pays out on (and by no means wins […]

    Like

  20. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  21. […] elles ne le font pas. Je n’économise pas l’argent qu’ils prétendent. Il fait ça généralement d’une manière très drôleainsi que de nombreux paris de 100 000 $ sur lesquels il ne paie jamais (et ne gagne jamais non […]

    Like

  22. […] a number of digital well being firms, proving that lots of them don’t save the cash they declare. He does this usually in a very funny way, together with a number of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  23. […] of digital well being corporations, proving that a lot of them don’t save the cash they declare. He does this normally in a really humorous method, together with plenty of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  24. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  25. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  26. […] a number of digital well being firms, proving that lots of them don’t save the cash they declare. He does this often in a really humorous manner, together with a lot of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  27. […] number of digital well being firms, proving that a lot of them don’t save the cash they declare. He does this often in a really humorous manner, together with a lot of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  28. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny way, along with lots of $100k bets which he never pays out on (and never wins […]

    Like

  29. […] a number of digital well being firms, proving that lots of them don’t save the cash they declare. He does this often in a really humorous means, together with a number of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  30. […] of digital well being corporations, proving that a lot of them don’t save the cash they declare. He does this normally in a really humorous means, together with a lot of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  31. […] of digital well being corporations, proving that a lot of them don’t save the cash they declare. He does this usually in a very funny way, together with a lot of $100k bets which he by no means pays out on (and by no means wins […]

    Like

  32. […] now several digital health companies, proving that many of them don’t save the money they claim. He does this usually in a very funny wayalong with lots of $100k bets which he never pays out on (and never wins […]

    Like

Leave a reply to What’s behind all these assessments of digital health? – Raymond G. Elmore Cancel reply