They Said What?

Home » wellness doesn't work » Stick a fork in it: Participants-vs-non-participants study design is dead

Stick a fork in it: Participants-vs-non-participants study design is dead

Do you know whether heartburn pills are safe for long-term use?

If instead of randomized control trials, the FDA simply allowed drug companies to compare the results of people who conscientiously used a drug to people who couldn’t be bothered, they could save a ton of money.

Not that I want to put ideas in their head.

But they don’t allow that, for the simple reason that active motivated people will always outperform inactive unmotivated people. Absent equal intent-to-treat — meaning comparing people actually taking the drug to people who think they are actually taking the drug — you can’t distinguish the effect of motivation from the effect of the drug.

As true as that is in drug research, it is even more true in activities, such as wellness, where motivation is paramount.  And yet the standard study design –participants vs. non-participants — compares all the motivated people to all the unmotivated people.

Sure, the Wellness Ignorati will claim the groups are “matched,” and on paper maybe their demographics are the same…but you can’t “match” the state of mind between participants and non-participants, as the Ignorati well know.

This American Journal of Managed Care essay, a more formal albeit less colorful version of an earlier TSW smackdown, means that quite literally every study done using this participants-vs-non-participants design is either largely or entirely invalid.

 


3 Comments

  1. Dell Dorn says:

    very good, CLEAR and informative post. Thanks

    Like

  2. temerick479 says:

    The wellness ignorati know better than this but also know the average benefit manager won’t know the difference.

    Like

    • whynobodybelievesthenumbers says:

      They also have a vested interest in keeping the average benefits manager in the dark. That’s why the Ignorati never publicly attack my stuff, other than Wellsteps, which has since learned not to. The last thing the Ignorati want to do is shine a light on their methodology.

      Like

In the immortal words of the great philosopher Pat Benatar, hit me with your best shot.