“Springbuk does not certify the information, nor does it guarantee the accuracy and completeness of such information.”
Home » Wearable Devices
Category Archives: Wearable Devices
The 2017 Deplorables Awards — Runners Up
It’s time for the 2017 Deplorables Awards, lovingly bestowed on those vendors who do the best job making other vendors look good.
The good news is that you don’t have to actually win the Deplorables Award to sue me. Runners-up are eligible too. Here is my address for hand-service delivery most of the year:
890 Winter Street #208, Waltham MA 02451
In case you decide to sue me between June 22 and August 8, use:
8 Paddock Circle, Chilmark, MA 02535
And don’t leave out my attorney:
Josh Gardner, GARDNER & ROSENBERG P.C., 33 Mount Vernon Street, Boston, Massachusetts 02108
I don’t know how much more I can do for you, other than lick the envelope. So go for it. Don’t make me beg.
But, remember, unlike your usual business model, in court you are required to actually tell the truth (I would be happy to explain to you how that works), meaning there is no chance of your winning — or likely even avoiding summary judgment, since none of the evidence is in dispute. It’s all your own writings. Oh, and I do my own cross, which means you won’t be able to find an expert witness. Anyone who knows enough about wellness to be an expert witness also knows enough about wellness to know that attempting to defend you would be a humiliating, on-the-record experience.
And there is always the chance that some annoying jerk might blog about it…
The 2017 Runners-Up
Imagine a four-square matrix with competence on one axis and integrity on the other. The people and organizations we’ll be highlighting today would intersect with the companies mentioned in Monday’s posting at only one single point.
Springbuk and Fitbit
As many of you recall, earlier in the year we analyzed the study done by Springbuk that secretly financed by Fitbit. Or maybe I need new glasses, because I just couldn’t find the disclosure in the Springbuk report that this paean to Fitbit was financed by Fitbit, the way Nero used to have the judges award him Olympic medals.
Coincidentally, the study showed Fitbit saving gobs of money because employees taking more than 100 steps a day spend less money than those taking fewer. However, a simple tally of one’s own footsteps shows that it is impossible not to take 100 steps a day unless you are both:
- in a hospital bed; and also
- on dialysis.
This 100 steps-a-day threshold was repeated many times in the study, with no explanation of how that number came to be. However, it turns out we owe these two outfits an apology. Fitbit and Springbuk have told a number of people privately (not publicly, in order to avoid an embarrassing news cycle) that they didn’t really mean to say that 100 steps a day constituted activity. They meant to say that taking 100 steps a day implied you had your Fitbit on. My apologies for failing to read their minds that their conclusions were based on reading people’s minds to determine whether they wore the Fitbit deliberately, or simply forgot/remembered/cared to put their Fitbit on.
They never did explain — privately or publicly or to anyone — how employees who took an average number steps during the baseline year could show huge savings by taking an average number of steps in the study year too.
They also never explained how these two statements didn’t completely contradict each other, even though I specifically asked them to in a personal letter, excerpted here:
Third, can you reconcile this statement…:
“The materials in this document represent the opinion of the authors and not representative of the views of Springbuk, Inc. Springbuk does not certify the information, nor does it guarantee the accuracy and completeness of such information.”
“This demonstration of impact achieved by integrating Fitbit technology into an employee wellness program reinforces our belief in the power of health data and measurement in demonstrating ROI,” said Rod Reasen, co-founder and CEO of Springbuk.
National Business Group on Health
Next up is the National Business Group on Health. Last year they made the list for criticizing the US Preventive Services Task Force for not demanding enough screenings, in a country that is drowning in them. Not content to rest on those laurels, this year they earned an Honorable Mention for inviting Dr. Oz to keynote on the role of quackery in corporate wellness, and perhaps tell us about his latest lose-weight-by-eating-chocolate miracle diet.
Health Enhancement Research Organization
HERO of course also earns a runner-up award. 2017 will be remembered as the year they finally came to grips with the realization that a business model based on fabricating outcomes requires that perpetrators possess that critical third IQ digit. Without that extra “1”, an organization trafficking in math that can at best be considered fuzzy is going to be outed.
This year’s set of lies? By way of background, their 2016 poison-pen letter insisted they had fabricated that data set showing that wellness loses money without disclosing that it was fabricated — and also never reviewed their fabricated data before publication. Early in the year, I had the insight that, wow, this “fabricated” Chapter in their guidebook is so much better than the other chapters that something is amiss. No one at HERO can analyze data competently…and yet, here it was, a competent data analysis.
I did something I had never thought to do before, which was look up the actual author of that chapter. It was Iver Juster MD. He was a great analyst even before he read all my books, took all my courses, and achieved all my certifications in Critical Outcomes Report Analysis.
So I called Iver. Here’s what I learned:
- Whereas Paul Terry and Ron Goetzel had insisted that Iver fabricated the data, Iver said, of course he didn’t — whatever made me think that? (“If it wasn’t real, I would have disclosed that,” he observed. Of course he would have. Iver has tremendous integrity.)
- The Board discussed and reviewed his chapter at length, and made helpful suggestions, for which he was quite grateful. This review process required “countless hours,” just as the HERO document says:
The number of transparent lies HERO tells could make a president blush. In the immortal words of the great philosopher LL Cool J, they lied about the lies they lied about.
Even though 2017 was an off-year for them in terms of the number of lies, they still told enough to be named a runner-up.
Wellness Corporate Solutions
Next is Wellness Corporate Solutions, famous for its crash-dieting contests. WCS now offers a water-drinking contest. The idea is to set up a “challenge” for your team to drink more water than other teams. They call this a “healthy competition.” I guess they didn’t get the memo that forcing yourself to drink when you don’t want to drink, just to make more money, is anything but healthy. Here is a novel idea: drink when you are thirsty. Evolution 1, WCS 0.
Perhaps as an encore, WCS, Dr. Oz and the National Business Group on Health could team up to offer a chocolate-eating contest.
I looked into this outfit to see where they get their ideas. The CEO previously ran something called the Washington Document Service. That qualifies her to run a wellness company. As Star Wellness says, to run a wellness company successfully, your background needs to be in sales, or “municipality administration.” After all, what is more central to administering a municipality than documents?
Wellsteps
What fun would a list of runners-up be without Wellsteps, the proud recipient of the 2016 Deplorables Award? While their streams of consciousness weren’t as memorable in 2017 as in 2016 (“It’s fun to get fat. It’s fun to be lazy“), they get credit for trying. Their 2017 weight-loss campaign was headlined: “This campaign is not really about weight loss, it is about helping you apply the behavioral secrets of those who have lost weight.”
So if your kids ever want you to teach them how to ride a bike, say: “It’s not really about riding a bike. It’s about helping you apply the secrets of people who have ridden bikes.”
And what secrets are we talking about? What person who has lost weight doesn’t brag to everyone or even write a book? If there is a secret to weight loss, like eating chocolate, Wellsteps owes it to the country to tell them. Don’t make us beg.
Odds and Ends
No Koop Award winner this year, but an honorable mention to past winners and runners up for their commitment to wellness:
- McKesson, for flooding the country with opioids;
- Pepsi, for flooding the world with sugar.
Sounds like in 2018 the logical winners would be Philip Morris, or maybe The Asbestos Corporation of America.
Veering briefly into the public sector, kudos to Representative Virginia Foxx, (R-NC5) for introducing the Required Employee DNA Disclosure Act. Even HERO thought it was a dumb idea…and their threshold for thinking something that increases wellness industry revenues is a dumb idea is quite high, having all rallied behind the Johnson & Johnson Fat Tax, in which companies would be required to disclose the weight of their employees.
Next up…the winner of the 2017 Deplorables Award
Share this:
Fitbit Throws a Bit of a Fit, Part 2: More Fitbit Tidbits
The “prequels” to this posting are:
Fitbit might just have taken the lead in the wellness industry’s race to the bottom. They are using the “dumb and dumber” defense to deflect their ethical shortcomings. This defense has been shown to work, in the sense that Ron Goetzel still has a job.
In Fitbit’s case, they have no choice. If they claim to be intelligent, that would mean they dramatically overstated the value of Fitbits deliberately, as opposed to out of pure, sheer, unadulterated ignorance. In turn, that would mean that the folks at Fitbit could be facing a little taxpayer-financed vacation in the federal hoosegow. That’s because public companies aren’t allowed to deliberately misrepresent their product to shareholders, which is precisely what this press release does. Stupid is OK. Dishonest isn’t.
Here are a few more morsels from that study:
- There were 22,259 employees in the employer population. Only 905 were in the study population. So the entire analysis of savings was based on projection from 4% of the population.
- For some unexplained reason, the control group –the people who did nothing at all — enjoyed a dramatic 9.3% reduction in medical claims costs, vs. an “expected” increase of 5.8%, a 15.1% swing. So doing nothing turns out to be a great strategy to achieve double-digit savings.
- Speaking of doing nothing, perhaps our favorite tidbit from this study is that an employee could stay in bed for up to 182 days a year — meaning take 100 steps a day or less, getting up just to eat and pee, as described in the original Springbuk study — and “save” 21.8%.
- It’s also possible that employees simply forgot to put on their Fitbits the other 183 days of the year, which is why they didn’t appear to take 100 steps on those days. However, that possibility is not acknowledged anywhere in the study. That could be because it wouldn’t make for much of a study to say: “We compared people who forgot to put on their Fitbit for fewer than 182 days to people who forgot to put on their Fitbit for more than 182 days.”
Therefore, whatever the other criticisms of this study, no one can accuse them of lying or even exaggerating when they say:
Speaking of which, let us now just focus on the 374 people (about 2% of their entire population) who did take more than 100 steps a day for a whopping 274 days out of the year or more. Their savings are massive:
Even the healthier subset of employees can reduce healthcare costs by a quarter by wearing a Fitbit, but that’s nothing compared to “low steps” employees who walk only 6477 steps a day, about the same as everyone else in the country. Those lucky employees can slash costs by more than half by continuing to walk an average number of steps, but this time wearing a Fitbit.
Oh, wait a sec. They were wearing a Fitbit in the baseline year too. Otherwise, how would we know how many steps they took? So they didn’t do anything in order to save massive sums of money.
Come again? This conclusion seems wacky even by Fitbit/Springbuk standards. So let me repeat it: these people did basically nothing in the study year that they didn’t also do in the baseline year…and yet they somehow set a record for greatest cost reduction ever achieved in a year, 50.7%.
Then, these employees broke their own record. This next chart is for employees with “>=365 days” of use over the course of a year. (Not sure how they could have worn a Fitbit for “greater than 365 days” since the baseline for this two-year study was 2013, but maybe every year is a leap year on Springbuk’s planet.)
You read that right: a 58.6% reduction in spending for those 133 people taking the average number of steps everyone else takes in both years.
How much is a 58.6% reduction in costs in terms of utilization reduction? That means that simply by continuing to be average, these 133 average everyday folks wiped out the equivalent of all their hospitalizations and ER visits and specialist visits besides. Of course, we won’t know because Springbuk never plausibility-tested the result. As they say in journalism, it was a story too good to check.
Or, if Springbuk and Fitbit understood the concept of attribution as described in Biostatistics 101, they would realize that one can attribute only reductions in wellness-sensitive medical events to a wellness program, since those are what a wellness program is designed to avoid. If only those events can be avoided, they must have wiped out heart attacks and diabetes for these 133 people, their spouses, and roughly 5000 of their closest friends.
If anyone is interested in the real health impact of activity tracking, I’d recommend this JAMA article. It’s the only one on the topic which is not financed by people connected to the industry. Researchers attached activity trackers to some at-risk overweight/obese people to see how much weight they would lose (which would mean a reduction in their risk and possibly a slight reduction in their healthcare costs). The study’s result? The study population gained weight.
Share this:
Fitbit Throws a Bit of a Fit, Part 1
This column originally appeared in the Corporate Health and Wellness Association blog but they were asked to remove it by Springbuk, which did the original analysis. Not because it is inaccurate — no inaccuracies have ever been pointed out despite multiple requests by me to do so — but because it was accurate.
So I’m re-posting it here.
This is a sequel to “Springbuk Wants Employees to Go to the Bathroom,” which should be read prior to reading this posting.
In wellness, it’s totally legal to lie to customers. Indeed, if you don’t, you’ll probably lose them, since your competitors are happy to do exactly that, and most customers aren’t going to notice anyway.
In securities, though, it is totally illegal to lie to shareholders or to pay someone to write a favorable but dishonest report on your product, with the intent of propping up the stock price.
This brings up to Fitbit, and a recent report on savings allegedly generated by their activity trackers, published by Springbuk. Let’s leave aside for a moment the value of activity trackers to users. I like mine enough to generally recommend they be offered and subsidized (not given away) as part of a corporate wellness program, but “like” is not the issue in this savings claim. The issue is whether the math works.
And it doesn’t.
The Springbuk report of savings and outcomes for Fitbit was impossible. Among the clinical issues is the study design itself: the report defines “active” as taking 100 steps a day. However, as the previous installment showed, it is impossible not to take 100 steps in a day without being so sick you can’t get out of bed. So rather than being the threshold for being counted as an “active” person, as the Springbuk study says, it should be the threshold for being a person who can get out of bed. And of course, people who can get out of bed will by definition have lower healthcare costs than people who can’t get out of bed, whether or not they wear a Fitbit.
Among the mathematical issues, it is not possible to reduce costs by 45.6% (one of the claims made) with a fitness device, because in the working-age population, only about 5% of hospital admissions are caused by lack of fitness.
Further, in addition to the apparent mathematical and clinical impossibility of Springbuk’s results, the author — and Fitbit — refused to respond to the following query.
Hi Mr. Daniels,
I have some questions about your report. Perhaps I’ve gotten some things wrong, so I’d love to hear from you in the next 3 business days, if I have.
First, isn’t it the case that anyone who is not in a wheelchair walks at least 100 steps a day, Fitbit or no Fitbit? Is that the threshold for “active” as opposed to “bedridden” ?
Second, Figure 2c indicates that the very fact of being in the “engaged” group, even if you never get out of bed, reduces costs by 30%+. How is this possible? A corollary: It would seem that all savings is being attributed to Fitbit, at least in the Fitbit interpretation. They also seem to be taking credit for this: “266 employees who used their Fitbit tracker for at least half the duration of the program decreased their healthcare costs by 45.6% on average.”
Third, can you reconcile this statement…:
“The materials in this document represent the opinion of the authors and not representative of the views of Springbuk, Inc. Springbuk does not certify the information, nor does it guarantee the accuracy and completeness of such information.”
“This demonstration of impact achieved by integrating Fitbit technology into an employee wellness program reinforces our belief in the power of health data and measurement in demonstrating ROI,” said Rod Reasen, co-founder and CEO of Springbuk.
Fourth, how is it possible to show basically no separation for 182 days of getting out of bed (taking 100 steps a day) from being bedridden, but massive separation for getting out of bed for 274 days? I can’t find the explanation of the exercise science that would lead to that result. It would seem that there is some huge physiological disadvantage to those extra 92 days of taking 100 steps.
Fifth, am I missing the disclosure that Fitbit paid you to do this study? I can’t find it anywhere. Or did you do this on a pro bono basis?
Sixth, would you have come up with this same result if you had been paid by a hedge fund that was shorting Fitbit stock and wanted to show no savings?
Seventh, since most wellness-related healthcare spending is unavoidable altogether by walking 100 steps a days or any other amount for 12 months, I’m wondering if you were able to determine approximately which elements of healthcare spending were reduced, in order to get a 45.6% reduction in costs? You would have to wipe out all hospitalizations, for example – and get roughly a 10% reduction in everything else.
Thanks very much. If you would like to reply, I’ll look forward to your reply by 5 PM EDT on Wednesday 5/24.
Assuming Fitbit paid Springbuk (that’s a big assumption — this obvious conflict of interest is not disclosed anywhere, so the reader has to decide whether Springbuk collected money, or whether they did this study out of the goodness of their heart), one of four outcomes is possible:
- Springbuk genuinely thinks, among other things, that walking 100 steps a day for 274 days reduces healthcare costs by 27% vs. walking 100 steps a day for only 182 days. No crime there, other than the one committed by the grade school that granted them a diploma. It’s unlikely they think this, because Springbuk says they are “obsessed with analytics” and that they sell “the leading health analytics software…[with] powerful insights.” So if Springbuk truly believes their own report, then congratulations are in order: they have accomplished more in this one analysis than most extremely stupid people accomplish in a lifetime. (Not an original line, as Veep fans know, but apropos nonetheless.)
- Springbuk wanted to show savings because they were being well-paid to do so, but Fitbit put no pressure on them when they gave them the check. Once again, doesn’t say much for Springbuk’s ethics, but Fitbit did not commit a crime.
- Fitbit paid Springbuk to lie for them, in order to impress prospects and customers. Once again, no crime. There wouldn’t be enough room in the prison system if lying in wellness were a crime. (See http://www.ethicalwellness.org for a list of wellness vendors that have agreed not to lie. It’s not very long.)
- Fitbit paid Springbuk to lie for them, in order to inter alia impress investors. This is not legal, any more than if Fitbit made up their own data for that reason. Since many Fitbit analyst reports make reference to savings of “up to $1500 per employee per year,” and since this study appears to be one of only two justifications for that statement (the other being equally suspect), there is a case to be made that Fitbit’s stock price would indeed be lower if they told the truth: that no disinterested researcher has ever found more than a trivial impact on employee health status or healthcare insurance cost.
We don’t know which of these four is the case. Is Springbuk dishonest, or just incompetent? Does Fitbit genuinely believe that wearing their device could magically reduce healthcare costs for US corporations by hundreds of billions of dollars, or are they willing to lie in order to boost revenues and their share price?
We look forward to hearing the answers to these questions once the financial media gets hold of this posting.
Header Photo – Copyright: dzejmsdin / 123RF Stock Photo
Opinions expressed in this column are those of Al Lewis individually. They do not necessarily represent the views of the Corporate Health and Wellness Association. Therefore all threats of lawsuits should be directed to the former, to which I say: “Go ahead. Make my day.”
Share this:
Did Fitbit just do something, um, illegal?
In wellness, it is perfectly legal to lie to customers and prospects. That’s in most vendors’ DNA.* (Not all vendors — we will soon be publishing an expanded list of honest ones, and for now would direct readers to http://www.ethicalwellness.org for the original list of honest ones.)
However, if you are a public company, it is quite illegal to lie to shareholders. It’s possible Fitbit did just that. If they did, they could face major SEC sanctions.
Did that just happen? Read this link and then you make the call.
PS This is the sequel to Springbuk Wants Employees to Go to the Bathroom, which should be read in conjunction with this link.
*The irony is that one of the biggest liars specializes in collecting employee DNA and then pretending that they can save a ton of money by getting employees to lose weight by telling them it’s pretty darn impossible to lose weight, because they have a gene for obesity. Yes, you read that right and, no, it doesn’t make any sense.
Update: The link was removed at Fitbit’s request. In a couple of weeks they will defend this report and explain why they or Springbuk never responded to the requests I made for more information before publishing it. Good luck with that.
Share this:
Springbuk wants employees to go to the bathroom
Oh, so lovely sittin’
Abso-bloomin’-lutely still
I would never budge till spring
Crept over me window sill
–Eliza Doolittle
Springbuk has found the key to dramatic reduction in healthcare spending: getting out of bed. It doesn’t matter whether it’s to eat, find the remote, or, of course, pee. Just do something, anything, other than lie in bed all day — and you count as an “active” person who saves a ton of money, as compared to people who never get out of bed. Yes, Springbuk classifies you as “active” if you take at least 100 steps a day:
My question, for now, is not about how all this money was allegedly saved. Rather, my question for now is: how did these “less active” people — a huge chunk of the total population studied –manage to take fewer than 100 steps a day in the first place? Consider this random floor plan:
Now overlay the steps you take just in order to get up in the morning. A one-way trip to the bathroom and then to the breakfast nook appears to require about 20 steps. You haven’t even had your coffee yet and already you are a fifth of the way to your daily goal.
Speaking of coffee, add in a few more trips to the bathroom and <voila> you’ve reached your steps goal. Conclusion: it is impossible not to walk 100 steps a day, unless you want to starve to death, burst your bladder, or work at Spacely’s Sprockets. We wouldn’t recommend any of those three, especially the last, a very high-stress environment.
As a sidebar, I would note that George Jetson ironically has a lower BMI than Fred Flintstone, thus showing that taking more than 100 steps a day doesn’t reduce weight. Nor does a paleo diet, apparently. (Portion sizes might have something to do with it.) On the other hand, Fred can power his own car, thus showing the value of maintaining health at any size.
The Economics of Getting out of Bed
And needless to say, Springbuk provides some very compelling economics about the cost savings impact of getting out of bed. 100 steps a day for only 274 days a year (meaning you can take a well-deserved breather on weekends, holidays, vacation days and Beethoven’s birthday) generates a dramatic 28% reduction in costs. Wow! Who knew that peeing, eating, and looking for the remote (try your fridge or dresser drawers) could be so beneficial to your health?
Springbuk has additional bad news and good news.
The bad news is that taking 100 steps a day for more than 182 days (as opposed to more than 274 days) makes only a 3% differential impact on health spending, vs. taking 100 steps a day for less than 182 days. Still, there is some good news, which is that staying in bed for half the year also generates a huge reduction in costs, 31% to be exact.
Springbuk didn’t mention this, but the only way both these findings could be consistent would be that people –we will call them “semi-active” — who take 100 steps on more than 182 days but fewer than 274 days must have ridiculously high, off-the-charts healthcare spending and presumably morbidity. Apparently, moving those semi-active people between “less active” and “active” swings overall healthcare spending for the entire population by 25%.
The implication, as any exercise physiologist would tell you, is that starting in January, you need to track the number of days on which you take 100 steps. If you get to 181 such days by late summer, but don’t think you can make it to 274 days by the end of the year, then your best bet, statistically speaking, is to stay in bed until the ball drops in Times Square. Your bosses will love you for it, because you’ll be saving them 31%.
Naturally, Springbuk’s findings contradict all the other findings on wearables showing trivial changes in activity due to wearables after a short burst of interest. These trivial changes predictable show only trivial improvements in health and costs.
And someone should tell the Einsteins at Springbuk what anyone with a triple-digit IQ could intuit and what every other study shows: that a typical American takes many times more than 100 steps a day. 6886 steps per day, according to one study. So Springbuk’s study is wrong, making them eligible for a Koop Award.
Springbuk’s analysis may be wrong for another reason too: It does not account for the health hazards of taking too many steps. (Yes, you need to click through for the punchline.)
An accurate line in this report
I can’t believe I missed this, but Pete Aren didn’t, and pointed it out on linkedin. There is indeed one accurate line in their report, buried in the footnotes:
Share this:
Is Wellness-Driven Life Insurance Hancock’s New Coke?
John Hancock Insurance recently announced a plan to sell life insurance based on healthy behaviors. You get a discount on life and disability insurance for exercising and reporting good blood values on an ongoing basis, not just once when you sign up.
While we have been quite vocal in saying wellness is a waste of money and potentially injurious to health and morale (and lately the two wellness trade associations themselves have candidly supported that position), we find Hancock’s strategy to be a shockingly good idea.
There are many distinctions between Hancock’s offering and health insurance. First, life and disability insurance are opt-in products. No one is forcing you to buy them in order to get health insurance at work, or fining you if you don’t. No one is violating USPSTF guidelines, screening the entire workforce, or making you get checkups that are worthless at best.
Second, the same numbers that don’t remotely add up for wellness add up quite elegantly for life and disability. Cut 50% out of your heart attack rate for the latter and you probably reduce overall claims payout by 5%. Cut 50% out of your heart attack rate for health insurance and you reduce overall claims payout by less than 1%. Additionally, Hancock can possibly accomplish that goal through underwriting. An employer doesn’t have that option. So besides being worth more, a 50% reduction is achievable.
Finally, they should be able to generate some good self-selection into this product. People have to be willing to give up some privacy, and our colleague Anna Slomovic is quoted on this topic in the article in the New York Times, but as long as you know what risk you are taking and as long as there is some recourse, it isn’t the same thing as being forced to reveal personal information for a wellness program.
Declare your independence from wellness intrusion. Quizzify.
One asterisk: the article says they are relying on Vitality to come up with the risk adjustments. I doubt seriously that is the case. Hancock has real grownup actuaries whose job it is to price these risk adjustments. We assume the article is wrong — Hancock isn’t going to rely on a vendor that can’t even quote Dee Edington correctly and doesn’t understand how to design a study.
Absent that asterisk, we are confident that they will be successful and wish them the best of luck.
Share this:
Dee Edington Drains The Life Out Of The Vitality Group’s Distortion Of His Work
The Vitality Group
Short Summary of Company:
“Vitality is an active, fully integrated wellness program designed to engage your employees on their Personal Pathway to better health. Employers can choose to introduce the Vitality experience with one of our comprehensive plans. Activate is designed to bring wellness into the workplace. Elevate includes all the components of Activate, plus additional engagement features.”
Materials Being Reviewed
The Vitality Group “wearables at work” presentation. This presentation describes the health risk reduction achievable through engaging members at workplaces by wearing activity trackers.
Summary of key figures and outcomes:
Questions for Vitality Group:
You appear to be claiming that people who are “not active” reduced their risk factors simply by being engaged, without actually doing or reporting anything. A health services researcher might say that instead of taking credit for both the 6-point decline in the study group and the 5-point decline in the de facto control group risk, in reality only the difference between the two groups (1 point) could be attributable to fitness activities. If you disagree, can you explain exactly what it is that makes people in the inactive group so successful even if they don’t do anything?
ANS:
The amount that could be attributable to fitness activities is the difference between the two groups compared. For clarification, we compared (1) individuals who were engaged in fitness activities with the Vitality program (who might also be using other program elements), with (2) those who were engaged in the Vitality program on other elements but were not recording fitness activities directly with us.
So the graphic focused only on the incremental difference between the described fitness and non-fitness cohorts. Both the fitness and non-fitness cohorts were participating in other aspects of the Vitality program to track and improve their health, but the non-fitness group did not record their fitness activities through Vitality. Individuals in the non-fitness group may also have engaged in some fitness activities but simply did not log any of these activities through the Vitality program.
Observation::
Thank you for that clarification. When I look at the “difference between the two groups compared” I am seeing a 5-point decline in the first group and a 6-point decline in the second group, netting out to 1% as an “incremental difference,” rather than the 13% and 22% declines you claim,, but perhaps readers will see it differently.
How does your claim of success adjust for dropouts, and the likelihood that dropouts would have worse performance than people who were willing to be measured twice?
ANS:
This analysis did not include an adjustment for dropouts as the intent was not to make assumptions about unknown risk factors. A more detailed investigation could include this as a refinement.
Are you familiar with the concept of the “natural flow of risk” described on this slide researched and prepared by the “father of wellness measurement,” Dee Edington?
Edington’s research shows that nearly 50% of people with >4 risk factors will eventually move to a lower risk category on their own. Having been exposed to this “natural flow of risk” data, do you still believe that the non-active and active members (both groups were selected on the basis of having >4 risk factors) declined in risk due to the program, or else could some or all of the decline be due to (a) self-selection into the active group; (b) ignoring discouraged dropouts; and (c) the natural flow of risk?
Response:
Yes, we did allow for this effect by looking at the net changes in overall risk groupings by level of activity in the Vitality program. In other words, the percentages shown account for the overall flow of risk, including those who improved over the period but also those who deteriorated. The graphic focused on the proportion of high risk people in each group, but did allow for people moving into the group over the period.
Dee Edington’s work found that expected natural migration is actually a deterioration in risk groups as people naturally flow to high risk.
Often there is a tendency in wellness to compare consistent cohort risk transitions to these expected natural migration increases. Although both cohorts in the analysis saw an overall net improvement in risk groups, this comparison to natural migration was not the intent of this analysis. Instead the intent was to compare the relative changes in the two cohorts. This analysis showed that the cohort who engaged in fitness activities through Vitality had a lower proportion of high risk individuals as of their first risk measure, but had a greater net improvement in risk groups as of the last measure than those who did not engage in fitness activity through Vitality
Observation:
Hmm…well we can’t both be right. I’m looking at the exact same Dee Edington slide you are, but I am seeing the population’s risk “naturally flow” in both directions, not just “a deterioration in risk groups as people naturally flow to high risk.” Obviously the validity of the alleged declines in your cohorts is dramatically different depending on whether one uses your interpretation of Dr. Edington’s work (in which case your results are outstanding) or mine (in which case except for 1%, they are due to the natural flow downward of the highest-risk segment).
Like Alvy Singer did with Marshall McLuhan in Annie Hall, I took the liberty of asking Dee Edington himself to referee our disagreement. This is his response:
“The correct interpretation of that slide and of my work is that the natural flow of risk in a population moves in both directions, and must be understood in order to gauge impact of an intervention. It is not valid to simply start with people who were high-risk and claim credit for all risk reduction in that cohort while ignoring people who migrate in the other direction.”
They Said What? 