They Said What?

Home » Wearable Devices

Category Archives: Wearable Devices

Fitbit Throws a Bit of a Fit, Part 2: More Fitbit Tidbits

The “prequels” to this posting are:

  1. Springbuk wants employees to go to the bathroom
  2. Fitbit Throws a Bit of a Fit, Part 1

Fitbit might just have taken the lead in the wellness industry’s race to the bottom. They are using the “dumb and dumber” defense to deflect their ethical shortcomings. This defense has been shown to work, in the sense that Ron Goetzel still has a job.

In Fitbit’s case, they have no choice. If they claim to be intelligent, that would mean they dramatically overstated the value of Fitbits deliberately, as opposed to out of pure, sheer, unadulterated ignorance. In turn, that would mean that the folks at Fitbit could be facing a little taxpayer-financed vacation in the federal hoosegow. That’s because public companies aren’t allowed to deliberately misrepresent their product to shareholders, which is precisely what this press release does.  Stupid is OK. Dishonest isn’t.

Here are a few more morsels from that study:

  • There were 22,259 employees in the employer population. Only 905 were in the study population. So the entire analysis of savings was based on projection from 4% of the population.
  • For some unexplained reason, the control group –the people who did nothing at all — enjoyed a dramatic 9.3% reduction in medical claims costs, vs. an “expected” increase of 5.8%, a 15.1% swing. So doing nothing turns out to be a great strategy to achieve double-digit savings.
  • Speaking of doing nothing, perhaps our favorite tidbit from this study is that an employee could stay in bed for up to 182 days a year  — meaning take 100 steps a day or less, getting up just to eat and pee, as described in the original Springbuk study — and “save” 21.8%.
  • It’s also possible that employees simply forgot to put on their Fitbits the other 183 days of the year, which is why they didn’t appear to take 100 steps on those days. However, that possibility is not acknowledged anywhere in the study. That could be because it wouldn’t make for much of a study to say: “We compared people who forgot to put on their Fitbit for fewer than 182 days to people who forgot to put on their Fitbit for more than 182 days.”

Therefore, whatever the other criticisms of this study, no one can accuse them of lying or even exaggerating when they say:


Speaking of which, let us now just focus on the 374 people (about 2% of their entire population) who did take more than 100 steps a day for a whopping 274 days out of the year or more. Their savings are massive:

Even the healthier subset of employees can reduce healthcare costs by a quarter by wearing a Fitbit, but that’s nothing compared to “low steps” employees who walk only 6477 steps a day, about the same as everyone else in the country. Those lucky employees can slash costs by more than half by continuing to walk an average number of steps, but this time wearing a Fitbit.

Oh, wait a sec. They were wearing a Fitbit in the baseline year too. Otherwise, how would we know how many steps they took? So they didn’t do anything in order to save massive sums of money.

Come again?  This conclusion seems wacky even by Fitbit/Springbuk standards. So let me repeat it: these people did basically nothing in the study year that they didn’t also do in the baseline year…and yet they somehow set a record for greatest cost reduction ever achieved in a year, 50.7%.

Then, these employees broke their own record. This next chart is for employees with “>=365 days” of use over the course of a year. (Not sure how they could have worn a Fitbit for “greater than 365 days” since the baseline for this two-year study was 2013, but maybe every year is a leap year on Springbuk’s planet.)

You read that right: a 58.6% reduction in spending for those 133 people taking the average number of steps everyone else takes in both years.

How much is a 58.6% reduction in costs in terms of utilization reduction? That means that simply by continuing to be average, these 133 average everyday folks wiped out the equivalent of all their hospitalizations and ER visits and specialist visits besides. Of course, we won’t know because Springbuk never plausibility-tested the result. As they say in journalism, it was a story too good to check.

Or, if Springbuk and Fitbit understood the concept of attribution as described in Biostatistics 101, they would realize that one can attribute only reductions in wellness-sensitive medical events to a wellness program, since those are what a wellness program is designed to avoid.  If only those events can be avoided, they must have wiped out heart attacks and diabetes for these 133 people, their spouses, and roughly 5000 of their closest friends.


If anyone is interested in the real health impact of activity tracking, I’d recommend this JAMA article. It’s the only one on the topic which is not financed by people connected to the industry. Researchers attached activity trackers to some at-risk overweight/obese people to see how much weight they would lose (which would mean a reduction in their risk and possibly a slight reduction in their healthcare costs).  The study’s result? The study population gained weight.

Fitbit Throws a Bit of a Fit, Part 1

This column originally appeared in the Corporate Health and Wellness Association blog but they were asked to remove it by Springbuk, which did the original analysis.  Not because it is inaccurate — no inaccuracies have ever been pointed out despite multiple requests by me to do so — but because it was accurate.

So I’m re-posting it here.



This is a sequel to “Springbuk Wants Employees to Go to the Bathroom,” which should be read prior to reading this posting.


In wellness, it’s totally legal to lie to customers. Indeed, if you don’t, you’ll probably lose them, since your competitors are happy to do exactly that, and most customers aren’t going to notice anyway.

In securities, though, it is totally illegal to lie to shareholders or to pay someone to write a favorable but dishonest report on your product, with the intent of propping up the stock price.

This brings up to Fitbit, and a recent report on savings allegedly generated by their activity trackers, published by Springbuk. Let’s leave aside for a moment the value of activity trackers to users. I like mine enough to generally recommend they be offered and subsidized (not given away) as part of a corporate wellness program, but “like” is not the issue in this savings claim. The issue is whether the math works.

And it doesn’t.

The Springbuk report of savings and outcomes for Fitbit was impossible. Among the clinical issues is the study design itself: the report defines “active” as taking 100 steps a day. However, as the previous installment showed, it is impossible not to take 100 steps in a day without being so sick you can’t get out of bed. So rather than being the threshold for being counted as an “active” person, as the Springbuk study says, it should be the threshold for being a person who can get out of bed.  And of course, people who can get out of bed will by definition have lower healthcare costs than people who can’t get out of bed, whether or not they wear a Fitbit.

Among the mathematical issues, it is not possible to reduce costs by 45.6% (one of the claims made) with a fitness device, because in the working-age population, only about 5% of hospital admissions are caused by lack of fitness.

Further, in addition to the apparent mathematical and clinical impossibility of Springbuk’s results, the author — and Fitbit — refused to respond to the following query.


Hi Mr. Daniels,

I have some questions about your report. Perhaps I’ve gotten some things wrong, so I’d love to hear from you in the next 3 business days, if I have.

First, isn’t it the case that anyone who is not in a wheelchair walks at least 100 steps a day, Fitbit or no Fitbit?   Is that the threshold for “active” as opposed to “bedridden” ?

Second, Figure 2c indicates that the very fact of being in the “engaged” group, even if you never get out of bed, reduces costs by 30%+.  How is this possible?  A corollary:  It would seem that all savings is being attributed to Fitbit, at least in the Fitbit interpretation. They also seem to be taking credit for this: “266 employees who used their Fitbit tracker for at least half the duration of the program decreased their healthcare costs by 45.6% on average.”

Third, can you reconcile this statement…:

“The materials in this document represent the opinion of the authors and not representative of the views of Springbuk, Inc. Springbuk does not certify the information, nor does it guarantee the accuracy and completeness of such information.”

…with this statement:

“This demonstration of impact achieved by integrating Fitbit technology into an employee wellness program reinforces our belief in the power of health data and measurement in demonstrating ROI,” said Rod Reasen, co-founder and CEO of Springbuk. 

Fourth, how is it possible to show basically no separation for 182 days of getting out of bed (taking 100 steps a day) from being bedridden, but massive separation for getting out of bed for 274 days?  I can’t find the explanation of the exercise science that would lead to that result. It would seem that there is some huge physiological disadvantage to those extra 92 days of taking 100 steps.

Fifth, am I missing the disclosure that Fitbit paid you to do this study? I can’t find it anywhere. Or did you do this on a pro bono basis?

Sixth, would you have come up with this same result if you had been paid by a hedge fund that was shorting Fitbit stock and wanted to show no savings?

Seventh, since most wellness-related healthcare spending is unavoidable altogether by walking 100 steps a days or any other amount for 12 months, I’m wondering if you were able to determine approximately which elements of healthcare spending were reduced, in order to get a 45.6% reduction in costs?  You would have to wipe out all hospitalizations, for example – and get roughly a 10% reduction in everything else.

Thanks very much.  If you would like to reply, I’ll look forward to your reply by 5 PM EDT on Wednesday 5/24.


Assuming Fitbit paid Springbuk (that’s a big assumption — this obvious conflict of interest is not disclosed anywhere, so the reader has to decide whether Springbuk collected money, or whether they did this study out of the goodness of their heart), one of four outcomes is possible:

  1. Springbuk genuinely thinks, among other things, that walking 100 steps a day for 274 days reduces healthcare costs by 27% vs. walking 100 steps a day for only 182 days. No crime there, other than the one committed by the grade school that granted them a diploma. It’s unlikely they think this, because Springbuk says they are  “obsessed with analytics” and that they sell “the leading health analytics software…[with] powerful insights.” So if Springbuk truly believes their own report, then congratulations are in order: they have accomplished more in this one analysis than most extremely stupid people accomplish in a lifetime. (Not an original line, as Veep fans know, but apropos nonetheless.)
  2. Springbuk wanted to show savings because they were being well-paid to do so, but Fitbit put no pressure on them when they gave them the check. Once again, doesn’t say much for Springbuk’s ethics, but Fitbit did not commit a crime.
  3. Fitbit paid Springbuk to lie for them, in order to impress prospects and customers. Once again, no crime. There wouldn’t be enough room in the prison system if lying in wellness were a crime. (See http://www.ethicalwellness.org for a list of wellness vendors that have agreed not to lie. It’s not very long.)
  4. Fitbit paid Springbuk to lie for them, in order to inter alia impress investors. This is not legal, any more than if Fitbit made up their own data for that reason. Since many Fitbit analyst reports make reference to savings of “up to $1500 per employee per year,” and since this study appears to be one of only two justifications for that statement (the other being equally suspect), there is a case to be made that Fitbit’s stock price would indeed be lower if they told the truth: that no disinterested researcher has ever found more than a trivial impact on employee health status or healthcare insurance cost.

We don’t know which of these four is the case. Is Springbuk dishonest, or just incompetent? Does Fitbit genuinely believe that wearing their device could magically reduce healthcare costs for US corporations by hundreds of billions of dollars, or are they willing to lie in order to boost revenues and their share price?

We look forward to hearing the answers to these questions once the financial media gets hold of this posting.

Header Photo – Copyright: dzejmsdin / 123RF Stock Photo

Opinions expressed in this column are those of Al Lewis individually. They do not necessarily represent the views of the Corporate Health and Wellness Association. Therefore all threats of lawsuits should be directed to the former, to which I say: “Go ahead. Make my day.”

Did Fitbit just do something, um, illegal?

In wellness, it is perfectly legal to lie to customers and prospects. That’s in most vendors’ DNA.*  (Not all vendors — we will soon be publishing an expanded list of honest ones, and for now would direct readers to http://www.ethicalwellness.org for the original list of honest ones.)

However, if you are a public company, it is quite illegal to lie to shareholders. It’s possible Fitbit did just that. If they did, they could face major SEC sanctions.

Did that just happen? Read this link and then you make the call.

PS This is the sequel to Springbuk Wants Employees to Go to the Bathroom, which should be read in conjunction with this link.


*The irony is that one of the biggest liars specializes in collecting employee DNA and then pretending that they can save a ton of money by getting employees to lose weight by telling them it’s pretty darn impossible to lose weight, because they have a gene for obesity. Yes, you read that right and, no, it doesn’t make any sense.



Update: The link was removed at Fitbit’s request.  In a couple of weeks they will defend this report and explain why they or Springbuk never responded to the requests I made for more information before publishing it.  Good luck with that.

Springbuk wants employees to go to the bathroom

Oh, so lovely sittin’
Abso-bloomin’-lutely still
I would never budge till spring
Crept over me window sill

–Eliza Doolittle


Springbuk has found the key to dramatic reduction in healthcare spending: getting out of bed. It doesn’t matter whether it’s to eat, find the remote, or, of course, pee. Just do something, anything, other than lie in bed all day — and you count as an “active” person who saves a ton of money, as compared to people who never get out of bed. Yes, Springbuk classifies you as “active” if you take at least 100 steps a day:

My question, for now, is not about how all this money was allegedly saved. Rather, my question for now is:  how did these “less active” people — a huge chunk of the total population studied –manage to take fewer than 100 steps a day in the first place? Consider this random floor plan:

Now overlay the steps you take just in order to get up in the morning. A one-way trip to the bathroom and then to the breakfast nook appears to require about 20 steps. You haven’t even had your coffee yet and already you are a fifth of the way to your daily goal.

Speaking of coffee, add in a few more trips to the bathroom and <voila> you’ve reached your steps goal.  Conclusion: it is impossible not to walk 100 steps a day, unless you want to starve to death, burst your bladder, or work at Spacely’s Sprockets. We wouldn’t recommend any of those three, especially the last, a very high-stress environment.

As a sidebar, I would note that George Jetson ironically has a lower BMI than Fred Flintstone, thus showing that taking more than 100 steps a day doesn’t reduce weight. Nor does a paleo diet, apparently. (Portion sizes might have something to do with it.)  On the other hand, Fred can power his own car, thus showing the value of maintaining health at any size.


The Economics of Getting out of Bed

And needless to say, Springbuk provides some very compelling economics about the cost savings impact of getting out of bed.  100 steps a day for only 274 days a year (meaning you can take a well-deserved breather on weekends, holidays, vacation days and Beethoven’s birthday) generates a dramatic 28% reduction in costs. Wow! Who knew that peeing, eating, and looking for the remote (try your fridge or dresser drawers) could be so beneficial to your health?


Springbuk has additional bad news and good news.

The bad news is that taking 100 steps a day for more than 182 days  (as opposed to more than 274 days) makes only a 3% differential impact on health spending, vs. taking 100 steps a day for less than 182 days. Still, there is some good news, which is that staying in bed for half the year also generates a huge reduction in costs, 31% to be exact.

Springbuk didn’t mention this, but the only way both these findings could be consistent would be that people –we will call them “semi-active” — who take 100 steps on more than 182 days but fewer than 274 days must have ridiculously high, off-the-charts healthcare spending and presumably morbidity.  Apparently, moving those semi-active people between “less active” and “active” swings overall healthcare spending for the entire population by 25%.

The implication, as any exercise physiologist would tell you, is that starting in January, you need to track the number of days on which you take 100 steps. If you get to 181 such days by late summer, but don’t think you can make it to 274 days by the end of the year, then your best bet, statistically speaking, is to stay in bed until the ball drops in Times Square. Your bosses will love you for it, because you’ll be saving them 31%.


Naturally, Springbuk’s findings contradict all the other findings on wearables showing trivial changes in activity due to wearables after a short burst of interest. These trivial changes predictable show only trivial improvements in health and costs.

And someone should tell the Einsteins at Springbuk what anyone with a triple-digit IQ could intuit and what every other study shows: that a typical American takes many times more than 100 steps a day.  6886 steps per day, according to one study. So Springbuk’s study is wrong, making them eligible for a Koop Award.

Springbuk’s analysis may be wrong for another reason too:  It does not account for the health hazards of taking too many steps.  (Yes, you need to click through for the punchline.)



An accurate line in this report

I can’t believe I missed this, but Pete Aren didn’t, and pointed it out on linkedin.  There is indeed one accurate line in their report, buried in the footnotes:

Is Wellness-Driven Life Insurance Hancock’s New Coke?

John_Hancock_Envelope_SignatureJohn Hancock Insurance recently announced a plan to sell life insurance based on healthy behaviors. You get a discount on life and disability insurance for exercising and reporting good blood values on an ongoing basis, not just once when you sign up.

While we have been quite vocal in saying wellness is a waste of money and potentially injurious to health and morale (and lately the two wellness trade associations themselves have candidly supported that position), we find Hancock’s strategy to be a shockingly good idea.

There are many distinctions between Hancock’s offering and health insurance. First, life and disability insurance are opt-in products. No one is forcing you to buy them in order to get health insurance at work, or fining you if you don’t. No one is violating USPSTF guidelines, screening the entire workforce, or making you get checkups that are worthless at best.

Second, the same numbers that don’t remotely add up for wellness add up quite elegantly for life and disability.  Cut 50% out of your heart attack rate for the latter and you probably reduce overall claims payout by 5%. Cut 50% out of your heart attack rate for health insurance and you reduce overall claims payout by less than 1%.  Additionally, Hancock can possibly accomplish that goal through underwriting. An employer doesn’t have that option.  So besides being worth more, a 50% reduction is achievable.

Finally, they should be able to generate some good self-selection into this product.  People have to be willing to give up some privacy, and our colleague Anna Slomovic is quoted on this topic in the article in the New York Times, but as long as you know what risk you are taking and as long as there is some recourse, it isn’t the same thing as being forced to reveal personal information for a wellness program.

Declare your independence from wellness intrusion. Quizzify.

Declare your independence from wellness intrusion. Quizzify.

One asterisk:  the article says they are relying on Vitality to come up with the risk adjustments. I doubt seriously that is the case.  Hancock has real grownup actuaries whose job it is to price these risk adjustments. We assume the article is wrong — Hancock isn’t going to rely on a vendor that can’t even quote Dee Edington correctly and doesn’t understand how to design a study.

Absent that asterisk, we are confident that they will be successful and wish them the best of luck.

 

Dee Edington Drains The Life Out Of The Vitality Group’s Distortion Of His Work

The Vitality Group

Short Summary of Company:

“Vitality is an active, fully integrated wellness program designed to engage your employees on their Personal Pathway to better health. Employers can choose to introduce the Vitality experience with one of our comprehensive plans. Activate is designed to bring wellness into the workplace. Elevate includes all the components of Activate, plus additional engagement features.”

Materials Being Reviewed

The Vitality Group “wearables at work” presentation. This presentation describes the health risk reduction achievable through engaging members at workplaces by wearing activity trackers.

Summary of key figures and outcomes:

Vitality members engaged in their program

Questions for Vitality Group:

You appear to be claiming that people who are “not active” reduced their risk factors simply by being engaged, without actually doing or reporting anything. A health services researcher might say that instead of taking credit for both the 6-point decline in the study group and the 5-point decline in the de facto control group risk, in reality only the difference between the two groups (1 point) could be attributable to fitness activities. If you disagree, can you explain exactly what it is that makes people in the inactive group so successful even if they don’t do anything?

ANS:

The amount that could be attributable to fitness activities is the difference between the two groups compared. For clarification, we compared (1) individuals who were engaged in fitness activities with the Vitality program (who might also be using other program elements), with (2) those who were engaged in the Vitality program on other elements but were not recording fitness activities directly with us.

So the graphic focused only on the incremental difference between the described fitness and non-fitness cohorts. Both the fitness and non-fitness cohorts were participating in other aspects of the Vitality program to track and improve their health, but the non-fitness group did not record their fitness activities through Vitality. Individuals in the non-fitness group may also have engaged in some fitness activities but simply did not log any of these activities through the Vitality program.

Observation::

Thank you for that clarification. When I look at the “difference between the two groups compared” I am seeing a 5-point decline in the first group and a 6-point decline in the second group, netting out to 1% as an “incremental difference,” rather than the 13% and 22% declines you claim,, but perhaps readers will see it differently.

How does your claim of success adjust for dropouts, and the likelihood that dropouts would have worse performance than people who were willing to be measured twice?

ANS:

This analysis did not include an adjustment for dropouts as the intent was not to make assumptions about unknown risk factors. A more detailed investigation could include this as a refinement.

Are you familiar with the concept of the “natural flow of risk” described on this slide researched and prepared by the “father of wellness measurement,” Dee Edington?

Dee Edington's Diagram

Edington’s research shows that nearly 50% of people with >4 risk factors will eventually move to a lower risk category on their own.   Having been exposed to this “natural flow of risk” data, do you still believe that the non-active and active members (both groups were selected on the basis of having >4 risk factors) declined in risk due to the program, or else could some or all of the decline be due to (a) self-selection into the active group; (b) ignoring discouraged dropouts; and (c) the natural flow of risk?

 Response:

Yes, we did allow for this effect by looking at the net changes in overall risk groupings by level of activity in the Vitality program. In other words, the percentages shown account for the overall flow of risk, including those who improved over the period but also those who deteriorated. The graphic focused on the proportion of high risk people in each group, but did allow for people moving into the group over the period.

Dee Edington’s work found that expected natural migration is actually a deterioration in risk groups as people naturally flow to high risk.

Often there is a tendency in wellness to compare consistent cohort risk transitions to these expected natural migration increases. Although both cohorts in the analysis saw an overall net improvement in risk groups, this comparison to natural migration was not the intent of this analysis. Instead the intent was to compare the relative changes in the two cohorts. This analysis showed that the cohort who engaged in fitness activities through Vitality had a lower proportion of high risk individuals as of their first risk measure, but had a greater net improvement in risk groups as of the last measure than those who did not engage in fitness activity through Vitality

Observation:

Hmm…well we can’t both be right.  I’m looking at the exact same Dee Edington slide you are, but I am seeing the population’s risk “naturally flow” in both directions, not just “a deterioration in risk groups as people naturally flow to high risk.”  Obviously the validity of the alleged declines in your cohorts is dramatically different depending on whether one uses your interpretation of Dr. Edington’s work (in which case your results are outstanding) or mine (in which case except for 1%, they are due to the natural flow downward of the highest-risk segment).

Like Alvy Singer did with Marshall McLuhan in Annie Hall, I took the liberty of asking Dee Edington himself to referee our disagreement. This is his response:

“The correct interpretation of that slide and of my work is that the natural flow of risk in a population moves in both directions, and must be understood in order to gauge impact of an intervention.  It is not valid to simply start with people who were high-risk and claim credit for all risk reduction in that cohort while ignoring people who migrate in the other direction.”

%d bloggers like this: