Category Archives: health/healthcare

Wearable Fitness Trackers: A Comparative Usability Evaluation

This guest post is from graduate students Haley Vaigneur and Bliss Altenhoff. Haley and Bliss compared the usability of two fitness trackers as part of a graduate course in health informatics taught by Kelly Caine.

trackers

Wearable fitness trackers allow users to track and monitor their health. While these devices originated as a way for doctors to monitor chronically ill patients’ vitals, they have recently been developed and marketed for to a more general, health-conscious market. Equipped with advanced sensors such as accelerometers, users’ activity and sleep can be automatically tracked and then compared with their logged fitness goals and daily diet. Users can then use their statistics to help create or maintain a healthier lifestyle. Two examples of such devices are the Jawbone Up and Fitbit Flex, shown above.

Wearable technology is popular and has the potential to dramatically impact health (e.g. long-term health and activity data tracking, immediate syncing with Electronic Health Records (EHRs)). But these benefits can only be realized if the user is able to effectively use and understand these devices. This was the motivation for focusing on two of the most popular models of fitness trackers: the JawBone Up and FitBit Flex and their accompanying smartphone apps.

This study examined the usability of these two devices and their accompanying smartphone apps by having 14 participants (7 for Jawbone Up, 7 for FitBit Flex) perform a think-aloud test on five key features: Setup, Setting Goals, Tracking Diet, Tracking Activity, and Setting an Alarm. Participants then kept the wearable for three days and were encouraged to incorporate it into their normal routine. On the third day, participants completed the System Usability Scale survey and an informal interview regarding their experiences using the wearable.

Some of the key Jawbone UP findings were:

  1. Adding food or drink items was somewhat difficult due to unintuitive organization and unpredictable bugs. For example, one participant attempted to add a food item by scanning the bar code of a Lunchable, but the app added a Dr. Pepper to the log.
  2. Participants struggled to find the alarm settings, with one conducting a general web search for help to understand the Smart Sleep Window settings and how to save alarm settings.
  3. None of the participants were able to figure out how to communicate to the band or app that they would like to begin a workout. They didn’t realize that the Stopwatch menu option was intended to time the workout.

Some of the key findings of the FitBit Flex were:

Setting goals
Setting goals
wheretotap
What do I tap?
  1. Participants felt that the wristband (when using the appropriate sized band) was not uncomfortable or revealing and they were proud to wear it because it made them feel healthy.
  2. Users had a difficult time figuring out where to go on the app to set their health goals at first. Their instinct was to find it on the app homepage, or Dashboard, but it was under the Account tab.
  3. Some users had difficulty putting on the wristband, and several noted that it fell off unexpectedly. Users were also confused about where to “tap” the wristband to activate it, based on the instructions given in the app. The picture can appear to instruct the user to tap below the black screen, when the user actually needs to tap the screen directly, and firmly.
  4. Users did not realize that after turning Bluetooth on their phone, they needed to return to the app to tell the phone and wristband to begin syncing. They also noted that leaving Bluetooth on all day drained their phone battery.

    init
    Bluetooth confusion

Based on time per task and number of errors the FitBit Flex performed better than the Jawbone Up on the five tasks. Users’ ultimate trust in the data, willingness to continue using the wearable, and general satisfaction with each wearable was heavily influenced by their initial experiences (first day). The positive initial think-aloud results for the FitBit Flex were also consistent with a more positive later experience and stronger acceptance of the wearable.

This study found that there is still much room for improvement in the usability of the accompanying smartphone apps. A major concern for these kinds of devices is keeping user interest and motivation, which can easily be lost through confusing or cumbersome designs. By striving to improve the human factors of the apps simultaneous to the capabilities of the actual wearables, there is great potential for greater user satisfaction, and thus more long-term use.

While activity tracking wearables are currently most popular with more tech-savvy, active people, these devices should be designed to be used by all ages and levels of experience users. These devices could change health monitoring drastically and give people the power and ability to make better choices, and live healthier lifestyles.

Haley Vaigneur is a graduate student in Industrial Engineering at Clemson University. Her concentration is Human Factors and Ergonomics, emphasizing on research in the healthcare field.

Bliss Altenhoff is a Doctoral Candidate studying Human Factors Psychology at Clemson University, where she received her M.S. in Applied Psychology in 2012.  She is a member of the Perception and Action (PAC) lab, where her research is concentrated on enhancing human perception and performance by enriching perceptual display technologies for laparoscopic surgeons. .

ACKNOWLEDGMENTS
This material is based upon work supported by the National Science Foundation under Grant No. 1314342. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Tylenol: A narrow margin of safety

I’m always surprised at how many people don’t know how dangerous Tylenol (or anything with acetaminophen/paracetamol) is. I can’t remember where I initally learned that just a few pills could cause liver failure, but it was at least 10 years ago. Of course, more than 10 years ago, I remember that Tylenol was the only painkiller allowed to be given out in my high school because it was so “safe.”

Anyway, it’s certainly been in the news lately and I have collected some interesting sources. First off, This American Life devoted an hour to investigating the problem, its history, and ongoing conflict. Absolutely worth a listen. Here is a teaser bit of the transcript:

Will Lee is one of the liver specialists and researchers who’s been calling attention to the dangers of acetaminophen. He is at the University of Texas Southwestern Medical Center and talked to reporter Sean Cole.

Will Lee: In one of the articles, we actually printed up the little coupon that I got with my Zocor prescription that says, $2 off on your next acetaminophen bottle. And one of the things on the coupon, it says safest. Not safer, or not safe, but safest. Well, this is the number-one drug killing Americans every year.

Sean Cole: Over-the-counter drug killing Americans.

Will Lee: Over-the-counter drug.

Sean Cole: Not the safest.

Will Lee: Not the safest, for sure. So I guess that’s– if you think I have a bee in my bonnet, that’s probably where it came from.

There are two human factors issues related to this “narrow margin of safety” that I’d like to highlight:

1. Warning design
2. Counter-intuitive doses for children

Warnings

If you’d like a good overview of warning design, see Wogalter, Conzola, & Smith-Jackson, 2002.

The warnings are printed in red on the lid of the container and say “Contain acetaminophen – Always read the label.”
warning
I know it took a long time to even have this, but I’d like to point out that this does not follow the best practices for warning design because it does not include the consequences or focus on the “hidden hazard.” Hidden hazards are really the reason warnings are important. Here are some examples:
  1. Does a knife need a warning that it is sharp? No, that’s not a hidden hazard. Now, if the knife is SO sharp it’s unexpectedly dangerous, then it needs a warning that conveys that information. I had a friend in high school who had a habit of testing knives with his fingertip. This never cuts you, just gives a feel for sharpness. Well, one time he had someone sharpen a swiss army knife for him – and what usually would have “felt sharp” went instantly through the skin down to the bone. That’s a hidden hazard.
  2. Does coffee need a warning that it’s hot? No, not if it’s just drink-ably hot. But yes if it is purposefully super-heated enough to cause 3rd degree burns. Watch this video to get new insight on the McDonald’s coffee story.

It is the narrow margin of safety that is the hidden hazard of acetaminophen. Nothing else available over the counter can kill you with such a small increase in dose. You can take 20x the recommended dose of Advil before it becomes threatening. I ignored the dosing for Advil once and took 6 in 3 hours rather than 2 – it made me sick and I regretted it, but would I deserve to die for that mistake? I could have died if it had been Tylenol instead of Advil. This hidden hazard is the most critical part of the warning – even if it’s on the label, I think it should be highlighted on the lid.

Which brings me to… the hidden hidden hazard that was created by the Tylenol company.

Counter-intuitive doses for children
In brief, for over a decade Tylenol provided medication labeled for infants and medication labeled for children (and of course, medication for adults). 99/100 people on the street would assume that these were in order of strength – least for the infants, more for the kids, and most for the adults. That assumption was correct for the kids and the adults. The counter-intuitive dose was for the infants: instead of being weaker because they were smaller, it was much higher. Their reasoning was that it is hard to get medication into an infant, so if it’s stronger you can get an effect with less. An article from Pro-Publica gives a more detailed summary.
An excerpt from Pro-Publica:

The two types of pediatric Tylenol had a counterintuitive difference. Drop for drop, the strength of Infants’ Tylenol far exceeded that of Children’s Tylenol.

In addition, the active ingredient in Tylenol, acetaminophen, has what the FDA deems a narrow margin of safety. The drug is generally safe at recommended doses, but the difference between the dose that helps and the dose that can cause serious harm is one of the smallest for any over-the-counter drug.

By confusing the pediatric products and administering too much of the infants’ version, parents could inadvertently overdose their children. Other manufacturers also made two children’s products with different concentrations of acetaminophen.

Between 2000 and 2009, the FDA received reports of 20 children dying from acetaminophen toxicity – a figure the agency said likely “significantly underestimates” the problem. Three deaths were tied directly to mix-ups involving the two pediatric medicines. Such errors may have caused some of the other deaths, but the agency has acknowledged that its data lacks sufficient detail to determine the precise cause.

This American Life included a heartbreaking case where parents were instructed to give their infant doses measured in terms of the children’s Tylenol (their doctor just said “Tylenol”), but since they had an infant they did what anyone would and used the Tylenol for infants, resulting in the death of their baby.

The big push seems to be for better warnings (with the example in the picture as “better.”) Indeed, this can have an impact – for example, as reported in the NY Times, removing infant versions of medication and saying a drug is “not for children under 2” reduced drug-related emergency room visits for that age range by half.

Excerpt from the NY Times article:

In 2007, amid mounting concern that infant cough and cold medicines were unsafe and misused, manufacturers voluntarily withdrew products intended for children younger than 2. The makers revised the labels on the rest of the medicines, which now warn parents that they should not be given to children younger than 4.

Government researchers said on Monday that those moves have had a remarkable effect: a significant decrease in emergency hospital visits by toddlers and infants with suspected medical problems after using these medicines.

Dr. Daniel Frattarelli, a former chairman of the committee on drugs at the American Academy of Pediatrics, praised the study, saying it showed that “the label is a very powerful tool for changing parent behavior.”

In the new study, published in the journal Pediatrics, researchers at the Centers for Disease Control and Prevention reviewed data from 63 hospitals to estimate the number of emergency visits from 2004 to 2011 by young children who had taken cough and cold medicines.

Children under 2 accounted for 4.1 percent of all emergency visits for suspected drug-related effects before the 2007 withdrawal, the researchers found, and accounted for 2.4 percent afterward.

However, even casual readers of the blog have probably noted how often I mention the Hierarchy of Safety: first try to design out the hazard, guard against the hazard, and warn. I’m not saying warnings aren’t important, but if we want to have the biggest impact we should be working on designing out or guarding against the hazard. Some ideas in that realm include bottle design that restricts the flow of liquid (this could prevent a child from dosing him or herself, but not parents from giving the wrong dose), and packaging “single servings” of medication, so that its obvious how much to give at one time. Although the treehugger in me isn’t a fan of more packaging, this also could provide more space for good warnings AND have those warnings in extremely close spatial and temporal proximity to use of the product.

 

Usability process not used for ACA website

slideA recently released report, done in March 2013, reveals the process of creating Healthcare.gov. Hindsight is always 20/20, but we’ve also worked hard to establish best practices for considering both engineering and the user in software development. These contributions need to be valued, especially for large scale projects. After looking through the slides, one thing I note is that even this improved approach barely mentions the end users of the website. There is one slide that states “Identify consumer paths; review and modify vignettes.” The two examples of this are users who have more or less complex needs when signing up for insurance. I don’t see any mention of involving actual users prior to release.

The NPR write-up states:

Consultants noted there was no clear leader in charge of this project, which we now know contributed to its disastrous release. And there was no “end-to-end testing” of its full implementation, something we now know never happened.

Some of this may fall on us, for not being convincing enough that human factors methods are worth the investment. How much would the public be willing to pay for a solid usability team to work with the website developers?

Human-Technology Interactions in Health

multisense

Coincidentally, the topic of social/human-technology interaction is in the news quite a bit today.  I’m pleased that the topic of the human factors implications of the social interaction with technology is getting more focus.

First, Dr. Wendy Rogers of Georgia Tech gets interviewed in the New York Times about her work on older adults and in-home helper robots:

Dr. Rogers has been experimenting with a large robot called the PR2, made by Willow Garage, a robotics company in Palo Alto, Calif., which can fetch and administer medicine, a seemingly simple act that demands a great deal of trust between man and machine.

“We are social beings, and we do develop social types of relationships with lots of things,” she said. “Think about the GPS in your car, you talk to it and it talks to you.” Dr. Rogers noted that people developed connections with their Roomba, the vacuum robot, by giving the machines names and buying costumes for them. “This isn’t a bad thing, it’s just what we do,” she said.

In a more ambitious use of technology, NPR is reporting that researchers are using computer-generated avatars as interviewers to detect soldiers who are susceptible to suicide. Simultaneously, facial movement patterns of the interviewee are recorded:

“For each indicator,” Morency explains, “we will display three things.” First, the report will show the physical behavior of the person Ellie just interviewed, tallying how many times he or she smiled, for instance, and for how long. Then the report will show how much depressed people typically smile, and finally how much healthy people typically smile. Essentially it’s a visualization of the person’s behavior compared to a population of depressed and non-depressed people.

While this sounds like an interesting application, I have to agree with with one of its critics that:

“It strikes me as unlikely that face or voice will provide that information with such certainty,” he says.

At worst, it will flood the real therapist with a “big data”-type situation where there may be “signal” but way too much noise (see this article).

Prescription Smartphone Apps

I recently published a study (conducted last year) on automation trust and dependence. In that study, we pseudo-wizard-of-oz’ed a smartphone app that would help diabetics manage their condition.

We had to fake it because there was no such app and it would be to onerous to program it (and we weren’t necessarily interested in the app, just a form of advanced, non-existent automation).

Now, that app is real.  I had nothing to do with it but there are now apps that can help diabetics manage their condition.  This NYT article discusses the complex area of healthcare apps:

Smartphone apps already fill the roles of television remotes, bike speedometers and flashlights. Soon they may also act as medical devices, helping patients monitor their heart rate or manage their diabetes, and be paid for by insurance.

The idea of medically prescribed apps excites some people in the health care industry, who see them as a starting point for even more sophisticated applications that might otherwise never be built. But first, a range of issues — around vetting, paying for and monitoring the proper use of such apps — needs to be worked out.

The focus of the article is on regulatory hurdles while our focus (in the paper) was how potential patients might accept and react to advice given by a smartphone app.

(photo: Ozier Muhammad/The New York Times)

Recent Research Potpourri

Just a small plug for some recent research from Anne and me. The topics are ones that we’ve discussed on the blog:  games and cognitive training and e-health tools.  First, Anne and colleagues recently published a paper showing that playing World of Warcraft can boost some measures of cognitive abilities in older adults:

The effectiveness of a game-based cognitive training intervention on multiple abilities was assessed in a sample of 39 older adults aged 60–77. The intervention task was chosen based on a cognitive task analysis designed to determine the attentional and multi-modal demands of the game. Improvements on a measure of attention were found for the intervention group compared to controls. Furthermore, for the intervention group only, initial ability scores predicted improvements on both tests of attention and spatial orientation. These results suggest cognitive training may be more effective for those initially lower in ability.

Meanwhile, my group has recently published a paper looking at the barriers to older adults’ adoption of electronic personal health records:

Electronic personal health records (PHRs) have the potential to both make health information more accessible to patients and function as a decision-support system for patients managing chronic conditions. Age-related changes in cognition may make traditional strategies of integrating and understanding existing (i.e., paper-based) health information more difficult for older adults. The centralized and integrated nature of health information, as well as the long-term tracking capabilities present in many PHRs, may be especially beneficial for older patients’ management of health. However, older adults tend to be late adopters of technology and may be hesitant to adopt a PHR if the benefits are not made clear (perceived usefulness). Toward the design of a useful PHR, a needs analysis was conducted to determine how people currently manage their health information, what they perceive as useful, and to identify any unmet needs. This paper describes two qualitative studies examining the health information needs of both younger and older adults. The first study used a 2-week diary methodology to examine everyday health questions or concerns, while the second study examined maintenance of health information and perceptions of PHRs through the use of a three-part interview. User’s perceptions of the usefulness of PHRs are provided as recommendations for the design of e-health technology, especially those targeted for older adult healthcare consumers. The results suggest that both older and younger adults would deem a PHR useful if it provides memory support in the form of reminders, provides tools to aid in comprehension of one’s health concerns, is interactive and provides automatic functions, and is highly accessible to authorized users, yet one’s information is kept secure and private.

Bad Usability Causes Cranky Babies

I peripherally heard about another Tylenol recall and assumed the recall was prompted by tainted medicine or something.  Anne just sent me a link to the story and it is apparently usability related.  The syringe-based dosing system, called SimpleMeasure, seems to be difficult to use.  Here is what NPR says:

the “SimpleMeasure” dosing system that’s supposed to make it easier to fill a syringe with the right amount of the grapey painkiller and fever-reducer is too complicated for some parents.

Fun with confusing medication names!

Check out this post from The Consumerist about how unhappy the FDA is with Durezol and Durasal.

A hint: It’s ok if you accidentally use Durezol when you wanted Durasal, but the penalty is high for using Durasal instead of Durezol!*

This link contains an explanation of the names:

When drugs are submitted to the FDA for approval, the Agency carefully screens their proprietary names for similarities. However, Durasal (salicylic acid) is an OTC medication that did not undergo the approval process. That is why the two names exist side-by-side in the pharmacies.

Thus far, the FDA is asking pharmacists to “be vigilant.” I think we know how that usually plays out.

 

*Durezol is eye drops. Durasal is wart remover.

Coming to APA 2011: A Conversation Hour on Use of Electronic Health Records in Clinical Practice

Drs. Kelly Caine (of guest post fame)  and Dennis Morrison will be presenting on human factors considerations for the design and use of electronic health records.  Audience participation is welcome as they discuss this important topic. See abstract below.

In this conversation hour we will discuss the use of electronic health records in clinical practice. Specifically, we will focus on how, when designed using human factors methods, electronic health records may be used to support evidence based practice in clinical settings. We will begin by giving a brief overview of the current state of electronic health records in use in behavioral health settings, as well as outline the potential future uses of such records. Next, we will provide an opportunity for the audience members to ask questions, thus allowing members to guide the discussion to the issues most relevant to them. At the conclusion of the session, participants will have a broader understanding of the role of electronic health records in clinical practice as well as a deeper understanding of the specific issues they face in their practice. In addition, we hope to use this conversation hour as a starting point to generate additional discussions and collaborations on the use of electronic health records in clinical practice, potentially resulting in an agenda for future research in the area of electronic health records in clinical behavioral health practice.

Kelly Caine is the Principal Reserach Scientist in the Center for Law, Ethics, and Applied Research (CLEAR) Health Information.

Dennis Morrison is the CEO of the non-profit Centerstone Research Institute.

Check out the full Division 21 program.

Treemapping Your Way to Healthier Food Choices

Renee Walker, MFA in Design, came up with an innovative use of treemaps to provide nutrition information – winning the Rethink the Food Label contest.

Can you imagine the horror of food companies once they realize how much of their treemap has to say SUGAR? This visualization is certainly easier than the rule of thumb I was taught: “If sugar is one of the first three ingredients, it’s a lot of sugar.” Just look at how easy it is to compare peanut butters in the above image.

You can go here to see all of the finalists in the label design competition.

Only tangentially related, the interactive stock market map is my favorite previous use of treemaps.


Image owned by Renee Walker.