Category Archives: expertise

Warning against overgeneralizing in UX

I enjoyed this article by Matt Gallivan, Experience Research Manager at AirBnB, about the tendency of experts to overgeneralize their knowledge. I try to watch out for it in my own life: When you’re an expert at one thing, it’s so easy to think you know more than you do about other areas.

Excerpt:

Because if you’re a UX researcher, you do yourself and your field no favors when you claim to have all of the answers. In the current digital product landscape, UX research’s real value is in helping to reduce uncertainty. And while that’s not as sexy as knowing everything about everything, there’s great value in it. In fact, it’s critical. It also has the added bonus of being honest.

Somewhat related, here is a fun page analyzing where and why AirBnB succeeds at usability.

Rock Climbing Human Factors – Harness attachment points

A friend of mine was recently rappelling from a climb, meaning that she had the rope through a device that was connected to her belay loop on her harness. As she rappelled, she yelled that her harness broke, and the waistband of the harness slid nearly to her armpits. Fortunately, she remained calm and collected, and was still able to rappell safely, if awkwardly, to the ground. On the ground, her partner saw that her waistband with belay loop had become disconnected from her leg loops. The leg loops were intact, though a keeper-strap that helps the leg loops stay centered was no longer connected.

So, what happened?

First, for the non-climbers, a primer. A climbing harness is composed of three major parts, attached to each other in various ways depending on the manufacturer. The first part is the waistband, which is load-bearing, meaning that it is meant to take the weight of a climber.

The second part of the harness is the belay loop, a load-bearing stitched circle that connects the waistband and leg loops and is also used to hold a belay device, to hold the climber’s weight when rappelling, and for anchoring to the ground or a wall when needed.

The last part of the harness is the leg loops, which are also load-bearing in the parts that connect to the belay loop and around the legs themselves.

Figure 1 shows the general composition of climbing harnesses, with these three parts diagrammed in the Base Concept.

harness_leg_loop_error
Figure 1. Simplified diagrams of climbing harnesses.

On most harnesses, the leg loops are kept connected to the belay loop by a “keeper strap.” This is usually a weak connection not meant to bear weight, but only to keep the leg loops centered on the harness (shown in blue in figure 1). In the case study that prompted this blog post, the keeper strap was connected through the belay loop, rather than the full-strength leg loops (figure 2.) When loaded, it came apart, separating the leg loops from the waistbelt. My own tests found that the keeper strap can be very strong, when it is loaded on the strap itself. But if the leg loops move so that the keeper buckle is loaded by the belay loop, it comes apart easily.

errorharness
Figure 2. Harness assembled with keeper strap bearing weight via the belay loop.

There are two ways to mis-attach leg loops to the belay loop of a harness. The first way is by connecting the leg loops back to the harness, after they were removed, using the keeper strap. The video below demonstrates this possibility. Once connected, the harness fits well and gives little indication the leg loops are not actually connected to bear weight.

The second (and I think more likely) way is by having the leg loops disconnected from the back of the harness, usually for a bathroom break or to get in and out of the harness. The leg loops are still connected in the front of the harness, but if a leg loop passes through the belay loop, suddenly the keeper strap is load bearing when the leg loops flip around. However, the harness does not fit differently nor does it look particularly different unless carefully inspected. Video below.

The non-load bearing parts of the harness are what determine the possibility for this error. In figure 1, some harnesses either do not allow disconnection of the leg loops in back or only allow their disconnection in tandem. When the leg loops are connected in this way, the front of the leg loops cannot be passed through the belay loop. Video demonstration below.

Back to figure 1, some harnesses allow the disconnection of leg loops for each leg. If these are disconnected, a loop may be passed through the front belay loop, resulting in the error in figure 2.

In sum, this error can be examined for likelihood and severity. It is not likely that the error occurs, however if it does occur it is likely it will go undiscovered until the keeper strap comes apart. For severity, the error could be lethal, although that is not likely. The waistbelt will hold the climber’s weight and having leg loops and a waistbelt is a (comfortable) redundancy. However, the sudden shock of suddenly losing support from the leg loops could cause loss of control, either for an un-backed-up rappell or while belaying another climber.

What are the alternatives?

  • Climbing is exploding, particularly climbing in gyms. The “gym” harnesses, with fewer components and gear loops (Figure 1), are a good option for most climbers now. However, there is little guidance about what harness one should buy for the gym vs. outdoor versatility so few probably know this harness is a good option.
  • Some harnesses are designed to be load-bearing at all points (i.e., “SafeTech” below). It is impossible to make an error in leg loop attachment.
  • safetech

  • Harnesses with permanently attached leg loops or loops that attach in the back with a single point are unlikely to result in the error.
  • Many climbers reading this are thinking “This would never happen to me” or “You’d have to be an idiot to put your harness together like that” or my usual favorite “If you wanted climbing to be perfectly safe, you shouldn’t even go.” Blaming the victim gives us a feeling of control over our own safety. However, there are other instances where gear was assembled or re-assembled incorrectly with tragic consequences. No one (or their child) deserves to pay with their life for a simple mistake that can be prevented through good design.

    Brilliant guard against accidents in indoor rock climbing

    For those who don’t follow news of climbing accidents as closely as I do, there has been a spate of accidents associated with the automatic belay devices (autobelays) installed at climbing gyms.

    These devices are handy to have around as they negate the need for a climbing partner, allowing one to exercise and train alone. The climber clips his or her harness into the device at the bottom of the wall, and it automatically retracts (like a seat belt) when you climb upward. At the top, you let go of the wall and the device lowers you slowly back to the ground. You are probably imagining that the accidents had to do with failures of the equipment – while that is not unheard of, the most recent issues have all been with climbers forgetting to clip into the system at all.

    The most recent tragedy occurred this past September, where an experienced climber died after a fall in a Texas gym, and it’s been listed as so common it happens at “every gym,” though not always resulting in a fall. Here is the facebook page with members of another gym discussing a similar accident.

    fb

    If you talk with climbers or read accident forums you will invariably be faced with a large contingent bent on blaming the victim. I’ll grant that it is hard to imagine forgetting to clip into a safety device and climb 30 feet up a wall, but that’s because I hardly ever do it. One characteristics these accidents share is that the victims were experienced and used the auto-belays frequently.

    When a procedure becomes automatic, it becomes more accurate and less effortful, but it also becomes less accessible to the conscious mind. When a step is skipped, but all other steps are unaffected, it’s especially hard to notice the skipped step in an automatic process. If caring more or working harder or “being more careful” could actually prevent this type of problem, we wouldn’t have any toddlers left in hot cars, perfectly good airplanes flown into the ground, or climbers falling because they didn’t clip into the autobelay.

    That brings me to the device I saw installed at a climbing gym last night.

    guardAbove: The guard in place, clipped to the wall and ready to go. Notice how it blocks the footholds of the climbs.

    photo 2Above: Nikki shows how to unclip the guard before attaching to her harness.

     

    photo 4Above: Clipped in and safely ready to go. Guard is on the ground and out of the way (it is ok to step on it!)

    Let me tell you why I think this is brilliant.

    • It’s highly visible.
    • It functions as a guard. This adheres to the hierarchy of safety: First, try to design out the hazard. Second, guard against the hazard. Last, warn. These are in order of effectiveness. Prior to this device, I had only seen signs on the wall saying “Clip in!” (And a year ago, even those didn’t exist.) This device physically blocks the start of the climbing routes, demanding interaction before one starts climbing.
    • Using it properly does not add any additional time or mess to climbing a route. If it weren’t there, the climber would still have to unclip the autobelay from an anchor close to the ground, etc. With it there, the climber does the same thing and once done, the guard becomes a flat mat that doesn’t get in anyone’s way.

    Is it perfect? No. You can also climb with a belayer on the same or nearby routes, and then it’s also blocking your way at the start of the climb. Some adaptation should be made by the route-setters at the gyms to minimize this. But overall, what a great and simple solution.

     

    Excerpts from the NASA ASRS

    One of my students last semester (thanks, Ronney!) turned me on the “Callback” publication from the NASA Aviation Safety Reporting System. These are almost all first person stories written as case studies of errors and accidents or near accidents. There aren’t so many that it falls under my list of neat databases, but it certainly is interesting reading.

    I’ve collected a few below to give a taste of the stories that are included. These are just the top level descriptions – click through to read the first person accounts.

    From Issue 381 Upside Down and Backwards

    1. “An aircraft Mode Selector Panel that “looks the same” whether right side up or upside down, and that can be readily installed either way, is a good example of a problematic design. Confronted with an inverted panel, this Cessna 560 Captain found out what happens when the wrong button is in the right place. “
    2. “Without detailed instructions and clear notation, nearly symmetrical parts can be installed incorrectly. Faced with the replacement of such a part, this CRJ 700 Maintenance Technician wound up with a case of component “misorientation.”

    From Issue 383 When Practice Emergencies Go Bad

    1. “…a C182 pilot performed a simulated engine failure while undergoing a practical examination. It appears that both the examiner and the examinee were so engrossed in the simulated emergency that they both tuned BEEEEP out BEEEEP the BEEEEP gear BEEEEP warning BEEEEP horn.”
    2. “When faced with a real engine failure, performing the Engine Secure Checklist reduces the chance of a fire on landing. However, actually performing the steps in the Engine Secure Checklist when the engine failure is not real can lead to a real problem.”

    From Issue 382 Fly the Airplane!

    1. “A review of recent ASRS reports indicates that failure to follow one of the most basic tenets of flight continues to be a concern when pilots are faced with distractions or abnormal situations.”

    From Issue 376 The Fixation Factor

    1. “The ability to maintain the “big picture” while completing individual, discrete tasks is one of the most critical aspects of working in the aviation environment. Preoccupation with one particular task can degrade the ability to detect other important information. This month’s CALLBACK looks at examples of how fixation adversely affects overall task management.”
    2. “Advanced navigation equipment can provide a wealth of readily available information, but as this Cirrus SR20 pilot learned, sometimes too much information can be a distraction.”

    From Issue 375 Motor Skills: Getting Off to a Good Start

    1. “The Captain of an air carrier jet experienced a very hot start when distractions and failure to follow normal flow patterns altered the engine start sequence.”
    2. “This pilot was familiar with the proper procedures for hand-propping, but despite a conscientious effort, one critical assumption led to a nose-to-nose encounter.”

    Photo credit smartjunco @ Flickr

     

     

     

    Verdict Reached for Air France Rio Crash

    The BBC has reported the incident analysis of the Air France crash that killed 228 people was due to lack of pilot skill in dealing with a high altitude stall.

    Here is a link to the BEA Report from the Bureau d’Enquetes et d’Analyses. It’s a frightening read, as they give a moment by moment analysis of the last minutes in the cockpit. No emergency was ever noted and there did not appear to be any mechanical failures. It appeared that the flight crew thought events were under control the entire time (despite the alarms.)

     

     

    Photo credit Vin Crosbie at Flickr.

    “Sully” Sullenberger to Speak at the HFES 2010 Conference

    I received word today that Captain Chesley “Sully” Sullenberger will give the keynote address at the 2010 Human Factors and Ergonomics Conference in San Francisco this October.

    Not only am I excited to hear him speak, I am excited because he is the perfect choice for a Human Factors audience: he has spoken publicly on interface and instruction issues in aviation and should have an interesting take on responding to an emergency in the midst of a complex task (albeit one he was well-trained for).

    Check out this clip of him on The Daily Show, where he talks about the disconnect between design for everyday use versus emergency use when referring to tabs on emergency manuals. The HF starts right at 3 minutes.

    Here is a 3-D recreation of the flight complete with audio tape.

    I often use this clip in presentations when discussing expert performance.

    Photo credit Jim Davidson

    HFBlog at HFES 2009 in San Antonio

    We were at the Human Factors and Ergonomics Society Annual conference in San Antonio.  This post is merely here to collect our six-part posts on various talks.  These only scratch the surface of what was presented.

    HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures

    I‘d like to highlight some of the talks I enjoyed last week and point our readers to their research.

    First up, we have:

    The Influence of Rating Method on Knowledge Structures.
    Chad C. Tossell, Rice U.; Brent A. Smith, U.S. Air Force Academy; Roger W. Schvaneveldt, Arizona State U., Polytechnic

    This talk was a great introduction to understanding how we organize information in the brain as we change from novices to experts in a task. This isn’t something I’ve done in my work, so I learned a great deal in the 15 minute presentation.

    In a nutshell, one can use a technique called Pathfinder to see how people link concepts associated with a job or task. These links can be analyzed to see how people change the organization of the knowledge as they become experts. For example, a novice would have a very different understanding of how the parts of an engine depend on one another, or at best only show topical links between parts of the system. An expert mechanic would organize the parts of the engine differently. Usually the knowledge structure of one expert is very similar to that of other experts (while novices vary greatly.)

    Unfortunately, using the Pathfinder technique is time consuming and arduous. The current talk focused on simplifying the technique while preserving the integrity of the findings.

    Continue reading HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures