Prominent figures warn of dangerous Artificial Intelligence (it’s probably a bad HF idea too)

Recently, some very prominent scientists and other figures have warned of the consequences of autonomous weapons, or more generally artificial intelligence run amok.

The field of artificial intelligence is obviously a computational and engineering problem: designing a machine (i.e., robot) or software that can emulate thinking to a high degree.   But eventually, any AI must interact with a human either by taking control of a situation from a human (e.g., flying a plane) or suggesting courses of action to a human.

I thought this recent news item about potentially dangerous AI might be a great segue to another discussion of human-automation interaction.  Specifically, to a detail that does not frequently get discussed in splashy news articles or by non-human-factors people:  degree of automation. This blog post is heavily informed by a proceedings paper by Wickens, Li, Santamaria, Sebok, and Sarter (2010).

First, to HF researchers, automation is a generic term that encompasses anything that carries out a task that was once done by a human.  Such as robotic assembly, medical diagnostic aids, digital camera scene modes, and even hypothetical autonomous weapons with AI.  These disparate examples simply differ in degree of automation.

Let’s back up for a bit: Automation can be characterized by two independent dimensions:

  • STAGE or TYPE:  What is it doing and how is it doing it?
  • LEVEL: How much it is doing?

Stage/Type of automation describes the WHAT tasks are being automated and sometimes how.  Is the task perceptual, like enhancing vision at night or amplifying certain sounds?  Or is the automation carrying out a task that is more cognitive, like generating the three best ways to get to your destination in the least amount of time?

The second dimension, Level, refers to the balance of tasks shared between the automation and the human; is the automation doing a tiny bit of the task and then leaving the rest to the user?  Or is the automation acting completely on its own with no input from the operator (or ability to override)?

If you imagine STAGE/TYPE (BLUE/GREEN) and LEVEL (RED) as the X and Y of a chart (below), it becomes clearer how various everyday examples of automation fit into the scheme.  As LEVEL and/or TYPE increase, we get a higher degree of automation (dotted line).

Degrees of automation (Adapted from Wickens et al., 2010)
Degrees of automation represented as the dotted line (Adapted from Wickens et al., 2010)

Mainstream discussions of AI and its potential dangers seem to be focusing on a hypothetical ultra-high degree of automation.  A hypothetical weapon that will, on its own, determine threats and act.  There are actually very few examples of such a high level of automation in everyday life because cutting the human completely “out of the loop” can have severely negative human performance consequences.

The figure below shows some examples of automation and where they fit into the scheme:

Approximate degrees of automation of everyday examples of automation
Approximate degrees of automation of everyday examples of automation

Wickens et al., (2010) use the phrase, “the higher they are, the farther they fall.”   This means that when humans interact with greater degrees of automation, they do fine if it works correctly, but will encounter catastrophic consequences when automation fails (and it always will at some point).  Why?  Users get complacent with high DOA automation, they forget how to do the task themselves, or they loose track of what was going on before the automation failed and thus cannot recover from the failure so easily.

You may have experienced a mild form of this if your car has a rear-backup camera.  Have you ever rented a car without one?  How do you feel? That feelings tends to get magnified with higher degrees of automation.

So, highly autonomous weapons (or any high degree of automation) is not only a philosophically bad/evil idea, it is bad for human performance!


Three months with Apple Watch

My watch face of the moment

First, a disclaimer: this isn’t a full-on review of the Watch. There are more qualified people to review the gadget.  The newest one comes from one of the best and most thorough hardware review sites: Anandtech.

One part of the review was particularly insightful:

Overall, I found that the fitness component of this watch to be a real surprise. I often hear that Apple is good at making things we didn’t know we wanted, but this is probably the first time I’ve really believed that statement. Going into the review, I didn’t really realize that I wanted a solid fitness tracker on a smartwatch, but now I’m really convinced that there is value to such features.

This has been my experience as well.  I’ve never cared to wear a fitness tracker but i’m surprised at how much I pore over the stats of my standing, activity, and workout levels.  The watch also provides a surprisingly effective level of motivation (badges & activity circles).

My activity level (for someone who sits at a desk most of the time) has dramatically increased since the watch (see right; yellow line is when I got the watch).IMG_0369

We used to think that smartphones were the “ubiquitous” technology but there are times I leave it behind.  The watch is always-on and there will be interesting use-cases and challenges in the future.  I’d love to start my car with my watch!

Some other random thoughts:

  • The fitness features are great but I wish there was a better way to view my data:
    • View splits on outdoor runs
    • View all my workouts instead of looking for them in the calendar view.
  • Many reviews i’ve read assume the watch will replace the phone.  But doing any extended activity really tires the shoulders!  My interactions are really limited to much less than 5-10 seconds.
  • I notice that haptic feedback on the wrist is much less jarring and easier to dismiss (i.e., not as disruptive) as vibrating phones on the body.
  • The Apple Watch is made for travel:
    • Most airlines have applets for the watch that make it so easy to keep track of gates, departures, & arrivals.
    • Boarding a plane with your watch feels very futuristic but most pass readers are on the right side and I wear my watch on the left resulting in very awkward wrist positions.  Even when the reader was on the left, it is facing upwards requiring me to turn my wrist downwards.
  • It is unobtrusive and looks like a watch, not like a gizmo on my wrist.
  • Apple Pay belongs on the watch.  I’ve used Apple Pay on my phone but it is much more seamless on the watch.
  • Notifications are great if you pare down what can notify you.  I only get notified of VIP mail (select senders) and text messages.
  • Controlling my thermostat, and other electrical devices from my wrist is pretty great.

Psychology Podcasts

Those who know me know I am a fiend for podcasts. Since I’m also a fiend for psychology, I can’t help but notice when it pops up in a podcast, even one not focused on psychology. I use many of them in my courses: for example, the This American Life episode on what having schizophrenia sounds like is a must listen when I hit the Abnormal Psychology chapter in Intro. The Radiolab Memory and Forgetting is a staple in my Cognitive class and I take advantage of the multi-disciplinarity of Human Factors to play clips from every area. Startup had a good one that illustrates what human factors looks like to a client.

Over the years I’ve compiled a list of my favorites relating to psychology. Some are clips from longer podcasts while some are dedicated to psychology (e.g., Invisibilia). Each one has a general area of psychology noted (although some hit two or more areas) and if it’s a clip I put the start and end time of the most related audio.

I hope you enjoy the resource and I will keep updating it as I find more. If you know of any I don’t have listed, please link to it in the comments for the blog and I’ll add it to the spreadsheet.

Rock Climbing Human Factors – Harness attachment points

A friend of mine was recently rappelling from a climb, meaning that she had the rope through a device that was connected to her belay loop on her harness. As she rappelled, she yelled that her harness broke, and the waistband of the harness slid nearly to her armpits. Fortunately, she remained calm and collected, and was still able to rappell safely, if awkwardly, to the ground. On the ground, her partner saw that her waistband with belay loop had become disconnected from her leg loops. The leg loops were intact, though a keeper-strap that helps the leg loops stay centered was no longer connected.

So, what happened?

First, for the non-climbers, a primer. A climbing harness is composed of three major parts, attached to each other in various ways depending on the manufacturer. The first part is the waistband, which is load-bearing, meaning that it is meant to take the weight of a climber.

The second part of the harness is the belay loop, a load-bearing stitched circle that connects the waistband and leg loops and is also used to hold a belay device, to hold the climber’s weight when rappelling, and for anchoring to the ground or a wall when needed.

The last part of the harness is the leg loops, which are also load-bearing in the parts that connect to the belay loop and around the legs themselves.

Figure 1 shows the general composition of climbing harnesses, with these three parts diagrammed in the Base Concept.

Figure 1. Simplified diagrams of climbing harnesses.

On most harnesses, the leg loops are kept connected to the belay loop by a “keeper strap.” This is usually a weak connection not meant to bear weight, but only to keep the leg loops centered on the harness (shown in blue in figure 1). In the case study that prompted this blog post, the keeper strap was connected through the belay loop, rather than the full-strength leg loops (figure 2.) When loaded, it came apart, separating the leg loops from the waistbelt. My own tests found that the keeper strap can be very strong, when it is loaded on the strap itself. But if the leg loops move so that the keeper buckle is loaded by the belay loop, it comes apart easily.

Figure 2. Harness assembled with keeper strap bearing weight via the belay loop.

There are two ways to mis-attach leg loops to the belay loop of a harness. The first way is by connecting the leg loops back to the harness, after they were removed, using the keeper strap. The video below demonstrates this possibility. Once connected, the harness fits well and gives little indication the leg loops are not actually connected to bear weight.

The second (and I think more likely) way is by having the leg loops disconnected from the back of the harness, usually for a bathroom break or to get in and out of the harness. The leg loops are still connected in the front of the harness, but if a leg loop passes through the belay loop, suddenly the keeper strap is load bearing when the leg loops flip around. However, the harness does not fit differently nor does it look particularly different unless carefully inspected. Video below.

The non-load bearing parts of the harness are what determine the possibility for this error. In figure 1, some harnesses either do not allow disconnection of the leg loops in back or only allow their disconnection in tandem. When the leg loops are connected in this way, the front of the leg loops cannot be passed through the belay loop. Video demonstration below.

Back to figure 1, some harnesses allow the disconnection of leg loops for each leg. If these are disconnected, a loop may be passed through the front belay loop, resulting in the error in figure 2.

In sum, this error can be examined for likelihood and severity. It is not likely that the error occurs, however if it does occur it is likely it will go undiscovered until the keeper strap comes apart. For severity, the error could be lethal, although that is not likely. The waistbelt will hold the climber’s weight and having leg loops and a waistbelt is a (comfortable) redundancy. However, the sudden shock of suddenly losing support from the leg loops could cause loss of control, either for an un-backed-up rappell or while belaying another climber.

What are the alternatives?

  • Climbing is exploding, particularly climbing in gyms. The “gym” harnesses, with fewer components and gear loops (Figure 1), are a good option for most climbers now. However, there is little guidance about what harness one should buy for the gym vs. outdoor versatility so few probably know this harness is a good option.
  • Some harnesses are designed to be load-bearing at all points (i.e., “SafeTech” below). It is impossible to make an error in leg loop attachment.
  • safetech

  • Harnesses with permanently attached leg loops or loops that attach in the back with a single point are unlikely to result in the error.
  • Many climbers reading this are thinking “This would never happen to me” or “You’d have to be an idiot to put your harness together like that” or my usual favorite “If you wanted climbing to be perfectly safe, you shouldn’t even go.” Blaming the victim gives us a feeling of control over our own safety. However, there are other instances where gear was assembled or re-assembled incorrectly with tragic consequences. No one (or their child) deserves to pay with their life for a simple mistake that can be prevented through good design.

    Parking sign re-design

    I’ll be the first to admit that I experience cognitive overload while trying to park. When there are three signs and the information needs to be combined across them, or at least each one needs to be searched, considered, and eliminated, I spend a lot of time blocking the street trying to decide if I can park.

    For example, there might be a sign that says “No parking school zone 7-9am and 2-4pm” combined with a “2 hour parking only without residential permit 7am-5pm” and “< —-Parking” to indicate the side of the sign that’s open. It’s a challenge to figure out where and how long I can park at 1pm or what happens at 7pm.

    Designer Nikki Sylianteng created new signs for parking in Los Angeles that incorporated all information into a single graphic.

    I still have some difficulty in going back and forth to the legend at the bottom, but probably just because I’ve never seen the signs before. Otherwise, one just needs to know the time and day of the week.

    An interview with her can be found in the LA Weekly where she describes mocking up a laminated example in NY and asking people for feedback on the street via sharpies. (Yay for paper prototypes!) An NPR story focused on the negative reactions of a few harried LA denizens, who predictably said “I like how it was,” but I’d like to see some timed tests of interpreting if it’s ok to park. I’d also like to suggest using a dual-task paradigm to put parkers under the same cognitive load in the lab as they might experience on the street.

    As for NY parking signs – I still can’t parse them.

    The Square Cash Disappearing Act

    Square Cash is a great service – it allows you to send money via an email with no service charge if you’re using your debit card. You can receive money without entering a PIN. I use it all the time to divide up restaurant bills among my friends. That said, I found a usability issue yesterday that I wanted to share.

    I needed to link my debit card to the app, so I followed their very simple instructions for entry. 

    The first screen asks for the card number. The number pad is telephone-order rather than number pad-order. This is on a phone, so that makes sense even if I’m much more used to entering these numbers using a keyboard.


    Next, the expiration date. On my card, the expiration date is 09/16/2016*, so I start to enter it.


    Here is the screen as you start to enter the date:

    I then proceeded to enter 09/16 as I looked at my card, then the CCV, and got an error message about an incorrect card number. Tried again. Same. Did this four times before I realized that the expiration date was month/year. It isn’t as though I’d never seen this, or been asked to enter just the month and year from a card, so I thought hard about what tricked me.

    I concluded it was the difference between the second and third screens – the guidance is there before you start typing, but as soon as you put in any number for the date, the guidance disappears. Since I was looking down at my card, I just entered what I saw and didn’t think enough to check – especially since it called for ##/##, which matched the month and day on my card, not ##/####, which could only be a month and year.

    You are welcome to blame the user for this one, but it would be a small fix to keep the background guide visible during entry.

    *No, I’m not dumb enough to put my real card number or expiration date in the pictures for this post. :-)

    Short Course in Anthropometry

    NPR just ran an extremely detailed article on the importance and study of anthropometry. The topic is the undue stress nursing places on the spine, even when “proper” lifting procedures are followed. Highlighed is the work of Bill Marras (recent Editor of the journal Human Factors), who developed a sensor rig for the forces experienced by the spine. Read for yourself, but if you want the high points:

    “Moving and lifting patients manually is dangerous even for veteran nursing staff, Marras says, for several reasons:

    • The laws of physics dictate that it’s easiest to lift something when it’s close to your body. But nursing employees have to stand at the side of the bed, relatively far from the patient.

    • Nursing employees also often bend over the patient. That’s important, because there’s a chain of bones along the spine, called facet joints, hidden under the little bumps protruding under the skin. Those bones interconnect and help absorb loads when standing straight. Marras says that when nurses lift as they’re bending, those bones disengage and their disks take most of the force. Those forces are “much, much higher than what you’d expect in an assembly line worker,” he says.

    • When nurses keep working under these loads, it causes microscopic tears in the “end plates,” which are films as thin as credit cards above and below each disc. Those tears lead to scar tissue, which can block the flow of nutrients into the disks — until, eventually, the disks start to collapse. “You could be doing this damage [to your back] for weeks or months or years, and never realize it,” says Marras. “The event that caused you to feel the problem is just the straw that broke the camel’s back.”

    The final conclusion was that people cannot lift other people safely. Assistive machines are needed, and as the article points out, hospitals do not have them.

    Not blaming the user since 2007!

    %d bloggers like this: