Category Archives: conference

Calibrating User’s Perception of Automation

Last week I had the pleasure of presenting in a symposium on automation in safety critical domains arranged by Dr. Arathi Sethumadhavan at the American Psychological Association annual meeting.  My fellow participants were:

  • Arathi Sethumadhavan, PhD (Medtronic)
  • Poornima Madhavan, PhD (Old Dominion University)
  • Julian Sanchez, PhD (Medtronic)
  • Ericka Rovira, PhD (United States Military Academy)

Everyone presented on issues related to human-automation interaction.  I do not have their permission to show their slides so this post is more generally a lay-person’s description of one aspect of automation research:  consequences of perceptions of automation reliability.

One of the most popular types of news items we post is stories of when people rely too much on unreliable automation with sometimes funny or tragic consequences.  For example, when people use in-car navigation/GPS systems and slavishly follow its directives without judging conditions for themselves.

This is a classic example of a mis-match between the user’s perception of how reliable the system is and how it actually is.  See the figure below:

from Gempler & Wickens, 1998

The Y-axis is how the user perceives the system’s reliability while the X-axis is the actual reliability of the system.  Let’s focus on the two zones in the upper left and lower right represent.  When the user perceives that the automation is more reliable than it actually is (RED CLOUD) they will over-trust the automation and perhaps rely too much on its occasionally faulty advice (this is where much of the GPS horror stories lie).  People may get their mis-judgements about the reliability from many sources (marketing literature, limited use, or recommendations).

For example, my digital camera has an auto mode that claims to be able to detect many types of settings (macro, landscape, night) and automatically adjust settings to suit.  However, in practice it seems less reliable than the marketing literature suggests.  The company exhorts me to TRUST iA (their name for automation)!

So in a few situations where I over-rely on iA, I end up with images that are too dim/bright, etc.  The system doesn’t tell me how it came to its decision leaving me out of the loop.  Now, I just don’t use iA mode.

The other zone (YELLOW CLOUD) is less studied but it represents situations where the automation is actually very reliable but people perceive it as not very reliable and so will depend on it less–even when their performance degrades as a result.  Examples are more difficult to come up with but one might be the example of health aids that doctors might use to assist in diagnosis of patients.

Finally, the line in the middle is proper calibration: perceived reliability is perfectly correlated with the actual reliability of the automation.  This is where we want to be most of the time.  When our calibration is perfect, we will rely on the automation when we should and NOT when we shouldn’t.

Getting people to properly calibrate their trust and dependence on automation is a complex human factors psychological problem.

Coming to APA 2011: A Conversation Hour on Use of Electronic Health Records in Clinical Practice

Drs. Kelly Caine (of guest post fame)  and Dennis Morrison will be presenting on human factors considerations for the design and use of electronic health records.  Audience participation is welcome as they discuss this important topic. See abstract below.

In this conversation hour we will discuss the use of electronic health records in clinical practice. Specifically, we will focus on how, when designed using human factors methods, electronic health records may be used to support evidence based practice in clinical settings. We will begin by giving a brief overview of the current state of electronic health records in use in behavioral health settings, as well as outline the potential future uses of such records. Next, we will provide an opportunity for the audience members to ask questions, thus allowing members to guide the discussion to the issues most relevant to them. At the conclusion of the session, participants will have a broader understanding of the role of electronic health records in clinical practice as well as a deeper understanding of the specific issues they face in their practice. In addition, we hope to use this conversation hour as a starting point to generate additional discussions and collaborations on the use of electronic health records in clinical practice, potentially resulting in an agenda for future research in the area of electronic health records in clinical behavioral health practice.

Kelly Caine is the Principal Reserach Scientist in the Center for Law, Ethics, and Applied Research (CLEAR) Health Information.

Dennis Morrison is the CEO of the non-profit Centerstone Research Institute.

Check out the full Division 21 program.

Hope to see you at APA 2011!

There will be an extensive program for the Applied Experimental Division of the American Psychological Association at their conference in D.C. from August 4-7, and I invite all to come! We’ve collected a who’s-who from established to up-and-coming researchers to bring you cutting edge work in human factors, ergonomics, automation, human machine systems, aviation, video gaming, and much more!

Below is a listing of the program. You can find the dates and times on the APA website by selecting Division 21 from the combo box.

Special Talks

1. Presidential Address by Pat DeLucia, Texas Tech University – “Engineering Psychology for Technology”

2. Invited Address by John W. Senders, University of Toronto
Winner of the Franklin V. Taylor Award for outstanding contribution to the field of Applied Experimental and Engineering Psychology – You can read about the talk Dr. Senders gave at APA 2010.

3. Invited Address by new APA Fellow Peter Pirolli, Palo Alto Research Center, CA
“Information Foraging in the Social Web”

4. Address by the winner of the Alluisi Early Career award – Bruce Walker, Georgia Institute of Technology
“Sonification and Auditory Displays for Assistive Technology: Science and Service”

5. Address by the winner of the Briggs Dissertation award – Michael A. Nees, Georgia Institute of Technology
“Flexibility of Representation in Working Memory for Nonspeech Sounds: Theoretical and Practical Implications”

Paper session on the Psychology of Aviation

1. Durso, F. T., Pop, V., Stearman, E. J., & Kazi, S. – “How to Create a Vigilance Decrement in NextGen Air Traffic Control”

2. King, R., Carretta, T. R., & Chappelle, W. – “Ab Initio Selection of Remotely Piloted Aircraft Operators”

3. Krajewski, J., Schnieder, S., Sommer, D., & Golz, M. – “Estimating Fatigue From Simulated Air Traffic Controller Communication”

4. Bleckley, K., & Broach, D. – “Selecting Air Navigation Service Providers for the Next Generation of Air Traffic Control”

5. Gorman, J. C. – “Team Coordination Dynamics in Uninhabited Air Vehicle Operations”

6. Nelson, E., Baker, K., Gee, S., Boehm-Davis, D. A. – “The Party Line: Who’s Listening?”

Paper session focusing on Behavioral Research at George Mason University

1. Baldwin, C. L. – “Design and Examination of In-Vehicle Auditory Collision Warning Messages”

2. Peterson, M. S. – “Guiding Attention”

3. Parasuraman, R. – “Neurogenetics of Individual Differences in Working Memory and Decision Making: Implications for Selection and Training”

4. Thompson, J. – “Recognizing Human Movement”

5. Shaw, T. – “Group-Level and Individual Differences Approaches to the Study of Sustained Attention”

6. Boehm-Davis, D. A. – “Do Pilots Really Need the Party Line?”

Conversation Hour on the Symbiosis of Basic and Applied Research

Discussants:

Doug Gillan, North Carolina State University – “In Psychology, Applied Research = Basic Research = Applied Research = ….”

Tim Nichols, Microsoft Game Studios, Redmond, WA –  “New Technologies and New Modes of Interaction: Collaboration Opportunities for Industry and Academic Researchers”

Invited Address on Current NHTSA Human Factors Research Priorities

Tim Johnson, Department of Transportation, Washington, DC

Symposium session with Games to Explain Human Factors – Come, Participate, Learn, and Have Fun

Chair – Ron G. Shapiro, Providence, RI

Conversation Hour on Electronic Health Records in Clinical Practice

Discussants:

Kelly E. Caine, Indiana University Bloomington

Dennis Morrison, Centerstone Research Institute, Bloomington, IN

Symposium on Human-Automation Interaction in Safety Critical Domains

Presenters:
1. Arathi Sethumadhavan, Medtronic, Inc., Mounds View, MN
2. Julian Sanchez, Medtronic, Inc., Mounds View, MN
3. Poornima Madhavan, Old Dominion University
4. Ericka Rovira, United States Military Academy
5. Richard Pak, Clemson University

Paper Session – Human Factors Potpourri

1. Brown, C. M., Kobus, D. A. – “Changes in Cognitive Performance During Extended Periods of Heavy-Load Carriage”

2. Smarr, C., Serrano-Baquero, D., Cullen, R. H., McBride, S. E., Beer, J. M., & Rogers, W. A. – “Using Knowledge Engineering to Understand Communication Processes in the Mowing of Citrus Groves”

3. Beer, J. M., McBride, S. E., Mitzner, T. L., Springman, J. M., & Rogers, W. A. – “Challenges in Patient Education: Teaching Older Adults Home Health Care Tasks”

4. McBride, S. E., Tsai, W., Knott, C. C., & Rogers, W. A. – “Understanding User Needs for an Osteoarthritis Management Tool”

5. Lane, S., Peterson, J. V., Taylor, E. J., Jackson, J. A. – “Understanding Factors Leading to Computer Loyalty”

Conversation Hour on the Activities of the Board on Human-Systems Integration in the National Academy of Sciences

Barbara Wanchisen, National Academies, Washington, DC
“Human-Systems Integration at The National Academies: Health, Military, and Aviation Applications”

 Posters:

1. Billings, D. R., Oleson, K. E., Chen, J. Y. C., & Hancock, P. A. – “Mitigating Inappropriate Trust in Human-Robot Interactions: A Review of Trust Calibration Strategies in the Literature”

2. Morrow, J., Adler, M. C., Schleicher-Dilks, S., Oliver, T., Andrews, A. P., & Golden, C. J. – “Neuropsychological and Demographic Correlates to Impairment in Simulated Driving”

3. Dan, C. S. – “Effects of Time of Automation Failures and Expectation on Trust in Automation”

Profiles in Human Factors: Dr. Elizabeth Blickensderfer, Embry-Riddle Aeronautical University

This post is from our series of human factors career profiles. Check them all out if you’re curious about what kinds of careers you can have in this field!

Dr. Elizabeth Blickensderfer received her Ph.D. in Human Factors & Organizational Psychology the University of Central Florida and is currently an Associate Professor at Embry-Riddle Aeronautical University in Daytona Beach, FL.

Anne: Hi Beth, thanks for agreeing to an interview. Would you tell me a little about your job and what you like most about it?

Beth: I am an associate professor in the Human Factors and Systems department. I teach 2-3 classes each semester (undergraduate and graduate), and work on various funded and unfunded research projects. For example, I am currently working on two FAA funded projects. The first one is examining human-machine interface issues related to certifying Unmanned Aerial Systems (UAS) to fly in U.S. airspace. The other FAA funded project is testing the efficacy of a training course which teaches pilots to better understand weather technololgy tools. Some other projects that I am involved with focus on human error in aviation maintenance and weather related issues in Helicopter Emergency Medical Service flights. I also oversee master’s theses on a variety of topics at Embry-Riddle.

One aspect that I LOVE about my job is that I am able to balance teaching with research. I love my students and interacting with them in the classroom and mentoring them as they develop their human factors/erogonomics knowledge and skills. I also love staying involved in sponsored research projects that involve interesting questions about a variety of human-machine interaction issues.

Anne: One of the commenter on our blog wanted to know why people would choose to work in academia versus industry. How did you decide on your career path and what pros and cons do you think there are for an academic job?

Beth: Again, I love teaching and my students. While most jobs involve the mentoring and coaching of junior colleagues, my position in academia allows me even more opportunities to mentor and also to instruct in the classroom. I am fortunate to be a faculty member at a teaching university that also values research. That way I am able to do both. Not all universities have the same culture, however. Another reason I enjoy academia is that I have considerable flexibility in where and when I do my work. This allows me to balance my career with having a family. A potential con to my position at my university is that I do not manage large scale research programs. That would be extremely difficult to do effectively with my teaching course load.

Anne: What kinds of jobs do your graduates, both undergrad and graduate, usually end up with when they leave your program?

Beth: Our alumni work everywhere! A few examples are the United Space Alliance, Rockwell-Collins, Symantec, Microsoft, Northrup Grumman, the FAA, NASA, various organizations within the Department of Defense, Boeing, and the list goes on.

Anne: Many universities never teach their psychology students about the field of human factors. If you had the opportunity to guest lecture to introductory psychology courses, what would you tell them about the field of human factors to get them interested?

Beth: Working in human factors and ergonomics means you get to combine many interests in one field –  you are working at the cutting edge of technology and seeing your work make a difference in the world, all while earning good money!

Anne: That’s true. We can have fun while being practical. So, who was the last speaker that you saw present and what did he or she talk about?

Beth: Yesterday I saw Ms. Anousheh Ansari, a successful engineer and entrepreneur. Ansari established the Ansari X Prize, a $10 million cash award for the first non-governmental launch of a reusable manned spacecraft twice within two weeks. In 2006, she participated in an 8 day expedition aboard the International Space Station. Ansari has published her life story in My Dream of Stars: From Daughter of Iran to Space Pioneer, a book coauthored with Homer Hickam.

Anne: Wow, that must have been fascinating! Speaking of books, what book are you currently reading?

Beth: “The Gift of an Ordinary Day: A Mother’s Memoir” by Katrina Kenison. It is all about treasuring the ordinary, unremarkable moments of everyday life the most.

Anne: So true. Well, here is one last question that I already know the answer to… What are you doing on April 7, 2011?

Beth: Ha! Hosting the 2011 Student Conference on Human Factors and Applied Psychology! All students interested in human factors are welcome, even if they do not wish to present and there is no need to pre-register.

Anne: Thanks so much for your time — I hope the conference goes well!

 

Background Image Photo Credit: iagoarchangel