During the conference I had a very personal experience with the effects of automation reliability on trust and subsequent behaviors. First, a bit of background. There is a large body of research examining how humans interact with automated systems (Global positioning systems, for example). Human-automation interaction is quite complex; being affected by many factors.
Julian Sanchez (of MITRE) presented a poster at the conference summarizing the literature; presenting how the many variables of human-automation interaction relate to each other (figure 1). One factor being extensively investigated is the issue of how much the user/operator (you & me) trusts the automation.
I have used Google Maps on my phone extensively; and in the many cities I’ve used it, it has been a reliable tool for directions. Since the phone includes a GPS chip, it can track my movements as I walk showing me my distance to my destination. However, it failed miserably in San Antonio…twice. First, I tried to find a restaurant near the River Walk and following the directions led me to go almost in the complete opposite direction. I was so confident in Google we spent 20 minutes walking around until we asked a local policeman for directions. One reason I was confident was that as we walked toward our destination, the phone confirmed that our position was nearing the GPS destination.
The second failure was when we tried to find the Cowboy bar. Automation researchers would say that I was complacent–I over-relied on the automation which indicted that my trust was not calibrated correctly. My high level of trust came from thinking, “San Antonio is a big city, it must be fully and accurately mapped…” as well as past successful navigation attempts. This is one consequence of ultra-high reliability systems: the effect they have on users expectations and trust. Ever since my return, I’ve needed to use Google Maps (on the web or phone) and I have found myself very uncertain of the stated locations and directions offered by Google Maps. I confirm Gmaps using the competing service (Bing Maps).
Sanchez, J. (2009). Conceptual model of human-automation interaction. Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting.