HFES Conference in San Antonio, Part 2 – Eliciting Knowledge Structures

I‘d like to highlight some of the talks I enjoyed last week and point our readers to their research.

First up, we have:

The Influence of Rating Method on Knowledge Structures.
Chad C. Tossell, Rice U.; Brent A. Smith, U.S. Air Force Academy; Roger W. Schvaneveldt, Arizona State U., Polytechnic

This talk was a great introduction to understanding how we organize information in the brain as we change from novices to experts in a task. This isn’t something I’ve done in my work, so I learned a great deal in the 15 minute presentation.

In a nutshell, one can use a technique called Pathfinder to see how people link concepts associated with a job or task. These links can be analyzed to see how people change the organization of the knowledge as they become experts. For example, a novice would have a very different understanding of how the parts of an engine depend on one another, or at best only show topical links between parts of the system. An expert mechanic would organize the parts of the engine differently. Usually the knowledge structure of one expert is very similar to that of other experts (while novices vary greatly.)

Unfortunately, using the Pathfinder technique is time consuming and arduous. The current talk focused on simplifying the technique while preserving the integrity of the findings.

How Pathfinder Works

Each concept is presented with every other possible concept in a task or system and the participant judges the relatedness of the two concepts.

pf

This creates a map. Below is an example of what one of these maps might look like, taken from the NON-Pathfinder program (and addictive web game) Funny Farm.

FFAs you can see, related concepts are linked together in multiple ways. (In the game you try to guess the unrevealed related concepts, shown above in orange.)

Chad was nice enough to let me show the comparison tool (called “Pairwise”) they used in the experiment, which was compared to the Pathfinder technique.

pairwiseHere, instead of going through each comparison one by one, the person can drag all the boxes to their degree of relatedness to the core concept shown (coordinated flight).

Results

They did not find as much of a time savings as they anticipated using this method, but came up with some good changes to make that should result in a more efficient technique. However, it did result in novice and expert knowledge maps that were not different from a Pathfinder analysis, which is a start for creating the alternative technique.

I enjoyed the creativity of the interface and look forward to seeing how Pairwise develops!

Primary Source

Tossell, C., Smith, A., & Schvaneveldt, R. (2009).  Influence of Rating Method on Knowledge Structures.   Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.