All posts by Anne McLaughlin

Associate Professor, Department of Psychology, North Carolina State University, Raleigh, NC

Calling the Media out for Misleading InfoViz

I was reading an article on my local news today and saw this graphic, apparently made for the article.

screenshot-2016-12-10-at-12-14-30-pm
Being from Alabama, and just a pattern-recognition machine in general, I immediately noticed it was an anomaly. The lightest pink surrounded on all sides by the darkest red? Unlikely. The writer helpfully provided a source though, from the FBI, so I could look at the data myself.

screenshot-2016-12-10-at-12-19-42-pm

There, right at the start, is a footnote for Alabama. It says “3 Limited supplemental homicide data were received.” Illinois is the only other state with a footnote, but because it’s not so different from its neighbors, it didn’t stand out enough for me to notice.

Florida was not contained in the FBI table and thus is grey – a good choice to show there were no data for that state. But as for Alabama and Illinois, it’s misleading to include known bad data in a graph that has no explanations. They should also be grey, rather than imply the limited information is the truth.

I looked up similar data from other sources to check how misleading the graphic was. Because wouldn’t it be nice if my home state had figured out some magic formula for preventing firearm deaths? Unfortunately, The Centers for Disease Control (CDC) statistics on gun deaths put Alabama in the top 4 for the most gun deaths. That’s quite the opposite of the optimism-inducing light pink in the first graphic. The graph below is for 2014 while the first graphic is for 2013, but in case you might be thinking there was some change, I also looked up 2012 (the CDC appears to publish data every two years). The CDC put firearm deaths per person in Alabama even higher that year than in 2014.
screenshot-2016-12-10-at-12-25-54-pm

In closing, I don’t think this graphic was intentionally misleading. Sure, there are plenty of examples where I would be happy to accuse malice instead of bad design. Most times it’s probably just people working under a deadline or with software tools that don’t allow custom corrections. We do have to be careful – I’d hate to see Alabama not receive aid to curb their firearm death rate based on poor information visualizations.

Institutional Memory, Culture, & Disaster

I admit a fascination for reading about disasters. I suppose I’m hoping for the antidote. The little detail that will somehow protect me next time I get into a plane, train, or automobile. A gris-gris for the next time I tie into a climbing rope. Treating my bike helmet as a talisman for my commute. So far, so good.

As human factors psychologists and engineers, we often analyze large scale accidents and look for the reasons (pun intended) that run deeper than a single operator’s error. You can see some of my previous posts on Wiener’s Laws, Ground Proximity Warnings, and the Deep Water Horizon oil spill.

So, I invite you to read this wonderfully detailed blog post by Ron Rapp about how safety culture can slowly derail, “normalizing deviance.”

Bedford and the Normalization of Deviance

He tells the story of a chartered plane crash in Bedford, Massachusetts in 2014, a take-off with so many skipped safety steps and errors that it seemed destined for a crash. There was plenty of time for the pilot stop before the crash, leading Rapp to say “It’s the most inexplicable thing I’ve yet seen a professional pilot do, and I’ve seen a lot of crazy things. If locked flight controls don’t prompt a takeoff abort, nothing will.” He sums up the reasons for these pilot’s “deviant” performance via Diane Vaughn’s factors of normalization (some interpretation on my part, here):

  • If rules and checklists and regulations are difficult, tedious, unusable, or interfere with the goal of the job at hand, they will be misused or ignored.
  • We can’t treat top-down training or continuing education as the only source of information. People pass on shortcuts, tricks, and attitudes to each other.
  • Reward the behaviors you want. But we tend to punish safety behaviors when they delay secondary (but important) goals, such as keeping passengers happy.
  • We can’t ignore the social world of the pilots and crew. Speaking out against “probably” unsafe behaviors is at least as hard as calling out a boss or coworker who makes “probably” racist or sexist comments. The higher the ambiguity, the less likely people take action (“I’m sure he didn’t mean it that way.” or “Well, we skipped that list, but it’s been fine the ten times so far.”)
  • The cure? An interdisciplinary solution coming from human factors psychologists, designers, engineers, and policy makers. That last group might be the most important, in that they recognize a focus on safety is not necessarily more rules and harsher punishments. It’s checking that each piece of the system is efficient, valued, and usable and that those systems work together in an integrated way.

    Thanks to Travis Bowles for the heads-up on this article.
    Feature photo from the NTSB report, photo credit to the Massachusetts Police.

    Thoughtful and Fun Interfaces in the Reykjavik City Museum

    I stopped over in Iceland on the way to a conference and popped in to the Reykjavik City Museum, not knowing what I’d find. I love the idea of technology in a museum, but I’m usually disappointed. Either the concepts are bad, the technology is silly (press a button, light some text), or it just doesn’t work, beaten into submission by armies of 4-year-olds.

    Not at the Settlement Exhibit in Reykjavik. There are two unique interfaces I want to cover, but I’ll start at the beginning with a more typical touchscreen that controlled a larger wall display. As you enter the museum, there are multiple stations for reading pages of the Sagas. These are the stories of their history, from the 9th to 11th centuries, and beautifully illustrated.
    njals_saga_miniature
    They have been scanned, so you can browse the pages (with translations) and not damage them. I didn’t have all day to spend there, but after starting some of the Sagas, I wished I had.

    Further in you see the reason for the location: the excavation of the oldest known structure in Iceland, a longhouse, is in the museum! Around it are typical displays with text and audio, explaining the structure and what life was like at that time.

    Then I moved into a smaller dark room with an attractive lit podium (see video below). You could touch it, and it controlled the large display on the wall. The display showed the longhouse as a 3-D virtual reconstruction. As you moved your finger around the circles on the podium, the camera rotated so you could get a good look at all parts of the longhouse. As you moved between circles, a short audio would play to introduce you to the next section. Each circle controlled the longhouse display, but the closer to the center the more “inside” the structure you can see. Fortunately, I found someone else made a better video of the interaction than I did:

    The last display was simple, but took planning and thought. Near the exit was a large table display of the longhouse. It was also a touch interface, where you could put your hand on the table to activate information about how parts of the house were used. Think of the challenges: when I was there, it was surrounded by 10 people, all touching it at once. We were all looking for information in different languages. It has to be low enough for everyone to see, but not so low it’s hard to touch. Overall, they did a great job.

    Be sure to do a stopover if you cross the Atlantic!

    Both videos come from Alex Martire on YouTube.

    So you want to go to school for Human Factors: Final Steps

    This is Post 4 in our ongoing series about graduate school in Human Factors. (Post 1 & Post 2 & Post 3)

    1. Prepare your materials and apply

    • Take the GRE. Most programs will require your GRE scores. You’ll want to do this early, in case you need to take it again. You can and should study for the GRE – no matter what people tell you, studying affects scores. Why is a good GRE so important? It is not only about getting admitted. GRE scores are often used in allocating fellowships, RAs, and TAs. A bonus fellowship could mean as much as a 30% increase in your funding offer.
    • Select at least 3 people to write letters of reference on your behalf. They should be faculty who know you well and can speak about your ability to succeed in graduate school.
      Do not include letter writers such as family, friends, pastors, or other “character references.” They hold little to no weight and may count against you if the review committee assumes you couldn’t find academic references.
    • When selecting letter writers, ask them if they can write, “a positive recommendation” instead of just “a recommendation.” You want an honest answer. A recommendation from a class instructor that just says “This person was in my class. They seemed interested. They received X grade” doesn’t mean much to the review committee. You should alert letter-writers ahead of the first deadline, at least a month preferably two.
    • Even for professors you know well, it never hurts to remind them of all the research activities you’ve had and what you learned from them. A page with a bulleted list will help jog the memory of your letter writer to help them write a detailed and personal letter.

    2. Wait!

    • You’ll probably hear in February about acceptance, but it may be as late as the end of March. If you were put on a waitlist, you might not know until just before the April 15th deadline. This is because schools may have put out offers and are waiting to hear if they are accepted before making an offer to you. There is no shame in coming from the waitlist – even the waitlists are very competitive for PhD programs.

    Wiener’s Laws

    The article “The Human Factor” in Vanity Fair is two years old, but since I can’t believe I missed posting it — here it is! It’s a riveting read with details of the Air France Flight 447 accident and intelligent discussion of the impact automation has on human performance. Dr. Nadine Sarter is interviewed and I learned of a list of flight-specific “laws” developed by Dr. Earl Wiener, a past-president of HFES.

    “Wiener’s Laws,” from the article and from Aviation Week:

    • Every device creates its own opportunity for human error.
    • Exotic devices create exotic problems.
    • Digital devices tune out small errors while creating opportunities for large errors.
    • Invention is the mother of necessity.
    • Some problems have no solution.
    • It takes an airplane to bring out the worst in a pilot.
    • Whenever you solve a problem, you usually create one. You can only hope that the one you created is less critical than the one you eliminated.
    • You can never be too rich or too thin (Duchess of Windsor) or too careful about what you put into a digital flight-guidance system (Wiener).
    • Complacency? Don’t worry about it.
    • In aviation, there is no problem so great or so complex that it cannot be blamed on the pilot.
    • There is no simple solution out there waiting to be discovered, so don’t waste your time searching for it.
    • If at first you don’t succeed… try a new system or a different approach.
    • In God we trust. Everything else must be brought into your scan.
    • It takes an airplane to bring out the worst in a pilot.
    • Any pilot who can be replaced by a computer should be.
    • Today’s nifty, voluntary system is tomorrow’s F.A.R.

    Kudos to the author, William Langewiesche, for a well researched and well written piece.

    So you want to go to school for Human Factors: The Approach Email

    This is Post 3 in our ongoing series about graduate school in Human Factors. (Post 1 & Post 2)

    Your initial email communication is your first impression and should be managed carefully. Address all communications formally and you may want someone to proof-read before you send it. That means:

    1. Address everyone by their proper title

    • Bad: “Hi Rich…” or, “Hey” or just launching into the message
    • Good: “Dr. McLaughlin,” or “Professor McLaughlin,”

    2. Be specific.

    • Bad: “I am very interested in your research on X. It is very interesting. The more I read about it, the more I am interested in it. It seems very interesting and important.”
    • Good: “I recently read a collection of your papers on X. It was very interesting to me as I saw connections with the topics I have been studying, such as Y.”

    3. Be succinct! Omit needless words.

    4. Stay on topic/avoid excessive personal anecdotes:

    • Bad: “After my house burned down and I lost everything, I sat back and thought about what I really wanted in life and discovered it was to work in your lab.”
    • Bad: writing a wall of text (e.g., one giant paragraph with no line breaks)
    • Good: “I was fortunate to learn about the field of human factors when we had a special topics course in Ergonomics at my university. For that class, I did [describe project] which lead me to your work on X.”

    5. Avoid inadvertently selfish language

    • Bad: “Your lab would help me in my interests and my career. It would be the best thing for me.”
    • Good: “I have experience in multiple statistical programs, including SPSS and MATlab. As a research assistant in Dr. X’s lab, I have experience with data entry, cleaning data, and analysis. Although I have not yet gotten to run participants through a study protocol, I have been allowed to observe the graduate students in that task.”

    6. Proofread for grammar and typos

    • Bad: your vs you’re, any misspelled words, and so on.

    7. Avoid carelessness: Sending an email to Dr. A but writing your emails addressed to Dr. B.

    Below is a sample “approach email” to the professor you are considering as an advisor. Yours will differ, but this is an example of the level of formality and what to include.

    Dear Dr. FutureAdvisor,

    I am a senior psychology major at My University and interested in pursuing a Ph.D. in Human Factors Psychology after graduation. I came across your research when I was collecting articles for a literature review on user trust in automated systems and am interested in applying to your lab to work on similar topics.

    In the past two years, I have worked as a research assistant in a lab here at My University and spent a summer in an NSF REU program at Bigger University. In the MU1 lab, I worked with Dr. So Andso on research into motivation changes across the lifespan. I learned to enter and clean data for analysis with SPSS and SAS, follow a research protocol to run participants, and write SPSS syntax. One specific project I worked on was investigating whether people over 65 reported different motivations for performance and whether they responded differently than younger adults to reinforcement schedules on an implicit learning task. This gave me an interest in aging but more generally an interest in individual differences.

    I am excited by the prospect of continuing in a research program after graduation and believe I would be a good fit for your lab. Please let me know if you will be accepting applications this year.

    Thank you for your time,
    YourName

    The optimal time for sending this email is the fall semester of your senior year. This gives you time to communicate, perhaps plan a visit, and let the faculty member know you’ll be applying to their program.

    Discussion of Human Factors on “Big Picture Science” podcast

    You all know I love podcasts. One of my favorites, Big Picture Science, held an interview with Nicholas Carr (a journalist) on over-reliance in automation. The entire podcast, What the Hack, also covers computer security. To skip to the HF portion, click here.

    • +points for mentioning human factors by name
    • +points for clearly having read much of the trust in automation literature
    • -points for falling back on the “we automate because we’re lazy” claim, rather than acknowledging that the complexity of many modern systems requires automation for a human to be able to succeed. Do you want to have that flight to NY on the day you want it? Then we have to have automation to help that happen – the task has moved beyond human ability to accomplish it alone.
    • -points for the tired argument that things are different now. Google is making us dumber. Essentially the same argument that happens with every introduction of technology, including the printing press. We aren’t any different than the humans that painted caves 17,300 years ago.

    For more podcasts on humans and automation, check out this recent Planet Money: The Big Red Button. You’ll never look at an elevator the same way.

    *While looking up support for the claim that people have always thought their era was worse than the previous, I found this blog post. Looks like I’m not the first to have this exact thought.

    So you want to go to school for Human Factors: General Sequence of Events

    This is Post 2 in our ongoing series about graduate school in Human Factors. (Post 1)

    In this post, we discuss a general to-do list for those considering graduate school in Human Factors. Comments from other faculty welcome!

    1. Get Involved in Research as Early as Possible

    • This can be through a senior project, a class at your university where students do a research project, or (optimally) by working as a research assistant in a lab.
    • If your university does not have these opportunities, look around (nearby universities). Many professors will take volunteer research assistants, including in the summer, and train you in their lab. This gives you both experience and a potential reference letter.

    2. Start Looking for Departments/Mentors and Evaluate Fit

    • Many programs or labs have information on their alumni. Do they have the kinds of jobs you want? Do their alumni work at places you would like to work?
    • You will work mainly with a single advisor in an apprenticeship model. However, it’s a good idea to consider programs where you match more than one professor.
    • Check out the research interests of potential advisors by reading some of their recent publications or look at their curriculum vitae (the academic term for resume; often found online). We often have an area of expertise but work in other areas as well. You don’t want to choose an advisor based on work from 20 years ago that isn’t being continued today.
    • It is highly unlikely that a potential advisor will initiate a new research area to fit your interests–be flexible in your interests.
    • Create a spreadsheet listing department, contact information/web address to apply, potential faculty (and their major research areas), application fee, deadline, required materials, and your rating of fit.

    3. Contact Prospective Mentors

    • When you have identified some potential programs, check their website to see which faculty are affiliated with the program and taking students.
    • Not all faculty take students every year. Some faculty list on their website whether they are taking students. If unsure, a short, formal email to the professor asking if they are accepting new students is appropriate.
    • Just because they are on a departmental website does not mean that they are affiliated with the HF program (that department may have other graduate programs) or that they are taking students that year. If it is unclear, email and ask. It isn’t helpful if, for example, you are applying to a psychology program but list an industrial engineering professor as your preferred mentor.
    • If you would like to evaluate potential fit between you and your potential mentor, you can ask if they are willing to meet with you in-person. Opinions vary, but Skype/video conference meetings may work.

    Our next post will give an example of the kind of formality expected in contacting a prospective advisor.