Facial expression is a key element of how we express emotion and how we perceive emotion in others. However, the non-verbal communication of people with autism is often quite different than that of neurotypical people. This makes it difficult for other people to interpret their cues and for them to interpret the non-verbal cues of others.
What exactly is different in the non-verbal communication of people with autism? Is it their facial expression, their tone of voice, or something else? Can people with autism learn to change their facial expressions if they are made more aware of the differences? Dr. Ofer Golan discussed these questions with Dr. Ruth Grossman of Emerson College, whose research focuses on how children with ASD integrate and produce verbal and nonverbal information.
Dr. Grossman’s research looks at various aspects of social interaction in adolescents with autism who have language and cognitive skills that are within the normal range and are typically in a general education classroom. These adolescents have significant educational and career potential but often fall short of realizing that potential due to difficulties integrating into their social environment.
Her groundbreaking research uses eye-tracking, both a standalone eye tracker that sits on the table and records what the subject is looking at, and eye-tracking glasses that are mobile and can record where he or she is looking in the environment. She also uses infrared motion capture to record the movement of facial features in different situations, acoustic analyses, video recordings, and video coding, to analyze expressions and the impact that they’re having on conversation partners.
“What we found is that there is an asynchrony between these modalities in individuals with autism that is different from what you find in neuro-typical individuals,” says Grossman. “The auditory, the spoken channel, and the facial expression channels are not as synchronized and predictive of each other as they are in neuro-typical individuals. And that is specifically true when there’s an emotional component to the conversation. The more emotive somebody with autism becomes, the more the synchrony between these channels does not quite line up. And that can lead to some of these less favorable impressions that we’ve been recording.”
It takes an extensive amount of post-processing to get meaningful insights from the data and aggregate the data across samples and individuals. Therefore, she sees this type of technology as a research tool that can help identify the types of intervention that might be helpful, rather than an intervention tool that can be used to work with individuals. “Anybody who’s ever tried to get somebody to smile naturally for the camera knows that there is no good way to tell somebody how to do that,” she explains. “It either looks right or it doesn’t. We can’t tell someone to move the left corner of their mouth a little bit further out—that’s not the way we think about it as a human perceiver. It either looks right or it doesn’t, and we really don’t break it down further than that. We also don’t want to teach somebody with autism to move their faces in a way that is unnatural to them.”
Instead, Grossman strives to raise awareness of all stakeholders in conversations and interactions with people with autism. “As an autistic person, you need to be aware that your facial expressions are not as clearly read, that the other person may not know what you’re really feeling, so you might need to say it, you might need to be explicit about it,” she says. “As a person interacting with somebody autistic, you need to be aware that what you’re perceiving may not be an actual representation of that person’s internal state or communicative intent. You may need to ask a few more questions, or you may need to stop yourself from giving in to that first impression instinct of wanting to extricate yourself from that conversation because it’s feeling awkward and hang in there a little bit longer, give it a chance and try to better understand the message that this person is trying to send you, with a greater understanding of some of these ambiguities and inconsistencies that come up in the facial and vocal expressions of people with autism.”
EmotiPlay is a research-based tool that helps therapists, teachers, and parents teach children with autism to understand emotion, a life skill that impacts almost every element of their lives.
