Never Trust a Face, say scientists

Do you consider yourself good at reading people’s facial expressions?  Some tech companies are hoping to get rich from software they claim can read people’s facial expressions to determine their emotions and reactions to products and services.  Their clients are big businesses eager to do whatever they can to gain insights about overall customer satisfaction, while others are lining up to use the new software as a screening tool during the hiring process.  But new research just out has demonstrated that whether it is person-to-person or via software scans, both come up with a giant fail at accurately knowing people’s emotions based on reading their facial expressions.

Overall conclusion to the research:

Facial expressions might not be reliable indicators of emotion. In fact, the findings of the new study suggests it might be more accurate to say we should never trust a person’s face.

“The question we really asked is: ‘Can we truly detect emotion from facial articulations?  And the basic conclusion is, no, you can’t.”

 

-Aleix Martinez, professor of electrical and computer engineering, Ohio State University

Study and results overview

The researchers analyzed the kinetics of muscle movement in the human face and compared those muscle movements with a person’s emotions. They found that attempts to detect or define emotions based on a person’s facial expressions were almost always wrong.

“Everyone makes different facial expressions based on context and cultural background.  And it’s important to realize that not everyone who smiles is happy. Not everyone who is happy smiles. I would even go to the extreme of saying most people who do not smile are not necessarily unhappy. And if you are happy for a whole day, you don’t go walking down the street with a smile on your face. You’re just happy. It is also true that sometimes people smile out of an obligation to the social norms.”

 

-Aleix Martinez, professor of electrical and computer engineering, Ohio State University

And what about the tech software claims of accurately reading people’s faces?

The research group analyzed some of those technologies and found them lacking.

“Some claim they can detect whether someone is guilty of a crime or not, or whether a student is paying attention in class, or whether a customer is satisfied after a purchase. What our research showed is that those claims are complete baloney. There’s no way you can determine those things. And worse, it can be dangerous.”

 

-Aleix Martinez, professor of electrical and computer engineering, Ohio State University

And what are the dangers with misreading people’s faces?

“The danger lies in the possibility of missing the real emotion or intent in another person, and then making decisions about that person’s future or abilities…

For example, consider a classroom environment, and a teacher who assumes that a student is not paying attention because of the expression on the student’s face. The teacher might expect the student to smile and nod along if the student is paying attention. But maybe that student, for reasons the teacher doesn’t understand — cultural reasons, perhaps, or contextual ones — is listening intently, but not smiling at all. It would be wrong for the teacher to dismiss that student because of the student’s facial expressions.”  source

Who should be made aware of the problems with trying to read people’s faces?

The researchers believe their findings could indicate that people — from hiring managers, to teachers and professors, to criminal justice experts — should consider more than just a facial expression when they evaluate another person.

And the takeaway?

After extensively analyzing facial expressions and emotion data, the research team concluded that it takes much more than facial expressions to correctly detect human emotion.


 

Research reference: “Facial expressions don’t tell the whole story of emotion”, Research findings presented at the annual meeting of the American Association for the Advancement of Science in Seattle, Washington, February 16, 2020.