“It’s scary how well synchronised we humans are when we’re together. How skilled we are at interpreting each other and how sensitive we are to deviations. It never ceases to amaze me”, says Mathias Broth, professor of language and culture.
Mathias Broth conducts research together with Leelo Keevallik, also professor of language and culture, and doctoral student Hannah Pelikan, into how we humans communicate with each other. They look at how we carry out acts with each other – or with a robot – and what speechless communication is.Mathias Broth and Leelo Keevallik.
When the researchers study wordless communication, the focus is on the details; therefore the respondents are always recorded. By studying video recordings, it becomes clear how people use their voice, face, and body to interact.
“I studied a French TV team, and how they communicated on air. When the TV producer realised the detail with which we studied him, and the apparently insignificant details we noticed, he was flattered”, says Mathias Broth.
But why do we need to know, in such detail, what we do when we communicate?
“With every little action we do, we maintain roles and norms. With every sigh and every glance. By studying this in detail, we register what is systematic in what people do. On a larger scale, these studies teach us about how people uphold society”, says Leelo Keevallik.
Er, ah, um – Sounds as a way of interacting
Leelo Keevallik’s research focusses on sounds that seemingly mean nothing – sounds that are used to express – for instance – effort, pain, and whether the food tastes good.
“For a long time, these sounds were considered insignificant, because they’re not words. But research has shown that these sounds have a relatively fixed form, and we use them as tools in our social interaction. When someone sighs at a particular moment, we know what it can mean”, she says.
Teaching robots to interact with people
But interaction doesn’t only take place between people, it can also involve humans and technology. Doctoral student Hannah Pelikan is studying how robots use sound to interact with people. She is assisted by the toy robot Cozmo. Cozmo hardly speaks at all, but can move, recognise faces and make sounds. Photo credit Thor Balkhed
“It’s very simple, but can still interact with people. How does it do that, how do we understand what the robot is doing and wants? And how do we use our understanding to interact with it?”
Those are the questions that Hannah Pelikan hopes to answer. She has studied how people react to Cozmo’s way of showing emotions by way of sounds.
“Wordless interaction might seem vague, but sounds are flexible in a way that words aren’t. With sounds, Cozmo can communicate faster and in more contexts. When Cozmo displays happiness, we humans generally interpret that as a positive response, and thus this works in many different contexts. For instance, it’s appropriate both as response to when Cozmo meets a new person and when it gets a toy.”
Words on the other hand are not as flexible in different contexts. It’s difficult to preprogram the robot so that it can use words to respond to everything a human could possibly say to it, according to Hannah Pelikan.
Among other things, her studies have shown that interaction with Cozmo proceeds well when the robot displays happiness, but that it stops when Cozmo displays sad emotions. In this case, the study participants retry what they have said or done. Showing sad emotions can be an efficient way for the robot to signal that something in the interaction has gone wrong.
Robots that function together with people
In order for robots to function in society, they must follow the unwritten rules that form the basis of human interaction. Photo credit Thor Balkhed
“If we want to build machines that are easy for us to interact with and that function in the long term, we must start out from our knowledge about human interaction, and study the interaction between robots and humans in practice.”
Translated by George Farrants.
