Gaze-based technology opens new worlds

There is a before and an after date in the life of 20-year-old Hampus Björklund. With a computer and his own gaze, he can now communicate with people around him. He can express his own thoughts and emotions. Through a project at LiU, occupational therapy researchers hope to make this technology available to more people.
Hampus ögonstyr sin dator
“A revolution.”
That is how Ingemar Björklund, Hampus’s father, describes the possibilities of the gaze-based assistive technology.
Hampus Björklund has had cerebral palsy since birth. He cannot walk, eat, speak or control his body. Until the age of 15, his verbal communication with his family consisted of them asking him yes–no questions: Are you hungry, Do you want to listen to music etc. From his sounds, or using basic technology with yes–no options, it was fairly easy to understand his responses.

Five years ago Hampus began his participation in the project ‘Gaze-based assistive technology in daily life activities for children with physical impairments’. It was started by Helena Hemmingsson, professor of occupational therapy at Linköping University, with funding from the Swedish Research Council, among others. The participants were ten children and adolescents between one and 15 years of age with severe, congenital impairments in central Sweden.

Monitors the child’s eye movements 

Professor Helena Hemmingsson

“From birth, their only means of communication are blinking, and looking up, down or to the side,” says Prof Hemmingsson.
Knowledge of eye gaze control goes back to the 1990s, and is used in many different fields of research. But there has not been extensive research on the opportunities it offers this group of children and adolescents.
“Because there hasn’t been any scientific knowledge within this field, the county administrative boards in Sweden have been cautious about purchasing expensive technology for these children. That in itself was a motivation for us. We wanted to systematically determine whether these children can use gaze control as an effective tool in everyday life – it could help them communicate better and lead a more rewarding life,” says Prof Hemmingsson.
With this technology, a camera in the computer monitors the child’s eye movements. The child learns to hold his or her gaze on one part of the computer screen, simulating a computer mouse, to guide the computer. This enables the child to express an emotion, say that something hurts, choose a film to watch or listen to music, all on his or her own initiative.
Parents, teachers and other staff who are involved with the children have been trained in the technology and in what sort of specific content the children might need to have programmed in. The children have learnt the technology through daily use, intuitively, just as very small children learn to use computer tablets.

Parents must be mindful

Hampus och hans föräldrar framför datorn

In the Björklund’s kitchen in Bålsta, Hampus’s mother Gisele shows several examples of how the technology can enable contact with her son. She explains how important it is that they are mindful of his needs, for instance adding photos of friends and family, anticipating what he might want to talk about, or what he specifically might need to convey.
“We realised that unless we add something to his computer, it’s not available to him. He might never have been asked a particular question he wanted to be asked, or been able to express a particular emotion.”
When Hampus was 15, his parents thought he might want to swear – he was a teenager after all. So they added some swearwords. Hampus appreciated this.
Before he couldn’t express himself verbally at all. Now he can say if he is in pain, and he can express what he feels. His mother tells about the time he asked his aunt who was visiting to come closer. He looked at the symbol for “I want to tell you something”. Then he focused on “I love you”, and after that, “Turn off the computer”, because he was tired.

A window has been opened 

In her office at Campus Norrköping, Prof Hemmingsson confirms what Hampus and his parents have experienced with the technology.
“A window in to these children has been opened. And most of all – they can take their own initiative to communicate. They can express an opinion or amuse themselves.”
The project has followed ten children for 1.5 years. Also, the parents and teachers have been interviewed on how the children use the technology, and what consequences it has had. The results show that all the children use it, especially to communicate with others, for games and for learning in school.
“The teachers told us it was an aha moment, they got a much more profound sense of the child’s personality. The communication affects how people around the children perceive them.”
However, the extent to which the children use the technology varies depending on their cognitive abilities.

Families have high hopes 

In the long term, Prof Hemmingsson hopes that it will be possible to interview the children and adolescents in the project about their experiences.
The technical advances raise new issues with the research team. For instance there are ethical matters to consider.
“With all this success, the families have high hopes that their children will develop and enjoy a better quality of life. The technology demands a lot of the parents as well. We have to give them support, understand their hopes but also support them if all their hopes aren’t realised. Eye gaze control offers many opportunities, but it might not help all children,” says Prof Hemmingsson.
Hampus Björklund mostly uses the technology in school; he is in the final year of special needs high school. At home he spends less time at the computer, because he is tired after school.
“However we see that more and more, he is interested in continuing. For 15 years his life was so much more isolated,” says his father Ingemar.
He turns to Hampus and asks if he thinks the computer is good. It takes a little while, as Hampus is tired, but then he focuses his gaze on “Yes”.

Text: Eva Bergstedt
Photo: Vibeke Mathiesen

Fact box: The project ‘Gaze-based assistive technology in daily life activities for children with physical impairments’ is an intervention study that has been underway for five years. It is doctoral student Maria Borgestig’s PhD project. She has been involved in all aspects of the project and will continue to do research in this field. Researcher Patrik Rytterström has been postdoc in the project, in charge of teacher interviews. A future project will include a larger number of children and collaboration with researchers from Australia. Prof Hemmingsson will be head researcher for this project.