Better results when AI and humans collaborate

Artificial intelligence that doesn’t take into account human operators creates problems. Now LiU researchers are building a system where AI and humans work together to control air traffic. 

Thor Balkhed

“Remember that paper clip that used to appear in Microsoft Word? There you have the problem. For a support system to help you, it has to understand what you’re doing. Otherwise it can be annoying”, says Magnus Bång, associate professor at LiU’s Department of Computer and Information Science.

Together with Jonas Lundberg, professor at LiU’s Department of Science and Technology, he manages the F-Auto (Flexible Automation) research project, where they investigate how AI can support air traffic controllers.

The two researchers have built a system that monitors air traffic and that, with the help of AI, can detect conflicts, for instance discover aircraft that are too close to each other, or whose flightpaths intersect.

“Conflicts like that should be able to be solved automatically, for instance by the AI system adjusting the aircraft’s route or speed”, explains Magnus Bång.


Traditionally, automation has often meant automating as much as the technology can handle. The rest is left to a human operator to sort out. But this approach has proven problematic. If the operator has no work tasks, he or she soon loses the overview of the situation. If the AI system suddenly reports that it cannot handle the situation and is passing responsibility to the operator, the operator has no idea what is going on, and it can be difficult to rapidly make good decisions.

The opposite situation can also arise. When the air traffic controller is busy solving an urgent problem, the AI system must not interrupt.

Instead of automating as much as possible, Magnus Bång and Jonas Lundberg argue that systems must be created where the operator is constantly kept informed. They use the term ‘Human-in-the-Loop-AI’.

“Our research question is ‘How do we make sure that the operator and the AI collaborate?’. We maintain that the AI needs to know what the air traffic controller is doing, and vice versa”, says Magnus Bång.

One step further

To find out what an air traffic controller does, they use eye tracking, where the air traffic controller wears special glasses, making it possible to follow his or her eye movements.

“The information is streamed in real time, so we understand what the controller is looking at. We can know which specific conflict they are working on thanks to their eye movements. That information is sent to the AI. If the controller fixes their eyes on a certain flight object, the AI knows that he or she has seen it.”

AI is a popular research field, however there is not much research like Bång’s and Lundberg’s.

“Of course there’s a lot of cognition research into the problems of automation, but we go one step further, bringing with us the cognitive-science approach when we build these support systems. We don't want to automate processes simply because it’s possible, we build the AI itself, based on a cognitive-science perspective.”

The aim of their system is to support the air traffic controller, leading to increased safety, among other things.

“One thing we’re trying to achieve is being able to detect if the operator has missed anything. When working together, the human and the technology should be able to increase efficiency, monitor a larger air space, and be able to land more aircraft.”

"Our AI system has an industry focus"

The research in the field concerns air traffic control, but the results will be applicable to many other fields.

“The automation of any work environment that requires a human leads to these problems, whether it’s in maritime transport, nuclear power plants or healthcare.”

The research conducted by Magnus Bång and Jonas Lundberg is in progress. Using the system in real-world situations is still in the future.

“Our AI system has an industry focus, but we’re probably ten years away from being able to use it in real-world situations. There are considerable validation processes to deal with. The security level in air transport is extremely high and of course we collaborate with the Swedish Civil Aviation Administration. At the same time both the EU and the US are very interested in automating parts of the air traffic system.”

ContactShow/Hide content

Latest news from LiUShow/Hide content