Amplify teamwork with automation

In the last 10 years, automation has taken great strides in ATM (Air Traffic Management). There is still a very large potential in further developing digitalisation and automation in air traffic control.

The purpose specifically in area A is overall to develop theory and tools for understanding human-automation collaboration within ATM. Within Part A, the focus of the research is on a human-machine concept (digital colleague) to be able to relieve air traffic controllers in e.g. digital ATS environment i.e. "Remote Tower Services". The purpose is specifically to be able to understand the interaction human-digital colleague, for various analyzes such as design of HMI, safety evidence, etc. Another purpose is to relieve the visual burden for ATCO, through our research on sound as a medium for information (sonification).

The goal in the first years is to (1) develop methodology with tool support to understand

human-automation collaboration, applied to test cases (2) define an operational concept for a digital assistant (virtual colleague) with a focus on interface to operator (3) design and test an initial sonication (sound-based design) for use with visualization in operator environments.

The automation program II is a continuation of a previous 5-year project. Actors in the project are LiU and LFV.

In industries such as nuclear power management and air traffic control, information rich high stakes environments have emerged through automation of human work together with centralization of control and coordination. This pattern is now also emerging in other parts of society, such as home health care, emergency response, and ambulance dispatch.

Since it has proven practically impossible in high stakes domains to plan for every contingency, humans must collaborate with automated systems to stay in control. Situations and plans must simultaneously be considered on several levels, ranging from the status of physical items or actions, to overarching, and sometimes conflicting, qualities and goals. The goal of this project is to research generic methods for visualization of complex situations to strengthen human-automation collaboration in real-time systems, through research on: visualization and interaction techniques specifically targeting this area, integrating the visualization techniques in existing automated systems, and experimentally evaluating the effectiveness and efficiency of the proposed methods.

The project is funded by the Swedish Research Council

Three main challenges have been identified to handle expected drone traffic 1) To be able to understand how well the airspace works in normal operations, with ongoing / planned traffic and traffic rules. 2) To be able to adjust the traffic rules, the airspace capacity, so that the traffic flows better, safety is maintained, through reusable traffic solutions. 3) To be able to see / predict the effect of specific planned traffic solutions before they are implemented so-called "What if", "probing" functionality.

The UTM CITY project is based on the research through design methodology. According to the methodology, a design (UTM concept) is constructed at the same time as a context is built up (simulated urban environment with drone traffic) together with problem owners and other stakeholders. In the project, we create an image of future traffic management, by designing an interactive simulation and visualization of traffic and tools in and between cities. Within UTM, there is data that varies over time, which makes it difficult to get an overview of the situation based on numbers and simple graphs. Thus, today there is a great need for technologies for UTM that support the user in effectively

overviewing data. In this project we will work with visual dashboards, which are based on and develop the area of information visualization.

Current ATM systems require one controller per 5-15 flights, which is seen as unviable for UTM (operations will become too expensive). The purpose of UTM is to manage drone traffic safely and efficiently beyond line of sight. UTM manages current and planned traffic, airspace restrictions and contingencies. For cities, combined missions in shared airspace, especially with destination hotspots (e.g. airports, hotels, logistics hubs), can cause interactions between missions (i.e. adverse effects, such as congestion, inefficiency, risk of separation infringements). UTM manages and minimizes these adverse effects. This requires interactivity between mission management (in particular route planning) and UTM.

The purpose of the project is to design, prototype, and test concepts for Unmanned Traffic Management (UTM) for Dubai City in an interactive visualization and traffic simulation. The goals of the project are (1) to visualize in 3D unmanned air traffic (delivery drones) for Dubai City. Moreover, goal (2) is to define an operational concept for unmanned traffic and (3) to use it to define optimal routes for parcel delivery drones.

The project is managed by LIU in close collaboration with University of Sharjah, Saab, LFV and Dubai Aviation Engineering Projects (DAEP).

We design and develop interactive visual sequence mining techniques to be applied to various system- and meta-data obtained from monitoring ATM systems with the aim to map and explore the underlying relations and processes within and between monitoring systems and identify patterns of events leading to interesting or unwanted outcomes. The aim is to:

• predict errors or failures and prevent them by scheduling troubleshooting visit in time
• make informed decisions regarding the intervals between scheduled maintenance visits

Goals

RESKILL is a five-year research project (2016-2021) that explores novel training methods and interface designs in the two domains of maritime piloting and digital (remote) tower operations. The goal is to increase operators’ competence, skill, and understanding of automation support systems. The research is organized in two approaches: exploring automation transparency principles and interactive visualization techniques to explain, to the operator, how automation works; and combining eye-tracking equipment and interactive visualizations techniques to explain, to the instructor, what the operator is looking at.

Main activities

RESKILL is conducted in close collaboration with domain experts and operators in the two domains tower air traffic control and maritime piloting. To gain an understanding of the research challenges and opportunities, research will initially employ ethnographic studies in the field, interviews, questionnaires, and eye-tracking data collections. This knowledge is used as input to develop concepts of novel training methods and innovative automation support systems. These concepts are then further refined, prototyped, and investigated in empirical studies, including workshops and simulations. Thereto, simulations are conducted in two simulators located at Campus Norrköping: the Wärtsilä Bridge Simulator and the Saab Tower Simulator.

Research funding

The RESKILL project is funded by public research and innovation funds from the Swedish Air Navigation Service Provider LFV, the Swedish Maritime Administration, and the Swedish Transport Administration.

Research benefits and expected outcomes

The purpose of RESKILL is to create and empirically investigate novel support systems and training methods for securing skills and knowledge (i.e. re-skilling) of operators in relation to an increased automation-dependent working environment envisioned in the future. The research is expected to result in increased training efficiency and operator competence and understanding of automation. The research comprises:

• The development of innovative and novel training support systems utilizing interactive visualizations that explain how different automation support systems work. These transparent systems support operators’ understanding of how the automation works and behaves. In the maritime domain, research is conducted on the predictor automation. The predictor automation is a ship navigation and maneuvering support system used by maritime pilots. In the tower air traffic control domain, research is conducted on the Digital Tower Assistant (DiTA). DiTA is envisioned as a digital companion to support tower air traffic controllers in handling airborne traffic movements in multi remote tower operations (MRTO).
• The development of innovative training support systems employing eye-tracking equipment and interactive visualizations to provide instructors with an understanding of operators’ visual activity and scan patterns.

Publications

2019
Westin, C., Lundin Palmerius, K., Johansson, J., & Lundberg, J. (2019). Concept of Reskilling for Automation Collaboration in Maritime Piloting. In IFAC PAPERSONLINE (Vol. 52, pp. 365–370). ELSEVIER. https://doi.org/10.1016/j.ifacol.2019.12.090

Westin, C., Vrotsou, K., Nordman, A., Lundberg, J., & Meyer, L. (2019). Visual Scan Patterns in Tower Control: Foundations for an Instructor Support Tool. Presented at the SESAR Innovation Days. https://www.sesarju.eu/sites/default/files/documents/sid/2019/papers/SIDs_2019_paper_42.pdf

Goals

MAHALO investigates the impact and relationship between the conformance and transparency of a conflict detection and resolution automation support system on air traffic controller understanding, acceptance, trust, and performance. MAHALO intends to answer a simple but profound question: should we develop automation that solve problems like the individual human (conformal), or should we develop automation that explain itself to the human (transparency).

Main activities

MAHALO will create a conflict detection and resolution support system based on artificial intelligence (AI) and machine learning (ML) methods for en-route air traffic control. A ML architecture will be used that allow the systems to identify and adapt to individual controllers’ problem-solving strategies (i.e. conformance). Domain transparency, Explainable AI, and ML interpretability methods will be explored to facilitate explanations of how the system works and reasons when solving problems. The system will be empirically tested with air traffic controllers in human-in-the-loop simulations. The impact on trust, acceptance, system understanding and performance will be measured and conclusions will be derived on the best trade-off between those two concepts.

Research funding

The MAHALO project has received funding from the SESAR Joint Undertaking (JU) under grant agreement No 892970. The JU receives support from the European Union’s Horizon 2020 research and innovation programme and the SESAR JU members other than the Union. The opinions expressed herein reflect the author’s view only.

Publications

2020
Westin, C., Hilburn, B., Borst, C., van Kampen, E-J., & Bång, M. (2020). Building Transparent and Personalized AI Support in Air Traffic Control. Paper presented at the 2020 IEEE/AIAA 39th Digital Avionics Systems Conference (DASC).