New AI-research to fuse physical and virtual worlds

Better digital visualisations and a world-class AI research environment. These are some of the goals of the new Linköping University-based research project NEST. This is one of nine wide-ranging so-called NEST projects which WASP is starting in Sweden in April. The project at Linköping University (LiU) will involve researchers taking on the biggest challenges in AI research. 

AI Artificial intelligence metamorworks

At LiU, there are several research projects on AI, autonomous systems and software and, not least of all within the framework for WASP, Sweden’s largest single research programme. The programme has a budget of SEK 1 billion, and LiU is the host university. Last year, WASP started the NEST (Novelty, Excellence, Syngergy, Teams) initiative, and took the decision to start a total of nine NEST projects. These will be carried out over subject boundaries, and all WASP universities will be involved in NEST.

Several LiU researchers

One of the NEST projects will be based at LiU and will involve several LiU researchers, such as Jonas Unger, professor of media and information technology, who will coordinate the project.

“The idea behind this is that we who work with it are world leading in our disciplines. This isn't just a single project – it's about building a new research environment”, he says.
At the turn of the year, it was announced that the nine NEST projects were to receive a grant of SEK 20 million each during a five year. The funding source is the Knut and Alice Wallenberg Foundation. This makes NEST a bigger initiative then the earlier initiatives within WASP, says Jonas Unger.

“The goal with NEST is to take on some of the biggest challenges within AI, which is about developing systems that can integrate data that comes from both the real and virtual worlds.”

Vulnerable systems

One area which would benefit greatly from integrating real data with virtual data, for example, is the automotive industry. Today, there are self-driving cars which can understand their surroundings with the help of sensors and cameras. Using these, the vehicle can understand how to navigate its surroundings. But there is a lot of manual work that goes on behind the technology in order to gather the necessary data. Furthermore, the technology has a certain vulnerability, as the data gathered often represent normal situations without disturbances.

“This leads to problems if you want to train the algorithm so that it can force the car to make an emergency stop if something pops up in front it. In the real-world data which is gathered, there probably aren't so many examples of that”, says Jonas Unger.

Using simulation, the researchers want to generate synthetic data which complements the data from real life.

Put into practice

To give the research results a practical application, the researchers are collaborating with several different actors: Ikea Communications (who work with product visualisiation), the company Arrivers (vehicle safety), the Swedish Transport Administration, AI Sweden, WASP WARA Media and Language.

For example, in their collaboration with the Swedish Transport Administration, the researchers want to see if it is possible to use other methods than pictures and sensor data to combine simulation with AI models.

"For example, this might involve combining input data that describe the flow of traffic in a city with data about the weather. If it rains or is icy, then the simulation can help determine where the risks for accidents are largest”, says Johan Unger.

Another example that the researchers want to study is simulation in product visualisation, and the technology called augmented reality, which is about laying digial pictures over reality. In collaboration with Ikea Communications, the researchers want to develop technology that allows customers to visualise how furniture will fit into their homes.

“This technology already exists, but there are many ways we can make significant improvements. Today, the technology is limited in terms of how realistic it looks when, for example, putting a sofa in your home. We also want to make it go faster in real time.”

Collaboration with Örebro

The NEST project at LiU is being run in collaboration with Örebro University. The collaborating researcher there is Amy Loutfi, professor of information technology. She says that developing AI technology that can adapt using data from both the virtual and physical worlds is at the heart of researchers’ methods.

“If we can bridge the gap between information from these two worlds, we will be able to make better visualisations, better predictions and, not least of all, develop better AI systems when it comes to interaction with people. We are seeing more and more how virtual environments and physical environments can be combined in order to get the best of both worlds”, says Amy Loutfi.

The initial work

The work is to begin in April 2022, and the first step for the NEST project – which is based at LiU – will be to establish a research environment for the long term.

“We hope that it will become a world-class distributed research environment here in eastern Sweden. We will work with the whole range of WASP instruments and international actors available to us”, says Amy Loutfi.

The researchers who are part of the NEST project at LiU are: Jonas Unger, professor of media and information technology, who will coordinate the project, as well as Michael Felsberg, professor of computer vision, and Fredrik Lindsten, associate professor in statistics and machine learning.

Short facts about the NEST-projects Show/Hide content

The letters AI is placed on a book shelf. A robothand is lifting the I letter.
iStock/style-photography style-photography

Nine new research projects start

During 2021, WASP decided to establish NEST (Novelty, Energy, Synergy and Teams).

The goal is to create multidisciplinary and focussed research enivronments to tackle current and coming challenges with AI, autonomous systems and software, making Sweden more competitive.

The programme consists of nine NEST projects, of which one is based at Linköping University. In the new year, it was announced that the NEST projects were to receive grants of 20 million each during a five-year period. The grant comes from WASP, which is primarily financed by the Knut and Alice Wallenberg Foundation.

The grant for the NEST project at LiU is to finance four PhD students who will work on four different research projects, of which three are at LiU and one at Örebro University.

World-leading research on AI

WASP (Wallenberg AI, Autonomous Systems and Software Program) is Sweden’s biggest single research initiative, with a total budget of SEK 5.5. billion.

The programme is supported by the Knut and Alice Wallenberg Foundation (KAW), which was started in 2015 to create a world-leading platform in AI, autonomous systems and software, as well as a feeder school for young researchers. It is about both undergraduate and postgraduate education.

The WASP research takes place through collaborations between industry and the university. Linköping University is the host for WASP, and other participating universities include Chalmers, KTH, Umeå, Lund, Örebro and Uppsala.


Show/Hide content

Latest news from LiU Show/Hide content