07 February 2023

In the 'Sound of Art' project, famous Nordic works of art have been enhanced with new music by different artists. The music was created after LiU researcher Niklas Rönnberg decoded the works of art through musical sonification. "It's been very exciting. Sound can make art accessible in a different way and hopefully create a new experience,” he says.

A person at the museum.
"I used my already existing code for analysing colour. I modified it so that the painting became a musical instrument." THOR BALKHED

Niklas Rönnberg is an associate professor and docent at the Department of Science and Technology. He mainly teaches sound technology on campus Norrkping and has been researching sonification since 2016. A research area in which he, together with some colleagues at KTH Royal Institute of Technology in Stockholm, is unique in Sweden.

"Sonification is the audio variant of visualisation, one might say, and for me it's a complement to something, such as a visualisation of data or an interaction. Sonification can take place in different ways, sometimes data is converted into sound, other times sound is created and connected to data," says Niklas Rönnberg. A person with earphones next to a computer. Photo credit THOR BALKHED

Decoding works of art

In the autumn of 2022, he became involved in a project initiated by the technology company Samsung. 'Sound of Art' aims to show how music and sound can enhance the experience of what we otherwise can only see. Four artists from Sweden, Norway, Denmark and Finland have interpreted a classic work of art from each country. The Swedish artist Esther has interpreted Hilma af Klint's painting 'Ynglingaåldern' (Youth) from 1907 – but instead of just making a new musical composition inspired by art, sonification was used. Niklas Rönnberg decoded all four works of art, so that different colours would give different notes and harmonies. Dark and mute colours give a lower pitch than lighter colours. Picture of a painting.Hilma af Klints "Youth". Photo credit THOR BALKHED

"I used my already existing code for analysing colour. I modified it so that the painting became a musical instrument. To put it simply, instead of striking a chord on a keyboard, you click on the painting and get a chord based on the colour you click."

He is pleased with the result achieved in a few intensive months.

"I'm pleased that I have managed to get musical instruments from a painting. It's a cool project, and it's a big thing for me as a researcher that my thoughts and ideas can reach people in this way."

"If you look at it purely as a sonification project, it's pretty simplified, but for it to be useful to a musician I can't have ten notes in each colour. That would produce endless combinations, some of which would no doubt sound awful."

Exploration in a public setting

It is not clear what the next step in the project will be.

"It would be exciting to be able to explore this in a public setting in a museum. There could be a big touch screen, next to the actual painting, where you could zoom in and explore not only what you see, but also sonification and sound."

The aim of Niklas Rönnberg's research is to investigate how what he calls musical sonification can be used, and where it would be most useful.

Simplify and tell

"Sound and sonification are different from visual things, and as a complement can simplify and tell us something different from what we see. Imagine working in a nuclear power plant or in an air traffic control tower with a lot of screens, where there are always situations where you risk missing something important. In this setting, a special sound could be a helpful and important complement. A person in a museum. Photo credit THOR BALKHED

In his research, Niklas Rönnberg is looking at how sound can be reintroduced in some control environments – for example in a machine room.

"In the past, the foreman would have been able to hear that something was wrong without having to look at the machine. In this day and age, we have lost some of that knowledge, so it would be super exciting to reintroduce some of the peripheral sound environment. Today, we could tell a story and facilitate a work situation using well-designed sounds that don't involve a machine rattling or the risk of hearing loss."

The most important conference in Norrkoping

The concept of sonification and most of the research into it occur in academia. Niklas Rönnberg is keen to change this. In late June, Linköping University will host the International Conference on Auditory Display (ICAD) in Norrköping, Sweden. This is the largest and most important conference in sonification. This year's theme is 'sonification for the masses'.

"Sonification is a bit obscure and that's a pity. It can be used in so many ways, and the conference this summer is about trying to reach out to people outside academia and making sonification both understandable and useful to everyone."

Brief facts: Niklas Rönnberg

Roles: Associate professor of sound technology and sonification researcher at the Department of Science and Technology, Campus Norrköping.
Academic journey: Student and employee at LiU since 1992. Master's Degree in Communication Science. Doctoral Degree in Technical Audiology, 2014.
Why the interest in sound: “Sound has always been a passion for me, I make a lot of sound and music in my free time. It's cool to be able to work with sound and music now, without being a musician”.

Contact

Latest news from LiU

Iontronic pump in thin blood vessels.

More effective cancer treatment with iontronic pump

When low doses of cancer drugs are administered continuously near malignant brain tumours using so-called iontronic technology, cancer cell growth drastically decreases. This is demonstrated in experiments with bird embryos.

Electronic medicine – at the intersection of technology and medicine

Swedish researchers have developed a gel that can form a soft electrode capable of conducting electricity. In the long term, they aim to connect electronics to biological tissue, such as the brain.

Green lights on the backside of a supercomputer.

Agreement signed – supercomputer Arrhenius to Linköping

The hosting agreement between NAISS at LiU and EuroHPC JU has been signed. This means that LiU is now officially hosting the European supercomputer Arrhenius, which will become one of the world’s fastest computers.