Sound can be used to simplify understanding of a visual representation of research data, or to improve the outcome of an interaction between human and a machine. To use sound in this way is called sonification.
We do research in musical sonification. Sonification means that sound is added as a complement to a visual representation to make the representation easier to comprehend, or to show new relationships in research data, or to increase the outcome of an interaction. Focus for our research is to explore the use of deliberately designed and composed musical sounds in sonification. Sonification can be used for sonifying measurements about climate change, bio sensor data in medical sciences, process control and transport management, and much more. A close collaboration with domain experts is needed for the sonification to be useful with regards to application and evaluation of the sonification. We welcome collaborations and suggestions for application areas for sonification.
Sonification and information visualization
Research produce data within many different research areas from medicine via decision support for air traffic control to climate change. These research results are usually presented for other researchers as well as the public via scientific texts and visual representations. However, these visual representations are often hard to comprehend and load the visual cognitive system. By using sound as a complementary modality, it should be able to ease the interpretation of the visual information. For example, sonification can reduce visual misinterpretations made by simultaneous brightness contrast, ease understanding of density in a visualization, or distinguish between different datasets shown in the same visual representation.
Sonification and decision support
Sonification might also be used in relation to automation and decision support. Within air traffic control, as an example, sonification can provide peripheral information that the air traffic controller does not have available on the visual display. Sonification could also provide auditive information about, for example, the status of different machines. This auditive information was something that technicians and operators were provided directly on the machine shop floor previously but is not available in today's quiet control rooms. Sonification might in these examples create a soundscape that provides constant available status information in the background, without forcing the operator to shift focus from ongoing work tasks.
Sonification and interaction
In a situation where a user interacts with a machine, sonification might be used to provide information to the user. This information might be transmitted by short auditive feedback, so called interface sounds, or by changes in a more holistic soundscape. Sonification might in such ways either directly affect the behavior of a user or create an immersive environment where the user's experience of the interaction is reinforced.
Musical elements in sonification
Basic research in sonification includes exploration and evaluation of the use of musical elements and structures, such as harmonics, pitch, amplitude, and tempo. We investigate what musical elements that are applicable to use in sonification and how these are experienced by users. Furthermore, we study what visual or interactive elements that are suitable for sonification, and the most appropriate mapping between these and changes of the musical elements in the sonification, that provide the best results and user experience.
Continue to DOI