04 September 2018

Hannes Ovrén shows in his doctoral thesis in computer vision how 3D models can be created from video films recorded with simple body-mounted cameras. The research opens new possibilities for both robots and humans, not least for the police and rescue services.

Research in computer vision has a major significance for the future of artificial intelligence: autonomous systems rely on the ability of robots and other systems to orient themselves and discover objects and people.

Hannes Ovrén’s contribution shows how to create a 3D model of the surroundings, based on video films taken with an inexpensive body-mounted camera. The model reproduces scale accurately, allowing measurements to be made.

“Currently, seeing robots move rather carefully, in order to keep track of where they are. In some cases, they may even have to stop in order to determine their location. This technology allows robots to move more freely and construct a model of the surroundings while moving”, says Per-Erik Forssén, docent at the Computer Vision Laboratory, and Hannes Ovrén’s principal supervisor.

Other fields of use can be found in, for example, police work or rescue work, where personnel with a body-mounted camera can recreate a crime scene or an accident location in three dimensions, with people and objects at the exact location they had at the instant the photograph was taken.

Rolling shutter problem

The problem with creating 3D images from simple video cameras has until now been that the camera must be stationary, preferably mounted on a tripod. If the camera moves, straight objects may appear to be curved in the image, or appear to be at different heights. Objects wobble, and a distorted image is obtained. This is because cheap cameras have a type of shutter known as a “rolling” shutter, which builds the image up in pixels row-by-row. Smartphones have this type of camera.

“Each image frame contains motion, but it is possible to improve the image significantly by modelling how the camera has moved and compensating for the motion”, says Hannes Ovrén.

To prevent the calculations from becoming too demanding, his method creates a curve, known as a “spline”, that describes how the camera has moved. This curve is constructed from spline knots, where each knot controls the appearance of the curve at a certain point in time. If the knots are placed more densely, the method can deal with more complex motion, but the calculations become more demanding.

Hannes Ovrén shows in the thesis that it is possible to use significantly fewer knots when the errors that arise due to the straightening and smoothing of the curve are modelled. In order to prevent the errors from becoming too large, the method also uses an inertial measurement unit attached to the camera. This is a small and cheap sensor that tracks acceleration, angular velocity and orientation relative to the ground.

Reliable model

“The measurements from the sensor are included in the calculations and we can in this way increase the distance between knots, reducing the size of the calculations”, says Hannes Ovrén.

The simplification means that the motion of the camera and the spline curve are not exactly the same. It is possible, however, to determine how the difference in pathway affects the magnitude of measurement errors, and in this way increase the reliability of the 3D model and the distances in it.

Hannes Ovrén will defend the thesis on 7 September 2018. He plans to continue work at FOI as a newly graduated doctor.

His thesis work has been part of the project “Learnable Camera Motion Models”, financed by the Swedish Research Council.

Continuous Models for Cameras and Inertial Sensors, Hannes Ovrén, Computer Vision Laboratory, Department of Electrical Engineering, Linköping University, 2018. Principal supervisor Per-Erik Forssén.

Translation George Farrants

Videos showing the construction of a 3D model from a GoPro sports camera: 

Video

Contact

Research

Latest news from LiU

A researcher is working together with a test subject.

Our sense of touch consists of 16 unique types of nerve cells

No less than 16 different types of nerve cells have been identified by scientists in a new study on the human sense of touch. Comparisons between humans, mice and macaques show both similarities and significant differences.

Two men in white lab coats with a computer in a lab.

Improving Alphafold to predict very large proteins

The AI tool Alphafold has been improved so that it can now predict the shape of very large and complex protein structures. Linköping University researchers have also succeeded in integrating experimental data into the tool.

Rinata Kazak looking down at her jacket.

LiU researcher ahead of UN climate summit - "I’m optimistic"

Azerbaijan will host the International Climate Summit this year. Although the country is heavily dependent on its oil production, holding the meeting there could actually be an advantage, according to Rinata Kazak, who will represent LiU.