Creation of a 3D model Photo credit: Hannes OvrénResearch in computer vision has a major significance for the future of artificial intelligence: autonomous systems rely on the ability of robots and other systems to orient themselves and discover objects and people. Hannes Ovrén’s contribution shows how to create a 3D model of the surroundings, based on video films taken with an inexpensive body-mounted camera. The model reproduces scale accurately, allowing measurements to be made.
Per-Erik Forssén and the small camera. Photo credit: Thor Balkhed“Currently, seeing robots move rather carefully, in order to keep track of where they are. In some cases, they may even have to stop in order to determine their location. This technology allows robots to move more freely and construct a model of the surroundings while moving”, says Per-Erik Forssén, docent at the Computer Vision Laboratory, and Hannes Ovrén’s principal supervisor.Other fields of use can be found in, for example, police work or rescue work, where personnel with a body-mounted camera can recreate a crime scene or an accident location in three dimensions, with people and objects at the exact location they had at the instant the photograph was taken.
Rolling shutter problem
The problem with creating 3D images from simple video cameras has until now been that the camera must be stationary, preferably mounted on a tripod. If the camera moves, straight objects may appear to be curved in the image, or appear to be at different heights. Objects wobble, and a distorted image is obtained. This is because cheap cameras have a type of shutter known as a “rolling” shutter, which builds the image up in pixels row-by-row. Smartphones have this type of camera.“Each image frame contains motion, but it is possible to improve the image significantly by modelling how the camera has moved and compensating for the motion”, says Hannes Ovrén.
To prevent the calculations from becoming too demanding, his method creates a curve, known as a “spline”, that describes how the camera has moved. This curve is constructed from spline knots, where each knot controls the appearance of the curve at a certain point in time. If the knots are placed more densely, the method can deal with more complex motion, but the calculations become more demanding.
Robot equipped with a cheap camera Photo credit: Thor BalkhedHannes Ovrén shows in the thesis that it is possible to use significantly fewer knots when the errors that arise due to the straightening and smoothing of the curve are modelled. In order to prevent the errors from becoming too large, the method also uses an inertial measurement unit attached to the camera. This is a small and cheap sensor that tracks acceleration, angular velocity and orientation relative to the ground.
Reliable model
“The measurements from the sensor are included in the calculations and we can in this way increase the distance between knots, reducing the size of the calculations”, says Hannes Ovrén.
The simplification means that the motion of the camera and the spline curve are not exactly the same. It is possible, however, to determine how the difference in pathway affects the magnitude of measurement errors, and in this way increase the reliability of the 3D model and the distances in it.
Hannes Ovrén will defend the thesis on 7 September 2018. He plans to continue work at FOI as a newly graduated doctor.
His thesis work has been part of the project “Learnable Camera Motion Models”, financed by the Swedish Research Council.
Continuous Models for Cameras and Inertial Sensors, Hannes Ovrén, Computer Vision Laboratory, Department of Electrical Engineering, Linköping University, 2018. Principal supervisor Per-Erik Forssén.
Translation George Farrants
Videos showing the construction of a 3D model from a GoPro sports camera:
