Accuware Dragonfly, a visual SLAM technology based on computer vision, provides accurate location to robots, drones, machines and vehicles. But, what is SLAM? How does it work?
This article wants to give a brief introduction to what SLAM is, how it works, what it’s for (and what it’s not for), and why it is important for the new industry revolution.
The importance of Simultaneous Localization and Mapping (SLAM) is constantly increasing, not only among the computer vision community, but across multiple industries. It is receiving specific interest from augmented and virtual reality industries, and from the robotics and automation sector.
SLAM is in fact now able to address localization problems that many industries have been facing along the years.
There is however large variety of SLAM systems available, from the academic world and from the industry: in this context, and to clarify the typical confusion around this new technology, it is worth exploring what SLAM means and how it works.
What is SLAM?
‘SLAM’ does not refer to a particular algorithm or specific software: it rather refers to the problem of simultaneously localisation (know the location/position and orientation) of a device with respect to its surroundings and at the same time create a map of the environment.
SLAM can be done in a number of different ways: SLAM is not strictly a computer vision topic, and it could also work with other technologies, such as lasers scanners and LiDARS.However, in this article, we will focus on visual SLAM, which is the most innovative technology. In fact, at Accuware, we have decided to focus on the development of Dragonfly, our visual SLAM system, to offer a valid alternative to other SLAM technologies that rely on specific hardware, such as LiDARs, and to create a brand new positioning algorithm.
Computing both the position of the device and the map, through the on-board camera, when neither are known, distinguishes the SLAM problem from other technologies.
For example, 3D mapping/reconstruction with a fixed camera rig is not SLAM, because while the map is being recovered, the positions of the cameras are already known and fixed. SLAM instead provides the ability to recover both device’s pose and the map structure, initially knowing neither of them.
It is important to note that this if one of the key features of SLAM: computing the pose creating the map in real time is in fact what makes SLAM different from other systems. This also means that the processing is typically “on the fly” so that the camera’s location is continuously known and updated.
Dragonfly, however, is able to also post-process existing videos: this is extremely useful in order to improve the accuracy inside some challenging environments, and to perform preliminary tests to estimate the final accuracy of the system, without being on site.
A Brief History of SLAM
The first researches on SLAM began among the robotics community: 1986 papers by Smith and Cheeseman are usually indicated as the first technical documents about SLAM, originally applied to wheeled robots on a flat ground. The first SLAM systems were combining different sensor readings (laser scanner or sonar, for example) with data from the control input (steering angle) and mechanical measurements (such as wheel rotations counts).
In recent years, instead, visual sensors have become a crucial aspect of SLAM research: the improvement of computer vision techniques and the high computation power of processors are opening a new era for SLAM.
Many studies on visual SLAM focused on the use of stereo cameras, or cameras in combination with other sensors (“sensor fusion”).
At Accuware we have decided instead to explore the pure computer vision SLAM, using only monocular cameras, without external sensors. While Stereo Cameras can be used with Accuware Dragonfly, they are not required.
Our scope has been to make SLAM a widely useful technology that does not require additional hardware or sensors. We wanted to deliver a precise location system based on visual information that can be derived from an existing on-board camera, removing the need of sensor fusion and of other hardware to be mounted on board of robots and drones.
How SLAM Works
Dragonfly analyzes the video stream coming from the device’s camera: it keep tracks of a set of points (“features”) through multiple camera frames and uses them to triangulate the 3D location of the device and create a virtual map of the environment. At the same time, Dragonfly can use the estimated point locations to calculate the camera’s pose.
With the use of a single monocular camera, carefully merging the different features detected over multiple frames, Dragonfly can elaborate the pose of the device (6-DOF) and map the structure of the surrounding ambient with high accuracy, up to an accuracy of 5 cm.
Dragonfly also includes the ability to improve the map quality over time, to increase the accuracy, and leverages loop closure: this automated procedure makes it possible to reduce the gradual accumulation of errors over time. The current location computed by Dragonfly can be associated to a preliminary known location inside the map (Visual or Virtual Markers), optimizing the map structure and reducing the accumulated error.
The map is then used to perform relocalisation: if the device experience a low tracking performance, which can lead to the system getting “lost”, Dragonfly is able to recognize a previously detected “feature” and use it as a marker to compute the relative location inside the map.
Relocalisation is also useful to start the positioning of the device from any place inside an existing map: the starting point is automatically recognized by Dragonfly, analyzing and recognizing the surrounding features.
SLAM in Real Applications
Now that we know how SLAM works, how can this system be useful in real life? How can Dragonfly be applied to actual projects?
Visual SLAM is nowadays needed in many different applications.
Dragonfly is used to remotely track the location of moving vehicles, such as forklifts, inside large environments. Dragonfly’s ability to dynamically update the map is extremely important in similar venues, which are subject to constant changes. Some of our customers are leveraging Dragonfly to monitor the usage of machines and ground robots (think about industrial cleaning machines, lawn mowers…), and others have been installing Dragonfly on board of flying drones to improve some operations inside GPS denied environments, such as inventory management.
We work with customers that have been developing self-driving robots and vehicles, and who are exploring autonomous navigation for drones as well.
In the era of automation, with the roll out of unmanned vehicles and with the beginning of commercial drones’ applications, Dragonfly is becoming an essential technology to provide positioning where GPS is not an option and where centimeter accuracy is necessary.