TUM Flyers

Team

Research: Technische Universität München (TUM)
End User: Schällibaum AG (SLB)

Vision-based MAV Navigation for Systematic Inspection of Structures

: Figure 1: Vision-based localization and 3D reconstruction of a handheld camera using LSD-SLAM

The aim of this project is to develop novel vision-based localization, 3D reconstruction, and navigation technologies for increasing the level of autonomy of MAV inspection systems and the systematic quality of inspections. The inspection of a structure such as a bridge with micro aerial vehicles (MAVs) is a highly challenging use-case for which we will develop a feasibility demonstration in this project. Inspection nowadays often is a costly and time-consuming process. Inspection teams need to climb through the structures, or special heavy equipment such as access lifts need to be employed. Current approaches using MAVs rely on GPS for autonomous navigation and require highly skilled and well trained pilots to maneuver the MAV close to structures. The vision-based technologies pursued in this project target at further automating and simplifying the use of MAVs in GPS-restricted or GPS-denied environments.

Systematic inspection and maintenance of the huge amount of structures in the EU such as industrial plants, wind power plants, or traffic infrastructure, e.g. bridges, is highly important for sustaining the productivity of Europe’s industry. Structures without timely maintenance treatments will likely require more costly treatments sooner than those properly maintained. Furthermore, sudden out-times of such structures due to deferred maintenance can lead to severe negative economic impact. In addition to their immense economic value, existing structures worldwide also may have major cultural value that must be preserved.

Nowadays, inspections of such structures are often done visually. Experienced inspection teams climb through the structures with the help of climbing equipment and access vehicles (manlift, bucket truck, etc.). This procedure is highly dangerous for the inspection teams, and is very time consuming and costly. It is also difficult for the team to perform a full and systematic inspection which may cause deficiencies to be overlooked. Finally, the inspection team typically has no accurate position information to exactly reference deficiencies, such that comparisons across multiple inspections are difficult to achieve.

In this project, our team TUM Flyers (composed of the TU Munich, Germany, and Schällibaum AG, Switzerland) aims at developing novel vision-based technologies for the systematic inspection of structures using micro aerial vehicles (MAVs). Such technologies could reduce maintenance time and costs and could increase systematicness and repeatability of inspections. In detail, this project aims

To develop vision-based localization in real-time to localize the MAV accurately in GPS-denied areas, in particular close to structures. The MAV concurrently and in real-time maps the structure in 3D for obstacle avoidance and path planning using the MAV’s on-board stereo vision sensors and its processing capabilities.
To develop a semi-autonomous assistive MAV flight mode for live inspection and mapping of structures. While the operator navigates the MAV close to structures, the MAV autonomously avoids obstacles perceived with its sensors.
To develop autonomous MAV waypoint navigation for systematic image collection, referencing, analysis, and reinspection of structures.

In the freestyle demonstration in Stage II.a, we will demonstrate core technologies for the inspection of structures with MAVs. We will showcase, how operators can control the MAV to perform a live inspection and to teach-in inspection views on a small-scale object. In Stage II.b, this technology will be used for an inspection of a medium-scale realistic industrial lab environment in an end-user driven task. For an inspection in this GPS-denied environment, the MAV needs to navigate at larger scale and close to obstacles.

Stage III conducts development and field tests for a feasibility demonstration of the inspection of a structure in an outdoor scenario. Bridges are highly challenging examples of such structures. This use-case scenario includes flying a few meters close to the structure of the bridge in order to collect inspection images, and combines major common challenges in the systematic inspection of a diverse set of structures with MAVs.