Project

Back to overview

Vision-based Aerial Swarms

Applicant Floreano Dario
Number 188457
Funding scheme Project funding (Div. I-III)
Research institution Laboratoire de systèmes intelligents EPFL - STI - IMT - LIS
Institution of higher education EPF Lausanne - EPFL
Main discipline Other disciplines of Engineering Sciences
Start/End 01.01.2020 - 30.06.2022
Approved amount 354'875.00
Show all

All Disciplines (2)

Discipline
Other disciplines of Engineering Sciences
Information Technology

Keywords (4)

Bio-inspired artificial intelligence; Flying robots; Vision-based flight control; Swarm intelligence

Lay Summary (German)

Lead
This research proposal aims at developing aerial swarms able to avoid collisions by means of on-board cameras and predictive capabilities. The drones will be capable to operate both indoor and outdoor and the methods developed will be particularly useful for situations in which the drones must fly at short distances to each other and among obstacles (e.g., urban environments).
Lay summary

The deployment of an autonomous group of drones (also called aerial swarms) is the next frontier in the science and technology of intelligent flying machines. Despite recent progress in coordination and communication methods, which have led to impressive displays of dancing drones lighting up night skies and theatre halls, these aerial systems are carefully orchestrated by ground computers that accurately monitor and control the motion of the drones. The few autonomous aerial swarms described so far, including our own, rely mostly on GPS (or similar technologies), intensive communication among the drones, and, when tested on real drones, struggle to prevent collisions. Recent work points to the feasibility of adding vision capabilities to aerial swarms.

This research proposal aims at developing aerial swarms able to avoid collisions by means of on-board cameras and predictive capabilities. This proposal is a short extension (2 years) of an ongoing project ending in 2019 where we have obtained promising, but preliminary, simulation results indicating the feasibility of the proposed solutions.

The main result of this project will be a group of drones which will coordinate only through vision and will have predictive capabilities which will reduce the collision risks among the drones. The drones will be capable to operate both indoor and outdoor. These methods will be particularly useful for dense swarms where drones fly at short distances to each other and among obstacles (e.g., urban environments). We also expect to develop novel methods for swarming with prediction and machine learning in robotic systems where large data collection is not practical. We expect that these methods will be applicable also beyond the specific swarming models used in this research proposal.

Direct link to Lay Summary Last update: 10.12.2019

Responsible applicant and co-applicants

Employees

Associated projects

Number Title Start Funding scheme
155907 Bio-inspired control of microflyers in highly complex environments 01.01.2015 Project funding (Div. I-III)

Abstract

The deployment of an autonomous aerial swarm, here intended as a decentralised system composed of multiple drones (at least 10) capable of autonomously and safely coordinating their trajectories during collective flight, is the next frontier in the science and technology of intelligent flying machines. Despite recent progress in coordination and communication methods, which have led to impressive displays of dancing drones lighting up night skies and theatre halls, these aerial systems are carefully orchestrated by ground computers that accurately monitor and control the motion of the drones. The few autonomous and decentralised aerial swarms described so far, including our own, rely mostly on external positioning technology (Motion Tracking Halls and GPS), intensive communication to share the range and bearing among the drones, and, when tested on real drones, struggle to prevent collisions induced by loss of communication packets and vehicle dynamics. Recent work by our lab and other labs point to the feasibility of adding vision to aerial swarms.This research proposal aims at developing fully decentralized aerial swarms with robust collision avoidance by means of on-board vision and predictive capabilities. This proposal is a short extension (2 years) of an ongoing project ending in 2019 where we have obtained promising, but preliminary, simulation results indicating the feasibility of the proposed solutions. Our approach builds on biologically inspired swarming methods, such as Reynold’s and Vicsek’s models of collective motion. The proposed work is organized along two parallel and related research lines.Within the first research line, our theoretical and experimental results indicate that vehicle dynamics and limited sensing, which reduces the information about the relative position of other drones, significantly affect the cohesion and collision risks in dense swarms. In this research proposal, we will enable drones to predict their trajectories and those of their neighbors under the constraints of specific vehicle dynamics and limited sensing. In particular, we plan to approach the problem within the theoretical framework of Model Predictive Control, a powerful computational strategy that can now be implemented in small and lightweight processing boards and is producing impressive results for the control of aggressive drone maneuvers. Within the second research line, we will start from our newly proposed method for training convolutional neural networks to produce decentralized and robust swarming control strategies from visual images collected by the drones during collective flight. However, those results were obtained in simulations and the drone were flown in environments with uniform background. In this research proposal, we will develop methods for transfer learning from simulations to indoor environments and to outdoor environments. In a first phase, we will use omnidirectional vision and published models of collective motion for training the neural networks. In a second phase, the visual layout and swarming strategies used for training the convolutional neural networks will incorporate the insights and model predictive control developed in the first research line. We plan to approach the transfer learning problem by using adversarial methods and on-policy imitation learning.We will validate the methods with a swarm of 10 vision-based quadcopters flying in indoor and outdoor environments. The results of the project will consist in decentralized aerial swarms with predictive control for reduced collision risks and with vision-based coordination among the drones. These methods will be particularly useful for dense swarms where drones fly at short distances to each other and among obstacles. We also expect to develop novel methods for swarming with model predictive control and for machine learning in robotic systems where large data collection is not practical. We expect that these methods will be applicable also beyond the specific swarming models used in this research proposal.
-