Introduction Visual odometry has made tremendous progress since Mars Exploration Rovers was exploited, but studies focus less on the platform itself. In order to make the state estimation better applied, we focus on details about the dynamical modeling of quadrotors and propose to make the most of all kinds of sensory data to acquire all states of any quadrotor platform. In the series of work, the identification of quadrotor dynamical system is prime, which is between Newton-Euler dynamics and rotor rpm and motor current. Then, an all-states estimator is designed to fuse different kinds of sensors, which leads to more accurate pose estimation and addictive generalized external force estimation without losing real-time and robustness. Related Research: VID-Fusion: Robust Visual-Inertial-Dynamics Odometry for Accurate External Force Estimation, Ziming Ding, Tiankai Yang, Kunyi Zhang, Chao Xu, Fei Gao, the International Conference on Robotics and Automation (ICRA 2021). [paper]
Project
Introduction Deep learning and reinforcement learning are bringing changes into robotics. In IARC challenge, we designed a hierarchical decision and control for a continuous multi-target problem: policy evaluation with action delay. We further proposed Time-in-Action reinforcement learning, mapping the state-action pair to the time accomplishing the action by its underlying controller. We also used fusing raw fisheye image and attitude data to detect and locate objects in a 3D environment with a deep neural network. Recently we utilized the deep learning method to leverage the performance of robotic navigation. We used the self-supervised learning method to complete obscured parts of obstacles and combine this map prediction during planning to accomplish fast and safe navigation. We are looking forward to learning biased sampling in trajectory planning. We are also interested in model-based reinforcement learning, as well as learning-based low-level controllers and learning-based state estimation. Video
Introduction High-speed autonomous navigation with micro aerial vehicles (MAVs) operating in unknown dynamic environments requires the vehicles to be capable of quick replans to avoid potential obstacles. The high operating speed, the short sensing range and the unknown dynamic environments all leave strictly limited reaction time to make motion plans of high qualities. Developing reasonable navigation frameworks and fast motion planning algorithms is an essential part of it, especially for those lightweight multi-rotors with limited onboard sensing and computation capabilities. The projects under this topic focus on developing effective yet computation-friendly motion planning methods to enable safe and high-speed navigation of MAVs, considering constraints and sensing and control uncertainties. Related Researches: TGK-Planner: An Efficient Topology Guided Kinodynamic Planner for Autonomous Quadrotors, Hongkai Ye, Xin Zhou, Chao Xu, Jian Chu, Fei Gao, IEEE Robotics and Automation Letter (RA-L). [paper] [code] EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotors, Xin Zhou, Zhepei Wang, Chao Xu, Fei Gao, IEEE Robotics and Automation Letter (RA-L with ICRA 2021 option). [paper] [code] Mapless-Planner: A Robust and Fast Planning Framework for Aggressive Autonomous Flight without Map Fusion, Jialin Ji, Zhepei Wang, Yingjian Wang, Chao Xu, Fei Gao, the International Conference on Robotics and Automation (ICRA 2021). [paper] EVA-Planner: Environmental Adaptive Quadrotor Planning, Lun Quan, Zhiwei Zhang, Chao Xu, Fei Gao, the International Conference on Robotics and Automation (ICRA 2021). [paper] [code] Video Links: TGK-Planner: https://www.bilibili.com/video/BV1gA411e7DH EGO-Planner: https://www.bilibili.com/video/BV1VC4y1t7F4 Mapless-Planner: https://www.bilibili.com/video/BV1mt4y1e7TS EVA-Planner: https://www.bilibili.com/video/BV1Zz4y1C7rt
Introduction The agility of quadrotors enables this machine to perform single-agent autonomous navigation in unknown environments and multi-agent precise formation control in open or known fields. However, few works combine both of them to present any real-world system which is capable of navigating quadrotor swarms sharing the same unknown space, especially with only onboard processing. This project is a systematic solution that enables high-performance traveling in cluttered environments for quadrotor swarms in field environments. In addition, it requires no external localization and computation or a pre-built map. Related Research: EGO-Swarm: A Fully Autonomous and Decentralized Quadrotor Swarm System in Cluttered Environments, Xin Zhou, Jiangchao Zhu, Hongyu Zhou, Chao Xu, Fei Gao, the International Conference on Robotics and Automation (ICRA 2021). [paper] [code]
Introduction AM-Traj: A waypoint-based trajectory generator. This project proposes a framework for large-scale waypoint-based trajectory generation, with highlights on its superior computational efficiency and simultaneous spatial-temporal optimality. Exploiting the implicitly decoupled structure of the problem, we conduct alternating minimization between boundary conditions and time durations of trajectory pieces. Algebraic convenience of both sub-problems is leveraged to escape poor local minima and achieve the lowest time consumption. Moreover, based on polynomial theory, an extremely fast feasibility checker is designed for various kinds of constraints. By incorporating it into our alternating structure, a constrained minimization algorithm is constructed to optimize trajectories on the premise of feasibility. Benchmark evaluation shows that our algorithm outperforms state-of-the-art waypoint-based methods regarding efficiency, optimality, and scalability. The algorithm can be incorporated in a high-level waypoint planner, which can rapidly search over a three-dimensional space for aggressive autonomous flights. The capability of our algorithm is experimentally demonstrated by quadrotor fast flights in a limited space with dense obstacles. Large-Scale Trajectory Generation in Flight Corridors: For quadrotor trajectory planning, describing a polynomial trajectory through coefficients and end-derivatives both enjoy their own convenience in energy minimization. We name them double descriptions of polynomial trajectories. The transformation between them, causing most of the inefficiency and instability, is formally analyzed in this paper. Leveraging its analytic structure, we design a linear-complexity scheme for both jerk/snap minimization and parameter gradient evaluation, which possesses efficiency, stability, flexibility, and scalability. With the help of our scheme, generating an energy optimal (minimum snap) trajectory only costs […]
Introduction In recent years, the evolution of quadrotor online planning makes drones fly out of laboratories and appear in numerous real-world applications. In this area, traditional gradient-based planners rely on a pre-built ESDF map to evaluate the risk of collision. However, construction ESDF consumes significant computation. This project is an ESDF-free gradient-based local planning framework, and we incorporate careful engineering considerations to make it lightweight and robust. Among the criteria of designing autonomous quadrotors, generating optimized trajectories and tracking the flight paths precisely are two critical components in the action aspect. In industrial applications, the planner and controller of a quadrotor are mostly independently designed, making it hard to tune the joint performance in different applications. To bridge this gap, we propose an efficient, receding horizon, local adaptive low-level planner as the middle layer between our original planner and controller. Our method is named as corridor-based model predictive contouring control (CMPCC) since it builds upon on MPCC and utilizes the flight corridor as hard safety constraints. It optimizes the flight aggressiveness and tracking accuracy simultaneously, thus improving our system’s robustness by overcoming unmeasured disturbances. Our method features its online flight speed optimization, strict safety and feasibility, and realtime performance and it is released as a low-level plugin (https://github.com/ZJU-FAST-Lab/CMPCC) for a large variety of quadrotor systems. Related Research: CMPCC: Corridor based Model Predictive Contouring Control for Aggressive Drone Flight, Jialin Ji, Xin Zhou, Chao Xu, Fei Gao, International Symposium on Experimental Robotics (ISER 2020). [paper] [code]
Introduction We present systematic solutions to some aerial applications with the integration of our algorithmic researches, including state estimation, onboard mapping, optimal trajectory generation and robust control. Here we introduce the aerial agile tracking as an instance. In Fast-Tracker, we propose a systematic solution that uses an unmanned aerial vehicle (UAV) to aggressively and safely track an agile target. The solution properly handles the challenging situations where the intent of the target and the dense environments are unknown to the UAV. Our work is divided into two parts: target motion prediction and tracking trajectory planning. In the target motion prediction method, we adopt polynomial regression based on the past target observations. Especially, Bernstein basis polynomial is used to enforce dynamical constraints in the regression method. The generated trajectory is extrapolated as the prediction of target future motion. In the tracking trajectory planner, we design a heuristic function for the kinodynamic searching method that considers both the current target observation and motion prediction. Afterwards, a flight corridor that consists of a sequence of connected free-space 3-D grids is formed based on the results of the path searching. The back-end optimizer then generates a spatial-temporal optimal safe trajectory within the flight corridor. What’s more, due to the occlusion of obstacles, the limited sensing range, and the uncertainty of the target’s intent, it is hard for the UAV always to locate the target. We design a strategy so that the UAV can re-locate the target as soon as possible. The proposed solution is […]
Approach one: Variational optical flowIntroductionIn this project, we propose a novel optical flow formulation for estimating two-dimensional velocity fields from an image sequence depicting the evolution of a passive scalar transported by a fluid flow. This motion estimator relies on a stochastic representation of the flow allowing to incorporate naturally a notion of uncertainty in the flow measurement. The Eulerian fluid flow velocity field is decomposed into two components: a large-scale motion field and a small-scale uncertainty component. We define the small-scale component as a random field. Subsequently, the data term of the optical flow formulation is based on a stochastic transport equation, derived from the formalism under location uncertainty proposed in Mémin (2014) and Resseguier et al. (2017a). In addition, a specific regularization term built from the assumption of constant kinetic energy involves the very same diffusion tensor as the one appearing in the data transport term. Opposite to the classical motion estimators, this enables us to devise an optical flow method dedicated to fluid flows in which the regularization parameter has now a clear physical interpretation and can be easily estimated. Experimental evaluations are presented on both synthetic and real-world image sequences. Results and comparisons indicate a very good performance of the proposed formulation for turbulent flow motion estimation.
IntroductionThis project mainly studies on the observation and motion parameters reconstruction of flapping flight, the stability analysis of insects hovering, and the flapping mechanisms design for the PIV experiments, aiming to improve the existing hull reconstruction and pose estimation algorithms, propose the analysis method of insect hovering under the active control under varying flapping frequency, and design flapping mechanisms with multiple freedoms for PIV experiments to implement more fined motion. As for hull reconstruction and pose estimation, the project reconstruct the hull of insect under the assumption that the insect body is rigid and its section is elliptical with the data of body radius, centerline and the wing outline. As for the experimental flapping motion system design, this project analyzes the design principles and keeps the Reynolds number and Strouhal number same in the real and experimental environment respectively, and describes how to the design bee-like and dragonfly-like flapping mechanisms and how the mechanisms are driven.Paper1.Y. Huang, J. Liang and C. Xu. Sability of the flapping-wing vehicle near hovering under active control by varying flapping frequency [C]. The Chinese Congress of Automation 2017, Jinan, Shandong, China, October 21-22, 2017.