PhD defence by Jonathan Binner Becktor

Supervisors
Principal supervisor: Professor Lazaros Nalpantidis, Department of Electrical and Photonics Engineering, DTU, Denmark
Co-supervisor: Emeritus Mogens Blanke, Department of Electrical and Photonics Engineering, DTU, Denmark
Co-supervisor: Associate Professor Evangelos Boukas, Department of Electrical and Photonics Engineering, DTU, Denmark

Assessment committee
Professor Ole Ravn, Department of Electrical and Photonics Engineering, DTU, Denmark
Associate Professor Renaud Detry, University of Leuven, Belgium
Associate Professor Anders Glent Buch, SDU Robotics, Denmark

Master of the Ceremony
Associate Professor Xinxin Zhang, Department of Electrical and Photonics Engineering, DTU, Denmark

Abstract
Marine incidents, such as groundings and collisions, are often caused by human error. To improve safety and reduce costs, the integration of autonomy into commercial vessels is desired. Autonomy can be implemented at different levels, from situational awareness modules to fully autonomous vessels. The ShippingLab project aims to explore the potential impact of robust perception on the safety, efficiency, and cost-effectiveness of autonomous ships in the maritime industry, which faces unique challenges such as frequent adverse weather conditions, as well as secrecy and a lack of publicly available data.

The current state of object detection in the maritime industry is plagued by multiple limitations that render the widely used methods inadequate. Firstly, the limited availability of high-quality data poses a major challenge. Secondly, the reliability and robustness of these methods are not up to the standards required for safe autonomous operations in this sector. The current methods fall short in ensuring the accuracy and consistency of object detection, which is critical in the maritime domain where operational safety is of utmost importance.

The thesis investigates the increasing need for robust perception in autonomous systems and the challenges posed by the maritime industry. It will present a solution of robust perception that significantly improves safety, efficiency, and costeffectiveness. Additionally, significant contributions have been made to robust and reliable maritime object detection through the use of Deep Learning methods. One challenge in real-world settings, including the maritime domain, is that input data is not always clean and curated. To handle noise and slight variations in input, data augmentation can be used, but there are no guarantees for the impact of these variations. The thesis explores the use of synthetic data and enables models to identify and re-annotate poor or incorrect samples, with the use of game engines to generate high-fidelity training data. The effectiveness of including this data is also studied, along with methods for selecting poor samples for revision and enabling semi-supervised learning through re-annotation/pseudo-labeling. A usable uncertainty metric for classification and outlier detection is crucial for the reliability and robustness of the object detector. The thesis examines methods for producing such a metric and its importance for the autonomous system. Overall, the thesis contributes to the advancement of robust perception in the maritime domain, making autonomous shipping safer and more efficient.