Detecting Marine pollutants and Sea Surface features with Deep learning in Sentinel-2 imagery

Abstract
Despite the significant negative impact of marine pollution on the ecosystem and humans, its automated detection and tracking from the broadly available satellite data is still a major challenge. In particular, most research and development efforts focus on one specific pollutant implementing, in most cases, binary classification tasks, e.g., detect Plastics or no Plastics, or target a limited number of classes, such as Oil Spill, Look-alikes and Water. Moreover, most developed algorithms tend to operate successfully only locally, failing to scale and generalize adequately towards operational deployments. Our aim is to address these challenges by introducing a holistic approach towards marine pollutant detection using remote sensing. We argue that constructing such operational solutions requires detectors trained and tested against different types of pollutants, various sea surface features and water-related thematic classes. We offer such a Marine Debris and Oil Spill (MADOS) dataset, composed of high-resolution multispectral Sentinel-2 (S2) data, consisting of 174 scenes captured between 2015 and 2022, with approximately 1.5 M annotated pixels, which are globally distributed and collected under various weather conditions. Moreover, we propose a novel Deep Learning (DL) framework named MariNeXt, based on recent state-of-the-art architectural advancements for semantic segmentation, which outperforms all baselines by at least 12 % in F1 and mIoU metrics. The extensive quantitative and qualitative validation justifies our choices and demonstrates the high potential of the proposed approach. We further discuss the underlying discrimination challenges among the competing thematic classes. Our dataset, code and trained models are openly available at https://marine-pollution.github.io/.

Acknowledgements
We thank Prof. Chuanmin Hu (University of South Florida) for fruitful discussions about the spectral behavior of floating marine debris and other floating materials. We thank our colleague Antonis Koutroumpas (NTUA) for his contribution to the Oil spill events collection. Ioannis Kakogeorgiou acknowledges the support by Hellenic Foundation for Research and Innovation (HFRI) and the General Secretariat of Research and Innovation (GSRI) under the 4th Call for HFRI Scholarships to PhD Candidates (grant: 11252). Part of this research was supported by the research project BiCUBES “Analysis-Ready Geospatial Big Data Cubes and Cloud-based Analytics for Monitoring Efficiently our Land & Water” funded by HFRI and the “Intelligent Early-Warning Oil Spill Detection and Prediction System for the Arabian Gulf and the Red Sea” funded by Saudi Aramco. Last, we thank NVIDIA for the support with the donation of GPU hardware.

Publisher
Elsevier BV

Journal
ISPRS Journal of Photogrammetry and Remote Sensing

DOI
10.1016/j.isprsjprs.2024.02.017

Additional Links
https://linkinghub.elsevier.com/retrieve/pii/S0924271624000625