Cloud and cloud shadow detection in remote sensing imagery

Analyzing optical remotely sensed imagery has numerous applications, including climate studies, weather forecasting, urban planning, agriculture, and natural resource/disaster management. All these applications rely on the data collected from air/space-borne sensors. On the one hand, transferring data from those sensors to ground stations is an expensive process from the time, bandwidth, storage, and computational points of view. On the other hand, no valuable information about the Earth's surface can be extracted from optical images that are heavily obscured by clouds and their shadows. Since, on average, 67% of the Earth's surface is covered by clouds, it seems that a considerable number of resources can be saved by transferring only images with no or minimal amount of cloud and shadow coverage. Additionally, cloud coverage by itself provides useful information about the climate and/or atmospheric parameters. Therefore, detecting clouds over a region is essential for assessing the atmospheric conditions of that region. Furthermore, cloud shadows can adversely impact the above-mentioned applications, since objects and structures covered by cloud shadows are not fully visible. Cloud and cloud shadow detection, along with cloud and cloud shadow coverage estimate, are thus among the most critical processes in the analysis of optical remotely sensed imagery. The objective of this dissertation is to address the problem of cloud and cloud shadow detection in optical remotely sensed imagery. Learning-based algorithms, especially deep learning-based ones, have proven to be promising for numerous computer vision and image processing tasks. Here, multiple deep learning models are proposed to learn the relevant cloud and cloud shadow features from satellite images. Furthermore, the problem of shadow segmentation in general RGB images is explored to set the stage for tackling cloud shadow detection. Two cloud detection datasets are prepared (and made publicly available) for the training and testing of deep learning models. These datasets—with more than 3,500 downloads to date—include various land and cloud types for a fairer benchmark evaluation of cloud detection algorithms. Additionally, a novel loss function is proposed for optimization of deep learning-based segmentation models that is useful when the object of interest exists only in some of the training/test images. Moreover, multiple off-line data augmentation algorithms are introduced to enhance the generalization ability of cloud and cloud shadow segmentation models. The research presented in this dissertation will help developing intelligent systems for filtering out images that are highly covered by clouds and their shadows after their acquisition. All the proposed algorithms and models can be deployed on-board satellites to facilitate more efficient data management. This avoids wasting resources by retaining and transmitting only those images with limited cloud coverage to the ground stations. In addition, by determining cloud/shadow-free regions in images, those areas can be used for further analysis.