Analyzing cancers in digitized histopathology images

Cancer refers to a group of diseases characterized by an uncontrolled proliferation of cells with underlying genetic mutations that can be arranged in solid masses forming tumors. The visual examination of morphological features in histopathological sections of a primary tumour, including appearance, spatial arrangement of cells as well as tissue architecture, forms the basis of diagnosis and prognostic for cancers. Currently, cancer diagnosis relies on a qualitative or semi-quantitative assessment of physical or digitized whole slide tissue images, which can be prone to subjectivity and relatively low agreement among experts. In this Thesis, after a thorough examination of the literature on studies that proposed deep learning models to analyze digital pathology images, we introduce novel automatic quantitative and prediction models that can potentially assist pathologists in the diagnosis of cancers. First, we introduce machine learning-based models in which we encode higher-order shape priors within multiple cost functions to quantify morphological features from digitized histopathology images. Specifically, we propose the first deep fully convolutional networks that integrate classification, segmentation, topological and geometrical priors combined with an uncertainty guided multi-loss function to analyze glandular structures in colon adenocarcinomas. Second, we present prediction models that automate experts’ cancer diagnosis procedure by evaluating the predictive relevance of clinically-derived features, learned features as well as a structured prediction model that mimics experts’ multi-magnification visual analysis of whole slide images. Finally, we present two strategies to improve the performance of learning-based prediction models by introducing automatic ways to identify abnormal tissue areas within multi-gigapixel whole slide images and reduce sensitivity to staining variability across datasets. The Thesis concludes with a discussion of the limitations of the proposed models and important directions for future research.