Diagnosing skin lesions from images with convolutional neural networks

Dermatological disorders are among the most common reasons for patients to visit general practitioners and are often diagnosed based on the visual properties of the affected skin. Machine classification of skin lesions from images offers the potential for a low cost, reproducible diagnosis that improves patients' access to dermatological expertise. This thesis studies automated approaches to diagnose skin disorders based on visual cues within colour images. The thesis begins with a review of the existing literature on visual diagnosis of skin disorders, compares the reported performance of humans and machines, and discusses general limitations of image-based diagnosis. This thesis then proposes five novel approaches that rely on convolutional neural networks (CNNs) to diagnosis skin lesions. The first two proposed works demonstrate that the parameters of a CNN, learned from non-skin images, transfer well to the tasks of skin lesion classification and image retrieval. The third work proposes a multi-resolution CNN architecture with end-to-end training that further increases classification accuracy. The final two works classify and localize several visual criteria that are commonly associated with melanoma from dermoscopy images, where a multi-task loss function in a multi-modal CNN architecture is proposed for classification, and a multi-label Dice score modified for imbalanced data is proposed to localize infrequently occurring melanoma-specific criteria. Finally, this thesis concludes with open questions that may benefit from further collaboration between dermatologists and computing researchers. This thesis demonstrates the potential role for CNNs as a common methodological building block to address the visual component of a variety of clinical problems within dermatology.