Learning shape-to-shape transformation

Many problems in computer graphics and geometric modeling, e.g., skeletonization, surface completion, and shape style transfer, can be posed as a problem of shape-to-shape transformation. In this thesis, we are interested in learning general-purpose shape transform, e.g., between 3D objects and their skeletons, between chairs and tables, and between letters of two different font styles, etc. With a point-based shape representation, we explore the problem of learning general-purpose shape-to-shape transformation, under two different settings: i). having shape-level supervision, ii). unsupervised. We present P2P-NET, a deep neural network, for learning shape transform under shape-level supervision. It is trained on paired shapes from the source and target domains, but without relying on point-to-point correspondences between the source and target point sets(i.e., point-level supervision). The architecture of the P2P-NET is that of a bi-directional point displacement network, which transforms a source point set to a prediction of the target point set with the same cardinality, and vice versa, by applying point-wise displacement vectors learned from data. For an unsupervised setting, we introduce LOGAN, a deep neural network aimed at learning general-purpose shape transforms from unpaired shape domains. It consists of an autoencoder to encode shapes from the two input domains into a common latent space, where the latent codes are overcomplete representations for shapes. The translator is based on a generative adversarial network (GAN), operating in the latent space, where an adversarial loss enforces cross-domain translation while a feature preservation loss ensures that the right shape features are preserved for a natural shape transform. We conduct ablation studies to validate each of our key designs and demonstrate superior capabilities in shape transforms on a variety of examples over baselines and state-of-the-art approaches. Several different applications enabled by our general-purpose shape transform solutions are presented to highlight the effectiveness, versatility, and potential of our networks in solving a variety of shape-to-shape transformation problems.