Learning to correspond and compose shape structures

Composing structures from different 3D shapes is a fundamental task in many computer graphics applications, especially for shape modeling and synthesis. Since many approaches to shape composition start by establishing a structural correspondence and segmentation, part-level structure analysis is a key component in a composition framework. However, due to the large discrepancy of geometry and topology, finding proper structure matching and resolving incompatibilities during composition are challenging. In this thesis, we decompose the structure composition problem into three sub-tasks, including finding part correspondences, co-segmentation, and composition optimization. Our goal is trying to find a general solution for structure composition between unsegmented 3D shapes with significant geometric and topological dissimilarities. First, we propose a fine-grained structure matching approach between two 3D shapes where the core evaluation mechanism only relies on recognizing shapes globally. Our key observation is that structure correspondences can be obtained by only performing shape recognition tasks. We demonstrate clear improvements over state-of-the-art approaches through test over sets of man-made models with rich geometric and topological variations. However, the proposed structure matching algorithm depends on part segmentation results. Co-segmentation would simultaneously solve the segmentation and correspondence problems to resolve the ambiguity caused by individual segmentation. We would focus on shape co-segmentation next. A deep network architecture for co-segmentation of a set of 3D shapes represented as point clouds is presented. We demonstrate that our network can provide consistent segmentation even when it is only trained on inconsistent data. After the correspondence and segmentation problems are properly settled, we propose a recursive neural network for shape composition. Our network takes as input sets of parts from two or more source 3D shapes and a rough initial placement of these parts. It outputs an optimized part structure for the composed shape, leading to high-quality geometry construction. We demonstrate that our network can significantly expand the range of composable shapes for assembly-based modeling through comparison with state-of-the-art alternatives.