EASI-Tex: edge-aware mesh texturing from single-image

We present a novel approach for single-image guided 3D mesh texturing, which employs an image diffusion model with judicious conditioning to transfer textures from a single image to a given 3D shape in a consistent manner. We condition a pre-trained Stable Diffusion generator with edges describing the mesh through ControlNet, and features extracted from the input image using IP-Adapter to produce textures that respect the underlying geometry of the mesh and the input texture without any optimization or training. However, we also introduce Image Inversion, a novel technique to quickly personalize the diffusion model using a single-image, for cases where the pre-trained IP-Adapter falls short in capturing all the details from the input image faithfully. Experimental results showcase the efficiency and effectiveness of our single-image guided edge-aware 3D mesh texturing approach, EASI-Tex, in preserving the details of the input texture on diverse 3D objects while respecting their geometry.