SKED: sketch-guided text-based 3D editing

With the recent advent of Text-to-3D models based on Text-to-image diffusion models, they are increasingly being integrated into Computer Graphics pipelines, facilitating creative 3D design tasks in unrestricted domains. Nevertheless, simple textual prompting is often insufficient for interactive and local manipulation of 3D content. Incorporating user-guided sketches offers a more intuitive control for editing and manipulation. However, since state-of-the-art Text-to-3D models rely on gradients generated by rendering arbitrary views for optimization, integrating sketches in these pipelines is not straightforward. This thesis presents SKED, a method for editing 3D objects represented as Neural Radiance Fields (NeRFs) conditioned on a text prompt and two sketches from different views. We ensure semantic alignment of the edit by using a pretrained text-to-image model as guidance. Additionally, our framework uses novel loss functions to make the edit adhere to the provided sketches. We demonstrate the effectiveness of our method through various qualitative and quantitative experiments.