STYLIZATION OF 3D MESHES USING 2D DIFFUSION MODELS

Summary from the TLDR AI newsletter

This research presents the 3DStyle-Diffusion model, a novel approach for detailed stylization of 3D meshes, integrating 2D Diffusion models for added control over appearance and geometry. It works by first parameterizing a 3D mesh's texture into reflectance and lighting, using implicit MLP networks, and then using a pre-trained 2D Diffusion model to align the rendered images with the text prompt and ensure geometric consistency.