3D FaceShell: Attribute Transfer in 3D Face Avatars as a VLM Defense Mechanism
Researchers introduce 3D FaceShell, a framework that applies subtle, learnable perturbations to 3D face avatars to mislead vision-language models (VLMs) from accurately inferring sensitive attributes, while preserving the avatar's visual fidelity and identity. The method uses a Gaussian shell optimized via multi-view embedding alignment to redirect VLM-based attribute inference. Experiments on celebrity face avatars and multiple black-box VLMs show that 3D FaceShell increases attribute injection and mismatch rates without compromising human-recognizable appearance.
Why it matters: This work proposes a novel defense against privacy risks posed by VLMs extracting sensitive information from 3D face avatars, operating directly on 3D representations rather than 2D images.
Full story at: arXiv Computer Vision ↗