VAULTGEMMA: THE WORLD'S MOST CAPABLE DIFFERENTIALLY PRIVATE LLM

Summary from the TLDR AI newsletter

VaultGemma is a model that Google trained from scratch with Differential Privacy (DP). DP offers a mathematically robust solution to user privacy that adds calibrated noise to present memorization. It has some trade-offs, like reducing training stability and significantly increasing batch size. There is still a utility gap between DP-trained and non-DP-trained models, but that gap can be systematically narrowed with more research on mechanism design for DP training.