DO TABULAR FOUNDATION MODELS REPAIR THEMSELVES?

Summary from the TLDR Data newsletter

Ablation results in tabular foundation models can be misleading when redundancy looks like causal importance. Measurements of direct and total effects across four models found little evidence of downstream self-repair, suggesting data teams should treat stable outputs after ablation as a reason to examine causality more carefully, not as proof that a layer is irrelevant.