MICROSOFT'S FULL-BANDWIDTH TRANSFORMERS

Summary from the TLDR AI newsletter

Full-bandwidth transformers fed the previous token's top-layer hidden state back into the model alongside the next token embedding. This allows latent computation to continue across decoding steps.