QWEN4'S ARCHITECTURE IS HERE EARLY, FIRING 6B PARAMETERS OUT OF 125B

Summary from the TLDR AI newsletter

Rather than adding experts, the model bolts on 51B parameters as a separate embedding indexed by two- and three-character fragments.