GOOGLE OPEN SOURCES 1.6T MOE MODEL (HUGGINGFACE HUB)

Summary from the TLDR AI newsletter

The switch transformer from Google was one of the first successful Mixture-of-Experts models. It now is available with code on the HuggingFace platform.