BITNET A4.8: 4-BIT ACTIVATIONS FOR 1-BIT LLMS

Summary from the TLDR AI newsletter

One of the key challenges with 1.58bit LLMs was the lack of hardware acceleration support. This work proposes 4.8bit activations to take advantage of the INT4/FP4 kernels in new hardware. It comes at no runtime cost.