SLICEGPT: STREAMLINING LARGE LANGUAGE MODELS EFFICIENTLY

Summary from the TLDR AI newsletter

SliceGPT introduces a novel post-training sparsification scheme to reduce the resource demands of large language models. By replacing weight matrices with smaller ones and reducing embedding dimensions, it can remove up to 25% of model parameters in major models like LLAMA2-70B and OPT 66B, while retaining up to 99% task performance.