TLDR AI · ENGINEERING & RESOURCES

STARCODER IN GGML

github.comGitHub repository

Summary from the TLDR AI newsletter

GGML is a framework for running 4-bit quantized models on the CPU. This means you can run really large models locally on your laptop. Starcoder is one of the very best open source program synthesis models. Scoring very well on the challenging OpenAI human eval benchmark. This pull request adds that model to the GGML framework which means you can run starcoder on commodity hardware.