TLDR Dev · QUICK LINKS
TOKENFLOOD
Summary from the TLDR Dev newsletter
Tokenflood is a load testing tool for instruction-tuned LLMs that simulates arbitrary load profiles by defining prompt lengths, prefix lengths, output lengths, and request rates.
TLDR Dev · QUICK LINKS
Tokenflood is a load testing tool for instruction-tuned LLMs that simulates arbitrary load profiles by defining prompt lengths, prefix lengths, output lengths, and request rates.