TLDR AI · HEADLINES & LAUNCHES

MODAL ADDS WEBSOCKETS

modal.com5 min read

Summary from the TLDR AI newsletter

Most serverless GPU offerings require one POST request per model inference. However, this is a challenge in cases where you want to stream model outputs. Websockets make streaming possible.