TLDR AI · HEADLINES & LAUNCHES
MODAL ADDS WEBSOCKETS
Summary from the TLDR AI newsletter
Most serverless GPU offerings require one POST request per model inference. However, this is a challenge in cases where you want to stream model outputs. Websockets make streaming possible.