INTRODUCING GPT-REALTIME AND REALTIME API UPDATES FOR PRODUCTION VOICE AGENTS

Summary from the TLDR AI newsletter

OpenAI's Realtime API is now generally available. It now supports MCP servers, image inputs, and phone calling, making voice agents more capable. The underlying model, gpt-realtime, is OpenAI's most advanced speech-to-speech model yet, showing improvements in following complex instructions, calling tools with precision, and producing speech that sounds natural and expressive. The Realtime API processes and generates audio directly through a single model and API, which reduces latency and preserves nuance in speech, producing more natural, expressive responses.