OpenAI Details GPT-Live’s Architecture for Continuous Stateful Voice Interaction

OpenAI recently published an engineering account of GPT-Live. It described how they designed the system to maintain continuous voice interaction while separating latency-sensitive media processing from broader application work. The live path contains the media pipeline and inference loop, while delegation, tool use, persistence, and other application logic run behind an asynchronous RPC boundary.

The design reflects a central challenge for real-time AI applications. Conversation must remain responsive even when other operations have variable latency or depend on external services. OpenAI’s account also describes dedicated, stateful inference for each session. Sessions reserve capacity on their assigned instance, but their context can move to another instance when capacity is being drained, or a conversation reaches its context limit.

OpenAI retained WebRTC as its media foundation, introducing WebRTC Abridged Roundtrip Protocol (WARP⁠) improvements and Instant Connect to reduce startup latency. Before launch, the company also conducted a "silent" test that processed authentic incoming Voice traffic while discarding the output. According to OpenAI, this identified load-related behavior that synthetic tests had missed.