fix(google): skip context replay on resumed realtime sessions#6000
fix(google): skip context replay on resumed realtime sessions#6000he-yufeng wants to merge 1 commit into
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
🚩 Session resumption handle is never cleared on failed resumption
The _recv_task at realtime_api.py:1042-1049 only updates _session_resumption_handle when session_resumption_update.resumable=True AND new_handle is present. If the server indicates resumable=False (e.g., handle expired), the stale handle is never cleared. This is a pre-existing issue not introduced by this PR, but the PR amplifies its impact: with a stale handle, _should_seed_initial_chat_context() will keep returning False on every subsequent reconnection attempt, meaning the initial chat context is never sent even though the server can't resume from the handle. Whether this is a practical problem depends on how the Google API handles expired handles (it may always reject the connection outright).
(Refers to lines 1042-1049)
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
🚩 history_config and session_resumption may have conflicting semantics
In _build_connect_config() at line 1086, when mutable_chat_context is False, history_config=HistoryConfig(initial_history_in_client_content=True) is set, telling the server that initial history will be provided via client content. However, when a resumption handle is also set (line 1122-1124), the config tells the server both to resume from a handle AND to expect client-sent history. Since _should_seed_initial_chat_context() now skips sending that history when a handle exists, there could be a protocol mismatch where the server expects client content that never arrives. This depends on the Google API's handling of these flags together and may not be an issue if session resumption takes precedence over history_config.
(Refers to lines 1086-1088)
Was this helpful? React with 👍 or 👎 to provide feedback.
| def _should_seed_initial_chat_context(self) -> bool: | ||
| return self._session_resumption_handle is None |
There was a problem hiding this comment.
🟡 Chat context updates during disconnection are silently lost on session resumption
When _session_resumption_handle is set and the connection drops, any calls to update_chat_ctx() (e.g., tool results completing, post-interruption sync at agent_activity.py:3533 or agent_activity.py:3654) store the new context in self._chat_ctx but return early without sending (realtime_api.py:611-613). On reconnect, _should_seed_initial_chat_context() returns False because the handle is non-None, so the updated context is never sent. The server restores old state from the handle, creating a permanent client/server context mismatch. Subsequent update_chat_ctx calls compute diffs against the local self._chat_ctx (which includes the "phantom" messages), so those messages are never sent to the server.
Concrete scenario: tool execution completes during reconnection
- Session active, handle "abc" received from server
- Connection drops →
_active_session = None - A tool finishes executing → agent calls
update_chat_ctx(ctx_with_tool_result) update_chat_ctxsees_active_session is None→ stores inself._chat_ctx, returns- Reconnection:
_should_seed_initial_chat_context()returns False → context NOT sent - Server restores from handle "abc" (old context without tool result)
- Client thinks server has tool result, server doesn't → permanent mismatch
Before this PR, the full self._chat_ctx (with updates) was always sent on reconnect, avoiding this mismatch.
Prompt for agents
The _should_seed_initial_chat_context method only checks if _session_resumption_handle is None, but doesn't account for whether self._chat_ctx was modified during disconnection (when _active_session is None). This creates a permanent context mismatch between client and server.
Relevant code locations:
- realtime_api.py:1073-1074 (_should_seed_initial_chat_context)
- realtime_api.py:610-613 (update_chat_ctx early return when no active session)
- realtime_api.py:872 (the check that gates initial context sending)
Possible approaches:
1. Add a flag like _chat_ctx_modified_while_disconnected that is set in update_chat_ctx when _active_session is None. Then _should_seed_initial_chat_context returns True if either handle is None OR the flag is set. Clear the flag after sending.
2. After successful session resumption (receiving session_resumption_update with resumable=True), compute the diff between what the server has and what self._chat_ctx has, and send any missing messages.
3. Track the chat_ctx state at the time the handle was obtained, and on reconnect compare against current self._chat_ctx to detect divergence.
Was this helpful? React with 👍 or 👎 to provide feedback.
he-yufeng
force-pushed
the
fix/google-realtime-resume-context
branch
from
cbc3f01 to
d4cfc32
Compare
|
Rebased onto current main and kept the focused resume-context guard intact. Ruff and py_compile pass. The focused pytest collection is currently blocked locally by an older installed livekit.protocol package missing agent_simulation; I did not widen the branch to environment or dependency changes. |
|
The red The job then kept running until GitHub canceled the runner several hours later. All style/type checks and blockguard tests are green. I am leaving the branch unchanged unless you see a link between this IPC process-pool failure and the resumed realtime session context path. |
1 participant
Summary
Fixes #5985
To verify