feat(ai): Implement session resumption and context window compression for live api#9795
feat(ai): Implement session resumption and context window compression for live api#9795
Conversation
🦋 Changeset detectedLatest commit: bbdcd1c The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Vertex AI Mock Responses Check
|
There was a problem hiding this comment.
Code Review
This pull request adds support for session resumption and context window compression in Live sessions. Key changes include the implementation of LiveSession.resumeSession(), the addition of SessionResumptionConfig and ContextWindowCompressionConfig interfaces, and updated server message handling for resumption updates. The review feedback identifies a logic error in the session connection handshake where the wrong setup message was transmitted and private state was incorrectly mutated due to a shallow copy. A leftover debugging log was also identified for removal.
| const setupMessage = { ...this._setupMessage }; | ||
| if (sessionResumption) { | ||
| setupMessage.setup.sessionResumption = sessionResumption; | ||
| } | ||
| this._webSocketHandler.send(JSON.stringify(this._setupMessage)); |
There was a problem hiding this comment.
The current implementation prepares a local setupMessage variable but then sends this._setupMessage. Furthermore, the mutation at line 107 (setupMessage.setup.sessionResumption = sessionResumption) actually mutates the private this._setupMessage property because the spread at line 105 is only a shallow copy. It is better to avoid mutating the stored setup message and ensure the correctly prepared object is sent to the handler.
| const setupMessage = { ...this._setupMessage }; | |
| if (sessionResumption) { | |
| setupMessage.setup.sessionResumption = sessionResumption; | |
| } | |
| this._webSocketHandler.send(JSON.stringify(this._setupMessage)); | |
| const setupMessage: _LiveClientSetup = { | |
| ...this._setupMessage, | |
| setup: { | |
| ...this._setupMessage.setup, | |
| ...(sessionResumption ? { sessionResumption } : {}) | |
| } | |
| }; | |
| this._webSocketHandler.send(JSON.stringify(setupMessage)); |
| await this.close(); | ||
| console.log('closed'); | ||
| await this._connectSession(sessionResumption); |
Based on #9624
Moved connection initiation logic from LiveGenerativeModel.connect() into the class so that the LiveSession.resumeSession() method can more easily access it. Since the LiveGenerativeModel.connect() method needs to wait for the connection to be opened successfully, and a constructor can't be async, the class provides a
connectionPromiseproperty for external code to tell when the connection has finished opening.