feat: add facial voice presentation hooks#272
Conversation
|
Follow-up implementation pushed. Added:
Validation:
Local reviewer fallback: APPROVED WITH RUNTIME SMOKE CAVEAT.
|
|
Update: added LiveKit-ready realtime NPC voice input hook. Implemented:
Validation:
px --yes markdownlint-cli2 docs/design/56-focused-npc-dialogue-portrait-lipsync-design.md: pass.
Local code-review fallback verdict: approved. The Unity client still does not call model/voice providers directly or store provider keys. LLM/voice output remains presentation/dialogue only; gameplay state still requires Nakama/Fusion validation. Remaining gap: live Play Mode smoke for actual mic UX should run after this branch is merged into the integration Unity workspace, because root dev is currently dirty and this feature branch is validated in the dedicated validation worktree. |
|
Update: pushed real voice-turn wiring, not just the client hook. Implemented now:
Validation:
Local code-review fallback verdict: approved. Unity still does not store provider keys or call Gemini/OpenAI directly. Cloud voice remains behind Nakama/api.dos.ai; local Windows speech is a development fallback only. |
|
Validation update for commit 2586604:
Local validation:
Local reviewer fallback verdict: approved with known Unity Editor/MCP noise. No provider API keys are exposed in Unity, and the new audio path remains presentation-only. |
What does this PR do?
Adds the first custom facial-animation and scoped voice presentation path for prototype NPC speech. Unity now requests voice session material through the existing Nakama gateway, plays a server-provided clip when available, and falls back to a local prototype tone that drives mouth blendshapes by audio amplitude.
Linked issue / ADR
Refs #262. Refs #25 for the later Ida Faber asset import validation pass.
Touched areas
Unity/)backend/nakama/)docs/design/) or ADRs (docs/adr/)Test plan
git diff --cached --checknpx.cmd --yes markdownlint-cli2 docs/design/56-focused-npc-dialogue-portrait-lipsync-design.mdmcpforunity://instancesreturned 0 instances).Unity/, due to the existing Package Manager failure:The "path" argument must be of type string. Received undefined. No packages loaded.No C# compiler errors were emitted before package resolution stopped.npm.cmd run buildinbackend/nakama, due to existing TypeScript config failureTS5103: Invalid value for '--ignoreDeprecations'withignoreDeprecations: "6.0".npm.cmd testinbackend/nakama, becausebuild/index.jsis absent after the blocked build.No Nakama runtime source was changed in this PR.
Server-authority check (mandatory if touching gameplay)
This PR only adds local speech presentation. Voice material is requested through Nakama/api.dos.ai session data and remains presentation only.
Reviewer pass
Local
code-reviewfallback verdict: APPROVED WITH VERIFICATION CAVEATS.Review summary:
SecondSpawnGatewayClient.GetVoiceSessionfor scoped voice material and does not embed provider keys.