feat(kernel-language-model-service): Add language model client#876
Merged
feat(kernel-language-model-service): Add language model client#876
Conversation
grypez
force-pushed
the
grypez/language-model-client
branch
from
f2a4ffd to
e3cf0b3
Compare
Contributor
grypez
force-pushed
the
grypez/language-model-client
branch
from
e3cf0b3 to
5296cb9
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
Author
|
@cursoragent review this PR for inconsistencies across docstrings, documentation and naming conventions |
|
Unable to authenticate your request. Please make sure to connect your GitHub account to Cursor. Go to Cursor |
grypez
force-pushed
the
grypez/language-model-client
branch
from
2dffa25 to
ea9f656
Compare
grypez
force-pushed
the
grypez/language-model-client
branch
4 times, most recently
from
216052f to
a21e0ed
Compare
Add @ocap/kernel-language-model-service — a new package that bridges the kernel with OpenAI-compatible language model backends. Core types and kernel service: - ChatParams / ChatResult / ChatService types for request/response shapes - makeKernelLanguageModelService: registers a kernel service object that dispatches chat calls to a provided chat function - LANGUAGE_MODEL_SERVICE_NAME constant Backends: - Open /v1 backend with SSE streaming (makeOpenV1NodejsService) - Ollama Node.js backend (OllamaNodejsService) Client API: - makeChatClient: returns an OpenAI-SDK-compatible client object that routes chat.completions.create() calls through a CapTP ChatService - makeSampleClient: convenience wrapper for single-token sampling Test utilities: - makeMockOpenV1Fetch: deterministic SSE mock for unit/integration tests Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add an integration test and e2e test that exercise the full kernel → LMS service → Ollama round-trip via a bundled vat. - lms-chat-vat: bundled vat that sends a single chat message and logs the response, used to verify the round-trip through the kernel - lms-chat.ts: shared test helper (runLmsChatKernelTest) - lms-chat.test.ts: uses a mock fetch — CI-safe, no network - lms-chat.e2e.test.ts: uses real fetch against local Ollama - agents.e2e.test.ts: json/repl agent e2e tests against local Ollama - test/suite.test.ts: pre-flight check that Ollama is running Lay out the package to match kernel-test: all vats, helpers, and test files live under src/; test/ holds only the Ollama pre-flight suite. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a chat-based capability-augmented agent alongside the existing
text-completion JSON agent. Uses ChatService.chat() with a messages[]
array instead of LanguageModel.sample() with a raw prompt string.
- makeChatAgent({ chat, capabilities }) implements the Agent interface
- Capabilities are described to the model via a JSON system prompt
- Model signals completion by invoking the auto-injected 'end' capability
- Capability results are injected as 'user' turns for model-provider compatibility
- Exported as @ocap/kernel-agents/chat, mirroring @ocap/kernel-agents/json
Usage:
const client = makeChatClient(lmsServiceRef, 'qwen2.5:0.5b');
const agent = makeChatAgent({
chat: (messages) => client.chat.completions.create({ messages }),
capabilities: { walletBalance, walletSend },
});
const result = await agent.task('what is my balance?');
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…gent Replace the custom JSON-in-system-prompt approach with the standard chat completions tool-calling interface: capabilities are passed as `tools`, responses arrive via `tool_calls`, and results are returned as `role: "tool"` messages. Add a `glm-4.7-flash` / llama.cpp e2e test and a `LMS_PROVIDER` constant so the suite can be aimed at either Ollama or llama.cpp without OOM. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
grypez
force-pushed
the
grypez/language-model-client
branch
from
a21e0ed to
70ae122
Compare
sirtimid
previously requested changes
Mar 31, 2026
Contributor
sirtimid
left a comment
sirtimid
left a comment
There was a problem hiding this comment.
Nice! Some comments to address.
| * @param params - The chat parameters. | ||
| * @returns A promise resolving to the full chat result. | ||
| */ | ||
| async #nonStreamingChat(params: ChatParams): Promise<ChatResult> { |
Contributor
There was a problem hiding this comment.
#nonStreamingChat, #streamingChat, and listModels never check response.ok. A 401/429/500 will produce cryptic JSON.parse or .map() errors instead of a clear HTTP error. This is a real production concern with LLM APIs (rate limits, auth failures).
packages/kernel-language-model-service/src/open-v1/base.test.ts
Outdated
Show resolved
Hide resolved
packages/kernel-language-model-service/src/open-v1/base.test.ts
Outdated
Show resolved
Hide resolved
packages/kernel-language-model-service/src/open-v1/base.test.ts
Outdated
Show resolved
Hide resolved
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Made-with: Cursor
Made-with: Cursor
Co-authored-by: Dimitris Marlagkoutsos <info@sirtimid.com>
grypez
commented
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a client constructor to
@ocap/kernel-language-model-service, allowing wiring ollama or any OpenAI-compatible language model server into the kernel as a first-class kernel service; and extends@ocap/kernel-agentswith a chat-completion-based agent that uses the standard tool-calling interface. Verified end-to-end with integration and e2e tests in@ocap/kernel-test-local.feat(kernel-language-model-service): Add /v1 API support and language model clientLanguage model service supported backends:
/v1/chat/completionsAPI is de facto standard across industry, as evidenced by spaghetti compatibility across LiteLLM, Ollama, llama.cpp, and anthropic.Vat clients (
src/client.ts):makeChatClientreturns an OpenAI-SDK-shaped object (client.chat.completions.create(...)) that routes calls through a CapTPChatServicereference viaE().makeSampleClientis the equivalent for raw token sampling.Test utilities (
src/test-utils/): Replaces the previous queue-based mock model withmakeMockOpenV1Fetch— a deterministic SSE mock that emits a configured sequence of token strings, keeping integration tests CI-safe and fast.makeMockSampleprovides the same for the sample path.Also updates
kernel-testto use the new chat/sample API: replaceslms-user-vat/lms-queue-vatwithlms-chat-vat/lms-sample-vatand the corresponding test files.feat(kernel-agents): chat-completion-based agent with tool-callingAdds
makeChatAgentto@ocap/kernel-agents(exported as@ocap/kernel-agents/chat), a capability-augmented agent built on the chat completions API. Capabilities are passed astools, responses arrive viatool_calls, and results are returned asrole: "tool"messages.test(kernel-test-local): kernel-LMS integration testsAdds two test variants that share a common
runLmsChatKernelTesthelper:lms-chat.test.ts: injectsmakeMockOpenV1Fetch— no network, runs undertest:dev.lms-chat.e2e.test.ts(local): uses realfetchagainst a local Ollama instance, run viatest:e2e:local.Both launch a subcluster with
lms-chat-vatthrough the kernel, register the LMS kernel service, and assert that the vat logs the model's response.Also adds
agents.e2e.test.tsexercising chat/json/repl agents end-to-end, andtest/suite.test.tsas a language-model-service pre-flight check, whether using ollama, llama-cpp, lite-llm, etc.Package layout is conformed to
kernel-test: all vats, helpers, and test files live undersrc/;test/holds only the pre-flight suite.Test plan
yarn workspace @ocap/kernel-test-local test:dev:quiet— unit + integration tests pass (no Ollama required)yarn workspace @ocap/kernel-language-model-service test:dev:quiet— all klms unit tests passyarn workspace @ocap/kernel-agents test:dev:quiet— all kernel-agents unit tests passllama3.2:3bpulled:yarn workspace @ocap/kernel-test-local test:e2e:localglm-4.7-flashpulled and served,yarn workspace @ocap/kernel-test-local test:e2e:local:llama-cppNote
Medium Risk
Moderate risk due to introducing new public exports and wiring for chat/tool-calling plus new Open /v1 and expanded Ollama service paths, which could affect downstream integrations and streaming behavior.
Overview
Adds a first-class chat-completions client layer to
@ocap/kernel-language-model-service(makeChatClient/makeSampleClient) plus a kernel-service wrapper (makeKernelLanguageModelService) so vats can call LMS backends via a stable/v1-style API.Introduces an OpenAI-compatible
/v1Node.js backend with parameter/response validation, JSON stripping, and SSE streaming support, and expands the Ollama backend to supportchatand both streaming and non-streaming rawsamplerequests (including tool-call argument parsing).Extends
@ocap/kernel-agentswith a newmakeChatAgentstrategy (exported as@ocap/kernel-agents/chat) that drives tool-calling loops, validates tool args via newkernel-utilsJSON-schema→Superstruct helpers, and records chat turns as experiences; updates local and kernel tests to exercise the new service/client paths and replaces the old queue-based LMS test utils with deterministicmakeMockOpenV1Fetch/makeMockSample.Written by Cursor Bugbot for commit 6b23244. This will update automatically on new commits. Configure here.