Open
Conversation
Introduce a browser-based settings experience and improve model path resolution. Adds a Settings panel (Settings.razor) with styles and JS (settings.css, settings.js) and services to persist settings and API keys (SettingsService, InferPageSettings, SettingsStateService). Wire the settings UX into NavBar and Home to request/show/save settings and apply them at runtime via Utils.ApplySettings; Program.cs now distinguishes CLI-provided config vs browser configuration (Utils.NeedsConfiguration). Update Utils to support manual capability overrides and sanitize model paths, and tweak app CSS and script includes. Improve LLMService local model resolution and path handling (fallback GenericLocalModel, ResolvePath) so unregistered or absolute model paths load correctly.
Several refactors and behavior fixes across the app: - NavBar: call SettingsState.RequestSettings directly from the settings button, simplify theme toggle and reload helpers, remove unused flag/methods. - Home: prevent adding image inputs when the selected model doesn't support vision and display an error. - Settings page: build backend options with Prepend for simpler ordering. - Program.cs: validate Self backend model at startup and exit if unsupported; adjust AddMaIN registration flow based on NeedsConfiguration and BackendType. - SettingsService: convert to constructor-injected style, consolidate load/save into generic dict helpers, and add typed methods for saving/getting API keys and model history. - SettingsStateService: shorten comment describing the event bus. - Utils: unify capability checks with a generic GetCapability<T>, make Reason mutually exclusive with ImageGen, and streamline environment variable handling when setting backend API keys. These changes simplify codepaths, centralize settings persistence logic, and enforce model capability checks earlier and at the UI level.
Add per-backend profile storage (model + capability flags) and use it in settings UI. SettingsService: introduce BackendProfile record and SaveProfileForBackendAsync/GetProfileForBackendAsync using a new storage key. Settings.razor: load profile on backend select, restore manual vision/reasoning/imagegen flags only for unregistered models, fallback to legacy model-history when no profile exists, and save the full profile instead of just the model. This enables preserving capability information per backend while remaining compatible with older model-only entries.
Add support for multimodal projector (.mmproj) names across the stack and ensure uploaded image files are treated as images. Implemented MMProjectName on several local vision models and added MmProjName property to InferPage settings, backend profiles, Utils, and ServiceConstants. Settings UI now exposes an MMProj File input for local unregistered vision models and saves/loads it per backend. LLMService now extracts image files from message.Files early (ChatHelper.ExtractImageFromFiles), resolves mmproj name from the model or chat properties to load LLava weights, and requires loaded weights before processing image messages. ChatHelper.ExtractImageFromFiles moves image files into message.Images and cleans up Files so they aren't misrouted to RAG/memory.
Home: when a custom Model Path is set for Self backend, bake the resolved full file path into a generic local model instance so the LLMService loads from the correct location (creates GenericLocal[Vision][Reasoning]Model variants with FileName set to the resolved path). This fixes cases where Chat.Properties don't reach the service and the model file must be embedded in the model object. Settings: add RegisteredMmProjPathHint to display the expected MMProj file path for registered local vision models (shows a "MMProj file: ..." hint derived from ResolvedModelPathPreview or fallback model directory), keeping the hint in sync with the "Will load:" preview. LLMService: when resolving an mmproj for image models, prefer the directory of a fully-qualified modelKey (custom model file path) and fall back to the configured models path; this ensures mmproj files are located next to custom model files.
Update settings dialog sizing in src/MaIN.InferPage/wwwroot/settings.css to use viewport-based width and constraints: set width to 40vw with a min-width of 480px and max-width 90vw, and increase max-height from 85vh to 95vh. This improves layout and usability across different screen sizes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1. Settings UI for InferPage
2. Custom Model Paths
3. Image & MMProj Support
4. Code Refactoring
5. LLamaSharp Upgrade