Release 2.0

Release 2.0 Latest

Changelog: Release 2.0

Added:

Model Configuration UI: Added dynamic dropdown selectors to the sidebar to switch between Chat and Vision models at runtime.

ChromaDB Document Clearing: Added a "Clear Synced Documents" control to the sidebar to clear user-specific vector embeddings from ChromaDB.

Optimized:

Verbatim Image Scaling: Implemented automatic scaling for high-resolution images (max 1024px) to drastically reduce vision tile segmentation, yielding an 8x-16x speedup during document syncs.

Memory Lifecycle Management: Enforced immediate model unloading with keep_alive=0 on Ollama client calls, preventing multiple models from residing in memory simultaneously.

Hardened System Constraints: Upgraded RAG prompts to prevent hallucinations, strictly forcing the model to state "I do not know" when facts are missing from the context.

Systematic Sidebar OCR: Configured the vision engine prompt to systematically transcribe sidebars and secondary sections (such as Languages, Packages, and Contributors).

Broadened Search Window: Increased retrieval search neighbors (k_neighbors) to 15 to ensure matching data chunks are not crowded out by duplicates.

Fixed:

Streamlit Session State Crash: Resolved a KeyError and AttributeError by initializing the standard model in the Streamlit session state on page load.

Container OOM Failures: Resolved Docker container terminations (Exit Code 137) by lowering the peak RAM and VRAM footprint.

Dependency Cleanup: Streamlined the deployment footprint by removing the obsolete CPU-based Tesseract OCR packages and configurations from the Dockerfile and requirements.txt.

Release 1.0

Changelog: Release 1.0

Added:

• Private Multimodal RAG: Introduced the ability to process and interact with both text documents and visual media (PNG/JPG) fully on-device.

• Dual-Pass Vision Engine:

  • Semantic Pass: Integrated moondream to interpret image layouts, UI structures, and diagrams.

  • Syntax Pass: Integrated Tesseract OCR for high-fidelity extraction of code blocks and technical text from images.

• Config-Driven Architecture:

  • config/parameters.json: Externalized control for LLM selection, VRAM-optimized chunking, and inference settings.

  • config/prompts.json: Customizable system personas and RAG response templates.

• Unified Deployment Script: Created run.sh to automate system diagnostics, Ollama model pulls, and container orchestration.

• Local Vector Storage: Implemented persistent storage using ChromaDB for fast semantic retrieval.

• User Management: Integrated MongoDB for secure session handling and user authentication.

Optimized:

• Hardware Balancing: Tuned the engine for 4GB VRAM hardware (GTX 1050 Ti), utilizing llama3.2:1b for efficient local inference.

• Project Structure: Modularized code into src/, utils/, and config/ directories for better maintainability.

Fixed:

• Ollama Network Bridge: Resolved connectivity issues between Docker containers and the host Ollama service on Linux systems.

• Context Integrity: Optimized chunking logic to ensure visual descriptions and extracted text remain linked in the vector database.

Pre-release Version 0.1

Pre-release Version 0.1

Features:

• On device document intelligence capabilities for .txt and .pdf files.
• User Login and Chat History support.
• On device VectorDB storage which is used for RAG context.