Multimodal KS

[HARSHDIPSAHA]

Fixes issue #16
Currently, the KnowledgeSpace AI agent relies only on text-based queries, while a large amount of
neuroscience metadata remains locked inside non-editable formats such as figures, tables, and
presentation screenshots. This PR introduces a multimodal pipeline that allows users to upload
images and automatically convert them into refined, high-signal search queries.

The feature includes:

• A React-based upload interface with an auto-expanding search box for long scientific queries.
• A FastAPI backend endpoint that performs OCR using Pytesseract.
• An intelligence layer using Gemini 2.0 Flash-Lite for zero-shot Named Entity Recognition (NER).
• A refined prompt that outputs only a clean, comma-separated list of scientific entities,
  preventing conversational noise from polluting the search engine.

Impact:

• Unlocks neuroscience metadata trapped in images.
• Improves dataset discovery precision and recall.
• Enables a fully multimodal search workflow for KnowledgeSpace.

[HARSHDIPSAHA]

@QuantumByte-01 please review.

[QuantumByte-01]

Interesting direction — extracting neuroscience terms from uploaded images via OCR and Gemini is a genuinely useful idea for researchers uploading paper figures. Three things needed before merging: pytesseract requires a system-level Tesseract installation which is not reflected in the Dockerfile or setup docs (add it to both), the frontend file upload UI needs styling and proper error states, and OCR accuracy on scientific figures without preprocessing will be poor — consider adding image preprocessing (contrast, binarization) before OCR. Please address these and reopen.

[HARSHDIPSAHA]

Hi @QuantumByte-01 , thanks for looking into this.

Interesting direction — extracting neuroscience terms from uploaded images via OCR and Gemini is a genuinely useful idea for researchers uploading paper figures. Three things needed before merging: pytesseract requires a system-level Tesseract installation which is not reflected in the Dockerfile or setup docs (add it to both), the frontend file upload UI needs styling and proper error states, and OCR accuracy on scientific figures without preprocessing will be poor — consider adding image preprocessing (contrast, binarization) before OCR. Please address these and reopen.

Perfectly valid points, I wil work on this..