Skip to content

feat: add LEANN connector support#183

Open
iamonuwa wants to merge 1 commit intopathwaycom:mainfrom
iamonuwa:add-leann-connector
Open

feat: add LEANN connector support#183
iamonuwa wants to merge 1 commit intopathwaycom:mainfrom
iamonuwa:add-leann-connector

Conversation

@iamonuwa
Copy link

Introduction

To contribute code to the Pathway project, start by discussing your proposed changes on Discord or by filing an issue.
Once approved, follow the fork + pull request model against the main branch, ensuring you've signed the contributor license agreement.

Context

This PR adds a LEANN output connector (pw.io.leann.write) to support LEANN vector indices in RAG pipelines. LEANN is a storage-efficient vector database that uses graph-based selective recomputation to achieve 97% storage reduction compared to traditional solutions.

As discussed in issue #173, LEANN is implemented as a Python output connector following the established pattern from issue #167. The connector accumulates documents and builds the LEANN index on pipeline completion, with support for both HNSW and DiskANN backends.

How has this been tested?

  • Unit tests (python/pathway/tests/test_io_leann.py): Test observer behavior including additions, deletions, empty text handling, metadata extraction, and index build logic with mocked LEANN dependencies.
  • Integration tests (integration_tests/leann/test_leann.py): End-to-end tests verifying index creation, search functionality, metadata columns, DiskANN backend, custom text columns, directory creation, and edge cases like empty tables.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature or improvement (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

  • My code follows the code style of this project,
  • My change requires a change to the documentation,
  • I described the modification in the CHANGELOG.md file.

@CLAassistant
Copy link

CLAassistant commented Jan 31, 2026

CLA assistant check
All committers have signed the CLA.

@zxqfd555 zxqfd555 self-assigned this Feb 4, 2026
@zxqfd555 zxqfd555 self-requested a review February 4, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants