feat(py): add Checks, Chroma, Pinecone, Cloud SQL PostgreSQL plugins with tests and Pydantic standardization #4328

yesudeep · 2026-01-30T09:44:03Z

Summary

This PR brings the Python SDK closer to feature parity with the canonical JavaScript implementation by adding several key plugins, standardizing Pydantic model patterns, and establishing robust testing conventions.

New Plugins (with JS Parity)

Cloud SQL PostgreSQL Plugin (NEW)

Full-featured vector store integration for Cloud SQL for PostgreSQL:

Parity: Matches JS implementation in API surface and runtime behavior.
Engine: PostgresEngine supporting Cloud SQL Connector with IAM and password authentication.
Indexes: Support for HNSW and IVFFlat vector indexes with configurable parameters.
Actions: PostgresRetriever and PostgresIndexer with metadata filtering and JSON support.
Utilities: Batch indexing, distance strategies (Euclidean, Cosine, Inner Product), and schema validation.

Checks Plugin

Google Checks AI Safety integration for content policy evaluation:

Parity: Evaluator returns scores per policy, matching JS behavior.
Features: Guardrails API client, middleware for violative content blocking, and support for all standard policy types.

Chroma Plugin

ChromaDB vector store integration for local RAG applications:

Features: Retriever and indexer actions, metadata filtering (where, whereDocument), and MD5-based ID generation.

Pinecone Plugin

Pinecone vector store integration for cloud-native RAG:

Features: Retriever and indexer actions, namespace support, and advanced metadata filtering.

Core SDK Improvements

Pydantic Model Standardization

Standardized all Pydantic models to use alias_generator=to_camel and populate_by_name=True.

Benefit: Ensures type checkers (ty, pyrefly, pyright) correctly resolve fields while maintaining camelCase compatibility with the Genkit JSON protocol.
Guideline: Documented in py/GEMINI.md.

Test Naming Convention & CI Stability

Established a strict naming convention to prevent module collection conflicts in large-scale test runs:

Format: {plugin_name}_{component}_test.py (e.g., cloud_sql_pg_engine_test.py).
Structure: Removed __init__.py from tests directories to ensure unique module paths.
Guideline: Added to py/GEMINI.md.

New Samples

rag-cloud-sql-pg: End-to-end RAG flow using Cloud SQL PostgreSQL.
rag-chroma: Local RAG sample using ChromaDB.
rag-pinecone: Cloud RAG sample using Pinecone.
checks-demo: Demonstration of AI safety guardrails.

Testing & Quality

All linters and tests pass with zero errors.

bin/lint: 0 errors (ty, pyrefly, pyright all pass)
Core genkit tests: 480 passed
Google GenAI tests: 163 passed
Checks plugin tests: 71 passed
Chroma plugin tests: 17 passed
Pinecone plugin tests: 17 passed
Cloud SQL PostgreSQL plugin tests: 52 passed

Total new tests added: ~160 tests

gemini-code-assist · 2026-01-30T09:45:24Z

Summary of Changes

Hello @yesudeep, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the Python Genkit library's capabilities by introducing robust features for model operation tracking, dynamic action management, and enhanced testing. It also broadens its ecosystem integration with new plugins for AI safety (Google Checks) and popular vector databases (ChromaDB, Pinecone), alongside practical RAG sample applications. These additions aim to provide developers with more control, flexibility, and tools for building advanced AI applications.

Highlights

Unified Operation Tracking for Models: Introduced a new generate_operation method in the Genkit AI core, providing a consistent Operation object for tracking model generations. This unifies the interface for both long-running background models and standard, immediate-response models, improving tracking, auditing, and enabling future retry/resume logic.
Dynamic Action Providers (DAP): Implemented Dynamic Action Providers (DAP) which allow for the dynamic loading and management of actions (like tools or models) from external sources at runtime. This feature includes caching mechanisms for performance and is crucial for integrating with systems like MCP (Model Context Protocol) servers or plugin marketplaces.
Enhanced Schema Definition: Added define_json_schema to the Genkit API, enabling direct registration of raw JSON schemas. This complements the existing define_schema for Pydantic models, offering more flexibility for external or custom schema definitions.
Comprehensive Model Testing Suite: A new test_models function has been added to the testing utilities, providing a standardized suite to validate model implementations. This suite covers basic generation, multimodal input, conversation history, system prompts, structured output, and tool calling, with automatic skipping for unsupported features.
New AI Safety Plugin: Google Checks: Integrated the Google Checks AI Safety API as a new plugin. This plugin offers content safety evaluation, guardrails middleware to automatically block unsafe content (both input and output), and configurable policies for various harm categories.
New Vector Store Plugins: ChromaDB and Pinecone: Introduced dedicated plugins for ChromaDB and Pinecone, enabling seamless integration of these popular vector databases for Retrieval-Augmented Generation (RAG) workflows. Both plugins support document indexing with embeddings and similarity-based retrieval, with Pinecone also offering namespace and metadata filtering.
RAG Sample Applications: Added new sample applications demonstrating RAG patterns using both ChromaDB and Pinecone. These samples illustrate how to index documents, retrieve relevant context, and augment LLM generations for more informed responses.
Documentation Updates: Updated documentation to include detailed feature parity analysis between JavaScript and Python Genkit implementations, and clarified Python import best practices.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This is a substantial pull request that adds several new features and plugins, significantly increasing feature parity with the JavaScript implementation. The additions of generate_operation, Dynamic Action Providers (DAP), and the model testing suite are well-implemented and thoroughly documented. The new plugins for Chroma, Pinecone, and Checks are also great additions to the ecosystem.

My review focuses on a few key areas for improvement:

Adherence to the project's own coding guidelines regarding import locations.
Performance optimization in the new checks plugin by avoiding unnecessary authentication token refreshes.
Improving code clarity and maintainability in the checks plugin by leveraging Pydantic's aliasing features for cleaner data model parsing.

Overall, this is excellent work that brings a lot of valuable functionality to the Python version of Genkit.

py/plugins/checks/src/genkit/plugins/checks/guardrails.py

py/packages/genkit/src/genkit/ai/_aio.py

py/plugins/checks/src/genkit/plugins/checks/guardrails.py

pavelgj · 2026-01-30T11:47:02Z

py/packages/genkit/src/genkit/core/plugin.py


    @abc.abstractmethod
-    async def init(self) -> list[Action]:
+    async def init(self, registry: 'Registry | None' = None) -> list[Action]:


registry was removed from plugins on purpose

how does one resolve embedders from a plugin then?

@huangjeff5

yesudeep · 2026-01-30T12:20:19Z

/gemini review

gemini-code-assist

Code Review

This is a substantial and well-structured pull request that adds significant new capabilities to the Python SDK, including production-ready vector store plugins for Chroma and Pinecone, comprehensive RAG samples, and important bug fixes. The work to ensure Python 3.14 compatibility and standardize plugin initialization signatures is also a valuable improvement. The code is generally of high quality with good documentation. I've identified one critical issue with a package dependency version, an incomplete implementation in the new checks plugin, and a couple of areas in the new RAG samples where the code can be simplified for better readability and maintainability. Overall, this is an excellent contribution.

py/pyproject.toml

py/plugins/checks/src/genkit/plugins/checks/plugin.py

py/samples/rag-chroma/src/main.py

py/samples/rag-pinecone/src/main.py

yesudeep · 2026-01-30T13:20:11Z

/gemini review

yesudeep · 2026-01-30T17:04:26Z

/gemini review

gemini-code-assist

Code Review

This is a substantial and well-executed pull request that adds significant new functionality with the Chroma, Pinecone, and Checks plugins. The standardization of Pydantic models using alias_generator=to_camel is a great improvement for code clarity and maintainability. The update to the Plugin.init signature to include the registry is a thoughtful change that enables more powerful and decoupled plugins, as demonstrated in the new vector store implementations. The comprehensive test coverage and addition of new samples are also commendable. I've left a few minor comments related to adhering to the new contribution guidelines introduced in this same PR. Overall, excellent work!

py/bin/generate_schema_typing

py/plugins/chroma/src/genkit/plugins/chroma/plugin.py

yesudeep · 2026-01-30T18:13:38Z

@huangjeff5 need your input here about the Plugin protocol

yesudeep · 2026-01-30T21:28:16Z

/gemini review

gemini-code-assist

Code Review

This is an impressive and substantial pull request that brings the Python SDK much closer to feature parity with the JavaScript implementation. The addition of the Checks, Chroma, Pinecone, and Cloud SQL PostgreSQL plugins is a major step forward.

I'm particularly impressed with:

The thorough documentation updates in GEMINI.md and the new plugins/README.md, which significantly improve developer guidance and discoverability.
The standardization of Pydantic models using alias_generator=to_camel, which enhances consistency and maintainability across the codebase.
The robust implementation of the new plugins, especially the thoughtful handling of authentication and database connections in the cloud-sql-pg plugin.
The pragmatic workarounds for Python 3.14 compatibility issues in noxfile.py and pyproject.toml.

I have a few minor suggestions for improvement:

There is some code duplication in the new checks plugin that could be refactored for better maintainability.
In several of the new vector store plugins (chroma, pinecone, cloud-sql-pg), there are opportunities to improve error handling by logging exceptions instead of silently passing, which would aid in debugging data quality issues.

Overall, this is a high-quality contribution that significantly enhances the capabilities and developer experience of the Python SDK. Great work!

py/plugins/checks/src/genkit/plugins/checks/plugin.py

py/plugins/chroma/src/genkit/plugins/chroma/plugin.py

py/plugins/pinecone/src/genkit/plugins/pinecone/plugin.py

py/plugins/cloud-sql-pg/src/genkit/plugins/cloud_sql_pg/plugin.py

Implements the resolve() method to create and return the guardrails evaluator action, matching the JS implementation in evaluation.ts. Key changes: - Add _create_evaluator_action() that creates the evaluator Action - Add _evaluate_datapoint() to call Checks API for content classification - Update list_actions() to return single checks/guardrails evaluator - Update tests to verify new behavior The evaluator evaluates content against all configured safety policies and returns scores with pass/fail status based on violation results.

This change standardizes all Pydantic models that need camelCase JSON serialization to use `alias_generator=to_camel` instead of manual `Field(alias='...')` patterns. This approach: - Ensures type checkers understand models correctly - Allows Pythonic constructor calls with snake_case field names - Automatically handles camelCase serialization/deserialization Changes: - Add guideline to GEMINI.md documenting the preferred pattern - Update google-genai plugin models (gemini.py, veo.py, lyria.py) - Update checks plugin guardrails models - Update all sample files with RpgCharacter and menu schemas - Add comprehensive tests for Checks, Chroma, and Pinecone plugins - Fix evaluator resolution in Checks plugin to return list of scores Test coverage added: - Checks plugin: evaluator, guardrails, metrics, middleware tests - Chroma plugin: retriever, indexer, helpers tests - Pinecone plugin: retriever, indexer, helpers tests

- Fix TODO format in generate_schema_typing to use proper issue link format - Replace type: ignore comments in Chroma plugin with proper casts - Use cast(Embeddings, ...) for embedding vectors - Use cast(Include, ...) for include fields - Use cast(Metadatas, ...) for metadata lists - Import chromadb types (Embedding, Embeddings, Include, Metadata, Metadatas) This follows GEMINI.md guidelines to fix type errors rather than suppress them.

- Change default k from 10 to 4 to match JS implementation - Add ignore_metadata_columns auto-population logic in retriever - Strip .gserviceaccount.com from IAM principal emails (matches JS) - Add close_connection() alias for close() (JS API parity) - Add from_engine_args() alias for from_connection_string() (JS API parity) - Add metadata_json_column existence check in _check_columns() - Update tests to reflect new default k=4

Renamed test files to have unique prefixes matching other plugins: - engine_test.py -> cloud_sql_pg_engine_test.py - indexes_test.py -> cloud_sql_pg_indexes_test.py - plugin_test.py -> cloud_sql_pg_plugin_test.py This fixes pytest collection errors where test modules with the same name across different plugins would conflict with each other.

The __init__.py file was causing the tests directory to be treated as a Python package, leading to module name conflicts during pytest collection in CI. Other plugins (checks, chroma) don't have __init__.py in their tests directories.

- Rename package to genkit-plugin-cloud-sql-pg to match GEMINI.md guidelines - Add genkit-plugin-cloud-sql-pg to workspace dependencies in pyproject.toml - This ensures the plugin is installed in CI environments during test collection - Update README and sample pyproject.toml with new package name - Regenerate uv.lock

huangjeff5 · 2026-02-02T16:49:41Z

py/plugins/xai/src/genkit/plugins/xai/plugin.py

        self._xai_client = XAIClient(api_key=api_key, **cast(dict[str, Any], xai_params))

-    async def init(self) -> list:
+    async def init(self, registry: 'Registry | None' = None) -> list:


No string types

huangjeff5 · 2026-02-02T16:49:49Z

py/plugins/vertex-ai/src/genkit/plugins/vertex_ai/model_garden/modelgarden_plugin.py

        self.model_locations = model_locations or {}

-    async def init(self) -> list[Action]:
+    async def init(self, registry: 'Registry | None' = None) -> list[Action]:


No string types

huangjeff5 · 2026-02-02T16:50:20Z

py/plugins/pinecone/LICENSE

@@ -0,0 +1,176 @@
+                              Apache License


Hm, is this the right license?

huangjeff5 · 2026-02-02T16:53:46Z

py/plugins/ollama/src/genkit/plugins/ollama/plugin_api.py

        self.client = partial(ollama_api.AsyncClient, host=self.server_address)

-    async def init(self) -> list:
+    async def init(self, registry: 'Registry | None' = None) -> list:


No string types

huangjeff5 · 2026-02-02T16:53:53Z

py/plugins/mcp/src/genkit/plugins/mcp/client/client.py

        return self.name

-    async def init(self) -> list[Action]:
+    async def init(self, registry: 'Registry | None' = None) -> list[Action]:


No string types

huangjeff5 · 2026-02-02T16:54:08Z

py/plugins/google-genai/src/genkit/plugins/google_genai/google.py

        )

-    async def init(self) -> list[Action]:
+    async def init(self, registry: 'Registry | None' = None) -> list[Action]:


No string types

huangjeff5 · 2026-02-02T16:54:11Z

py/plugins/google-genai/src/genkit/plugins/google_genai/google.py

        )

-    async def init(self) -> list[Action]:
+    async def init(self, registry: 'Registry | None' = None) -> list[Action]:


No string types

huangjeff5 · 2026-02-02T17:14:29Z

.github/workflows/python.yml

+          # Run nox with Python 3.12 to avoid workspace compatibility issues
+          # with chromadb (which doesn't support Python 3.14 yet).
+          # Nox will still create the test environment with the target Python version.
+          uv run --python 3.12 --directory py nox -s "tests-${{ matrix.python-version }}"


So we're actually checking python 3.12 for all tests?

yesudeep requested a review from MengqinShen January 30, 2026 09:44

github-project-automation bot added this to Genkit Backlog Jan 30, 2026

github-actions bot added docs Improvements or additions to documentation feature New feature or request python Python config labels Jan 30, 2026

yesudeep requested a review from zarinn3pal January 30, 2026 09:45

gemini-code-assist bot reviewed Jan 30, 2026

View reviewed changes

py/plugins/checks/src/genkit/plugins/checks/guardrails.py Outdated Show resolved Hide resolved

py/packages/genkit/src/genkit/ai/_aio.py Outdated Show resolved Hide resolved

py/plugins/checks/src/genkit/plugins/checks/guardrails.py Outdated Show resolved Hide resolved

MengqinShen approved these changes Jan 30, 2026

View reviewed changes

yesudeep changed the title ~~yesudeep/feat/quirky instrument~~ feat(py): add Chroma, Pinecone, and Checks plugins with DAP support Jan 30, 2026

yesudeep changed the title ~~feat(py): add Chroma, Pinecone, and Checks plugins with DAP support~~ feat(py): add Chroma, Pinecone, Checks plugins with DAP and testing support Jan 30, 2026

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch 7 times, most recently from 1176500 to e47ca92 Compare January 30, 2026 11:42

pavelgj requested a review from huangjeff5 January 30, 2026 11:45

pavelgj reviewed Jan 30, 2026

View reviewed changes

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch 4 times, most recently from d596cdf to 1a01b91 Compare January 30, 2026 12:16

yesudeep changed the title ~~feat(py): add Chroma, Pinecone, Checks plugins with DAP and testing support~~ feat(py): add Chroma, Pinecone, RAG samples, Python 3.14 compatibility, and bug fixes Jan 30, 2026

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch from 1a01b91 to 1b409e0 Compare January 30, 2026 12:20

gemini-code-assist bot reviewed Jan 30, 2026

View reviewed changes

py/pyproject.toml Show resolved Hide resolved

py/plugins/checks/src/genkit/plugins/checks/plugin.py Outdated Show resolved Hide resolved

py/samples/rag-chroma/src/main.py Outdated Show resolved Hide resolved

py/samples/rag-pinecone/src/main.py Outdated Show resolved Hide resolved

gemini-code-assist bot reviewed Jan 30, 2026

View reviewed changes

py/bin/generate_schema_typing Outdated Show resolved Hide resolved

py/plugins/chroma/src/genkit/plugins/chroma/plugin.py Outdated Show resolved Hide resolved

py/plugins/chroma/src/genkit/plugins/chroma/plugin.py Outdated Show resolved Hide resolved

yesudeep changed the title ~~feat(py): add Checks, Chroma, Pinecone plugins with tests and Pydantic standardization~~ feat(py): add Checks, Chroma, Pinecone, Cloud SQL PostgreSQL plugins with tests and Pydantic standardization Jan 30, 2026

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch from eb96dda to eed6433 Compare January 30, 2026 18:06

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch 2 times, most recently from f89b80e to f1792fc Compare January 30, 2026 21:27

gemini-code-assist bot reviewed Jan 30, 2026

View reviewed changes

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch 3 times, most recently from a25f306 to 1df7af8 Compare January 30, 2026 21:51

yesudeep added 9 commits January 30, 2026 17:38

feat(py): checks

5d2ecd8

fix(py/cloud-sql-pg): fix pyrefly bad-specialization error in engine.py

740c494

yesudeep force-pushed the yesudeep/feat/quirky-instrument branch from 1df7af8 to 740c494 Compare January 31, 2026 01:41

huangjeff5 reviewed Feb 2, 2026

View reviewed changes

py/plugins/pinecone/LICENSE

@@ -0,0 +1,176 @@

Apache License

Copy link

Contributor

huangjeff5 Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, is this the right license?

huangjeff5 reviewed Feb 2, 2026

View reviewed changes

feat(py): add Checks, Chroma, Pinecone, Cloud SQL PostgreSQL plugins with tests and Pydantic standardization #4328

Are you sure you want to change the base?

feat(py): add Checks, Chroma, Pinecone, Cloud SQL PostgreSQL plugins with tests and Pydantic standardization #4328

Uh oh!

Conversation

yesudeep commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

New Plugins (with JS Parity)

Cloud SQL PostgreSQL Plugin (NEW)

Checks Plugin

Chroma Plugin

Pinecone Plugin

Core SDK Improvements

Pydantic Model Standardization

Test Naming Convention & CI Stability

New Samples

Testing & Quality

Uh oh!

gemini-code-assist bot commented Jan 30, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yesudeep commented Jan 30, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yesudeep commented Jan 30, 2026

Uh oh!

yesudeep commented Jan 30, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yesudeep commented Jan 30, 2026

Uh oh!

yesudeep commented Jan 30, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yesudeep commented Jan 30, 2026 •

edited

Loading