chore(py): Simplify Dev UI integration and remove unused abstractions

[huangjeff5]

Problem

Previously, run_main() was running reflection server and running your code, both as sync tasks on the same event loop. Web frameworks typically own the event loop and provide their own hook. This means we (as genkit team) have to provide a custom implementation of that hook to support the framework. With FastAPI, we needed to provide a genkit_lifespan() hook. Genkit users also have to add that hook boilerplate into their code.

The reflection server now starts automatically in a background daemon thread when GENKIT_ENV=dev is set, right at Genkit() construction. The thread owns its own asyncio event loop so it never conflicts with FastAPI's, Flask's, or any other framework's loop. The registry is already protected by a threading.RLock so concurrent access from the background thread is safe.

In the case of a web app like FastAPI - Since both the Reflection Server and FastAPI have the ability to call a plugin from their own eventloop, there are model plugin bugs because the underlying model client errors out when executing from a different eventloop than the one that it was created in. Because of that, plugin authors need to create and cache a client on every running eventloop that executes the model client code.

We provide a convenience method to make that easier. Here's how Plugin code can be executed on either loop. Plugin authors need to write something like this now:

`
from genkit.core._loop_local import _loop_local_client

class MyPlugin(Plugin):
def init(self, **params):
self._client_params = params
self._runtime_client = _loop_local_client(
lambda: AsyncClient(**self._client_params)
)
... (reuse self._runtime_client() later to access the SDK client
`

Before:

Scripts needed run_main():

ai = Genkit(plugins=[GoogleAI()], model='googleai/gemini-2.0-flash')

@ai.flow()
async def my_flow(input: str) -> str:
    return (await ai.generate(prompt=input)).text

async def main():
    print(await my_flow('Hello'))

ai.run_main(main())  # starts reflection server + blocks for Dev UI

FastAPI apps needed an additional lifespan plugin — even for local development only:

from genkit.plugins.fastapi import genkit_lifespan

ai = Genkit(...)
app = FastAPI(lifespan=genkit_lifespan(ai))  # only needed for Dev UI

After

FastAPI apps — no lifespan needed:

ai = Genkit(...)  # Dev UI starts here automatically
app = FastAPI()

Future (Plugin Authors)

Re-export from loop-local client to genkit.plugin package (containing set of utilities and types commonly used by plugin authors)

from genkit.plugin import loop_local_client

Scripts — unchanged:

ai = Genkit(...)

@ai.flow()
async def my_flow(input: str) -> str:
    return (await ai.generate(prompt=input)).text

async def main():
    print(await my_flow('Hello'))

ai.run_main(main())  # still needed to keep the process alive for Dev UI

Changes

• Genkit.__init__(): starts the reflection server in a background daemon thread when GENKIT_ENV=dev
• run_main(): simplified — no longer starts the server itself, just runs the user coroutine then blocks on Ctrl+C to keep the daemon thread alive
• genkit_lifespan(): deprecated to a no-op with DeprecationWarning — existing code keeps working
• web/manager/ directory removed — the internal abstraction that existed to run multiple ASGI servers on the same event loop, which was the previous approach to this problem
• web-multi-server, web-endpoints-hello, web-short-n-long samples removed
• Dead types removed: ToolRequestLike, MatchableAction, ResourceFn Protocol, ASGIApp Protocol
• Error wire format names simplified: GenkitReflectionApiErrorWireFormat → ReflectionError, GenkitReflectionApiErrorWireFormat => RefectionErrorDetails

[gemini-code-assist]

Summary of Changes

Hello @huangjeff5, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly simplifies the integration of the Genkit Dev UI, particularly for web applications, by automating the reflection server's startup process. It removes previous complexities and boilerplate code, making the developer experience smoother and the internal codebase more maintainable. The changes streamline how Genkit instances manage their lifecycle and interact with different application types, focusing on a more intuitive 'set it and forget it' approach for development environments.

Highlights

• Simplified Dev UI Integration: The Genkit Dev UI reflection server now starts automatically in a background daemon thread when GENKIT_ENV=dev is set, directly upon Genkit() instance creation. This eliminates the need for explicit setup in web frameworks like FastAPI.
• Reduced Boilerplate for Web Applications: The genkit_lifespan() function for FastAPI is now deprecated and acts as a no-op, removing the requirement for users to add boilerplate code to their FastAPI applications solely for local Dev UI integration.
• Streamlined run_main() Functionality: The run_main() method has been simplified; it no longer manages the reflection server directly. Instead, it focuses on running the user's coroutine and then blocking to keep the background reflection server thread alive during development.
• Removed Internal Server Management Abstractions: The entire web/manager/ directory, which contained abstractions for running multiple ASGI servers on the same event loop, has been removed, simplifying the internal architecture.
• Cleanup of Unused Types and Samples: Several unused type protocols (ToolRequestLike, MatchableAction, ResourceFn Protocol, ASGIApp Protocol) have been removed, along with associated web-multi-server and web-short-n-long samples, reducing code complexity and maintenance overhead.
• Standardized Error Wire Format: Error wire format names have been simplified from GenkitReflectionApiErrorWireFormat to ErrorResponse and GenkitReflectionApiDetailsWireFormat to ErrorDetails, improving clarity and consistency.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• py/packages/genkit/src/genkit/ai/_base_async.py
  • Added imports for asyncio, contextlib, socket, and threading.
  • Implemented _is_port_available and _find_free_port for dynamic port discovery.
  • Modified GenkitBase.__init__ to automatically start the Dev UI reflection server in a background daemon thread when GENKIT_ENV=dev.
  • Simplified run_main to execute the user's coroutine and then block, relying on the background thread for Dev UI server management.
• py/packages/genkit/src/genkit/blocks/resource.py
  • Removed ResourceFn and MatchableAction Protocols.
  • Updated type hints and removed cast operations related to the removed Protocols.
• py/packages/genkit/src/genkit/blocks/tools.py
  • Removed ToolRequestLike Protocol.
  • Updated type hint for tool_response to directly use ToolRequestPart.
• py/packages/genkit/src/genkit/core/error.py
  • Renamed GenkitReflectionApiDetailsWireFormat to ErrorDetails.
  • Renamed GenkitReflectionApiErrorWireFormat to ErrorResponse.
  • Updated references to the new error wire format names throughout the file.
• py/packages/genkit/src/genkit/core/reflection.py
  • Added imports for os and signal.
  • Modified handle_terminate to send a SIGTERM to the current process, replacing the previous terminate_all_servers call.
• py/packages/genkit/src/genkit/web/manager/init.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/_adapters.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/_base_server.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/_info.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/_manager.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/_ports.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/_server.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/manager/signals.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/packages/genkit/src/genkit/web/typing.py
  • Removed Protocol and runtime_checkable imports.
  • Removed the ASGIApp Protocol definition.
• py/packages/genkit/tests/genkit/core/error_test.py
  • Updated imports and assertions to reflect the renaming of error wire format classes.
• py/packages/genkit/tests/genkit/web/manager/signals_test.py
  • Removed the entire file, as the web/manager module and its abstractions are no longer used.
• py/plugins/fastapi/src/genkit/plugins/fastapi/lifespan.py
  • Deprecated genkit_lifespan to a no-op function.
  • Removed all logic related to starting a reflection server and RuntimeManager within the lifespan context.
• py/samples/web-fastapi-bugbot/src/main.py
  • Removed imports related to Lifespan and genkit_lifespan.
  • Updated FastAPI app initialization to remove the lifespan argument, reflecting the new automatic Dev UI startup.
• py/samples/web-multi-server/LICENSE
  • Removed the entire file, as the web-multi-server sample is no longer relevant.
• py/samples/web-multi-server/README.md
  • Removed the entire file, as the web-multi-server sample is no longer relevant.
• py/samples/web-multi-server/pyproject.toml
  • Removed the entire file, as the web-multi-server sample is no longer relevant.
• py/samples/web-multi-server/run.sh
  • Removed the entire file, as the web-multi-server sample is no longer relevant.
• py/samples/web-multi-server/src/main.py
  • Removed the entire file, as the web-multi-server sample is no longer relevant.
• py/samples/web-short-n-long/LICENSE
  • Removed the entire file, as the web-short-n-long sample is no longer relevant.
• py/samples/web-short-n-long/README.md
  • Removed the entire file, as the web-short-n-long sample is no longer relevant.
• py/samples/web-short-n-long/pyproject.toml
  • Removed the entire file, as the web-short-n-long sample is no longer relevant.
• py/samples/web-short-n-long/run.sh
  • Removed the entire file, as the web-short-n-long sample is no longer relevant.
• py/samples/web-short-n-long/src/main.py
  • Removed the entire file, as the web-short-n-long sample is no longer relevant.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request significantly simplifies the integration of the Genkit Dev UI by automating the startup of the reflection server in a background daemon thread when running in development mode (GENKIT_ENV=dev). This removes the need for framework-specific boilerplate like genkit_lifespan for FastAPI. The cleanup of unused internal abstractions (web/manager) and dead types improves maintainability. My feedback focuses on improving the efficiency of the health check loop and ensuring consistent binding behavior in the port utility.

[huangjeff5]

/gemini review

[gemini-code-assist]

Code Review

This pull request is a great simplification of the Dev UI integration. Moving the reflection server to a background daemon thread with its own event loop is a clean solution that removes a lot of complexity and boilerplate for developers using frameworks like FastAPI. The removal of the web/manager abstraction and related code is a significant improvement for maintainability. The new tests for the background server behavior are thorough and cover the important aspects of the new design, including thread safety.

I have one major piece of feedback regarding a potential race condition in how free ports are selected for the reflection server. Other than that, the changes look solid.

[huangjeff5]

/gemini review

[gemini-code-assist]

Code Review

This pull request significantly simplifies the Dev UI integration by refactoring how the reflection server is started. The new approach of using a background daemon thread with its own event loop is a great improvement, as it resolves event loop conflicts with web frameworks like FastAPI and simplifies the user code by removing the need for run_main or framework-specific lifespan hooks in many cases. The removal of the complex web/manager abstraction and the simplification of run_main make the core library cleaner and easier to understand. The introduction of the _loop_local_client helper is a clever solution to ensure that SDK clients are created and used on the correct event loop, and its application across the anthropic, google-genai, and compat-oai plugins is consistent and well-executed. The new tests for the background server, including concurrency tests, are thorough and provide good coverage for the new functionality.

One major aspect of this PR is the removal of a large number of plugins (checks, cloudflare-workers-ai, cohere, deepseek, evaluators, huggingface, mcp). This was not mentioned in the pull request description. While this is a significant change, I've focused my review on the modified code. For future large-scale changes like this, it would be beneficial to document them in the PR description to provide full context.

Overall, the code changes are of high quality and represent a substantial improvement to the framework's developer experience and internal architecture. I have not found any issues of medium or higher severity.