Fix inconsistent response format in RunnableRails (#313)#1391
Open
divya-garak wants to merge 2 commits intoNVIDIA-NeMo:developfrom
Open
Fix inconsistent response format in RunnableRails (#313)#1391divya-garak wants to merge 2 commits intoNVIDIA-NeMo:developfrom
divya-garak wants to merge 2 commits intoNVIDIA-NeMo:developfrom
Conversation
Fixes NVIDIA-NeMo#472: The langchain runnable was not called with the runnable config that defines callbacks, breaking Langfuse tracing integration. Changes: - Store RunnableConfig and kwargs from invoke() for use in passthrough function - Pass stored config to underlying runnable to preserve callbacks - Add test to verify callback passthrough functionality This ensures that callbacks (like Langfuse tracing) are properly propagated to the underlying LLM runnable wrapped in RunnableRails.
Fixes NVIDIA-NeMo#313: Adds consistent_output_format parameter to RunnableRails to allow users to enforce consistent response formats. Changes: - Add consistent_output_format parameter with options: "preserve", "always_dict", "always_string" - Add _format_output_consistently() method to handle format conversion - Update invoke() method to use consistent formatting - Add comprehensive tests for all formatting modes The inconsistency occurred when: - Normal passthrough returned original format (e.g., string) - Rails triggered returned dict format (e.g., {"output": "blocked"}) Now users can choose: - "preserve" (default): Maintains current behavior for backward compatibility - "always_dict": Always returns dict format for predictable structure - "always_string": Always returns string format when possible for simplicity This resolves the inconsistency while maintaining backward compatibility.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #313: This PR resolves the inconsistent response format issue in RunnableRails where response formats would change between string and dict depending on whether rails were triggered.
Problem
The inconsistency occurred in these scenarios:
{"output": "blocked message"}This made it difficult to predict and handle response formats consistently in applications.
Solution
Added a new
consistent_output_formatparameter toRunnableRailsconstructor with three options:"preserve"(default): Maintains current behavior for backward compatibility"always_dict": Always returns dict format for predictable JSON-like structure"always_string": Always returns string format when possible for simplicityChanges Made
consistent_output_formatparameter toRunnableRails.__init__()_format_output_consistently()method to handle format conversioninvoke()method to apply consistent formatting to all return pathsUsage Examples
Testing
This enhancement provides users with control over response formats while maintaining full backward compatibility.