Skip to content

fix: escape special characters in SchemaTransformProcessor JSON templates#250

Merged
andreatgretel merged 2 commits intomainfrom
fix/schema-transform-json-escaping
Jan 28, 2026
Merged

fix: escape special characters in SchemaTransformProcessor JSON templates#250
andreatgretel merged 2 commits intomainfrom
fix/schema-transform-json-escaping

Conversation

@andreatgretel
Copy link
Contributor

📋 Summary

Fixes GitHub issue #227 where SchemaTransformProcessor fails intermittently with JSONDecodeError when LLM-generated content contains quotes, backslashes, newlines, or other special characters that break JSON parsing.

🔄 Changes

🐛 Fixed

  • Added _json_escape_record() function in schema_transform.py that properly escapes all string values before template rendering using json.dumps

🧪 Tests

  • Added test_process_with_special_characters_in_llm_output - tests quotes, newlines, backslashes, tabs
  • Added test_process_with_mixed_special_characters - tests complex mixed content
  • Updated test_process_with_json_serialized_values to expect proper JSON serialization

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to:


🤖 Generated with AI

@andreatgretel andreatgretel requested a review from a team January 27, 2026 12:37
@andreatgretel andreatgretel added the bug Something isn't working label Jan 27, 2026
@andreatgretel andreatgretel force-pushed the fix/schema-transform-json-escaping branch from fcb7d6b to 5ee0fcb Compare January 27, 2026 16:34
@andreatgretel andreatgretel changed the base branch from main to johnny/refactor/207-repackaging-with-subpackages January 27, 2026 16:36
@johnnygreco johnnygreco force-pushed the johnny/refactor/207-repackaging-with-subpackages branch from 824d8ed to 177d38d Compare January 27, 2026 18:32
@andreatgretel andreatgretel changed the base branch from johnny/refactor/207-repackaging-with-subpackages to main January 28, 2026 13:20
@andreatgretel andreatgretel force-pushed the fix/schema-transform-json-escaping branch from 5ee0fcb to 5f8a1bb Compare January 28, 2026 13:27
…ates

Fixes GitHub issue #227 where SchemaTransformProcessor fails with
JSONDecodeError when LLM-generated content contains quotes, backslashes,
newlines, or other special characters that break JSON parsing.

The fix properly escapes all string values before template rendering
using json.dumps to handle all JSON-special characters.
@andreatgretel andreatgretel force-pushed the fix/schema-transform-json-escaping branch from 5f8a1bb to 27571cc Compare January 28, 2026 13:59
@andreatgretel andreatgretel merged commit 406928d into main Jan 28, 2026
46 checks passed
@andreatgretel andreatgretel deleted the fix/schema-transform-json-escaping branch February 5, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants