fix(python-driver): add null-guards in ANTLR parser and relax runtime version pin#2372
fix(python-driver): add null-guards in ANTLR parser and relax runtime version pin#2372uesleilima wants to merge 3 commits intoapache:masterfrom
Conversation
… version pin Fix two related issues in the Python driver's ANTLR4 parsing pipeline: 1. Add null-guards in ResultVisitor methods (visitAgValue, visitFloatLiteral, visitPair, visitObj, handleAnnotatedValue) to prevent AttributeError crashes when the ANTLR4 parse tree contains None child nodes. This occurs with vertices that have complex properties (large arrays, special characters, deeply nested structures). (apache#2367) 2. Relax antlr4-python3-runtime version constraint from ==4.11.1 to >=4.11.1,<5.0 in both pyproject.toml and requirements.txt. The 4.11.1 pin is incompatible with Python >= 3.13. The ANTLR ATN serialized format is unchanged between 4.11 and 4.13, so the generated lexer/parser files are compatible. Validated with antlr4-python3-runtime==4.13.2 on Python 3.11-3.14. (apache#2368) Also replaces shadowing of builtin 'dict' in handleAnnotatedValue with 'd', and uses .get() for safer key access on parsed vertex/edge dicts. Closes apache#2367 Closes apache#2368
Verify that malformed and truncated agtype strings raise AGTypeError (or recover gracefully) rather than crashing with AttributeError. This tests the null-guards added to the ANTLR parser visitor. Made-with: Cursor
tcolo
left a comment
There was a problem hiding this comment.
@uesleilima thanks alot of the patch - can you please address the issues in the review from claude code?
## Code Review: `fix(python-driver)`: add null-guards in ANTLR parser and relax runtime version pin
### Summary
This PR fixes two real bugs: a crash on `AttributeError` when deserializing complex vertex properties (#2367), and a version pin that blocks Python 3.13+ (#2368). The approach is sound and the test coverage is meaningful. A few things worth discussing before merging.
---
### 🔴 Issues to Address
**1. `visitFloatLiteral` — silent fallback may mask malformed input**
When `ctx.getChild(0)` returns `None`, the node is genuinely malformed. Silently returning a default value hides the problem — a test confirms no `AttributeError`, but an incorrect result is arguably worse. Consider raising `AGTypeError` consistently with how `visitPair` handles missing nodes:
```python
# Suggested
child = ctx.getChild(0)
if child is None:
raise AGTypeError("Malformed float literal: missing child node")2. handleAnnotatedValue — type validation failure mid-path construction
When a type check fails mid-path (e.g., vertex dict check passes, edge dict check raises AGTypeError), the partially constructed Path object may already have been mutated. It's discarded on exception so it's minor, but a comment explaining the intended behavior would help.
🟡 Suggestions
1. Add a comment explaining the version constraint
The change from ==4.11.1 to >=4.11.1,<5.0 is the right call — ANTLR4 maintains format compatibility within a major version. A brief inline comment in pyproject.toml will help the next person who wonders why it's not pinned more tightly:
# ANTLR4 runtime is format-compatible within major versions; tested on 4.13.2 with Python 3.11–3.14
"antlr4-python3-runtime>=4.11.1,<5.0"2. Malformed input tests should assert the expected outcome, not just the absence of AttributeError
# Current pattern proves the old crash is gone, but doesn't pin the new contract
try:
result = handler.parse(malformed_input)
except AttributeError:
pytest.fail("Should not raise AttributeError")
# Better: assert what *should* happen
with pytest.raises(AGTypeError):
handler.parse(malformed_input)
# OR if graceful None is expected:
result = handler.parse(malformed_input)
assert result is None3. visitObj silently skips None values — visitPair hard-fails on them
This inconsistency could cause silent data loss (a property disappearing) instead of a parse error. Either align the behavior or add a comment explaining the intentional difference.
✅ What Looks Good
- The
visitAgValuenull-checks are clean and correctly handle the annotated-type path. - Replacing
dict["key"]with.get("key")inhandleAnnotatedValueis the right call for externally-provided data. - Fixing the shadowed
dictbuiltin is a good catch — subtle but real. - The nine new test cases cover solid breadth of the real-world inputs that triggered the original bug.
- Both
pyproject.tomlandrequirements.txtare kept in sync on the version change — these often drift.
Verdict: Request Changes
The core fix is correct and needed. Two points before merging:
- Clarify/fix
visitFloatLiteral— silent0.0default vs. raisingAGTypeError. - Tighten the malformed-input tests to assert the expected outcome, not just the absence of
AttributeError.
The version pin relaxation is ready to merge as-is.
Paste this into **Files changed → Review changes → Leave a comment** on the PR and select **Request changes**.
- visitFloatLiteral: raise AGTypeError on malformed child node instead of silently returning a fallback value - visitObj: add comment documenting that visitPair's validation makes the None-guard defensive-only - handleAnnotatedValue: add comment explaining partial-construction behavior on type-check failure - pyproject.toml: add comment explaining ANTLR4 version range rationale - Tests: assert AGTypeError (or graceful recovery) for malformed and truncated inputs, not just absence of AttributeError Made-with: Cursor
Summary
Fix two related issues in the Python driver's ANTLR4 parsing pipeline:
1. ANTLR4 agtype parser crashes on vertices with complex properties (#2367)
The
ResultVisitorinbuilder.pycrashes withAttributeError: 'NoneType' object has no attribute 'stop'when deserializing certain vertex records whose properties contain large arrays, long text fields with special characters, or deeply nested structures.Root cause: Several visitor methods access child nodes (
.symbol,.getText(),.accept()) without null-checking first. When the ANTLR4 parser tree containsNonenodes, these calls crash.Fix: Add null-guards in
visitAgValue,visitFloatLiteral,visitPair,visitObj, andhandleAnnotatedValue. Also replaces shadowing of builtindictwithdinhandleAnnotatedValue, and uses.get()for safer key access on parsed vertex/edge dicts.2.
antlr4-python3-runtime==4.11.1is incompatible with Python >= 3.13 (#2368)The exact version pin
==4.11.1causes runtime errors on Python 3.13+ due to deprecated/removed CPython internals that ANTLR 4.11.x relies on.Fix: Relax the constraint from
==4.11.1to>=4.11.1,<5.0in bothpyproject.tomlandrequirements.txt. The ANTLR ATN serialized format is unchanged between 4.11 and 4.13, so the generated lexer/parser files are compatible. Validated withantlr4-python3-runtime==4.13.2on Python 3.11-3.14.Tests
Added 9 new test cases in
test_agtypes.py:All 12 tests pass with
antlr4-python3-runtime==4.13.2.Closes