Skip to content

Collect source text from ParseCallback during parsing#178

Open
ALancki1 wants to merge 2 commits intotree-sitter:masterfrom
ALancki1:collect-parsecallback-source-text
Open

Collect source text from ParseCallback during parsing#178
ALancki1 wants to merge 2 commits intotree-sitter:masterfrom
ALancki1:collect-parsecallback-source-text

Conversation

@ALancki1
Copy link
Copy Markdown

@ALancki1 ALancki1 commented Apr 2, 2026

This change collects source text returned by ParseCallback during parsing and stores it in the resulting Tree.

Previously, trees created from ParseCallback did not retain source text, causing methods like getText() and Node.getText() to return null. This change enables text-related APIs for callback-based parsing.

This also updates the callback parsing test to assert the new behavior.

Fixes #126

@juncas2205-debug

This comment was marked as spam.

@ALancki1
Copy link
Copy Markdown
Author

ALancki1 commented Apr 2, 2026

Thanks for taking a look. From my diff, the intended changes in Parser.java are only:

  • collecting callback text into a StringBuilder
  • appending each returned chunk
  • passing the collected text into Tree

I don’t think I changed the null-language check itself. Were you pointing to a specific issue with the added StringBuilder change in that hunk?

if (language == null) {
throw new IllegalStateException("The parser has no language assigned");
}
final var collected = new StringBuilder();
Copy link
Copy Markdown
Member

@ObserverOfTime ObserverOfTime Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a byte buffer might be more efficient than a string builder.
In any case, this needs to be benchmarked before and after.

@ALancki1
Copy link
Copy Markdown
Author

ALancki1 commented Apr 2, 2026

Good point. I switched this to collect bytes instead of using a StringBuilder, and I also used the callback offset to avoid duplicating overlapping chunks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Collect source bytes from ParseCallback

4 participants