Skip to content

Use the new compiler for building images#4407

Draft
doriable wants to merge 11 commits intomainfrom
port-new-compiler
Draft

Use the new compiler for building images#4407
doriable wants to merge 11 commits intomainfrom
port-new-compiler

Conversation

@doriable
Copy link
Copy Markdown
Member

@doriable doriable commented Mar 23, 2026

This ports our image building path to use the new compiler and handles all the necessary test changes, etc.

The new compiler brings a lot of improvements:

  • Richer diagnostics and error reporting
  • Support for Editions 2024 features
  • Incremental compilation (although this is mostly for the LSP, which is already using the new compiler)

This PR is in draft:

Fixes #4193

This ports our image building path to use the new compiler and handles
all the necessary test changes, etc.

The new compiler brings a lot of improvements:
- Richer diagnostics and error reporting
- Support for Editions 2024 features
- Incremental compilation (although this is mostly for the LSP, which is
  already using the new compiler)
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 23, 2026

The latest Buf updates on your PR. Results from workflow Buf CI / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMar 30, 2026, 10:14 PM

stefanvanburen added a commit that referenced this pull request Mar 23, 2026
This adds edition 2024 support to the LSP, notably in completion,
semantic tokens, and hover.

Intended to be a fast-follow to #4407.
stefanvanburen added a commit that referenced this pull request Mar 23, 2026
This adds edition 2024 support to the LSP, notably in completion,
semantic tokens, and hover.

Intended to be a fast-follow to #4407.
Comment on lines +271 to 274
fmt.Sprintf(
`%s:5:1:detected cyclic import while importing "a/a.proto"`,
filepath.FromSlash("testdata/cyclicimport/b/b.proto"),
),
Copy link
Copy Markdown
Member Author

@doriable doriable Mar 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love this diagnostic, and the fact that the diagnostics in general are unable to provide the more robust details in a single FileAnnotation... e.g. this is the full report shows a much clearer cycle: https://github.com/bufbuild/protocompile/blob/main/experimental/ir/testdata/imports/cycle.proto.yaml.stderr.txt

That being said, if the user is printing the output to text, then they'll get the full information. This is only on the individual FileAnnotation (which would be used for things like in-line github annotations, the json output for analysis, etc.) -- and would provide a clear place where the import cycle was detected (although more details, in this case in particular would be nice). Will noodle this one a little.

stefanvanburen added a commit that referenced this pull request Mar 24, 2026
This adds edition 2024 support to the LSP, notably in completion,
semantic tokens, and hover.

Intended to be a fast-follow to #4407.
Copy link
Copy Markdown
Member

@stefanvanburen stefanvanburen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a couple minor things, this is looking good!

alreadySeen,
nonImportFilenames,
seen,
xslices.ToStructMap(paths),
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: call this once outside the loop?

Comment on lines +86 to +90
defer func() {
if retErr != nil {
retErr = errors.Join(retErr, moduleFile.Close())
}
}()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we want to close the moduleFile regardless of if we have an error, because readObjectCloserToSourceFile will copy over its contents but not close it.

Suggested change
defer func() {
if retErr != nil {
retErr = errors.Join(retErr, moduleFile.Close())
}
}()
defer moduleFile.Close()

}

if len(targetFileInfos) == 0 {
// If we had no no target files within the module after path filtering, this is an error.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// If we had no no target files within the module after path filtering, this is an error.
// If we had no target files within the module after path filtering, this is an error.

Just since we're nearby.

bufctl.ExitCodeFileAnnotation,
"", // no stdout
filepath.FromSlash(`testdata/protofileref/noworkspaceormodule/fail/import.proto:3:8:import "`)+`google/type/date.proto": file does not exist`,
fmt.Sprintf(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and all others) is likely a breaking change. I am not sure if this matters on stderr, I can't remember, but buf was designed to output errors in the standard file:line:column:message format, which is relied upon by many tools.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a sweep, and I think it's mostly ALE -> stdout. That being said, this is still a valid concern and @stefanvanburen had a good suggestion, which I am fully supportive of:

  • Add a new error-format=pretty, which would provide the new error report format
  • Detect if the output is TTY, which we already do for logging, and if it is, then we default to error-format=pretty, and other wise, keep the default as error-format=text. This would keep CI's consistent, ALE, etc.
  • This would apply to anything that produces compiler diagnostics and file annotations, so I can do a bit of work to structure this for lint, breaking

Let me know if this approach seems sound to you!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed this offline, putting some notes here:

  • No need for a new format name, instead we'll just default to this report format for TTY (and keep the old format for non-TTY)
  • No changes to lint, breaking's stdout output -- this will only be used for stderr compiler output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

When will edition 2024 features be supported?

3 participants