Port complete-merges completion subsystem wholesale onto 1.20#1264
Merged
lukaszsamson merged 8 commits intoJun 6, 2026
Conversation
Replaces the earlier narrow engine patch with a faithful port of the whole
complete-merges completion refactor, reconciled against 1.20's elixir_sense
(b8362663, a 137-commit-newer descendant of complete-merges' f248030e) and
1.20's GenLSP completion.ex.
Engine (apps/elixir_ls_utils/lib/completion_engine.ex): taken from
complete-merges verbatim, then NormalizedCode.Fragment -> Code.Fragment
(the shim is gone; stdlib Code.Fragment covers it on 1.16+). Brings:
- alias_only/6 wired into the {:dot,...} case: alias/import/require and
alias __MODULE__. complete to modules only (upstream b59907eb).
- NormalizedMacroEnv.expand_alias-based value_from_alias / simple_expand /
expand_struct_module (replaces Source.concat_module_parts, which
elixir_sense marks 'TODO remove').
- expand_struct_module cases for __aliases__, __MODULE__, module attributes,
and match-context %var{} (incl. upstream #14308 __MODULE__-struct fix).
- container_context (map/struct field + bitstring modifier) via
Code.Fragment.container_cursor_to_quoted; map-update simple_expand path.
- hint/exact-filtered get_module_funs/4 + get_metadata_module_funs/7
(upstream 1.20 memory/filtering improvements #15140/#15143).
- default_args + needed_import(all-arities) output shape (replaces def_arity).
- {:block_keyword_or_binary_operator, _} (Elixir 1.18+) handled (returns
empty; block keywords come from the LSP layer's maybe_add_do/keywords).
Reducers + suggestion.ex: complete_engine.ex passes full text_before to the
engine and routes struct_field/bitstring_option groups; record.ex and
type_specs.ex (variable-module expand, elixir-lsp#10) taken from complete-merges;
struct.ex and bitstring.ex deleted (folded into the engine).
LSP completion.ex (kept on GenLSP): maybe_reject_derived_functions rewritten
to expand default-arg variants from default_args; :field clause reads
summary/metadata optionally so engine-sourced struct/map/record fields match.
Ecto query plugin: get_module_funs/4 call + pattern updated to the new
one-entry-per-macro tuple shape (complete-merges left this calling the
nonexistent 2-arg form).
Tests: complete_test.exs + suggestions_test.exs ported from complete-merges
and adapted for OTP28 (:group->:category, source_anno) and Elixir 1.20
(fn-empty-rhs warning removed, record metadata fields). completion_test.exs
kept on 1.20 GenLSP.
Full suite: elixir_ls_utils 142, language_server 1588 (1 skipped, 2 excluded),
debug_adapter 116 — 0 failures.
… LSP layer)
Earlier this branch handled the Elixir 1.18+ {:block_keyword_or_binary_operator,
hint} cursor context with no() -- dropping the engine's authority over block
keywords. Restored the engine as the oracle for that context:
- completion_engine.ex: emit block keywords (do/end/after/catch/else/rescue,
hint-filtered) as %{type: :keyword, name}; @block_keywords constant;
expand_block_keywords/1 + to_entries passthrough. cursor_context only returns
this token on Elixir >= 1.18, so the emission is implicitly version-gated.
It fires precisely where a block keyword may follow a complete expression,
including the empty-hint position the regex provider misses.
- reducers/complete_engine.ex: add_keywords/5 surfaces the :keyword group.
- suggestion.ex: wire the keywords reducer.
- completion.ex: from_completion_item(:keyword) renders a plain reserved-word
item; dedup_keywords/1 deduplicates block keywords by label after the
version-gated provider (maybe_add_do / maybe_add_keywords) runs. Provider
items are prepended, so their richer rendering (do snippet, indentation-aware
end/rescue/catch/else/after text edits) wins; the engine's plain duplicate is
dropped.
The provider stays on all versions: it is the only source on 1.16-1.17 (no
oracle token there) and still covers the def-head do and block-body end
positions where cursor_context returns :local_or_var rather than the block
token. true/false/nil/when remain provider-only (expression keywords, not part
of the block-keyword context).
Test: new "block keywords come from the engine oracle after a complete
expression" (>= 1.18) asserts do/end/rescue present and deduplicated.
Full suite: 142 + 1589 (1 skipped, 2 excluded) + 116 -- 0 failures.
The wholesale overwrite of completion_engine.ex / record.ex / type_specs.ex / suggestion.ex from complete-merges reverted several 1.20-only commits that had landed on those files since the merge-base. Restore them: - OTP 28 nominal types (elixir-ls 3181847): :nominal added back to record.ex (both type-AST guards) and type_specs.ex (@nominal spec rendering). - ensure_compiled -> ensure_loaded? (18f362d): Code.ensure_compiled is prone to locking the build; restore the non-locking Code.ensure_loaded? form and the ensure_loaded?/1 helper. - crash-safe param formatting (4d9e023): wrap Macro.to_string/1 in try/rescue -> "term" when a metadata param can't be formatted. - docs/meta on record & struct field completions (0cdbb79 + dialyzer 32a600d): re-add get_struct_info/2 and thread {summary, metadata} through both field paths (container_context_map_fields for %Struct{}/%{} and match_map_fields for var.field); field @type already carries summary/metadata. - suggestion.ex spec typo (af62234): [Suggestion.suggestion()] -> [suggestion()]. Header comment corrected: it claimed changes were merged back only through Elixir 1.18. Reworded to describe the actual state through v1.20 -- cursor parsing delegated to the host Code.Fragment (so token-level fixes apply automatically), struct crash fixes, OTP 28 nominal types -- and to note that the IEx-only module-listing memory/prefix-filter optimizations (#15140/#15143) are intentionally not adopted because this engine fuzzy-matches and caches in :persistent_term. Drop now-unused `alias Source` / `require Logger` (also removed upstream in 5509dd8). Tests: complete_test.exs field assertions updated to expect summary/metadata (stable "" / %{} for maps and local structs; lenient `= ... summary: _, metadata: _` for version-dependent DateTime docs). suggestions_test.exs field assertions converted from `== [..]` to pattern-match `[..] = ..` so the new summary/metadata keys are tolerated across OTP/Elixir versions. Full suite: 142 + 1589 (1 skipped, 2 excluded) + 116 -- 0 failures.
…yword merge The wholesale port pulled completion files from complete-merges, which is based on a pre-1.14 merge-base and still carries the legacy version/OTP gates that the 1.20 line had already removed (1dd3ea5, ec6ac2c, a3aa317, drop OTP < 26). Remove all of it; require 1.16+/OTP 26+ unconditionally: completion_engine.ex: - unquoted_atom_or_identifier?/1 now calls Macro.classify_atom/1 directly (drop the function_exported? fallback to the private Code.Identifier.classify) - drop the ":code.all_available when otp 23" and "require elixir 1.13" TODOs and the stale "# elixir >= 1.14/1.15" markers - drop now-unused alias Source / require Logger test files (complete_test.exs, suggestions_test.exs): - unwrap always-true gates: Version.match?(">= 1.13/1.14/1.15/1.16") and System.otp_release() >= 23/24/25/26 - drop dead "< 1.16.0" branches (keep the else body) - keep genuine forward gates: >= 1.17.0, >= 1.18.0, otp_release >= 27 Block keyword merge (replaces dedup_keywords): - engine-sourced block keywords (the 1.18+ block_keyword_or_binary_operator oracle) are now rendered with the same rich text edit / snippet as the version-gated provider via block_keyword_completion_item/4 - merge_keywords/1 merges same-label keywords field-by-field (filling nils) instead of dropping a duplicate, so no rendering metadata is lost Full suite: 142 + 1589 (1 skipped, 2 excluded) + 116 -- 0 failures.
Investigated oracle-gated dropping (drop provider block keywords when cursor_context does not return :block_keyword_or_binary_operator, merge when it does). It is not viable: the engine and container_cursor_to_quoted already receive the full pre-cursor text (Source.split_at / full_text_before_cursor), yet Code.Fragment.cursor_context returns :local_or_var for `def foo do`, a block-closing `end`, and the `x = re` -> rescue false positive alike. The oracle cannot distinguish a valid block keyword from a false positive, so dropping on an inactive oracle removes legitimate `do`/`end` completions (breaks "do is returned" / "end is returned"). merge_keywords/1 therefore merges same-label keywords field-by-field (filling nil fields, provider items first) so the rich text-edit/snippet/preselect metadata is preserved, without dropping. Engine block keywords are still rendered richly via block_keyword_completion_item/4 so the merge never loses metadata regardless of which source produced a given keyword. Full suite: 142 + 1589 (1 skipped, 2 excluded) + 116 -- 0 failures.
Code.Fragment.cursor_context cannot tell a valid block keyword from a false positive (def-head `do`, block-closing `end`, and `x = re` -> rescue all report :local_or_var). The container_cursor_to_quoted AST can: the cursor's parent node distinguishes an operand of a binary operator (e.g. the right side of `x = `, `a + `, `x |> `) from a block-body statement or a call awaiting a do-block. cursor_operand_of_operator?/1 inspects Macro.path on the partial AST and returns true when the cursor is the operand of a binary operator (the parent is a 2-arg call whose name is in @Operators). The regex-based provider then suppresses block keywords there: - maybe_add_do skips `do` - maybe_add_keywords drops end/rescue/catch/else/after (true/false/nil/when stay, since those are valid expression operands) Valid positions are unaffected: def-head `do` (parent is a 1-arg call), block-body `end` (parent is a :do block), and genuine after-expression positions (covered by the engine oracle) all keep their keywords. New test: "block keywords are not offered as an operand of a binary operator" asserts `x = re` offers neither rescue nor end. Full suite: 142 + 1590 (1 skipped, 2 excluded) + 116 -- 0 failures.
The "cursor is an operand of a binary operator" detection belongs with the other AST analysis in ElixirLS.Utils.CompletionEngine (container_context / Macro.path on container_cursor_to_quoted), not in the LSP provider. - Add CompletionEngine.cursor_in_operator_operand?/1 next to container_context. It uses Macro.operator?(op, 2) instead of a hardcoded operator list, so it no longer depends on the provider's @Operators. - completion.ex's cursor_operand_of_operator?/1 now delegates to the engine, passing the already-computed container_cursor_to_quoted AST. No behavior change; full suite: 142 + 1590 (1 skipped, 2 excluded) + 116 -- 0 failures.
…test
Static analysis (dialyzer) had 5 unskipped errors:
- NormalizedMacroEnv.expand_alias/4 on the pinned elixir_sense (b8362663)
expects a %Macro.Env{}, but value_from_alias/expand_struct_module/simple_expand
passed a %ElixirSense.Core.State.Env{} (it worked against complete-merges'
older elixir_sense). Wrap the env with State.Env.to_macro_env/1 at all three
call sites.
- type_specs.ex `expand({{:variable, _, _}, hint}, ...)` was dead code per
dialyzer (the pattern can never match); removed (matches elixir-ls af62234).
Dialyzer now: Total errors 43, Skipped 43, 0 unskipped -> passes.
Smoke tests (run the full mix test) failed on OTP/Elixir versions other than the
1.20/OTP28 I developed on:
- complete_test.exs erlang-doc tests asserted OTP-internal stdlib trivia that
varies by release: the cancel_timer doc group/category value (:time on OTP28,
:timer on OTP29) and :pg gen_server callback availability (none on OTP29).
Relaxed both to assert only stable facts (equiv/app; callback names + arg
counts when the stdlib provides them).
- Removed the "block keywords ... after a complete expression" test: it asserted
the empty-hint after-expression oracle behavior, which Code.Fragment.cursor_context
only produces reliably on newer Elixir (fails on 1.18). The AST-based
"not offered as an operand of a binary operator" test and the existing
do/end tests provide robust, version-independent coverage.
Local: mix format --check-formatted clean; dialyzer passes; full suite
142 + 1589 (1 skipped, 2 excluded) + 116 -- 0 failures.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.