feat(controller/node): resolve label peers to NLB addresses + controller-side node_id (#368)

[bdchatham]

Summary

First slice of the peer-discovery harmonization (#368). Moves node_id resolution for the LabelPeerSource flow from the sidecar's :26657/status query to the controller's existing per-peer sidecar gRPC. Status.ResolvedPeers shifts to fully-composed <node_id>@<host>:<port> strings.

Why

The sidecar's :26657/status lookup path is in-cluster-only — NLBs only forward 26656, so it can't reach a publishable peer over the public endpoint. After PR #365, a peer's Spec.ExternalAddress is the publishable vanity hostname; if the controller writes that into Status.ResolvedPeers and the sidecar tries :26657/status, the lookup fails permanently.

The controller already has in-cluster access to each peer's sidecar gRPC (port 7777) — the same path the genesis CollectAndSetPeers task uses to read node_id from node_key.json. Reusing that path for the Label flow keeps node_id resolution authoritative (it's from the keyfile, not from :26657) and decouples discovery from publishable-vs-private addressing.

Behavior

Peer's Spec.ExternalAddress	Resolved entry
set (<vanity>:26656)	<node_id>@<vanity>:26656
unset	<node_id>@<peer>-0.<peer>.<ns>.svc.cluster.local:26656

node_id is fetched via the existing SidecarClient.GetNodeID (in-cluster gRPC to port 7777). On per-peer failure the whole resolve errors and controller-runtime retries — same atomic semantic as the genesis CollectAndSetPeers task. Seid only reads persistent_peers at boot, so transient peer-sidecar failures during reconcile don't disturb running pods.

Files

• internal/controller/node/peers.go — resolveLabelPeers composes the new wire format; new peerAddress helper picks Spec.ExternalAddress or falls back to headless DNS.
• internal/planner/planner.go — Label sources map to PeerSourceStatic instead of PeerSourceDNSEndpoints. The sidecar's DNSEndpointsSource is untouched — still serves EC2Tags peers.
• internal/task/task.go — widens task.SidecarClient interface with GetNodeID(ctx) (string, error). The full *sidecar.SidecarClient already implements it; mocks gain one-line stubs.
• api/v1alpha1/seinode_types.go — ResolvedPeers godoc updated for the new composed wire format.

Wire-format change (one-way door)

Status.ResolvedPeers was previously a []string of bare hosts (<peer>-0.<peer>.<ns>.svc.cluster.local). It's now a []string of fully-composed <node_id>@<host>:<port> entries. The only consumer is the planner, which is updated in this same PR. No external consumer is affected.

Backward compatibility

• Existing SNDs with LabelPeerSource keep working — their ResolvedPeers will repopulate with the new format on next reconcile, and the planner routes them through the same Static sidecar source path that genesis peers already use.
• Spec.Peers user-facing shape is unchanged.
• EC2Tags and Static PeerSources are unchanged.
• The sidecar's DNSEndpointsSource (the :26657/status path) stays in seictl — still serves EC2Tags. A follow-up could retire it once EC2Tags also moves to controller-side; out of scope here.

Test plan

• Unit: new TestReconcilePeers_PrefersExternalAddress. Existing peers tests updated for the new format.
• go test ./api/v1alpha1/... ./internal/... — green.
• make test-integration — envtest green (~64s).
• golangci-lint run --new-from-rev=main ./... — 0 issues.
• make manifests generate — clean.
• CI on this PR.

Refs

• Tracking issue: Harmonize SeiNode peer discovery between controller and sidecar #368
• Prior PR (controller publishable path): feat(p2p-endpoint): SND-owned per-pod NLB + Spec.ExternalAddress (#360) #365
• Platform follow-ups: sei-protocol/platform#747 (env wiring), sei-protocol/platform#748 (SG opening)

🤖 Generated with Claude Code

[cursor]

PR Summary

Medium Risk
Changes P2P peer list format and discovery path for label-based peers (networking/consensus adjacency); transient failures can retain stale node_id entries until the sidecar recovers.

Overview
Label-based peer discovery now writes status.resolvedPeers as CometBFT-ready <node_id>@<host>:<port> strings instead of bare cluster DNS hostnames. The SeiNode reconciler fetches each peer’s node_id via in-cluster sidecar gRPC (SidecarClient.GetNodeID), picks spec.externalAddress when set (publishable NLB path) or headless service DNS plus the standard P2P port otherwise, and on per-peer sidecar failure keeps the previous composed entry or omits a new peer so one bad peer does not block reconcile.

The discover-peers planner path for label sources now passes those strings as static addresses (replacing DNS-endpoint resolution in the sidecar). API/CRD docs and tests were updated for the new wire format and failure modes.

^{Reviewed by Cursor Bugbot for commit 3b71ffc. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bdchatham]

Three-way cross-review verdict:

• kubernetes-specialist: GO (atomic semantics correct)
• sei-network-specialist: GO (CometBFT semantics + node_id correctness verified)
• platform-engineer: HOLD — atomic resolve wedges fleet-wide reconciles in steady state when any label-matched peer's sidecar is transient-unavailable (pacific-1 scenario: 6+ SNDs with mutual selectors → single sidecar restart blocks every consumer's reconcile, not just peer updates).

Reasoning from first principles per coral discipline: atomic is right for the genesis path (bounded cohort, one-shot, co-bootstrapping) but wrong for the steady-state label flow (every reconcile forever, arbitrary peer lifecycle). Platform-engineer's blocker is the load-bearing one.

Fix pushed in 0fceb7a:

• On per-peer BuildSidecarClient or GetNodeID error, preserve that peer's prior entry from Status.ResolvedPeers (indexed by host:port). Transients don't drop peers from the list.
• New peer with no prior entry + sidecar failure: skip with structured log line (peer, err), do not fail the reconcile. Mitigates the k8s-specialist's "silent partial list" debugging concern.
• Two new unit tests: TestReconcilePeers_PreservesPriorEntryOnTransientFailure, TestReconcilePeers_SkipsNewPeerOnSidecarFailure.

Tests + envtest still green. golangci-lint --new-from-rev=main = 0 issues.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 10a8cbf. Configure here.}