Skip to content

Allow per-component configuration in Flux topology#8783

Open
GGraziadei wants to merge 1 commit into
apache:masterfrom
GGraziadei:8710-flux-component-conf
Open

Allow per-component configuration in Flux topology#8783
GGraziadei wants to merge 1 commit into
apache:masterfrom
GGraziadei:8710-flux-component-conf

Conversation

@GGraziadei
Copy link
Copy Markdown
Contributor

What is the purpose of the change

Add per-component configuration in the flux topology def grammar.

How was the change tested

@GGraziadei GGraziadei changed the title init Allow per-component configuration in Flux topology Jun 2, 2026
@rzo1 rzo1 requested review from reiabreu and rzo1 June 3, 2026 07:25
}

@Test
public void testComponentConfig() throws Exception {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make a test where we are overriding topology-level configs?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thank you for the comment. I totally agree, for coverage tree convenience, I am not adding a new test case, but compacting the scenario you proposed in this test case.

@rzo1
Copy link
Copy Markdown
Contributor

rzo1 commented Jun 5, 2026

Nice, well-scoped feature that rides on Storm's native component-config mechanism.

  1. Bolt config is applied once per incoming stream, not once per bolt. The block applying memory/CPU/tasks/config runs on every iteration of the streams loop, so a bolt with N inputs calls validateFields() + addConfigurations() N times on the same declarer. It's idempotent, so not a bug, and it matches the existing pattern for setMemoryLoad/setCPULoad/setNumTasks (just redundant=
  2. Validation asymmetry. buildConfig() applies the topology-wide config: with no ConfigValidation, while this PR validates per-component config. Fail-fast is the better behavior, but the asymmetry (an invalid-typed value passes topology-wide yet fails per-component) may surprise users. Consider validating the topology config too in a follow-up, or noting the difference.
  3. README clarity. "Known Storm configuration keys are validated …" is accurate but could read as blanket validation. Suggest adding: "Unknown/custom keys are not validated and are passed through verbatim; validation runs client-side at build time."
  4. Nit: new HashMap<String, Object>() in VertexDef → diamond <> to match the codebase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flux support per-component configuration in BoltDef and SpoutDef

3 participants