[FLINK-39016][Runtime/REST] Add configurable TTL for ExecutionGraph cache independent of web refresh interval

[Myracle]

What is the purpose of the change

This pull request introduces a new configuration option web.execution-graph.cache-ttl that allows users to configure the TTL (Time-to-Live) for the ExecutionGraph cache independently from the web.refresh-interval.
Previously, the ExecutionGraph cache TTL was implicitly tied to the web refresh interval, which made it difficult for users who need real-time job state synchronization (e.g., for monitoring dashboards or orchestration systems) to get fresh ExecutionGraph data without affecting the overall web UI refresh behavior.
With this change, users can:

• Set web.execution-graph.cache-ttl to a small value (or 0) to always fetch fresh ExecutionGraph data
• Keep the web UI refresh interval at a reasonable value for normal dashboard usage
• Maintain backward compatibility by defaulting to the web.refresh-interval when not explicitly configured

Brief change log

• Added new configuration option web.execution-graph.cache-ttl in WebOptions with fallback to web.refresh-interval
• Extended RestHandlerConfiguration to parse and hold the new ExecutionGraph cache TTL parameter with non-negative validation
• Updated RestEndpointFactory.createExecutionGraphCache() to use the dedicated TTL configuration instead of refresh interval
• Added unit tests covering default fallback behavior, custom values, zero value, and negative value validation
• Updated web configuration documentation to include the new option

Verifying this change

This change added tests and can be verified as follows:

• Added RestHandlerConfigurationTest#testExecutionGraphCacheTTLDefault() to verify the default fallback to web.refresh-interval
• Added RestHandlerConfigurationTest#testExecutionGraphCacheTTLCustomValue() to verify independent configuration of cache TTL
• Added RestHandlerConfigurationTest#testExecutionGraphCacheTTLZeroValue() to verify zero value support for real-time synchronization scenarios
• Added RestHandlerConfigurationTest#testExecutionGraphCacheTTLNegativeValue() to verify negative value validation throws IllegalArgumentException

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
• The S3 file system connector: (no)

Documentation

• Does this pull request introduce a new feature? (yes)
• If yes, how is the feature documented? (docs / JavaDocs )

[flinkbot]

CI report:

• 3744421 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[davidradl]

In the comments it says "0 (or a very small value)" but the other comments only talk of 0. I think we should be explicit in the text as to what the smallest number is that would cause caching.

[Myracle]

Good catch! You're right that the Javadoc and withDescription were inconsistent. Looking at the DefaultExecutionGraphCache implementation, only a value of exactly 0 deterministically disables caching (since currentTime < currentTime is always false). A "very small value" would only probabilistically result in cache misses, not guarantee them. I've updated the Javadoc to remove the ambiguous "(or a very small value)" phrasing, making it consistent with the withDescription text — both now explicitly state that setting this to 0 disables caching.

[davidradl]

The only reference to web.refresh-interval is see in the docs is web.refresh-interval.

The config parameter starts with web. implies it is for the Web UI. There are no other parameters starting with web.execution-graph - this is a Flink internal concept. How should I understand the context of where this config option implies?

[Myracle]

Thanks for the review! You are absolutely right. The web. prefix is reserved for Web UI-facing configurations, and execution-graph is a Flink internal concept that doesn't fit in this namespace. I've moved the config option to RestOptions with the key rest.cache.execution-graph.timeout, following the same naming convention as the existing rest.cache.checkpoint-statistics.timeout. The documentation has been updated accordingly.

[Varka808]

This zero value test can be merged into testExecutionGraphCacheTTLCustomValue.

[Varka808]

Please rebase the latest master branch and squash all commits.

[reswqa]

Thanks, LGTM.