From 3a79d6496806f5d9597e83aa7518c5c0bd6a4bc1 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Thu, 2 Jan 2025 10:49:16 +0100 Subject: [PATCH 1/4] Add concepts page on JVM argument overrides --- modules/concepts/pages/overrides.adoc | 70 +++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/modules/concepts/pages/overrides.adoc b/modules/concepts/pages/overrides.adoc index 300264a21..de3e3c8d3 100644 --- a/modules/concepts/pages/overrides.adoc +++ b/modules/concepts/pages/overrides.adoc @@ -150,3 +150,73 @@ The `podOverrides` will be merged onto the following resources the operators dep They will *not* be applied to: * Jobs, that are used to setup systems the product depends on e.g. create a database schema for Superset or Airflow. + +[#jvm-argument-overrides] +== JVM argument overrides + +You can configure the JVM arguments used by JVM based tools. +This is often times needed to e.g. configure a HTTP proxy or other network settings. + +As with other overrides, the operator generates a set of JVM arguments that are needed to run the tool, you can specify additional arguments, which are merged on top of the ones the operator generated. +As some JVm arguments are mutually exclusive (think of `-Xmx123m` and `-Xmx456m`), you also have the option to remove JVM arguments - either by specifying the exact argument or a regex. + +The merging mechanism is applied <- <- and works as following: + +Let user be the user-specified arguments and operator be the arguments generated by the operator. + +1. All arguments listed in user `remove` are removed from operator +2. All arguments matching any regex from user `removeRegex` are removed from operator +3. All arguments from user `add` are added to operator + +You can check the resulting effective JVM arguments by looking at the ConfigMap containing the config for the roleGroup (although some tools read the JVM arguments from environmental variable). + +=== Simple example + +One simple usage of this functionality is to add some JVM arguments, in this case needed for a special network setup: + +[source,yaml] +---- +kind: NifiCluster +spec: + # ... + nodes: + jvmArgumentOverrides: + add: # Add some networking arguments + - -Dhttps.proxyHost=proxy.my.corp + - -Dhttps.proxyPort=8080 + - -Djava.net.preferIPv4Stack=true +---- + +=== Advanced example + +The following more advanced setups shows how the garbage collector can be changed, the JVM memory configs can be changed and how roleGroups can override roles. + +[source,yaml] +---- +kind: NifiCluster +spec: + # ... + nodes: + config: + resources: + memory: + limit: 42Gi # We define some memory config, so that we can override it further down + jvmArgumentOverrides: + remove: + - -XX:+UseG1GC # Remove argument generated by operator + add: # Add some networking arguments + - -Dhttps.proxyHost=proxy.my.corp + - -Dhttps.proxyPort=8080 + - -Djava.net.preferIPv4Stack=true + roleGroups: + default: + replicas: 1 + jvmArgumentOverrides: + # We need more memory! + removeRegex: # They need to match the entire string, not only a part! + - -Xmx.* # So this will match "-Xmx123m", but not "-foo-Xmx123m" + - -Dhttps.proxyPort=.* # Remove arguments from the role, so that we can override it + add: # After we removed some arguments we can add the correct ones + - -Xmx40000m + - -Dhttps.proxyPort=1234 # Override arguments from the role +---- From 387268d2cff2b21c2d361e6e2542b5302c8f5e1d Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Tue, 7 Jan 2025 11:07:57 +0100 Subject: [PATCH 2/4] Apply suggestions from code review Co-authored-by: Andrew Kenworthy <1712947+adwk67@users.noreply.github.com> Co-authored-by: Malte Sander --- modules/concepts/pages/overrides.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/concepts/pages/overrides.adoc b/modules/concepts/pages/overrides.adoc index de3e3c8d3..d17060b0c 100644 --- a/modules/concepts/pages/overrides.adoc +++ b/modules/concepts/pages/overrides.adoc @@ -155,12 +155,12 @@ They will *not* be applied to: == JVM argument overrides You can configure the JVM arguments used by JVM based tools. -This is often times needed to e.g. configure a HTTP proxy or other network settings. +This is often needed to e.g. configure a HTTP proxy or other network settings. -As with other overrides, the operator generates a set of JVM arguments that are needed to run the tool, you can specify additional arguments, which are merged on top of the ones the operator generated. -As some JVm arguments are mutually exclusive (think of `-Xmx123m` and `-Xmx456m`), you also have the option to remove JVM arguments - either by specifying the exact argument or a regex. +As with other overrides, the operator generates a set of JVM arguments that are needed to run the tool. You can specify additional arguments which are merged on top of the ones the operator generated. +As some JVM arguments are mutually exclusive (think of `-Xmx123m` and `-Xmx456m`), you also have the option to remove JVM arguments - either by specifying the exact argument or a regex. -The merging mechanism is applied <- <- and works as following: +The merging mechanism is applied <- <- and works as follows: Let user be the user-specified arguments and operator be the arguments generated by the operator. @@ -168,7 +168,7 @@ Let user be the user-specified arguments and operator be the arguments generated 2. All arguments matching any regex from user `removeRegex` are removed from operator 3. All arguments from user `add` are added to operator -You can check the resulting effective JVM arguments by looking at the ConfigMap containing the config for the roleGroup (although some tools read the JVM arguments from environmental variable). +You can check the resulting effective JVM arguments by looking at the ConfigMap containing the config for the roleGroup (although some tools read the JVM arguments from environmental variables). === Simple example From 8f8e78d5f5d3635d29099724e4da9413a1885c54 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Tue, 7 Jan 2025 11:10:24 +0100 Subject: [PATCH 3/4] Reword merge mechanism --- modules/concepts/pages/overrides.adoc | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/modules/concepts/pages/overrides.adoc b/modules/concepts/pages/overrides.adoc index d17060b0c..128b55c20 100644 --- a/modules/concepts/pages/overrides.adoc +++ b/modules/concepts/pages/overrides.adoc @@ -160,13 +160,11 @@ This is often needed to e.g. configure a HTTP proxy or other network settings. As with other overrides, the operator generates a set of JVM arguments that are needed to run the tool. You can specify additional arguments which are merged on top of the ones the operator generated. As some JVM arguments are mutually exclusive (think of `-Xmx123m` and `-Xmx456m`), you also have the option to remove JVM arguments - either by specifying the exact argument or a regex. -The merging mechanism is applied <- <- and works as follows: +The merging mechanism is applied <- <- and works as follows: -Let user be the user-specified arguments and operator be the arguments generated by the operator. - -1. All arguments listed in user `remove` are removed from operator -2. All arguments matching any regex from user `removeRegex` are removed from operator -3. All arguments from user `add` are added to operator +1. All arguments listed in user specified `remove` are removed from operator generated +2. All arguments matching any regex from user `removeRegex` are removed from operator generated +3. All arguments from user specified `add` are added to operator You can check the resulting effective JVM arguments by looking at the ConfigMap containing the config for the roleGroup (although some tools read the JVM arguments from environmental variables). From bea00886a81e84acdf348d29ef0a721df479783d Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Wed, 15 Jan 2025 09:26:39 +0100 Subject: [PATCH 4/4] Update modules/concepts/pages/overrides.adoc --- modules/concepts/pages/overrides.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/modules/concepts/pages/overrides.adoc b/modules/concepts/pages/overrides.adoc index 128b55c20..0c4375b83 100644 --- a/modules/concepts/pages/overrides.adoc +++ b/modules/concepts/pages/overrides.adoc @@ -163,7 +163,8 @@ As some JVM arguments are mutually exclusive (think of `-Xmx123m` and `-Xmx456m` The merging mechanism is applied <- <- and works as follows: 1. All arguments listed in user specified `remove` are removed from operator generated -2. All arguments matching any regex from user `removeRegex` are removed from operator generated +2. All arguments matching any regex from user removeRegex are removed from operator generated. + The regex needs to match the entire argument, not only a substring 3. All arguments from user specified `add` are added to operator You can check the resulting effective JVM arguments by looking at the ConfigMap containing the config for the roleGroup (although some tools read the JVM arguments from environmental variables).