feat: Support OPA role mapping (#582)

* First skeleton of opa integration

* WIP implementation opa-role-mapping

* Add SecurityManager dynamically

* Better OPA_IMPORT

* Security manager own file in Docker-Images. Fixing python expression

* making clippy happy for now

* Updating some approaches

* Adding more rules, more sophisticated handling of stuff

* Defaults are working

* Better interfering of package path

* Happy Clippy

* update OpaSupersetSecurityManager import path

* import new opa_authorizer module

* Removing some ToDo's. Better comments

* Adding OpaRolesCache with 10 minutes default

* pre commit becomes happy

* create opa test basics

* rename test directory

* fix test-definition

* fix typo

* fix opa test scaffold

* Adding rule_name to be defined by the user. defaults to empty string

* Adding default to rule_name

* StackableOpaRule to string as we interfere from CRDs

* Adding ttl to crds. Default to 10.

* cache_ttl now Duration type. Converted to seconds in superset_config.py

* wip: integration tests

* integrate feedback

* create basic user role test

* First documentation draft

* Adding reference to opa user-info-fetcher

* Updating changelog

* fix rego and first check in integration test

* fix formatting issues

* Making rust fmt happy

* lint with ruff formatter

* use TtlCache from operator-rs

* Regenerate charts

* Making pre-commit happy

* apply typos and formatting corrections

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* update chart

* adress feedback in PR and rename envs

* fix changelog

* Update opa tests.

* support custom image for opa tests

* create and assign new role via API

* fix typos

* add some comments

* opa kuttl test is green (again)

* silence most of Pyright errors and warnings

* Update rust/crd/src/lib.rs

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* Renaming fields and structs

* format code

* make field required

* pass on opa endpoint instead of base url to the authorizer

* Update docs/modules/superset/pages/usage-guide/security.adoc

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* Update docs/modules/superset/pages/usage-guide/security.adoc

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* Update docs/modules/superset/pages/usage-guide/security.adoc

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* Update docs/modules/superset/pages/usage-guide/security.adoc

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* update security.adoc

* move constant

* rename opa dimension

* revert changes to smoke test

* Update tests/templates/kuttl/opa/40_superset.yaml.j2

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* remove unused image field

* add serde cache defaults

* Update rust/operator-binary/src/authorization/opa.rs

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* Update rust/operator-binary/src/superset_controller.rs

Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>

* add missing EOF

* add vector aggregator config map

* add openshift ns patch

---------

Co-authored-by: Benedikt Labrenz <benedikt@labrenz.org>
Co-authored-by: Sebastian Bernauer <sebastian.bernauer@stackable.de>
Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>

Author: 4 people

CHANGELOG.md

@@ -6,13 +6,15 @@
 
 - Run a `containerdebug` process in the background of each Superset container to collect debugging information ([#578]).
 - Aggregate emitted Kubernetes events on the CustomResources ([#585]).
+- Support OPA role mapping as optional custom security manager for Superset ([#582]).
 - Support for version `4.1.1` ([#595]).
 
 ### Changed
 
 - Default to OCI for image metadata and product image selection ([#586]).
 
 [#578]: https://github.com/stackabletech/superset-operator/pull/578
+[#582]: https://github.com/stackabletech/superset-operator/pull/582
 [#585]: https://github.com/stackabletech/superset-operator/pull/585
 [#586]: https://github.com/stackabletech/superset-operator/pull/586
 [#595]: https://github.com/stackabletech/superset-operator/pull/595

Cargo.lock

@@ -23,7 +23,7 @@ serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 serde_yaml = "0.9"
 snafu = "0.8"
-stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "stackable-operator-0.85.0" }
+stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "stackable-operator-0.86.0" }
 strum = { version = "0.26", features = ["derive"] }
 tokio = { version = "1.40", features = ["full"] }
 tracing = "0.1"

Cargo.nix

@@ -71,6 +71,46 @@ spec:
                           - authenticationClass
                         type: object
                       type: array
+                    authorization:
+                      description: |-
+                        Authorization options for Superset.
+
+                        Currently only role assignment is supported. This means that roles are assigned to users in OPA but, due to the way Superset is implemented, the database also needs to be updated to reflect these assignments. Therefore, user roles and permissions must already exist in the Superset database before they can be assigned to a user. Warning: Any user roles assigned with the Superset UI are discarded.
+                      nullable: true
+                      properties:
+                        roleMappingFromOpa:
+                          description: Configure the OPA stacklet [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) and the name of the Rego package containing your authorization rules. Consult the [OPA authorization documentation](https://docs.stackable.tech/home/nightly/concepts/opa) to learn how to deploy Rego authorization rules with OPA.
+                          properties:
+                            cache:
+                              default:
+                                entryTimeToLive: 30s
+                                maxEntries: 10000
+                              description: Configuration for an Superset internal cache for calls to OPA
+                              properties:
+                                entryTimeToLive:
+                                  default: 30s
+                                  description: Time to live per entry
+                                  type: string
+                                maxEntries:
+                                  default: 10000
+                                  description: Maximum number of entries in the cache; If this threshold is reached then the least recently used item is removed.
+                                  format: uint32
+                                  minimum: 0.0
+                                  type: integer
+                              type: object
+                            configMapName:
+                              description: The [discovery ConfigMap](https://docs.stackable.tech/home/nightly/concepts/service_discovery) for the OPA stacklet that should be used for authorization requests.
+                              type: string
+                            package:
+                              description: The name of the Rego package containing the Rego rules for the product.
+                              nullable: true
+                              type: string
+                          required:
+                            - configMapName
+                          type: object
+                      required:
+                        - roleMappingFromOpa
+                      type: object
                     clusterOperation:
                       default:
                         reconciliationPaused: false

Cargo.toml

@@ -126,6 +126,71 @@ Further information for specifying an AuthenticationClass for an OIDC provider c
 Superset has a concept called `Roles` which allows you to grant user permissions based on roles.
 Have a look at the {superset-security}[Superset documentation on Security^{external-link-icon}^].
 
+[opa]
+=== OPA role mapping
+
+Stackable ships a custom security manager that makes it possible to assign roles to users via the Open Policy Agent integration.
+The roles must exist in the Superset database before they can be assigned to users.
+If a role is not present in the Superset database, an error will be logged by the security manager and the user login will proceed without it.
+Also the role names must match exactly the output of the Rego rule named `user_roles`.
+In the following example, a rego package is defined that assigns roles to the users `admin` and `guest`.
+
+[source,yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: superset-opa-regorules
+  labels:
+    opa.stackable.tech/bundle: "true"
+data:
+  roles.rego: |
+    package superset
+
+    default user_roles := []
+
+    user_roles := roles if {
+        some user in users
+        roles := user.roles
+        user.username == input.username
+    }
+    users := [
+        {"username": "admin", "roles": ["Admin", "Test"]}, #<1>
+        {"username": "guest", "roles": ["Gamma"]} #<2>
+    ]
+----
+
+<1> Assign the roles `Admin` and `Test` to the `admin` user. The `Test` role is not a standard Superset role and must be created before the assignment.
+<2> Assign the `Gamma` role to the `guest` user.
+
+OPA rules can make use of the xref:opa:usage-guide:user-info-fetcher[user-info-fetcher] integration.
+
+The following snippet shows how to use the OPA security manager in a Superset stacklet.
+
+[source,yaml]
+----
+apiVersion: superset.stackable.tech/v1alpha1
+kind: SupersetCluster
+metadata:
+  name: superset-with-opa-role-mapping
+spec:
+  clusterConfig:
+    authorization:
+      roleMappingFromOpa:
+        configMapName: superset-opa-regorules # <1>
+        package: superset
+        cache: # <2>
+          entryTimeToLive: 10s # <3>
+          maxEntries: 5 # <4>
+----
+
+<1> ConfigMap name containing rego rules
+<2> Mandatory Opa caching. If not set, default settings apply.
+<3> Time for cached entries per user can live. Defaults to 30s.
+<4> Number of maximum entries, defaults to 1000. Cache will be disabled for maxEntries: 0.
+
+IMPORTANT: Any role assignments done in the Superset UI are discarded and will be overridden by the OPA security manager.
+
 === Superset database
 
 You can view all the available roles in the web interface of Superset and can also assign users to these roles.

crate-hashes.json

@@ -7,7 +7,9 @@ use snafu::{OptionExt, ResultExt, Snafu};
 use stackable_operator::{
     commons::{
         affinity::StackableAffinity,
+        cache::UserInformationCache,
         cluster_operation::ClusterOperation,
+        opa::OpaConfig,
         product_image_selection::ProductImage,
         resources::{
             CpuLimitsFragment, MemoryLimitsFragment, NoRuntimeLimits, NoRuntimeLimitsFragment,
@@ -89,6 +91,12 @@ pub enum SupersetConfigOptions {
     AuthLdapTlsCertfile,
     AuthLdapTlsKeyfile,
     AuthLdapTlsCacertfile,
+    CustomSecurityManager,
+    AuthOpaRequestUrl,
+    AuthOpaPackage,
+    AuthOpaRule,
+    AuthOpaCacheMaxEntries,
+    AuthOpaCacheTtlInSec,
 }
 
 impl SupersetConfigOptions {
@@ -119,6 +127,7 @@ impl FlaskAppConfigOptions for SupersetConfigOptions {
             SupersetConfigOptions::LoggingConfigurator => PythonType::Expression,
             SupersetConfigOptions::AuthType => PythonType::Expression,
             SupersetConfigOptions::AuthUserRegistration => PythonType::BoolLiteral,
+            // Going to be an expression as we default it from env, if and only if opa is used
             SupersetConfigOptions::AuthUserRegistrationRole => PythonType::StringLiteral,
             SupersetConfigOptions::AuthRolesSyncAtLogin => PythonType::BoolLiteral,
             SupersetConfigOptions::AuthLdapServer => PythonType::StringLiteral,
@@ -136,6 +145,13 @@ impl FlaskAppConfigOptions for SupersetConfigOptions {
             SupersetConfigOptions::AuthLdapTlsCertfile => PythonType::StringLiteral,
             SupersetConfigOptions::AuthLdapTlsKeyfile => PythonType::StringLiteral,
             SupersetConfigOptions::AuthLdapTlsCacertfile => PythonType::StringLiteral,
+            // Configuration options used by CustomOpaSecurityManager
+            SupersetConfigOptions::CustomSecurityManager => PythonType::Expression,
+            SupersetConfigOptions::AuthOpaRequestUrl => PythonType::StringLiteral,
+            SupersetConfigOptions::AuthOpaPackage => PythonType::StringLiteral,
+            SupersetConfigOptions::AuthOpaRule => PythonType::StringLiteral,
+            SupersetConfigOptions::AuthOpaCacheMaxEntries => PythonType::IntLiteral,
+            SupersetConfigOptions::AuthOpaCacheTtlInSec => PythonType::IntLiteral,
         }
     }
 }
@@ -179,6 +195,17 @@ pub struct SupersetClusterConfig {
     #[serde(default)]
     pub authentication: Vec<SupersetClientAuthenticationDetails>,
 
+    /// Authorization options for Superset.
+    ///
+    /// Currently only role assignment is supported. This means that roles are assigned to users in
+    /// OPA but, due to the way Superset is implemented, the database also needs to be updated
+    /// to reflect these assignments.
+    /// Therefore, user roles and permissions must already exist in the Superset database before
+    /// they can be assigned to a user.
+    /// Warning: Any user roles assigned with the Superset UI are discarded.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub authorization: Option<SupersetAuthorization>,
+
     /// The name of the Secret object containing the admin user credentials and database connection details.
     /// Read the
     /// [getting started guide first steps](DOCS_BASE_URL_PLACEHOLDER/superset/getting_started/first_steps)
@@ -242,6 +269,22 @@ impl CurrentlySupportedListenerClasses {
         }
     }
 }
+#[derive(Clone, Deserialize, Serialize, Eq, JsonSchema, Debug, PartialEq)]
+#[serde(rename_all = "camelCase")]
+pub struct SupersetOpaRoleMappingConfig {
+    #[serde(flatten)]
+    pub opa: OpaConfig,
+
+    /// Configuration for an Superset internal cache for calls to OPA
+    #[serde(default)]
+    pub cache: UserInformationCache,
+}
+
+#[derive(Clone, Debug, Deserialize, Eq, JsonSchema, PartialEq, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct SupersetAuthorization {
+    pub role_mapping_from_opa: SupersetOpaRoleMappingConfig,
+}
 
 #[derive(Clone, Debug, Deserialize, Eq, PartialEq, Serialize)]
 #[serde(rename_all = "camelCase")]
@@ -476,6 +519,14 @@ impl SupersetCluster {
         }
     }
 
+    pub fn get_opa_config(&self) -> Option<&SupersetOpaRoleMappingConfig> {
+        self.spec
+            .cluster_config
+            .authorization
+            .as_ref()
+            .map(|a| &a.role_mapping_from_opa)
+    }
+
     /// Retrieve and merge resource configs for role and role groups
     pub fn merged_config(
         &self,

deploy/helm/superset-operator/crds/crds.yaml

@@ -0,0 +1 @@
+pub mod opa;

docs/modules/superset/pages/usage-guide/security.adoc

@@ -0,0 +1,55 @@
+use std::collections::BTreeMap;
+
+use stackable_operator::{client::Client, commons::opa::OpaApiVersion, time::Duration};
+use stackable_superset_crd::{SupersetCluster, SupersetOpaRoleMappingConfig};
+
+pub const OPA_IMPORTS: &[&str] =
+    &["from opa_authorizer.opa_manager import OpaSupersetSecurityManager"];
+
+pub struct SupersetOpaConfigResolved {
+    opa_endpoint: String,
+    cache_max_entries: u32,
+    cache_ttl: Duration,
+}
+
+impl SupersetOpaConfigResolved {
+    pub async fn from_opa_config(
+        client: &Client,
+        superset: &SupersetCluster,
+        opa_config: &SupersetOpaRoleMappingConfig,
+    ) -> Result<Self, stackable_operator::commons::opa::Error> {
+        let opa_endpoint = opa_config
+            .opa
+            .full_document_url_from_config_map(client, superset, None, OpaApiVersion::V1)
+            .await?;
+
+        Ok(SupersetOpaConfigResolved {
+            opa_endpoint,
+            cache_max_entries: opa_config.cache.max_entries.to_owned(),
+            cache_ttl: opa_config.cache.entry_time_to_live.to_owned(),
+        })
+    }
+
+    // Adding necessary configurations. Imports are solved in config.rs
+    pub fn as_config(&self) -> BTreeMap<String, String> {
+        BTreeMap::from([
+            (
+                "CUSTOM_SECURITY_MANAGER".to_string(),
+                "OpaSupersetSecurityManager".to_string(),
+            ),
+            (
+                "AUTH_OPA_REQUEST_URL".to_string(),
+                self.opa_endpoint.to_owned(),
+            ),
+            (
+                "AUTH_OPA_CACHE_MAX_ENTRIES".to_string(),
+                self.cache_max_entries.to_string(),
+            ),
+            (
+                "AUTH_OPA_CACHE_TTL_IN_SEC".to_string(),
+                self.cache_ttl.as_secs().to_string(),
+            ),
+            ("AUTH_OPA_RULE".to_string(), "user_roles".to_string()),
+        ])
+    }
+}