Skip to content

feat(ai-platform): add weaviate-service feature with Ray-optional reconcile path #82

Merged
kupratyu-splunk merged 21 commits intofeature/weaviate-servicefrom
kiran/weaviate-service-k0s
Apr 7, 2026
Merged

feat(ai-platform): add weaviate-service feature with Ray-optional reconcile path #82
kupratyu-splunk merged 21 commits intofeature/weaviate-servicefrom
kiran/weaviate-service-k0s

Conversation

@kbhos-splunk
Copy link
Copy Markdown
Collaborator

Description

This PR adds a new weaviate-service feature to AIPlatform so we can run a Weaviate proxy-only service without requiring Ray Serve. It also makes Ray reconciliation optional when the platform features do not need Ray, and wires RELATED_IMAGE_WEAVIATE_PROXY through operator manifests and Helm values.

Related Issues

  • Related to #

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test improvement
  • CI/CD improvement
  • Chore (dependency updates, etc.)

Changes Made

  • Added weaviate-service as a supported feature in API/CRD schema and feature registry.
  • Implemented new feature package: pkg/ai/features/weaviateservice (factory, reconciler logic, unit tests).
  • Made Ray optional in AIPlatform reconcile/setup flow:
  • Skip Ray watches when Ray CRDs are absent.
  • Skip Ray stages for Ray-independent feature sets.
  • Add Ray cleanup stage for weaviate-only mode.
  • Added RELATED_IMAGE_WEAVIATE_PROXY support in:

Testing Performed

  • Unit tests pass (make test)
  • Linting passes (make lint)
  • Integration tests pass (if applicable)
  • E2E tests pass (if applicable)
  • Manual testing performed

Test Environment

  • Kubernetes Version:
  • Cloud Provider:
  • Deployment Method:

Test Steps

  1. Build/deploy updated operator image and apply manifests/CRDs.
  2. Deploy AIPlatform with feature weaviate-service and verify creation for weaviate service.
  3. Verify weaviate-service pods/services become healthy and Ray resources are not required in weaviate-only mode.
    Run Go test suite locally:
    go test ./...

Documentation

  • Updated inline code comments
  • Updated README.md (if adding features)
  • Updated API documentation
  • Updated deployment guides
  • Updated CHANGELOG.md
  • No documentation needed

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published
  • I have updated the Helm chart version (if applicable)
  • I have updated CRD schemas (if applicable)

Breaking Changes

Impact:
None expected.

Migration Path:
Migration Path: No mandatory migration. Existing saia/seca behavior remains; weaviate-service is additive.

Screenshots/Recordings

Additional Notes

Reviewer Notes

Please pay special attention to:


Commit Message Convention: This PR follows Conventional Commits

kupratyu-splunk
kupratyu-splunk previously approved these changes Apr 6, 2026
Copy link
Copy Markdown
Collaborator

@kupratyu-splunk kupratyu-splunk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure we get proper logs for all the success and failure scenarios.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new weaviate-service feature for AIPlatform to support running a Weaviate proxy-only service without requiring Ray Serve, and makes Ray reconciliation/watches optional when the selected feature set does not depend on Ray.

Changes:

  • Add weaviate-service to CRD/API schemas and feature registry, plus a new feature implementation package with unit tests.
  • Make Ray reconciliation conditional (skip Ray stages/watches when Ray isn’t needed or Ray CRDs are absent) and add a Ray cleanup path for Ray-less mode.
  • Wire RELATED_IMAGE_WEAVIATE_PROXY through Helm/operator manifests and cluster setup tooling; add helper scripts for k0s/EKS stack setup.

Reviewed changes

Copilot reviewed 30 out of 32 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
tools/cluster_setup/splunk-operator-cluster.yaml Updates Splunk Enterprise related image tag used in cluster setup manifest.
tools/cluster_setup/k0s_weaviate_service_cluster_with_stack.sh Adds k0s install/delete script for a weaviate-service-focused stack, including proxy image wiring.
tools/cluster_setup/eks_weaviate_service_cluster_with_stack.sh Adds EKS install/delete script for a weaviate-service-focused stack, including proxy image wiring.
tools/cluster_setup/cluster-config.yaml Adds images.weaviate.proxyImage to config to drive RELATED_IMAGE_WEAVIATE_PROXY.
tools/cluster_setup/artifacts.yaml Updates CRD schemas and operator env vars in the generated artifacts; adds RELATED_IMAGE_WEAVIATE_PROXY.
pkg/ai/reconciler.go Makes Ray stages optional based on features; passes feature env into AIService spec; adds Ray cleanup.
pkg/ai/reconciler_test.go Adds unit tests for feature env propagation and Ray-required detection logic.
pkg/ai/raybuilder/builder.go Skips Ray scaling/config generation for Ray-independent features.
pkg/ai/raybuilder/builder_test.go Adds unit test coverage for Ray-independence mapping.
pkg/ai/features/weaviateservice/impl.go Implements reconciler for weaviate-service feature (proxy Deployment/Service, env construction, defaults).
pkg/ai/features/weaviateservice/impl_test.go Adds unit tests for URL normalization, env overrides, and AIPlatformRef-based defaults.
pkg/ai/features/weaviateservice/factory.go Adds feature factory wiring for weaviate-service.
pkg/ai/features/weaviateservice/factory_test.go Tests factory construction.
pkg/ai/features/registry.go Registers weaviate-service in the feature factory registry.
internal/controller/aiplatform_controller.go Makes Ray watches conditional on Ray CRD presence.
helm-chart/splunk-ai-platform/values.yaml Documents feature-level env values in AIPlatform chart values.
helm-chart/splunk-ai-platform/templates/aiplatform.yaml Renders feature-level env into the AIPlatform CR.
helm-chart/splunk-ai-operator/values.yaml Adds weaviateProxyImage Helm value for operator env injection.
helm-chart/splunk-ai-operator/templates/deployment.yaml Injects RELATED_IMAGE_WEAVIATE_PROXY into operator Deployment.
helm-chart/splunk-ai-operator/crds/ai.splunk.com_aiservices.yaml Updates Helm-packaged AIService CRD schema for env + new feature enum.
helm-chart/splunk-ai-operator/crds/ai.splunk.com_aiplatforms.yaml Updates Helm-packaged AIPlatform CRD schema for env + new feature enum.
helm-chart/splunk-ai-operator/Chart.yaml Bumps dependency versions (cert-manager, splunk-operator).
helm-chart/splunk-ai-operator/Chart.lock Updates locked dependency versions/digests.
go.sum Updates Go module checksums for dependency bumps.
go.mod Bumps cert-manager and related indirect dependencies.
docs/weaviate_service_readme.md Adds documentation for the weaviate-service feature and install/verify steps.
config/samples/k0s-cluster-config-sample.yaml Adds sample config for k0s weaviate-service stack deployment.
config/manager/kustomization.yaml Wires new related image env var into manager deployment patch (currently appears malformed).
config/crd/bases/ai.splunk.com_aiservices.yaml Updates base AIService CRD schema for env + new feature enum.
config/crd/bases/ai.splunk.com_aiplatforms.yaml Updates base AIPlatform CRD schema for env + new feature enum.
api/v1/zz_generated.deepcopy.go Updates deepcopy generation for new FeatureSpec.Env map field.
api/v1/aiplatform_types.go Adds FeatureSpec.Env and extends feature enum to include weaviate-service.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@kbhos-splunk
Copy link
Copy Markdown
Collaborator Author

@copilot review

kupratyu-splunk
kupratyu-splunk previously approved these changes Apr 6, 2026
@kupratyu-splunk
Copy link
Copy Markdown
Collaborator

You will have to add the branch name in https://github.com/splunk/splunk-ai-operator/blob/kiran/weaviate-service-k0s/.github/workflows/codeql-analysis.yml#L18 for running the code scanning.

Add feature/weaviate-service branch to CodeQL analysis
@kupratyu-splunk kupratyu-splunk requested a review from spl-arif April 7, 2026 10:18
@kbhos-splunk
Copy link
Copy Markdown
Collaborator Author

I'll address minio cleanup of unused code in next MR.

@kupratyu-splunk kupratyu-splunk merged commit 2f719ae into feature/weaviate-service Apr 7, 2026
3 checks passed
@kupratyu-splunk kupratyu-splunk deleted the kiran/weaviate-service-k0s branch April 7, 2026 10:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants