docs(install): add documentation for generic Kubernetes deployment#408
docs(install): add documentation for generic Kubernetes deployment#408lexfrei wants to merge 3 commits intocozystack:mainfrom
Conversation
Add comprehensive guide for deploying Cozystack on non-Talos Kubernetes distributions (k3s, kubeadm, RKE2) using the isp-full-generic bundle. Documentation includes: - Host and sysctl requirements - Kubernetes configuration for each distribution - Step-by-step installation instructions - Example Ansible playbooks - Troubleshooting section Related: cozystack/cozystack#1950 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
✅ Deploy Preview for cozystack ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📝 WalkthroughWalkthroughThe PR restructures the Kubernetes installation docs to be distribution-agnostic: it renames and broadens the index page, separates Talos-specific guidance into its own subsection, and adds a comprehensive new guide for deploying Cozystack on generic Kubernetes distributions (k3s, kubeadm, RKE2, etc.). Changes
Sequence Diagram(s)(omitted — changes are documentation-only and do not introduce new multi-component control flow) Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related issues
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @lexfrei, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances Cozystack's documentation by providing a dedicated guide for deploying the platform on generic Kubernetes distributions. This expansion allows users to integrate Cozystack with existing Kubernetes clusters or choose non-Talos environments, offering greater flexibility and accessibility beyond the previously recommended Talos Linux setup. The new guide ensures users have all necessary information, from prerequisites to troubleshooting, for a successful deployment. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds valuable documentation for deploying Cozystack on generic Kubernetes distributions. The new guide is comprehensive, covering prerequisites, host configuration, Kubernetes setup for different distributions, and installation steps, complete with Ansible examples and a troubleshooting section. The changes are well-structured and clear. I've identified a few areas for improvement in the new documentation to enhance correctness and prevent potential user issues, including a likely version typo, potentially risky sysctl values, and inconsistent network configuration examples.
| fs.inotify.max_user_watches = 2099999999 | ||
| fs.inotify.max_user_instances = 2099999999 | ||
| fs.inotify.max_queued_events = 2099999999 |
There was a problem hiding this comment.
The inotify limits are set to an extremely high and identical value of 2099999999. This is likely a copy-paste error and poses a risk of kernel memory exhaustion, as each watch consumes non-swappable kernel memory. The values for max_user_watches, max_user_instances, and max_queued_events serve different purposes and should not be identical.
It is recommended to use more conventional and safer values, for example:
fs.inotify.max_user_watches = 524288fs.inotify.max_user_instances = 512fs.inotify.max_queued_events = 16384
Please update these values in both the 99-cozystack.conf example and the Ansible playbook example (lines 266-268).
| fs.inotify.max_user_watches = 2099999999 | |
| fs.inotify.max_user_instances = 2099999999 | |
| fs.inotify.max_queued_events = 2099999999 | |
| fs.inotify.max_user_watches = 524288 | |
| fs.inotify.max_user_instances = 512 | |
| fs.inotify.max_queued_events = 16384 |
There was a problem hiding this comment.
These values are intentional and required for Cozystack.
Cozystack runs FluxCD, multiple controllers, LINSTOR, KubeOVN/Cilium, and nested Kubernetes clusters — all creating massive inotify watch counts. Standard values (524288) cause too many open files errors.
These values were validated in production. They work because:
- Watches are allocated on-demand, not pre-allocated
- Kernel only uses memory for actual watches
- Cozystack architecture genuinely requires high counts
| ipv4-pod-cidr: "10.42.0.0/16" | ||
| ipv4-pod-gateway: "10.42.0.1" | ||
| ipv4-svc-cidr: "10.43.0.0/16" |
There was a problem hiding this comment.
The CIDR ranges in the cozystack-config.yaml example are inconsistent with those provided in the kubeadm configuration example. The documentation correctly states that these values must match the Kubernetes cluster configuration. To prevent deployment failures for users following the guide, these examples should be consistent.
kubeadmconfig usespodSubnet: "10.244.0.0/16"andserviceSubnet: "10.96.0.0/16".cozystack-config.yamlexample usesipv4-pod-cidr: "10.42.0.0/16"andipv4-svc-cidr: "10.43.0.0/16".
The same inconsistency is present in the Ansible deployment playbook variables (lines 295-296). I suggest aligning the cozystack-config.yaml and Ansible playbook examples with the kubeadm configuration.
| ipv4-pod-cidr: "10.42.0.0/16" | |
| ipv4-pod-gateway: "10.42.0.1" | |
| ipv4-svc-cidr: "10.43.0.0/16" | |
| ipv4-pod-cidr: "10.244.0.0/16" | |
| ipv4-pod-gateway: "10.244.0.1" | |
| ipv4-svc-cidr: "10.96.0.0/16" |
There was a problem hiding this comment.
Fixed in 95848e8. Added explicit warning about distribution-specific CIDR defaults:
- k3s:
10.42.0.0/16,10.43.0.0/16 - kubeadm:
10.244.0.0/16,10.96.0.0/16 - RKE2:
10.42.0.0/16,10.43.0.0/16
The example now clearly states it's for k3s, with a note to adjust for other distributions.
|
|
||
| Cozystack has been tested on: | ||
|
|
||
| - **k3s** v1.32+ (recommended for single-node and edge deployments) |
There was a problem hiding this comment.
The supported k3s version v1.32+ appears to be a typo, as this version of Kubernetes (and therefore k3s) has not been released. The latest stable Kubernetes version is v1.30. Please verify and correct the minimum supported k3s version. For consistency with kubeadm and RKE2 in this document, v1.28+ would be a reasonable value.
| - **k3s** v1.32+ (recommended for single-node and edge deployments) | |
| - **k3s** v1.28+ (recommended for single-node and edge deployments) |
There was a problem hiding this comment.
k3s v1.32+ is correct. Kubernetes 1.32 was released in December 2024, and k3s versions follow Kubernetes releases.
This documentation was written and tested with k3s v1.35 which is current. The v1.32+ requirement reflects the minimum tested version for the isp-full-generic bundle.
Add explicit warning that pod/service CIDRs must match the Kubernetes distribution defaults: - k3s: 10.42.0.0/16, 10.43.0.0/16 - kubeadm: 10.244.0.0/16, 10.96.0.0/16 - RKE2: 10.42.0.0/16, 10.43.0.0/16 Note: inotify limits (2099999999) are intentional and required for Cozystack operation - standard values are insufficient. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@content/en/docs/install/kubernetes/_index.md`:
- Around line 27-29: The phrase "command line" in the third list item should be
hyphenated as "command-line"; update the text in the list item that currently
reads "[Using talosctl]({{% ref \"./talosctl\" %}}), a specialized command line
tool for managing Talos" to use "command-line" so the item reads that talosctl
is "a specialized command-line tool for managing Talos."
In `@content/en/docs/install/kubernetes/generic.md`:
- Around line 97-104: The table separator rows use compact pipes (e.g.
|-----------|-------------|) and trigger MD060; update the separator lines for
both Markdown tables so there are spaces around each pipe (e.g. | ----------- |
------------- |) to satisfy markdownlint. Locate the header/separator rows for
the two tables (the lines immediately under the header row with dashes) and add
a single space before and after each pipe character; keep the dash counts
unchanged so column widths remain the same.
🧹 Nitpick comments (2)
content/en/docs/install/kubernetes/generic.md (2)
166-168: Pin Cozystack manifests to a specific version for reproducible installs.Using
releases/latestcan change behavior without notice. Consider pinning to a version and documenting how to upgrade.♻️ Proposed change (example)
-kubectl apply -f https://github.com/cozystack/cozystack/releases/latest/download/cozystack-crds.yaml +kubectl apply -f https://github.com/cozystack/cozystack/releases/download/vX.Y.Z/cozystack-crds.yaml-kubectl apply -f https://github.com/cozystack/cozystack/releases/latest/download/cozystack-operator-generic.yaml +kubectl apply -f https://github.com/cozystack/cozystack/releases/download/vX.Y.Z/cozystack-operator-generic.yaml- kubernetes.core.k8s: - src: https://github.com/cozystack/cozystack/releases/latest/download/cozystack-crds.yaml + kubernetes.core.k8s: + src: https://github.com/cozystack/cozystack/releases/download/vX.Y.Z/cozystack-crds.yaml- kubernetes.core.k8s: - src: https://github.com/cozystack/cozystack/releases/latest/download/cozystack-operator-generic.yaml + kubernetes.core.k8s: + src: https://github.com/cozystack/cozystack/releases/download/vX.Y.Z/cozystack-operator-generic.yamlAlso applies to: 217-219, 299-301, 331-333
322-327: Document the Ansible collection dependency foransible.utils.ipaddr.The filter used to compute
ipv4-pod-gatewayrequires theansible.utilscollection. If it isn’t present, the playbook fails. Please add a note in prerequisites or replace the computation with a static value.
- Hyphenate "command-line" in _index.md - Add spaces to table separators (MD060) Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
Any possible to automate everything with k3s server manifests? Running steps of commands isn't the k3s way. |
|
@cheese in progress. Ansible collection will be published later. |
2 participants
Summary
Add comprehensive documentation for deploying Cozystack on non-Talos Kubernetes distributions (k3s, kubeadm, RKE2) using the
isp-full-genericbundle introduced in PR cozystack/cozystack#1939.Changes
content/en/docs/install/kubernetes/generic.mdcontent/en/docs/install/kubernetes/_index.mdto include link to generic guideDocumentation Includes
Related
isp-full-genericbundleTest Plan
Summary by CodeRabbit