From 8f42085c9600f5519a08519c16dbe203fcd28251 Mon Sep 17 00:00:00 2001 From: Rashmi Ranjan Sahoo Date: Sat, 26 Jul 2025 12:38:40 +0530 Subject: [PATCH 1/3] chore(docs): improved grammar, structure, and formatting of README --- README.md | 129 +++++++++++++++++++++++++++++------------------------- 1 file changed, 70 insertions(+), 59 deletions(-) diff --git a/README.md b/README.md index f44b6ea..063d22d 100644 --- a/README.md +++ b/README.md @@ -1,106 +1,117 @@ # MLSysOps Framework -The *MLSysOps Framework* is the open‑source outcome of the EU‑funded MLSysOps -Horizon Europe project (Grant ID 101092912), running from Jan 2023 to Dec 2025. -Its aim is to deliver an AI‑enabled, agent‑based platform for autonomic, -cross‑layer management of compute, storage, and network resources across cloud, -edge, and IoT environments. +The **MLSysOps Framework** is the open-source result of the Horizon Europe _MLSysOps_ project (Grant ID 101092912), running from Jan 2023 to Dec 2025. It delivers an AI-enabled, agent-based platform for autonomic, cross-layer management of compute, storage, and network resources across cloud, edge, and IoT environments. -## Key Objectives +--- -- Provide an *open, AI‑ready framework* for scalable, trustworthy, - explainable system operation across heterogeneous infrastructures. -- Enable *continual ML learning* and retraining during runtime via - hierarchical agents. -- Support *portable, efficient execution* using container innovation and - modular, FaaS-inspired offloading. -- Promote *green, resource‑efficient, and secure operations* while - maintaining `QoS`/`QoE` targets. -- Facilitate realistic evaluation using real-world deployments in smart‑city - and precision‑agriculture scenarios. +## 🚀 Key Objectives -## Core Components +- Provide an **AI-ready, open framework** for scalable and explainable operations across heterogeneous infrastructures. +- Enable **continual ML learning and retraining** at runtime using hierarchical agents. +- Support **portable and efficient execution** via containers and modular, FaaS-style offloading. +- Promote **green, resource-efficient, and secure operations** while maintaining QoS/QoE. +- Enable **realistic evaluation** through deployments in smart city and precision agriculture scenarios. -- Hierarchical Agent Architecture: Interfaces with orchestration/control - systems and exposes an ML‑model API for plug‑and‑play explainable/re-trainable - models. +--- -- Telemetry & Control Knobs: Collects metrics across the continuum and adjusts - configuration (e.g., compute, network, storage, accelerator usage) - dynamically. +## 🧩 Core Components -- Distributed FaaS‑style Executor: Enables function offloading across tiers to - optimize latency, energy, and performance. +- **Hierarchical Agent Architecture** + Interfaces with orchestration/control systems and supports plug-and-play ML models. -- Explainable ML & Reinforcement Learning Module: Offers transparent decisions, - highlighting input factors influencing agent actions. +- **Telemetry & Control Knobs** + Collects metrics and dynamically tunes compute, network, storage, and accelerators. -- Use-cases: Includes real applications focusing on smart cities and agriculture. +- **Distributed FaaS-style Executor** + Offloads functions across layers to optimize latency, energy, and performance. -## Repository Contents +- **Explainable ML & RL Modules** + Provides transparent decision-making and insight into agent behavior. -| Directory | Description | -|----------|-------------| -| `agents/` | Core autonomic agents with policy-based plugins and ML/analytics | -| `orchestrators/` | Scripts to facilitate testbed setup | -| `mlsysops-cli/` | Tool to manage MLSysOps-related descriptors (agents, applications, etc.)| -| `northbound-api/` | Glue API from the CLI to the core Agent framework| -| `docs/` | Internal and public-facing documentation | +- **Real-world Use Cases** + Includes smart city and precision agriculture applications. -## Getting Started +--- + +## 📁 Repository Structure + +| Directory | Description | +| ----------------- | ---------------------------------------------------------------------- | +| `agents/` | Core autonomic agents with policy plugins and ML analytics | +| `orchestrators/` | Scripts and tools for testbed orchestration | +| `mlsysops-cli/` | Command Line Interface to manage agents, applications, and deployments | +| `northbound-api/` | API layer connecting CLI with the core framework | +| `docs/` | Design documents, usage guides, and contribution guidelines | + +--- + +## 🛠️ Getting Started ### Prerequisites -- Kubernetes v1.26+ +- Kubernetes `v1.26+` - `kubectl`, `karmada` -- Python 3.10+ +- Python `3.10+` - Access to a 4-node testbed environment ### Quick Start -Install the CLI tool: +Install the CLI: ```bash pip install mlsysops-cli ``` -Given an `ansible` inventory to setup 4 nodes in `inv.yml`, you can deploy the framework: +Clone the repository: ```bash -mls framework deploy-all --inventory inv.yml +git clone https://github.com/RR-Sahoo/mlsysops-framework.git +cd mlsysops-framework ``` -Create and deploy an example application: +Launch the test environment: ```bash -mls framework create-app-test-description -mls apps deploy-app --path mlsysops-app-test-description.yaml +make deploy-testbed ``` -See docs/ for detailed component setup guides. +Use the CLI: -## Documentation +```bash +mlsysops-cli --help +``` -Check the full documentation at [docs.mlsysops.eu](https://docs.mlsysops.eu) +--- -## Contributing +## 🤝 Contributing We welcome contributions from the community! -Browse [good first issues](https://github.com/mlsysops-eu/mlsysops-framework/issues?q=is%3Aissue%20state%3Aopen%20label%3Agood-first-issue) +- See [CONTRIBUTING.md](docs/CONTRIBUTING.md) for guidelines. +- Read our [Code of Conduct](docs/CODE_OF_CONDUCT.md). +- For design details, refer to [docs/design-overview.md](docs/design-overview.md). + +--- + +## 👥 Maintainers + +The MLSysOps Framework is maintained by: + +- [Your Name / Org 1](mailto:your.email@example.com) +- [Your Name / Org 2](mailto:another.email@example.com) + +See [MAINTAINERS.md](docs/MAINTAINERS.md) for the full list. -Review our [CONTRIBUTING.md](https://docs.mlsysops.eu/developer-guide/contribute/) +--- -Follow our [CODE_OF_CONDUCT.md](https://github.com/mlsysops-eu/mlsysops-framework/blob/main/docs/developer-guide/Code-of-Conduct.md) +## 📄 License -## License +This project is licensed under the Apache 2.0 License. See the [LICENSE](LICENSE) file for details. -This project is licensed under the Apache 2.0 License. +--- -## Acknowledgements +## 📢 Acknowledgements -This framework is developed as part of the Horizon Europe MLSysOps Project -(Grant ID 101092912), coordinated by the University of Thessaly, with -contributions from 12 European partners across academia and industry. +This work is funded by the European Union’s Horizon Europe program under Grant Agreement No. 101092912 (MLSysOps). -Learn more at [mlsysops.eu](https://mlsysops.eu) +--- From 83a4148b3336f551fd641e1e2a2cc10d059ec05f Mon Sep 17 00:00:00 2001 From: Rashmi Ranjan Sahoo Date: Sat, 26 Jul 2025 13:03:39 +0530 Subject: [PATCH 2/3] docs: update CONTRIBUTING and developer guide for clearer onboarding --- CONTRIBUTING.md | 14 +- docs/developer-guide/contribute.md | 298 +++++++++++++--------------- docs/developer-guide/development.md | 34 +++- 3 files changed, 173 insertions(+), 173 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 94502f3..129bdf9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,3 +1,13 @@ -# Contribute to the `MLSysOps` framework +# Contributing to the MLSysOps Framework -Please see [Contributing](/docs/developer-guide/contribute.md) +Thank you for your interest in contributing to the MLSysOps project! + +To ensure a smooth contribution process, please review our [Contributing Guide](docs/developer-guide/contribute.md), which includes: + +- How to set up your development environment +- Coding conventions and style guidelines +- How to submit issues and pull requests +- Review and approval process +- Community standards + +We appreciate your time and effort in helping us improve MLSysOps. diff --git a/docs/developer-guide/contribute.md b/docs/developer-guide/contribute.md index 4129bf3..64b971c 100755 --- a/docs/developer-guide/contribute.md +++ b/docs/developer-guide/contribute.md @@ -4,188 +4,158 @@ title: "Contributing" description: "Contributing guidelines" --- -The MLSysOps framework is an open-source project licensed under the [Apache -License -2.0](https://github.com/mlsysops-eu/mlsysops-framework/blob/main/LICENSE). -We welcome anyone who would be interested in contributing to `MLSysOps framework`. -As a first step, please take a look at the following document. -The current document provides a high level overview of `MLSysOps framework`'s code structure, along with a few guidelines regarding contributions to the project. +# 🛠️ How to Contribute -## Table of contents: +Welcome! 👋 We're thrilled you're considering contributing to the **MLSysOps Framework**. This guide will help you get started. -1. [Code organization](#code-organization) -2. [How to contribute](#how-to-contribute) -3. [Opening an issue](#opening-an-issue) -4. [Requesting new features](#requesting-new-features) -5. [Submitting a PR](#submitting-a-pr) -6. [Style guide](#style-guide) -7. [Contact](#contact) +--- + +## 📌 Table of Contents + +- [Ground Rules](#-ground-rules) +- [Code of Conduct](#-code-of-conduct) +- [Ways to Contribute](#-ways-to-contribute) +- [How to Submit a PR](#-how-to-submit-a-pr) +- [Writing Good Commit Messages](#-writing-good-commit-messages) +- [Issue Labels](#-issue-labels) +- [Branching Strategy](#-branching-strategy) +- [Contact](#-contact) -## Code organization +--- -The MLSysOps framework is structured as follows: +## 📋 Ground Rules -TBC +- Be respectful and considerate to others. +- Make sure your contributions align with the project's goals. +- Write clear, concise, and grammatically correct Markdown. +- Ensure documentation and code are easy to read and understand. +- When in doubt, open an issue and ask for guidance. -## How to contribute +--- -There are plenty of ways to contribute to an open source project, even without changing or touching the code. -Therefore, anyone who is interested in this project is very welcome to contribute in one of the following ways: +## 🤝 Code of Conduct -1. Using `MLSysOps framework`. Try it out yourself and let us know your experience. Did everything work well? Were the instructions clear? -2. Improve or suggest changes to the documentation of the project. Documentation is very important for every project, hence any ideas on how to improve the documentation to make it more clear are more than welcome. -3. Request new features. Any proposals for improving or adding new features are very welcome. -4. Find a bug and report it. Bugs are everywhere and some are hidden very well. As a result, we would really appreciate it if someone found a bug and reported it to the maintainers. -5. Make changes to the code. Improve the code, add new functionalities and make `MLSysOps framework` even more useful. +By participating in this project, you agree to abide by our [Code of Conduct](../../CODE_OF_CONDUCT.md). Please take a moment to read it. -## Opening an issue +--- + +## ✨ Ways to Contribute + +We welcome contributions of all kinds, including: + +- 📖 Improving documentation and fixing typos +- 💡 Suggesting new features or enhancements +- 🐛 Reporting bugs +- 🧪 Writing or improving tests +- 💻 Fixing issues +- 🌐 Translating the project +- 🧱 Improving code structure or consistency + +--- -We use Github issues to track bugs and requests for new features. -Anyone is welcome to open a new issue, which is either related to a bug or a request for a new feature. +## 🔀 How to Submit a PR -### Reporting bugs +1. **Fork the repository** + Click the **Fork** button on the top-right of this repo's GitHub page. -In order to report a bug or misbehavior in `MLSysOps framework`, a user can open a new issue explaining the problem. -For the time being, we do not use any strict template for reporting any issues. -However, in order to easily identify and fix the problem, it would be very helpful to provide enough information. -In that context, when opening a new issue regarding a bug, we kindly ask you to: +2. **Clone your forked repo** -- Mark the issue with the bug label -- Provide the following information: + ```bash + git clone https://github.com/YOUR_USERNAME/mlsysops-framework.git + cd mlsysops-framework + ``` + +3. **Create a new branch** + + ```bash + git checkout -b your-feature-name + ``` + +4. **Make your changes** + Ensure your code follows existing style guidelines and passes all checks. + +5. **Commit your changes** + + ```bash + git commit -m "feat: Add a descriptive message" + ``` + +6. **Push your branch** + + ```bash + git push origin your-feature-name + ``` + +7. **Open a Pull Request** + Go to the GitHub page of your fork, and click "Compare & pull request". + Provide a clear title and description of your changes. + +--- - 1. A short description of the bug. - 2. The respective logs both from the output and containerd. - 3. Framework's version manifest (either the commit hash or the version manifest file). - 4. The execution environment (CPU architecture, VMM etc.). - 5. Any particular steps to reproduce the issue. -- Keep an eye on the issue for possible questions from the maintainers. +## ✍️ Writing Good Commit Messages -A template for an issue could be the following one: +Follow the conventional commit format: + +```bash +(scope): short description ``` -## Description -An explanation of the issue -## System info +**Examples:** + +- `feat(docs): add contributing guidelines` +- `fix(auth): resolve login redirect issue` +- `chore: update dependencies` + +**Types:** + +- `feat` – A new feature +- `fix` – A bug fix +- `docs` – Documentation only changes +- `style` – Changes that do not affect the meaning of the code +- `refactor` – Code changes that neither fix a bug nor add a feature +- `test` – Adding missing tests +- `chore` – Changes to the build process or tools + +--- + +## 🏷️ Issue Labels + +Our GitHub issues use the following labels to help contributors find where they can help: + +- `good first issue` – Great for newcomers! +- `help wanted` – We need help with this. +- `bug` – Confirmed bugs. +- `enhancement` – Feature or improvement requests. +- `question` – Discussions or clarifications. + +--- + +## 🌱 Branching Strategy + +We follow a simple Git branching model: + +- `main` – Stable, production-ready code. +- `dev` – Development branch. All features and bug fixes should branch from and merge back into dev. + +**Example workflow:** -- Version: -- Arch: -- VMM: -- ... +```bash +# Start from dev +git checkout dev +git pull -## Steps to reproduce -A list of steps that can reproduce the issue. +# Create your feature branch +git checkout -b feat/component-xyz ``` -### Requesting new features - -We will be very happy to listen from users about new features that they would like to see in `MLSysOps framework`. -One way to communicate such a request is using Github issues. -For the time being, we do not use any strict template for requesting new features. -However, we kindly ask you to mark the issue with the enhancement label and provide a description of the new feature. - -## Submitting a PR - -Anyone should feel free to submit a change or an addition to the codebase of `MLSysOps framework`. -Currently, we use Github's Pull Requests (PRs) to submit changes to `MLSysOps framework`'s codebase. -Before creating a new PR, please follow the guidelines below: - -- Make sure that the changes do not break the building process of `MLSysOps framework`. -- Make sure that all the tests run successfully. -- Make sure that no commit in a PR breaks the building process of `MLSysOps framework` -- Make sure to sign-off your commits. -- Provide meaningful commit messages, describing shortly the changes. -- Provide a meaningful PR message - -As soon as a new PR is created the following workflow will take place: - - 1. The creator of the PR should invoke the tests by adding the `ok-to-test` label. - 2. If the tests pass, request from one or more `MLSysOps framework`'s [maintainers](https://github.com/nubificus/MLSysOps framework/blob/main/MAINTAINERS) to review the PR. - 3. The reviewers submit their review. - 4. The author of the PR should address all the comments from the reviewers. - 5. As soon as a reviewer approves the PR, an action will add the appropriate git trailers in the PR's commits. - 6. The reviewer who accepted the changes will merge the new changes. - -## Labels for the CI - -We use github workflows to invoke some tests when a new PR opens for `MLSysOps framework`. -In particular, we perform the following workflows tests: - -- Linting of the commit message. Please check the [git commit message style](#git-commit-messages) below for more info. -- Spell check, since `MLSysOps framework` repository contains its documentation too. -- License check -- Code linting. -- Building artifacts for amd64 and aarch64. -- Unit tests -- End-to-end tests - -For a better control over the tests and workflows that run in a PR, we define -three labels which can be used: - -- `ok-to-test`: Runs a full CI workflow, meaning all lint tests (commit - message, spellcheck, license), Code linting, building for x86 and aarch64, - unit tests and at last end-to-end tests. -- `skip-build`: Skips the building workflows along with unit and end-to end tests - running all the linting tests. This is useful when - the PR is related to docs and it can help for catching spelling errors etc. In - addition, if the changes are not related to the codebase, running the - end-to-end tests is not required and saves some time. -- `skip-lint`: Skips the linting phase. This is particularly useful on draft - PRs, when we want to just test the functionality of the code (either a bug - fix, or a new feature) and defer the cleanup/polishing of commits, code, and - docs to when the PR will be ready for review. - -**Note**: Both `skip-build` and `skip-lint` assume that the `ok-to-test` label -is added. - -## Style guide - -### Git commit messages - -Please follow the below guidelines for your commit messages: - -- Limit the first line to 72 characters or less. -- Limit all the other lines to 80 characters -- Follow the [Conventional Commits](https://www.conventionalcommits.org/) - specification and, specifically, format the header as `[optional scope]: - `, where `description` must not end with a fullstop and `type` - can be one of: - - - *feat*: A new feature - - *fix*: A bug fix - - *docs*: Documentation only changes - - *style*: Changes that do not affect the meaning of the code (white-space, - formatting, missing semi-colons, etc) - - *refactor*: A code change that neither fixes a bug nor adds a feature - - *`perf`*: A code change that improves performance - - *test*: Adding missing tests - - *build*: Changes that affect the build system or external dependencies - (example scopes: gulp, broccoli, `npm`) - - *ci*: Changes to our CI configuration files and scripts (example scopes: - `Travis`, `Circle`, `BrowserStack`, `SauceLabs`) - - *chore*: Other changes that don't modify src or test files - - *revert*: Reverts a previous commit -- In case the PR is associated with an issue, please refer to it, using the git trailer `Fixes: #Nr_issue` -- Always sign-off your commit message - -Since the MLSysOps framework comprises code written in various programming -languages we use the following styles for each: - -### Golang code style - -We follow `gofmt`'s rules on formatting GO code. Therefore, we ask all -contributors to do the same. Go provides the `gofmt` tool, which can be used -for formatting your code. - -### Python -TBC - -### C -TBC - -## Contact - -Feel free to contact any of the -[maintainers](https://github.com/mlsysops-eu/mlsysops-framework/blob/main/MAINTAINERS) -or using one of the below email addresses: - -- info@mlsysops.eu +--- + +## 📬 Contact + +If you have questions, suggestions, or just want to say hi, feel free to: + +- Open an issue +- Start a discussion +- Reach out via email + +Thank you for contributing to MLSysOps Framework! 💙 diff --git a/docs/developer-guide/development.md b/docs/developer-guide/development.md index d9a6c3c..535d400 100755 --- a/docs/developer-guide/development.md +++ b/docs/developer-guide/development.md @@ -1,14 +1,34 @@ -title: Setup a Dev environment ------- +# Setup a Development Environment -Most of the steps are covered in the [installation](../../installation) document. -Please refer to it for: +Most of the environment setup steps are already covered in the [Installation Guide](../../installation.md). Please follow that document first. -TBC +--- -The next step is to clone and build the `MLSysOps framework`: +## Clone the MLSysOps Framework + +Once you've completed the prerequisites, clone the repository and navigate into it: ```bash -git clone https://github.com/mlsysops-eu/mlsysops-framework +git clone https://github.com/mlsysops-eu/mlsysops-framework.git cd mlsysops-framework ``` + +--- + +## Build the Framework + +Depending on your development needs, you may want to build various components or run them locally. Follow the instructions in the respective README files of each module (e.g., mlsysops-cli, agents, etc.) for component-specific development workflows. + +--- + +## Recommendations + +- Use a Python virtual environment for CLI or backend development. +- Use pre-commit to enforce linting and formatting rules (see Contributing Guide). +- For advanced setup like deploying agents, check relevant module README files under the `agents/` or `mlsysops-cli/` directories. + +--- + +## Need Help? + +If you encounter issues during setup, feel free to [open an issue](https://github.com/mlsysops-eu/mlsysops-framework/issues) or refer to the [Developer Guide Index](../developer-guide/index.md). From f7762c1cfc51a12f071e6142f4340cd63e012140 Mon Sep 17 00:00:00 2001 From: Rashmi Ranjan Sahoo Date: Sat, 26 Jul 2025 14:19:53 +0530 Subject: [PATCH 3/3] chore: revert CONTRIBUTING.md changes to avoid conflict --- CONTRIBUTING.md | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 129bdf9..94502f3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,13 +1,3 @@ -# Contributing to the MLSysOps Framework +# Contribute to the `MLSysOps` framework -Thank you for your interest in contributing to the MLSysOps project! - -To ensure a smooth contribution process, please review our [Contributing Guide](docs/developer-guide/contribute.md), which includes: - -- How to set up your development environment -- Coding conventions and style guidelines -- How to submit issues and pull requests -- Review and approval process -- Community standards - -We appreciate your time and effort in helping us improve MLSysOps. +Please see [Contributing](/docs/developer-guide/contribute.md)