diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 80c4471325..2e65517964 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -10,32 +10,33 @@ The key directories are: ### Top level structure -/content - The main directory containing all Learning Paths and install guides as markdown files -/themes - HTML templates and styling elements that render the content into the final website -/tools - Python scripts for automated website integrity checking -config.toml - High-level Hugo configuration settings +- `/content` - The main directory containing all Learning Paths and install guides as markdown files +- `/themes` - HTML templates and styling elements that render the content into the final website +- `/tools` - Python scripts for automated website integrity checking +- `config.toml` - High-level Hugo configuration settings -### Content organization: +### Content organization -The /content directory is organized into: +The `/content` directory is organized into: -- learning-paths/ Core learning content organized by categories: - -- embedded-and-microcontrollers/ MCU, IoT, and embedded development topics - -- servers-and-cloud-computing/ Server, cloud, and enterprise computing topics - -- mobile-graphics-and-gaming/ Mobile app development, graphics, and gaming - -- cross-platform/ Cross-platform development and general programming topics, these appear in multiple categories on the website - -- laptops-and-desktops/ Desktop application development, primarily Windows on Arm and macOS - -- automotive/ Automotive and ADAS development - -- iot/ IoT-specific Learning Paths +- `learning-paths/` - Core learning content organized by categories: + - `embedded-and-microcontrollers/` - MCU, IoT, and embedded development topics + - `servers-and-cloud-computing/` - Server, cloud, and enterprise computing topics + - `mobile-graphics-and-gaming/` - Mobile app development, graphics, and gaming + - `cross-platform/` - Cross-platform development and general programming topics, these appear in multiple categories on the website + - `laptops-and-desktops/` - Desktop application development, primarily Windows on Arm and macOS + - `automotive/` - Automotive and ADAS development + - `iot/` - IoT-specific Learning Paths -- install-guides/ - Tool installation guides with supporting subdirectories organized by tool categories like docker/, gcc/, license/, browsers/, plus an _images/ directory for screenshots and diagrams +- `install-guides/` - Tool installation guides with supporting subdirectories organized by tool categories like `docker/`, `gcc/`, `license/`, `browsers/`, plus an `_images/` directory for screenshots and diagrams These are special directories and not used for regular content creation: - migration/ Migration guides and resources, this maps to https://learn.arm.com/migration - lists/ Content listing and organization files, this maps to https://learn.arm.com/lists - stats/ Website statistics and analytics, this maps to https://learn.arm.com/stats -The /content directory is the primary workspace where contributors add new Learning Paths as markdown files, organized into category-specific subdirectories that correspond to the different learning path topics available on the site at https://learn.arm.com/. +- `migration/` - Migration guides and resources, this maps to https://learn.arm.com/migration +- `lists/` - Content listing and organization files, this maps to https://learn.arm.com/lists +- `stats/` - Website statistics and analytics, this maps to https://learn.arm.com/stats + +The `/content` directory is the primary workspace where contributors add new Learning Paths as markdown files, organized into category-specific subdirectories that correspond to the different learning path topics available on the site at https://learn.arm.com/. ## Content requirements @@ -43,19 +44,20 @@ Read the files in the directory `content/learning-paths/cross-platform/_example- ### Content structure -Each Learning Path must have an _index.md file and a _next-steps.md file. The _index.md file contains the main content of the Learning Path. The _next-steps.md file contains links to related content and is included at the end of the Learning Path. +Each Learning Path must have an `_index.md` file and a `_next-steps.md` file. The `_index.md` file contains the main content of the Learning Path. The `_next-steps.md` file contains links to related content and is included at the end of the Learning Path. Additional resources and 'next steps' content should be placed in the `further_reading` section of `_index.md`, NOT in `_next-steps.md`. The `_next-steps.md` file should remain minimal and unmodified as indicated by "FIXED, DO NOT MODIFY" comments in the template. -The _index.md file should contain the following front matter and content sections: +The `_index.md` file should contain the following front matter and content sections: Front Matter (YAML format): + - `title`: Imperative heading following the [verb] + [technology] + [outcome] format -- `weight`: Numerical ordering for display sequence, weight is 1 for _index.md and each page is ordered by weight, no markdown files should have the same weight in a directory +- `weight`: Numerical ordering for display sequence, weight is 1 for `_index.md` and each page is ordered by weight, no markdown files should have the same weight in a directory - `layout`: Template type (usually "learningpathall") - `minutes_to_complete`: Realistic time estimate for completion - `prerequisites`: List of required knowledge, tools, or prior learning paths -- `author`: Main contributor's name, multiple authors can be listed separated using - on new lines +- `author`: Main contributor's name, multiple authors can be listed separated using `-` on new lines - `subjects`: Technology categories for filtering and search, this is a closed list and must match one of the subjects listed on https://learn.arm.com/learning-paths/cross-platform/_example-learning-path/write-2-metadata/ - `armips`: Relevant Arm IP, stick to Neoverse, Cortex-A, Cortex-M, etc. Don't list specific CPU models or Arm architecture versions - `tools_software_languages`: Open category listing Programming languages, frameworks, and development tools used @@ -64,7 +66,8 @@ Front Matter (YAML format): ### Further reading curation -Limit further_reading resources to four to six essential links. Prioritize: +Limit `further_reading` resources to four to six essential links. Prioritize: + - Direct relevance to the topic - Arm-specific Learning Paths over generic external resources - Foundation knowledge for target audience @@ -74,14 +77,16 @@ Limit further_reading resources to four to six essential links. Prioritize: Avoid overwhelming readers with too many links, which can cause them to leave the platform. All Learning Paths should generally include: -Title: [Imperative verb] + [technology/tool] + [outcome] -Introduction paragraph: Context + user goal + value proposition -Prerequisites section with explicit requirements and links -Learning objectives: three to four bulleted, measurable outcomes with action verbs -Step-by-step sections with logical progression -Clear next steps/conclusion + +- Title: [Imperative verb] + [technology/tool] + [outcome] +- Introduction paragraph: Context + user goal + value proposition +- Prerequisites section with explicit requirements and links +- Learning objectives: three to four bulleted, measurable outcomes with action verbs +- Step-by-step sections with logical progression +- Clear next steps/conclusion For title formatting: + - MUST use imperative voice ("Deploy", "Configure", "Build", "Create") - MUST include SEO keywords (technology names, tools) - Examples: "Deploy applications on Arm servers", "Configure Arm processors for optimal performance" @@ -91,19 +96,21 @@ The term "Learning Path" should always be capitalized. ### Writing style Voice and Tone: + - Second person ("you", "your") - NEVER first person ("I", "we") - Active voice - AVOID passive constructions - Present tense for descriptions - Imperative mood for commands - Confident and developer-friendly tone - Encouraging language for complex tasks -- Use inclusive language: +- Use inclusive language: - Use "primary/subordinate" instead of "master/slave" terminology - Don't use gendered examples or assumptions - Be mindful of cultural references that might not translate globally - Focus on clear, accessible language for all developers ### Sentence structure and clarity + - Average 15-20 words per sentence - Split complex sentences for scalability - Plain English - avoid jargon overload @@ -113,6 +120,7 @@ Voice and Tone: - Parallel structure in all lists ### Readability and section flow + - Flag any section over 700 words and suggest natural split points - Warn if more than 300 words appear between code examples - Identify paragraphs with sentences averaging over 20 words @@ -127,11 +135,21 @@ Voice and Tone: - Walls of text cause people to bounce from the page - If you're explaining 3+ things in one section, split it into separate sections - Each code block should be preceded by one to three sentences explaining what it does. +- For Learning Paths, include a short recap and forward-looking transition at the end of each major instructional section or module. Use a consistent heading such as: + + ## What you’ve accomplished and what’s next: + + In this section: + - Briefly summarize what the user has learned or completed + - Briefly describe what the user should expect in the next section or suggest further exploration + + Keep this concise and encouraging. Do not repeat earlier content verbatim. ### Word choice and style + - Use these preferred terms and phrases for consistency: - Numbers and units: Spell out numbers one through five (one, two, three, four, five), after this use numerals (6, 7, 8...). Use proper spacing for units: "1 GB", "23 MB/day" (not "1GB", "23MB/day"). Use "K" for thousands: "64K" (not "64k"). Use abbreviations for data rates: "Gbps" (not "Gb per second"). - - Common phrases: "To [action]" (not "Follow the steps below to [action]"), "for example" (not "e.g."), "that is" (not "i.e."), "because" (not "since"), "also" (not "in addition"), "to" (not "in order to"), "see" (not "refer to"), "use" (not "utilize" or "leverage"), "need" (not "require"), "can" or "might" (not "may"), "set up" as verb, "setup" as noun, "therefore" (not "ergo"), "namely" (not "viz."), "avoid" (not "try not to"). + - Common phrases: "To [action]" (not "Follow the steps below to [action]"), "for example" (not "e.g."), "that is" (not "i.e."), "because" (not "since"), "also" (not "in addition"), "to" (not "in order to"), "see" (not "refer to"), "use" (not "utilize" or "leverage"), "need" (not "require"), "can" or "might" (not "may"), "set up" as verb, "setup" as noun, "therefore" (not "ergo"), "namely" (not "viz."), "avoid" (not "try not to"). - Avoid condescending language: Don't use "simply", "just", "obviously", "clearly" - what's simple to you might not be to the learner. - Acknowledge when something can be tricky: Use phrases like "this step can be confusing at first" to validate learner experience. - Normalize errors: Use phrases like "if you see this error, here's how to fix it" to reassure learners that errors are part of the learning process. @@ -141,6 +159,7 @@ Voice and Tone: ## Content structure and consistency ### Cross-file and quality assurance + - Use the same technical terms consistently throughout all sections - Apply the word choice and style guidelines uniformly across all files - Maintain consistent capitalization of product names, technologies, and concepts @@ -166,6 +185,7 @@ Voice and Tone: ## Formatting and code samples ### Heading guidelines + - Use sentence case for all headings (first word capitalized, rest lowercase except proper nouns) - Heading types: - Conceptual headings: When explaining technology/motivation ("What is containerization?") @@ -178,6 +198,7 @@ Voice and Tone: - H4: Specific technical details or troubleshooting ### Code samples and formatting + - ALWAYS provide explanation before code blocks - Format: [What it does] → [Code] → [Expected outcome] → [Key parameters] - Use markdown tags for programming languages like bash, python, yaml, json, etc. @@ -188,13 +209,20 @@ Voice and Tone: - Use shortcodes for common pitfalls, warnings, important notes. ## Arm naming and architecture terms + - Use Arm for the brand in prose (for example, "Arm processors", "Arm servers"). - Use arm64 or aarch64 for the CPU architecture; these are acceptable and interchangeable labels. Prefer whichever term a tool, package, or OS uses natively. - Always use "Arm" (not "ARM") in all contexts except when referring to specific technical terms that require the original casing. - ARM64 is used by Windows on Arm and Microsoft documentation, so it is acceptable to use ARM64 when specifically referring to Windows on Arm. - In code blocks, CLI flags, package names, file paths, and outputs, keep the exact casing used by the tool (for example, --arch arm64, uname -m → aarch64). +## Product name emphasis + +- Product names and technical technologies (for example, LiteRT, XNNPACK, KleidiAI, SME2) should appear in regular text. +- Avoid using italics or bold to emphasize product or technology names unless they are part of a heading or a UI label. + ## Hyperlink guidelines + - Use the full path format for internal links: `/learning-paths/category/path-name/` (e.g., `/learning-paths/cross-platform/docker/`). Do NOT use relative paths like `../path-name/`. - Use the full URL for external links that are not on learn.arm.com, these open in a new tab. - When creating Learning Path content: @@ -206,32 +234,47 @@ Voice and Tone: - Some links are useful in content, but too many links can be distracting and readers will leave the platform following them. Include only necessary links in the content; place others in the "Next Steps" section at the end. Flag any page with too many links for review. ## Avoid looking like AI-generated content + - Warning signs of over-bulleting: More than 3 consecutive sections using bullet lists, bullet points that could be combined into narrative paragraphs, lists where items don't have parallel structure, bullet points that are actually full sentences better suited for paragraphs. - Use flowing narrative instead of excessive bullets. - Use natural writing patterns: Vary sentence length, use transitional phrases, include contextual explanations, add relevant examples, connect ideas logically. - Use conversational elements: Instead of "Execute the following command:", use "Now that you've configured the environment, run the following command to start the service:". Instead of "This provides benefits:", use "You'll notice several advantages with this approach, particularly when working with...". +### Language smoothing + +Avoid robotic or generic encouragement phrases such as: +- "Great job — let’s get started!" +- "Great job — your environment is ready!" + +Use calm, natural transitions that focus on what happens next. + +Also avoid multiple consecutive sentences starting with "This" (for example, "This ensures...", "This creates...", "This provides..."). Vary sentence structure to maintain natural flow. + ## AI-specific guidelines for content creation and editing ### Context awareness + - Consider the learner's likely environment (development vs. production, local vs. cloud) - Recognize when content assumes x86 defaults and suggest Arm alternatives - Flag when third-party tools may have limited Arm support - Suggest Arm-native alternatives when available (e.g., Arm compilers, optimized libraries) ### Technical depth consistency + - Maintain appropriate complexity level throughout the Learning Path - Avoid oversimplifying for Advanced skill level content - Don't assume prior knowledge beyond stated prerequisites - Balance theoretical explanation with practical implementation ### Platform-specific considerations + - Default to Arm-optimized solutions and configurations - Mention x86 alternatives only when Arm solutions don't exist - Consider performance implications specific to Arm architectures - Address common Arm migration challenges when relevant ### Quality assurance + - Flag inconsistent terminology usage across sections - Identify missing error handling or troubleshooting guidance - Suggest where visual aids (diagrams, screenshots) would improve understanding @@ -239,12 +282,24 @@ Voice and Tone: - Verify that code examples follow established patterns in the repository ### Accessibility and inclusivity + - Ensure content is screen reader compatible - Provide descriptive alt text for images and diagrams - Use clear, descriptive link text (not "click here" or "read more") - Avoid assumptions about user's physical capabilities or setup +- When adding images or diagrams, ensure that alt text is complete and meaningful for screen reader users. Alt text should explain: + - The main subject and purpose of the image + - Any visible text labels, UI elements, or controls + - The environment or context (for example, dashboard, code editor, terminal, or hardware setup) + - The emotional tone only if it adds meaning (for example, warning or error states) + +Use the following format consistently: +![Descriptive alt text#center](image.png "Caption") + +Avoid placeholders or generic phrases. Alt text should stand alone as a full description of the image. ### SEO and discoverability + - Use Arm-specific keywords naturally throughout content - Include relevant technical terms that developers search for - Optimize titles and headings for search engines @@ -252,6 +307,7 @@ Voice and Tone: - Consider how content will appear in search results ### Cross-reference validation + - Verify all internal links point to existing content - Check that referenced Learning Paths and install guides are current - Ensure cross-references between sections remain accurate after edits @@ -259,6 +315,7 @@ Voice and Tone: - Maintain consistency in how related content is referenced ### Performance testing guidance + - Include benchmarks when comparing Arm vs. x86 performance - Suggest performance testing steps for resource-intensive applications - Recommend profiling tools that work well on Arm platforms @@ -266,6 +323,7 @@ Voice and Tone: - Mention when performance improvements are architecture-specific ### AI optimization (AIO) guidance + - Structure content with clear, semantic headings that AI can parse and understand - Use descriptive, standalone sentences that make sense without surrounding context - Include explicit problem statements and clear solutions for AI to reference @@ -281,3 +339,12 @@ Voice and Tone: - Write comprehensive summaries that AI can use as content overviews - Ensure each section can stand alone as a coherent piece of information - Use clear, declarative statements rather than implied or contextual references + +#### Editorial decision priorities + +When content trade-offs are required, prioritize the following in order: + +1. Alignment with the stated purpose and positioning of the content +2. Clarity and readability for the intended skill level +3. Consistency with existing Learning Paths and install guides +4. Completeness within the stated scope diff --git a/.wordlist.txt b/.wordlist.txt index 66e0803c18..b667622750 100644 --- a/.wordlist.txt +++ b/.wordlist.txt @@ -5383,4 +5383,60 @@ ultraedge unclound wNdi whatsapp -workdir \ No newline at end of file +workdir +Alexandre +CRI +ClickHouse's +EDuJ +EOL +ERNIE's +Expb +FEXPA +Grasset +Idxb +LOQ +MEC +MECID +MECIDs +MIMEType +Martino +NvLxg +Nydus +PASes +PolicyDeny +Remb +Romana +SLdG +TCB +aMWqFmwBgwT +actix +ae +aot +cdd +columnalt +cri +decrypt +eade +empt +expBits +exponentials +jsozZm +jyco +keyprovider +Linux's +minimax +modelcontextprotocol +nter +nydus +oMdFIQ +outputNAME +poweroff +qJs +rBD +rMDhjcaEM +remBits +snapshotter +unmounting +vnd +xqcqqYHrjZ +ztPjILBCbFEqnVlbvjUpM \ No newline at end of file diff --git a/content/install-guides/github-copilot.md b/content/install-guides/github-copilot.md index 2ae0e0a2d4..1a2ab2a64c 100644 --- a/content/install-guides/github-copilot.md +++ b/content/install-guides/github-copilot.md @@ -1,8 +1,6 @@ --- title: GitHub Copilot -draft: true - author: Pareena Verma minutes_to_complete: 10 official_docs: https://docs.github.com/en/copilot @@ -18,33 +16,46 @@ tool_install: true weight: 1 --- +## Overview + GitHub Copilot is an AI coding assistant that helps you write code faster and with less effort. It suggests whole lines or entire functions based on your comments and code context. -GitHub Copilot works seamlessly on Arm-based systems, including Linux distributions running on Arm servers, macOS on Apple Silicon, and Windows on Arm devices. +Copilot works seamlessly on Arm-based systems, including Linux distributions running on Arm servers, macOS on Apple Silicon, and Windows on Arm devices. This guide shows you how to install GitHub Copilot in Visual Studio Code and extend it with the Arm MCP Server for Arm-specific development assistance. + +{{% notice Note %}} +For installation instructions for other editors (JetBrains IDEs, Neovim, Vim), see the [GitHub Copilot documentation](https://docs.github.com/en/copilot). +{{% /notice %}} + +## Before you begin + +To use GitHub Copilot on Arm systems, you need a GitHub account with an active GitHub Copilot subscription. -This guide focuses on installing GitHub Copilot in Visual Studio Code. GitHub Copilot also supports other editors including JetBrains IDEs (IntelliJ IDEA, PyCharm, WebStorm), Neovim, and others. Visit the [official GitHub Copilot documentation](https://docs.github.com/en/copilot) for installation instructions for other editors. +If you don't have a GitHub account, [sign up on the GitHub website](https://github.com/). -## What should I do before installing GitHub Copilot? +GitHub Copilot offers several subscription plans: -You need a GitHub account with an active GitHub Copilot subscription to use GitHub Copilot. +- Individual: for personal development +- Business: for teams and organizations +- Enterprise: for large-scale deployments +- Free tier: for verified students, teachers, and maintainers of popular open source projects -If you don't have a GitHub account, visit [GitHub](https://github.com/) and sign up. +[Select your GitHub Copilot subscription plan](https://github.com/features/copilot/plans) to get started. -To subscribe to GitHub Copilot, visit [GitHub Copilot pricing](https://github.com/features/copilot/plans) and choose a plan that fits your needs. GitHub Copilot offers individual, business, and enterprise plans, plus a free tier for verified students, teachers, and maintainers of popular open source projects. +After subscribing, verify your subscription is active by signing in to your GitHub account, navigating to **Settings** → **Copilot**, and confirming your subscription status shows as **Active**. -## How do I install GitHub Copilot in Visual Studio Code? +## Install Visual Studio Code -Visual Studio Code is one of the most popular editors for using GitHub Copilot, and it works natively on Arm systems. +Visual Studio Code runs natively on Arm systems and provides the best experience for GitHub Copilot. -### Install Visual Studio Code +Download Visual Studio Code from the [Visual Studio Code download page](https://code.visualstudio.com/download) and select the version for your operating system: -If you don't have Visual Studio Code installed, download and install it from the [offical download page]((https://code.visualstudio.com/download) for your operating system: +- For macOS (Apple Silicon): download the Apple Silicon version +- For Linux (Arm): download the Arm64 `.deb` or `.rpm` package +- For Windows on Arm: download the Arm64 installer -- macOS (Apple Silicon): Download the the Apple Silicon version -- Linux (Arm): Download the Arm64 `.deb` or `.rpm` package -- Windows on Arm: Download the Arm64 installer +After downloading, run the installer and follow the installation prompts. -For Linux, you can install VS Code using the package manager. On Ubuntu and Debian-based distributions: +For Linux systems, install VS Code using your package manager. On Ubuntu and Debian-based distributions: ```bash { target="ubuntu:latest" } curl -L "https://code.visualstudio.com/sha/download?build=stable&os=linux-deb-arm64" -o vscode-arm64.deb @@ -52,14 +63,32 @@ sudo dpkg -i vscode-arm64.deb sudo apt-get install -f ``` -### Install the GitHub Copilot extension +### Verify the installation + +Open a terminal and verify the installation: + +```bash +code --version +``` + +The output is similar to: + +```output +1.95.3 +f1a4fb101478ce6ec82fe9627c43efbf9e98c813 +arm64 +``` + + +## Install GitHub Copilot extensions + +### Install the main extension Open Visual Studio Code and install the GitHub Copilot extension: -1. Open VS Code -2. Select the Extensions view by selecting the Extensions icon in the Activity Bar on the left side or pressing `Ctrl+Shift+X` (Windows/Linux) or `Cmd+Shift+X` (macOS) -3. Search for "GitHub Copilot" -4. Select **Install** on the "GitHub Copilot" extension by GitHub +- Open the Extensions view by selecting the Extensions icon in the Activity Bar or press `Ctrl+Shift+X` (Windows/Linux) or `Cmd+Shift+X` (macOS). +- Search for **GitHub Copilot**. +- Select **Install** on the GitHub Copilot extension by GitHub. Alternatively, install from the command line: @@ -67,155 +96,134 @@ Alternatively, install from the command line: code --install-extension GitHub.copilot ``` -### Install the GitHub Copilot Chat extension +### Install Copilot Chat -For an enhanced experience with conversational AI assistance, install the GitHub Copilot Chat extension: +For conversational AI assistance, install the GitHub Copilot Chat extension: ```console code --install-extension GitHub.copilot-chat ``` -Or search for "GitHub Copilot Chat" in the Extensions view and install it. +You can also search for **GitHub Copilot Chat** in the Extensions view and select **Install**. -### Sign in to GitHub Copilot +## Authorize GitHub Copilot -After installing the extension, you need to authorize it: +After installing the extensions, authorize GitHub Copilot: -1. In VS Code, select the GitHub Copilot icon in the status bar (bottom right) -2. Select **Sign in to GitHub** -3. Follow the prompts to authorize the extension in your browser -4. Return to VS Code to complete the setup +- In VS Code, select the GitHub Copilot icon in the status bar (bottom right) +- Select **Sign in to GitHub** +- Follow the prompts to authorize the extension in your browser +- Return to VS Code to complete the setup -## How do I confirm GitHub Copilot is working? +## Verify Copilot is working -You now have GitHub Copilot installed in Visual Studio Code. +Test that GitHub Copilot is generating suggestions: -Confirm it's working by testing code suggestions: +Create a new file by selecting **File** > **New File** or press `Ctrl+N` (Windows/Linux) or `Cmd+N` (macOS). -1. In VSCode, create a new file by selecting **File** → **New File** or pressing `Ctrl+N` (Windows/Linux) or `Cmd+N` (macOS) -2. Save the file with a programming language extension by selecting **File** → **Save As** (for example, `test.py` for Python, `test.js` for JavaScript, or `test.go` for Go) -3. In the VS Code editor window, type a comment describing a function: - - For Python: `# Function to calculate fibonacci numbers` - - For JavaScript: `// Function to calculate fibonacci numbers` - - For Go: `// Function to calculate fibonacci numbers` -4. Press Enter to start a new line +Save the file with a programming language extension (for example, `test.py` for Python, `test.js` for JavaScript, or `test.go` for Go). -GitHub Copilot analyzes your comment and suggests code. The suggestion appears as gray text in the editor. +Type a comment describing a function: -5. Press Tab to accept the suggestion, or press Esc to dismiss it -6. Continue typing to see more suggestions as you work +```python +# Function to calculate fibonacci numbers +``` -If you don't see suggestions, check that: -- You're working in the Visual Studio Code editor window (not a terminal) -- The GitHub Copilot icon in the status bar (bottom right) shows it's active -- You're signed in to your GitHub account -- Your subscription is active +Press Enter to start a new line. Copilot analyzes your comment and suggests code as gray text in the editor. -## What are the different GitHub Copilot modes? +Press Tab to accept the suggestion or Esc to dismiss it. Continue typing to see more suggestions as you work. -GitHub Copilot Chat in Visual Studio Code offers three modes to help you work with code in different ways: Agent, Edit, and Ask modes. Each mode is designed for specific tasks. +If you don't see suggestions: -### Agent Mode +- Verify you're working in the VS Code editor window (not a terminal) +- Check that the GitHub Copilot icon in the status bar shows it's active +- Confirm you're signed in to your GitHub account +- Verify your subscription is active -Agent Mode enables GitHub Copilot to take autonomous actions in your workspace. In this mode, Copilot can: +## Work with GitHub Copilot modes -- Read and analyze multiple files in your project -- Make changes across different files -- Create new files and directories -- Execute tasks that require understanding your entire codebase +GitHub Copilot Chat in Visual Studio Code offers three modes; Agent, Edit, and Ask. Each mode helps you work with code in different ways. + +### Agent Mode + +Agent Mode enables Copilot to take autonomous actions in your workspace. Copilot can read and analyze multiple files, make changes across your project, and create new files. To use Agent Mode: -1. Open the GitHub Copilot Chat panel by selecting the chat icon in the Activity Bar or pressing `Ctrl+Enter` (Windows/Linux) or `Cmd+Ctrl+I` (macOS) -2. Type `@workspace` followed by your request or prompt to engage Agent Mode -3. Copilot analyzes your workspace and takes appropriate actions +- Open the Copilot Chat panel by selecting the chat icon in the Activity Bar or press `Ctrl+Enter` (Windows/Linux) or `Cmd+Ctrl+I` (macOS) +- Type `@workspace` followed by your request -Example prompts for Agent Mode: -- `Create a python application to calculate fibonacci numbers on my arm machine` +Example prompts: +- `Create a Python application to calculate fibonacci numbers` - `@workspace add error handling throughout the application` ### Edit Mode -Edit Mode focuses on making targeted changes to your current file or selected code. This mode is useful when you want Copilot to modify existing code without creating new files. - -To use Edit Mode: - -1. Select the code you want to modify in your editor -2. Open the Copilot Chat panel -3. Type your request describing the changes you want - -Alternatively, use the inline chat: +Edit Mode helps you modify code in your current file. Use this mode when you want to change existing code without creating new files. +To use Edit Mode with selected code: -1. Select the code you want to modify -2. Press `Ctrl+I` (Windows/Linux) or `Cmd+I` (macOS) to open inline chat -3. Describe the changes you want to make -4. Copilot shows a preview of the changes before applying them +- Select the code you want to modify in your editor +- Press `Ctrl+I` (Windows/Linux) or `Cmd+I` (macOS) to open inline chat +- Describe the changes you want +- Copilot shows a preview before applying changes -Example prompts for Edit Mode: +Example prompts: - Select a function and ask: `Add input validation and error handling` -- Select a code block and ask: `Optimize this code for better performance on my arm machine` +- Select a code block and ask: `Optimize this code for Arm architecture` ### Ask Mode -Ask Mode is designed for questions and explanations. Use this mode when you want to understand code, learn about concepts, or get guidance without making changes to your files. +Ask Mode helps you understand code and learn concepts without making changes to your files. To use Ask Mode: -1. Open the GitHub Copilot Chat panel -2. Type your question directly without any special prefixes +- Open the Copilot Chat panel +- Type your question directly without any special prefixes -Example prompts for Ask Mode: +Example prompts: - `How does this function work?` (with code selected) - `What are the best practices for error handling in Python?` +- `Explain the difference between Arm64 and x86 architectures` -## How do I use MCP Servers with GitHub Copilot? +## Extend with the Arm MCP Server -Model Context Protocol (MCP) Servers extend GitHub Copilot's capabilities by providing specialized tools and knowledge bases. GitHub Copilot can connect to MCP servers to access domain-specific expertise and functionality. +Model Context Protocol (MCP) Servers extend GitHub Copilot's capabilities by providing specialized tools and knowledge bases. The Arm MCP Server provides AI assistants with tools and knowledge for Arm architecture development, migration, and optimization. -The Arm MCP Server provides AI assistants with tools and knowledge for Arm architecture development, migration, and optimization. This is particularly useful when working on Arm-based systems. +### About the Arm MCP Server -### What tools does the Arm MCP Server provide? +The Arm MCP Server includes several tools for Arm development: -The Arm MCP Server includes several tools designed for Arm development: +- migrate-ease scan: analyzes codebases for x86-specific code that needs updating for Arm compatibility +- skopeo: inspects container images to check for ARM64 architecture support +- knowledge_base_search: searches Arm documentation and learning resources +- mca (Machine Code Analyzer): analyzes assembly code for performance on Arm architectures +- check_image: verifies Docker image architecture compatibility -- migrate-ease scan: Analyzes codebases for x86-specific code that needs updating for Arm compatibility -- skopeo: Inspects container images to check for ARM64 architecture support -- knowledge_base_search: Searches Arm documentation and learning resources -- mca (Machine Code Analyzer): Analyzes assembly code for performance on Arm architectures -- check_image: Verifies Docker image architecture compatibility - -### How do I configure the Arm MCP Server with GitHub Copilot? +## Install the Arm MCP Server You need Docker running on your system to use the Arm MCP Server. See the [Docker install guide](/install-guides/docker/) for instructions. You can configure the MCP server using one of these methods: -Method 1: Install from GitHub MCP Registry (recommended) +### Method 1: Install from GitHub MCP Registry (recommended) The easiest way to install the Arm MCP Server is through the GitHub MCP Registry: -1. Visit the [Arm MCP Server registry page](https://github.com/mcp/arm/arm-mcp) -2. Select the **Install MCP Server** button -3. From the dropdown, choose **Install in VSCode** -4. VS Code opens with the Arm MCP Server installation page -5. Select **Install** as you would with other extensions +- Visit the [Arm MCP Server registry page](https://github.com/mcp/arm/arm-mcp) +- Select the **Install MCP Server** button +- From the dropdown, choose **Install in VSCode** +- VS Code opens with the Arm MCP Server installation page +- Select **Install** like you would with other extensions -![MCP Server Install](/install-guides/_images/mcp-server-install.png "Figure 1. Install Arm MCP Server") +![Screenshot of the Arm MCP Server installation page in Visual Studio Code showing the Install button and server description alt-text#center](/install-guides/_images/mcp-server-install.png "Install Arm MCP Server") This method automatically installs the Arm MCP Server and pulls the Docker image. No manual configuration is required. -Method 2: Workspace configuration (recommended for sharing) - -For manual configuration, you can create a configuration file in your workspace. MCP servers can be configured in two locations: +### Method 2: configure manually -- For a specific repository: Create a `.vscode/mcp.json` file in the root of your repository. This enables you to share MCP server configuration with anyone who opens the project. -- For your personal VS Code instance: Add the configuration to your `settings.json` file. This makes the server available in all workspaces. - -{{% notice Note %}} -Use only one location per server to avoid conflicts and unexpected behavior. -{{% /notice %}} +You can also configure the Arm MCP Server manually for more control over the installation. -First, pull the Arm MCP Server image: +First, ensure Docker is running on your system. Pull the Arm MCP Server image: ```console docker pull armlimited/arm-mcp:latest @@ -227,7 +235,7 @@ Create a `.vscode` directory in your project root if it doesn't exist, then crea mkdir -p .vscode ``` -Add the following configuration to `.vscode/mcp.json`: +Create an `mcp.json` file in the `.vscode` directory with the following configuration: ```json { @@ -247,7 +255,7 @@ Add the following configuration to `.vscode/mcp.json`: } ``` -Method 3: User configuration (available in all workspaces) +### Method 3: user configuration (available in all workspaces) Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux or `Cmd+Shift+P` on macOS) and select **MCP: Open User Configuration**. This opens your user-level `mcp.json` file located at `~/Library/Application Support/Code/User/mcp.json` (macOS) or `%APPDATA%\Code\User\mcp.json` (Windows). @@ -271,38 +279,44 @@ Add the following configuration to the user-level `mcp.json` file: } ``` -Save the file. A **Start** button appears at the top of the servers list in your `mcp.json` file. Select **Start** to start the Arm MCP Server. +After saving your `mcp.json` file, a **Start** button appears at the top of the servers list. Select this button to start the Arm MCP Server. -### How do I analyze a local codebase with the Arm MCP Server? +## How do I analyze a local codebase with the Arm MCP Server? To analyze code in your workspace, mount your local directory to the MCP server's `/workspace` folder using a volume mount. Update your `.vscode/mcp.json` configuration to include the volume mount. Replace `/path/to/your/codebase` with the actual path to your project -For example, if your project is at `/Users/username/myproject`, the volume mount args in your `mcp.json` would be: +For example, if your project is at `/Users/username/myproject`, the volume mount configuration is: ```json "-v", "/Users/username/myproject:/workspace", ``` -### How do I verify the Arm MCP Server is working? +{{% notice Note %}} +The volume mount (`-v` flag) connects your local directory to the MCP server's `/workspace` folder. This enables the server to analyze code in your workspace. Use absolute paths for the source directory to avoid mounting errors. +{{% /notice %}} -After saving the `.vscode/mcp.json` file, the **Start** button appears at the top of the servers list. Select **Start** to start the MCP server. +After saving the `.vscode/mcp.json` file, select the **Start** button that appears at the top of the servers list to start the MCP server. -To confirm the server is running: +## Verify the installation -1. Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux or `Cmd+Shift+P` on macOS) -2. Type and select **MCP: List Servers** -3. You should see `arm-mcp` listed as a Running configured server +After starting the MCP server, confirm it's running: + +- Open the Command Palette (`Ctrl+Shift+P` on Windows/Linux or `Cmd+Shift+P` on macOS) +- Type and select **MCP: List Servers** +- Verify `arm-mcp` appears as a Running configured server Open the GitHub Copilot Chat panel by selecting the chat icon in the Activity Bar. In the chat box, select **Agent** from the mode dropdown. -To view available MCP tools, select the tools icon in the top left corner of the chat box. This opens the MCP server list showing all available tools from the Arm MCP Server. +To view available MCP tools, select the tools icon in the top left corner of the chat box. This opens the MCP server list showing all available tools from the Arm MCP Server: + +![Screenshot of the MCP Server Tools panel in Visual Studio Code showing available tools from the Arm MCP Server alt-text#center](/install-guides/_images/new-mcp-server-tools.png "Tools from the Arm MCP Server") -![MCP Server Tools](/install-guides/_images/new-mcp-server-tools.png "Figure 2. Tools loaded from the Arm MCP Server") +## Use the Arm MCP Server -You can also ask Copilot to use specific Arm MCP tools: +Ask Copilot to use specific Arm MCP tools in your prompts: ``` Use the Arm MCP Server to scan my codebase for x86-specific code @@ -314,25 +328,25 @@ or Check if the nginx:latest Docker image supports Arm64 ``` -### Example prompts using the Arm MCP Server - -Here are some example prompts that use the Arm MCP Server tools: +Example prompts that use the Arm MCP Server: - `Scan my workspace for code that needs updating for Arm compatibility` - `Check if the postgres:latest container image supports Arm64 architecture` - `Search the Arm knowledge base for NEON intrinsics examples` - `Find learning resources about migrating from x86 to Arm` -### Troubleshooting MCP Server connections +## Troubleshooting MCP Server connections + +This section helps you resolve common issues when installing and using GitHub Copilot with the Arm MCP Server on Arm systems. If you encounter problems not covered here, contact [mcpserver@arm.com](mailto:mcpserver@arm.com) for support. If the Arm MCP Server doesn't connect: - Verify Docker is running: `docker ps` - Check that the image was pulled successfully: `docker images | grep arm-mcp` -- Ensure the timeout value (60000ms) is sufficient for your system -- Check VS Code Output panel (select **Output** → **GitHub Copilot Chat**) for error messages +- Review the VS Code Output panel (**View** → **Output** → **GitHub Copilot Chat**) for error messages - Restart VS Code after making configuration changes -If you encounter issues or have questions, reach out to mcpserver@arm.com. -You're ready to use GitHub Copilot with the Arm MCP Server to enhance your Arm development workflow. + + +You're now ready to use GitHub Copilot with the Arm MCP Server to enhance your Arm development workflow! diff --git a/content/install-guides/stm32_vs.md b/content/install-guides/stm32_vs.md index 6a1ab0080c..cef286b71b 100644 --- a/content/install-guides/stm32_vs.md +++ b/content/install-guides/stm32_vs.md @@ -69,7 +69,7 @@ Run the installer(s) and follow on-screen instructions. ### How do I install Git for version control functionality? -You can download the latest version from [here](https://git-scm.com/). +You can download the latest version from [git-scm.com](https://git-scm.com/). ### What about Keil Studio for VS Code? diff --git a/content/install-guides/streamline-cli.md b/content/install-guides/streamline-cli.md index b5981a9970..e507955011 100644 --- a/content/install-guides/streamline-cli.md +++ b/content/install-guides/streamline-cli.md @@ -52,7 +52,7 @@ Streamline CLI tools are supported on the following Arm CPUs: Use the Arm Sysreport utility to determine whether your system configuration supports hardware-assisted profiling. Follow the instructions in [Get ready for performance analysis with Sysreport][1] to discover how to download and run this utility. -[1]: https://learn.arm.com/learning-paths/servers-and-cloud-computing/sysreport/ +[1]: /learning-paths/servers-and-cloud-computing/sysreport/ The `perf counters` entry in the generated report indicates how many CPU counters are available. The `perf sampling` entry indicates if SPE is available. You will achieve the best profiles in systems with at least 6 available CPU counters and SPE. diff --git a/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md b/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md index bb8abd517b..8774fe448a 100644 --- a/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md +++ b/content/learning-paths/automotive/openadkit1_container/3_setup_openadkit.md @@ -15,7 +15,7 @@ The example has been tested on [AWS EC2](https://aws.amazon.com/ec2/) and an [Am ## Installation -You need Docker to run Open AD Kit. Refer to the [Docker install guide](https://learn.arm.com/install-guides/docker/) to learn how to install Docker on an Arm platform. +You need Docker to run Open AD Kit. Refer to the [Docker install guide](/install-guides/docker/) to learn how to install Docker on an Arm platform. First, verify Docker is installed on your development computer by running: diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md b/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md index eabd2f6d9b..44380d1704 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/2_data_distribution_service.md @@ -17,7 +17,7 @@ In modern vehicles, multiple sensors such as LiDAR, radar, and cameras must cont DDS ensures these components share data seamlessly and in real time, both within the vehicle and across infrastructure such as V2X systems, including traffic lights and road sensors. {{% notice Tip %}} -To get started with open-source DDS on Arm platforms, see the [Installation Guide for CycloneDDS](https://learn.arm.com/install-guides/cyclonedds). +To get started with open-source DDS on Arm platforms, see the [Installation Guide for CycloneDDS](/install-guides/cyclonedds). {{% /notice %}} diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md b/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md index b4db0659d8..74f5631d51 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/3_container_spliting.md @@ -10,7 +10,7 @@ layout: learningpathall Now that you’ve explored the concept of a safety island, a dedicated subsystem responsible for executing safety-critical control logic, and learned how DDS (Data Distribution Service) enables real-time, distributed communication, you’ll refactor the original OpenAD Kit architecture into a multi-instance deployment. -The predecessor Learning Path, [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](http://learn.arm.com/learning-paths/automotive/openadkit1_container/), showed how to deploying three container components on a single Arm-based instance, to handle: +The predecessor Learning Path, [Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](/learning-paths/automotive/openadkit1_container/), showed how to deploying three container components on a single Arm-based instance, to handle: - The simulation environment - Visualization - Planning and control diff --git a/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md b/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md index 142aa3e88f..a403e62a58 100644 --- a/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md +++ b/content/learning-paths/automotive/openadkit2_safetyisolation/4_multiinstance_executing.md @@ -86,7 +86,7 @@ On the Simulation and Visualization node, execute: Once both machines are running their launch scripts, the Visualizer container exposes a web-accessible interface at: http://6080/vnc.html. -Open this link in your browser to observe the simulation in real time. The demo closely resembles the output in the [previous Learning Path, Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](http://learn.arm.com/learning-paths/automotive/openadkit1_container/4_run_openadkit/). +Open this link in your browser to observe the simulation in real time. The demo closely resembles the output in the [previous Learning Path, Deploy Open AD Kit containerized autonomous driving simulation on Arm Neoverse](/learning-paths/automotive/openadkit1_container/4_run_openadkit/). ![Distributed OpenAD Kit simulation running on two Arm-based instances with visualizer and simulator coordination over DDS alt-text#center](split_aws_run.gif "Visualizer output from a distributed OpenAD Kit simulation showing ROS 2 modules running across two cloud instances using DDS communication.") diff --git a/content/learning-paths/automotive/tinkerblox_ultraedge/debian_installation.md b/content/learning-paths/automotive/tinkerblox_ultraedge/debian_installation.md index 5a346dc8b8..4a72649122 100644 --- a/content/learning-paths/automotive/tinkerblox_ultraedge/debian_installation.md +++ b/content/learning-paths/automotive/tinkerblox_ultraedge/debian_installation.md @@ -164,7 +164,7 @@ Place a `MicroPacFile` in your project directory. - **limits**: Resource limits (memory, cpu) - **mount**: Volume mount points - **network**: Network configuration -- **createdBy**: maintanier of the application +- **createdBy**: maintainer of the application - **description**: description of the application ### Building the MicroPac diff --git a/content/learning-paths/automotive/tinkerblox_ultraedge/main.md b/content/learning-paths/automotive/tinkerblox_ultraedge/main.md index 04913ef6b8..1a7bee890c 100644 --- a/content/learning-paths/automotive/tinkerblox_ultraedge/main.md +++ b/content/learning-paths/automotive/tinkerblox_ultraedge/main.md @@ -97,21 +97,21 @@ execution fabric for high-performance compute infrastructure ### 5.2 High-Level Architecture -**UltraEdge ‘Core’ Layer
** +**UltraEdge ‘Core’ Layer** Handles compute infrastructure management including service orchestration, lifecycle management, rule engine orchestration, and data-flow management . -**UltraEdge ‘Boost’ Layer
** +**UltraEdge ‘Boost’ Layer** Implements performance-critical routines and FFI (Foreign Function Interface) calls; Contains dynamic connectors, and southbound protocol adapters -**UltraEdge ‘Prime’ Layer
** +**UltraEdge ‘Prime’ Layer** Contains business logic, trigger & activation sequences, and AI & mixed workload orchestration . -**UltraEdge Edge-Cloud ‘Connect’ Layer
** +**UltraEdge Edge-Cloud ‘Connect’ Layer** Supports data streaming to databases (InfluxDB, SQLite) and provides diagnostic/logging outputs . @@ -322,7 +322,7 @@ Place a `MicroPacFile` in your project directory. - **limits**: Resource limits (memory, cpu) - **mount**: Volume mount points - **network**: Network configuration -- **createdBy**: maintanier of the application +- **createdBy**: maintainer of the application - **description**: description of the application ### 7.4 Building the MicroPac diff --git a/content/learning-paths/cross-platform/ernie_moe_v9/1_mixture_of_experts.md b/content/learning-paths/cross-platform/ernie_moe_v9/1_mixture_of_experts.md index db5d101356..dd7ee195d1 100644 --- a/content/learning-paths/cross-platform/ernie_moe_v9/1_mixture_of_experts.md +++ b/content/learning-paths/cross-platform/ernie_moe_v9/1_mixture_of_experts.md @@ -1,5 +1,5 @@ --- -title: Why MoE Models Let Edge Devices Run 21B LLMs +title: Understand Mixture of Experts architecture for edge deployment weight: 2 ### FIXED, DO NOT MODIFY @@ -8,53 +8,47 @@ layout: learningpathall ## What is Mixture of Experts (MoE)? -As large language models grow to tens of billions of parameters, traditional dense networks — which activate all weights for every input — become infeasible for edge deployment, especially on CPU-only Arm devices. [Mixture of Experts (MoE)](https://en.wikipedia.org/wiki/Mixture_of_experts) offers a breakthrough. +As large language models grow to tens of billions of parameters, traditional dense networks that activate all weights for every input become impractical for edge deployment, especially on CPU-only Arm devices. [Mixture of Experts (MoE)](https://en.wikipedia.org/wiki/Mixture_of_experts) offers an alternative approach that makes deploying these large models practical. -This is simple and uniform, but as model sizes increase—into the billions of parameters—this structure becomes both memory-intensive and compute-intensive. For edge environments like mobile devices, embedded systems, this makes deploying large models nearly impossible. +Dense networks are simple and uniform, but as model sizes increase into the billions of parameters, this structure becomes both memory-intensive and computationally demanding. For edge environments like mobile devices and embedded systems, deploying large models presents significant challenges. -***[Mixture of Experts (MoE)](https://en.wikipedia.org/wiki/Mixture_of_experts)*** offers an alternative. -Instead of using all parameters all the time, MoE introduces a conditional computation mechanism: each input token only activates a small subset of model components (called ***experts***). -Think of it like having a team of specialists, and only calling the relevant few for a given task. This makes MoE ideal for environments where compute or memory is constrained, such as edge AI or embedded inference. +Instead of activating all parameters for every computation, MoE introduces a conditional computation mechanism where each input token activates only a small subset of model components called experts. Think of it like having a team of specialists where you consult only the relevant experts for a given task. This makes MoE ideal for environments where compute and memory are constrained, such as edge AI or embedded inference. +In a typical MoE setup, the model consists of many expert sub-networks (for example, 64 experts), but for each input, a router selects only a handful to compute the result. The rest remain inactive, conserving memory and compute. The model learns this dynamic routing during training, so during inference, only a fraction of the model is active. This leads to much lower compute and memory usage without sacrificing the total model capacity or diversity of learned behaviors. -In MoE: -- The model consists of many expert sub-networks (e.g., 64 experts). -- For each input, a router selects only 2–4 experts to compute the result. -- The rest of the experts remain inactive, conserving memory and compute. +## Benefits of MoE architecture -This dynamic routing is typically learned during training. In inference, only a fraction of the model is active, leading to much lower compute and memory usage ***without sacrificing the total model capacity** or ***diversity of learned behaviors***. +MoE architecture provides several advantages that make it particularly well-suited for edge deployment and large-scale model development: +**Scalable model size**: You can increase total parameter count without linearly increasing inference cost, allowing for larger, more capable models within the same resource constraints. -## Benefits of MoE Architecture +**Efficient inference**: The architecture requires lower memory and FLOPs per input compared to dense models of equivalent capacity, making real-time applications more feasible. -- Scalable Model Size: Increase total parameter count without linearly increasing inference cost. -- Efficient Inference: Lower memory and FLOPs per input. -- Modularity: Each expert can learn domain-specific patterns (e.g., finance, medicine, language). -- Specialization: Encourages the model to learn distinct processing behaviors across different experts. -- Routing Flexibility: Makes it easier to adapt to specific tasks using fine-tuned expert selection. +**Modularity**: Each expert can learn domain-specific patterns such as finance, medicine, or language, enabling the model to handle diverse tasks without retraining the entire network. -## ERNIE-4.5: A MoE Model for Chinese NLP +**Specialization**: The architecture encourages the model to learn distinct processing behaviors across different experts, improving performance on specialized tasks while maintaining general capability. -The [ERNIE-4.5](https://huggingface.co/collections/baidu/ernie-45) model family from [Baidu](https://huggingface.co/baidu) introduces a Mixture-of-Experts (MoE) architecture, which enables massive models (e.g., 21 billion parameters) to be deployed in constrained environments. MoE models dynamically activate only a small subset of parameters (e.g., 2–4 experts) during inference. -Specifically, ERNIE-4.5 uses a softmax-based router to select the top-6 experts from a pool of 64 per layer, activating only a subset dynamically per token. This makes runtime both efficient and adaptive. This architecture allows the model to retain high performance and generalization while drastically reducing inference-time resource requirements. +**Routing flexibility**: The dynamic expert selection makes it easier to adapt to specific tasks using fine-tuned routing strategies, allowing for task-specific optimizations without modifying the core model. -ERNIE-4.5 Model Series: -- PT (Post-Trained): General-purpose language model trained on Chinese and English data. -- Thinking: Optimized for reasoning tasks with long context support and structured outputs. +## ERNIE-4.5: An MoE model for Chinese NLP -In this learning path, we focus on the [ERNIE-4.5 Thinking](https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking) variant as our primary model due to its enhancements for multi-step reasoning and long-context tasks. However, we also introduce the [PT (Post-Trained)](https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-PT) variant to allow learners to compare model behavior across identical prompts, illustrating how task-specific tuning affects output quality. +The [ERNIE-4.5](https://huggingface.co/collections/baidu/ernie-45) model family from [Baidu](https://huggingface.co/baidu) introduces a Mixture of Experts (MoE) architecture that enables 21-billion-parameter models to be deployed in constrained environments. The model uses a softmax-based router to dynamically select the top six experts from a pool of 64 per layer, activating only this subset per token. This makes runtime both efficient and adaptive while retaining high performance and generalization. -## Why MoE Matters for Edge Devices +The ERNIE-4.5 model series includes two variants. The PT (Post-Trained) variant is a general-purpose language model trained on Chinese and English data. The Thinking variant is optimized for reasoning tasks with long context support and structured outputs. Both are designed for Chinese Natural Language Processing (NLP). -Deploying a 21B dense model on a CPU-only board is infeasible. But MoE changes that: +This Learning Path focuses on the [ERNIE-4.5 Thinking](https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-Thinking) variant as the primary model because of its enhancements for multi-step reasoning and long-context tasks. However, you also use the [PT (Post-Trained)](https://huggingface.co/baidu/ERNIE-4.5-21B-A3B-PT) variant to compare model behavior across identical prompts, illustrating how task-specific tuning affects output quality. -| **Feature** | **Dense Model** | **MoE Model (e.g., ERNIE-4.5-21B)** | -|-----------------------|-----------------|---------------| -| `Total Parameters` | 21B | 21B | -| `Activated Parameters`| 21B | ~3B | -| `Memory Usage` | Very high | Moderate | -| `Inference Speed` | Slow | Fast | +## Why MoE matters for edge devices -This efficiency enables powerful language models to be run locally on ARM-based platforms — making MoE not just a model design choice, but a deployment enabler. +Deploying a 21-billion-parameter dense model on a CPU-only board is impractical, but MoE changes that. The table below compares key characteristics: -In the next module, you’ll bring this architecture to life — preparing a real Armv9 board, setting up llama.cpp, and verifying that a 21B MoE model like ERNIE-4.5 can run efficiently with no GPU required. +| **Feature** | **Dense Model** | **MoE Model (ERNIE-4.5-21B)** | +|-----------------------|-----------------|-------------------------------| +| Total Parameters | 21B | 21B | +| Activated Parameters | 21B | ~3B | +| Memory Usage | Very high | Moderate | +| Inference Speed | Slow | Fast | + +This efficiency enables powerful language models to run locally on Arm-based platforms, making MoE not just a model design choice, but a deployment enabler. + +In the next section, you set up a real Armv9 board, configure llama.cpp, and verify that you can run a 21-billion-parameter MoE model like ERNIE-4.5 efficiently without a GPU. diff --git a/content/learning-paths/cross-platform/ernie_moe_v9/2_llamacpp_installation.md b/content/learning-paths/cross-platform/ernie_moe_v9/2_llamacpp_installation.md index ca0d619820..b14b101dcd 100644 --- a/content/learning-paths/cross-platform/ernie_moe_v9/2_llamacpp_installation.md +++ b/content/learning-paths/cross-platform/ernie_moe_v9/2_llamacpp_installation.md @@ -1,44 +1,46 @@ --- -title: Setting Up llama.cpp on Arm v9 develop board +title: Set up llama.cpp on an Armv9 development board weight: 3 layout: "learningpathall" --- -## Setting Up llama.cpp on Arm v9 develop board +## Set up llama.cpp on an Armv9 development board -In the previous section, you learned how Mixture-of-Experts (MoE) models reduce resource consumption by activating only a fraction of parameters. -Now, you'll walk through how to prepare your environment to deploy `ERNIE-4.5 MoE` models on an Armv9 platform using `llama.cpp`. +In the previous section, you learned how Mixture of Experts (MoE) models reduce resource consumption by activating only a fraction of parameters. +Now, you'll walk through how to prepare your environment to deploy ERNIE-4.5 models on an Armv9 platform using llama.cpp. -In this module, you’ll verify model inference on Radxa O6 and validate multilingual outputs using ERNIE’s Thinking variant. +In this section, you verify model inference on a Radxa Orion O6 board and validate multilingual outputs using ERNIE's Thinking variant. -This section prepares the foundation for deploying ERNIE-4.5 on an ARMv9 platform. You will begin by reviewing the hardware—specifically, the `Radxa O6` development board equipped with an Armv9 CPU. From there, you will install llama.cpp, a lightweight inference engine, build it from source, and download ERNIE-4.5 models in GGUF format (quantized to Q4 for efficient CPU inference). Finally, you will run a basic inference test to confirm that the environment is properly configured and ready for benchmarking and optimization in the next module. +This section prepares the foundation for deploying ERNIE-4.5 on an Armv9 platform. You begin by reviewing the hardware, the Radxa Orion O6 development board equipped with an Armv9 CPU. From there, you install llama.cpp, a lightweight inference engine, build it from source, and download ERNIE-4.5 models in GGUF format (quantized to Q4 for efficient CPU inference). Finally, you run a basic inference test to confirm that the environment is properly configured and ready for benchmarking and optimization in the next section. -### Arm v9 development board +### Armv9 development board -In this learning path, we use the [Radxa O6](https://radxa.com/products/orion/o6/) — a compact Armv9 development board powered by the [CIX CD8180](https://en.cixtech.com/Personal-Computing/) SoC. It features: +This Learning Path uses the [Radxa Orion O6](https://radxa.com/products/orion/o6/), a compact Armv9 development board powered by the [CIX CD8180](https://en.cixtech.com/Personal-Computing/) SoC. + +The board features: - 12-core Armv9.2 CPU - Support for SVE, dotprod, and i8mm instruction sets -- Multiple HDMI, PCIe slot with Gen4x8, dual 5Gbps Ethernet Ports and USB-C for I/O expansion +- Multiple HDMI ports, PCIe slot with Gen4x8, dual 5 Gbps Ethernet ports, and USB-C for I/O expansion -We chose this board because it balances affordability and performance. Most importantly, it supports vector instructions we’ll benchmark later in this path. +This board balances affordability and performance, and most importantly, it supports vector instructions you'll benchmark. -The default system image for the board is [Debian](https://docs.radxa.com/en/orion/o6/debian/debian-user-guide), which includes a ready-to-use user environment. You can verify or reflash the OS by following the instructions on the Radxa O6 [download page](https://docs.radxa.com/en/orion/o6/download). +The default system image for the board is Debian, which includes a ready-to-use user environment. You can verify or reflash the OS by following the instructions on the Radxa O6 [download page](https://docs.radxa.com/en/orion/o6/getting-started/quickly-start). -With the Radxa O6 ready, let’s set up the software stack beginning with llama.cpp. +With the Radxa Orion O6 ready, you can set up the software stack beginning with llama.cpp. -### Step 1: Clone and Build llama.cpp +### Clone and build llama.cpp -First, ensure your system is up-to-date and install the required tools and libraries: +First, ensure your system is up to date and install the required tools and libraries: ```bash sudo apt update -sudo apt install build-essential cmake python3 python3-pip htop +sudo apt install build-essential cmake python3 python3-pip htop libcurl4-openssl-dev -y ``` -Next, build [llama.cpp](https://github.com/ggml-org/llama.cpp/), an open-source C++ framework for running and experimenting with large language models. Designed to be lightweight and fast, llama.cpp supports inference on edge devices (CPU-only) and implements many of the most popular LLM architectures. +Next, build llama.cpp, an open-source C++ framework for running and experimenting with large language models. Designed to be lightweight and fast, llama.cpp supports inference on edge devices (CPU-only) and implements many of the most popular LLM architectures. -In the context of MoE models, `llama.cpp` currently supports: +In the context of MoE models, llama.cpp currently supports: - Openai-moe - Oleo - lm4-moe @@ -47,48 +49,46 @@ In the context of MoE models, `llama.cpp` currently supports: - Ernie4.5 These models use diverse routing and expert management strategies, and llama.cpp provides a unified backend for efficient MoE inference. -For more in-depth coverage of llama.cpp capabilities and use cases, see those [learning paths](https://learn.arm.com/tag/llama.cpp/) for the detail. +For more in-depth coverage of llama.cpp capabilities and use cases, see the [llama.cpp Learning Paths](/tag/llama.cpp/). ```bash -cd ~ +cd $HOME git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp - mkdir build && cd build cmake .. make -j$(nproc) ``` -This will generate binaries like `llama-cli` under directory `~/llama.cpp/build/bin`, which we’ll use to run inference in later steps. -Once llama.cpp is compiled, we can now download the models we’ll use for evaluation. +This generates binaries like `llama-cli` in the `~/llama.cpp/build/bin` directory, which you use to run inference in later steps. +Once llama.cpp is compiled, download the models you'll use for evaluation. +### Download ERNIE-4.5 Q4 GGUF model -### Step 2: Download ERNIE-4.5 Q4 GGUF Model +In this Learning Path, you use [ERNIE-4.5](https://huggingface.co/collections/baidu/ernie-45) to deploy on Armv9. -In this learning path, you will use [ERNIE-4.5](https://huggingface.co/collections/baidu/ernie-45) to deploy in Arm v9. -Download both model variants so you can experiment later: +Download both model variants so you can experiment with each one: ```bash -mkdir -p ~/models/ernie-4.5 -cd ~/models/ernie-4.5 +mkdir -p $HOME/models/ernie-4.5 && cd $HOME/models/ernie-4.5 wget https://modelscope.cn/models/unsloth/ERNIE-4.5-21B-A3B-PT-GGUF/resolve/master/ERNIE-4.5-21B-A3B-PT-Q4_0.gguf wget https://modelscope.cn/models/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF/resolve/master/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf ``` -You can see the size of both models are 12 GB and quantized to Q4, making them suitable for CPU-only inference. +You can see the size of both models is 12 GB, and they're quantized to Q4, making them suitable for CPU-only inference. {{% notice Note %}} -The Q4 quantized models reduce memory footprint and allow CPU‑only inference — you’ll still need around 12 GB of RAM for good performance. +The Q4 quantized models reduce memory footprint and allow CPU-only inference. You still need around 12 GB of RAM for good performance. {{% /notice %}} -While both the Thinking and PT variants of ERNIE-4.5 share the same MoE architecture, they are fine-tuned for different objectives. The Thinking model is optimized for logical reasoning and structured generation, making it the main focus of subsequent benchmarking and hardware optimization. You are encouraged to install both variants and observe behavioral differences using the same prompt. +While both the Thinking and PT variants of ERNIE-4.5 share the same MoE architecture, they're fine-tuned for different objectives. The Thinking model is optimized for logical reasoning and structured generation, making it the main focus of subsequent benchmarking and hardware optimization. You're encouraged to install both variants and observe behavioral differences using the same prompt. -### Step 3: Run a Basic Inference Test +### Run a basic inference test Navigate to the build directory and run the following command to verify that the model loads correctly and supports multilingual input: ```bash -cd ~/llama.cpp/build +cd $HOME/llama.cpp/build ./bin/llama-cli \ --jinja \ -m ~/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf \ @@ -98,37 +98,27 @@ cd ~/llama.cpp/build ``` Note the flags: -- ***-p***: Passes the input prompt directly as a string. -- ***-c 4096***: Sets the context length (in tokens). A longer context allows the model to “remember” more input text, which is crucial for long-form tasks. Here we use the recommended 4096 tokens. -- ***-t 12***: Specifies the number of CPU threads used for inference. You should match this number to the physical cores (or logical threads) available on your system to maximize performance. -- ***--jinja***: Enables Jinja‑style prompt templates. Many Chinese‑oriented MoE models rely on this template format for structured inputs. +- `-p`: Passes the input prompt directly as a string. +- `-c 4096`: Sets the context length (in tokens). A longer context allows the model to "remember" more input text, which is crucial for long-form tasks. Here, you use the recommended 4096 tokens. +- `-t 12`: Specifies the number of CPU threads used for inference. You should match this number to the physical cores (or logical threads) available on your system to maximize performance. +- `--jinja`: Enables Jinja-style prompt templates. Many Chinese-oriented MoE models rely on this template format for structured inputs. -If everything is set up correctly, you will see metadata output from llama.cpp indicating the model’s architecture and size: +If everything is set up correctly, you see metadata output from llama.cpp indicating the model's architecture and size: -``` +```output print_info: model type = 21B.A3B print_info: model params = 21.83 B print_info: general.name = Ernie-4.5-21B-A3B-Thinking ``` -Once inference is complete, the expected output will look like this (in Chinese): +Once inference is complete, the expected output looks like this (in Chinese): ![img1 alt-text#center](meo_result.png "Introduce Mixture of Experts in Chinese") -This answer demonstrates the model’s multilingual and structured reasoning ability. It begins with a narrative explanation introducing the concept of Mixture of Experts (MoE), followed by a well-organized markdown-style summary with section headers and bullet points: -- Concept breakdown: It describes MoE as a model combining multiple specialized sub-models (experts) and a routing mechanism (gateway) to activate only a few experts per input. -- Workflow explanation: It explains how routing selects top experts based on input features and how the system reduces computation by only activating a small number of experts per token. -- Chinese NLP examples: It gives examples such as word segmentation and translation, explaining how different experts may focus on distinct subtasks. - -This confirms: -- The GGUF model is successfully loaded. -- The llama.cpp build functions as expected. -- CPU-only inference on Armv9 is working. +This answer demonstrates the model's multilingual and structured reasoning ability. It begins with a narrative explanation introducing the concept of Mixture of Experts (MoE), followed by a well-organized markdown-style summary with section headers and bullet points. The response describes MoE as a model combining multiple specialized sub-models (experts) and a routing mechanism (gateway) to activate only a few experts per input. It explains how routing selects top experts based on input features and how the system reduces computation by activating only a small number of experts per token. The answer also gives examples such as word segmentation and translation, explaining how different experts might focus on distinct subtasks in Chinese NLP applications. -#### Why This Prompt Matters +This confirms the GGUF model is successfully loaded, the llama.cpp build functions as expected, and CPU-only inference on Armv9 is working. -This prompt, “Please introduce Mixture of Experts in Chinese.”, was chosen for its dual pedagogical value: -- ***Bilingual Capability Check***: The instruction is issued in English, but the answer is expected in Chinese. This helps confirm that ERNIE-4.5’s multilingual support is active and effective. -- ***MoE Behavior Engagement***: The topic itself — explaining “Mixture of Experts” — requires combining multiple sub-skills: technical understanding, translation, and structured explanation. This likely triggers different experts within the model to contribute during inference. Even though routing isn’t explicitly logged, the richness and precision of the output suggest that MoE routing is functioning as designed. This kind of prompt increases the likelihood of multiple experts being activated simultaneously—e.g., language generation, machine learning knowledge, and Chinese translation. +The prompt "Please introduce Mixture of Experts in Chinese." was chosen because it tests two important capabilities. First, it verifies bilingual capability since the instruction is in English but the answer is expected in Chinese, confirming that ERNIE-4.5's multilingual support works correctly. Second, it engages MoE behavior by having the model explain a technical concept in Chinese, which likely activates different experts within the model during inference. Even though routing isn't explicitly logged, the richness and precision of the output suggest that MoE routing functions as designed. This type of prompt increases the likelihood of multiple experts being activated simultaneously, including language generation, machine learning knowledge, and Chinese translation experts. -By using a single prompt, you verify setup correctness, observe output quality, and gain insight into MoE inference characteristics — all essential elements before moving on to hardware-specific performance tuning. +You've now successfully set up llama.cpp on Armv9 hardware, deployed the ERNIE-4.5 model, and confirmed that multilingual inference works correctly. With this foundation in place, you're ready to explore expert routing behavior and benchmark performance optimizations in the next sections. diff --git a/content/learning-paths/cross-platform/ernie_moe_v9/3_erine_moe.md b/content/learning-paths/cross-platform/ernie_moe_v9/3_erine_moe.md index 1dc710952b..4c17416623 100644 --- a/content/learning-paths/cross-platform/ernie_moe_v9/3_erine_moe.md +++ b/content/learning-paths/cross-platform/ernie_moe_v9/3_erine_moe.md @@ -1,46 +1,49 @@ --- -title: Comparing ERNIE PT vs Thinking and Expert Routing +title: Compare ERNIE model behavior and expert routing weight: 4 layout: "learningpathall" --- -## Compare ERNIE PT vs Thinking Model Behavior +## Compare ERNIE PT and Thinking model behavior Now that both ERNIE-4.5 models are installed and verified, you can compare their output behavior on the same task. -In this module, you’ll compare the inference styles of PT and Thinking models, and learn how to inspect internal MoE expert routing behavior during generation. +In this section, you compare the inference styles of PT and Thinking models, and learn how to inspect internal MoE expert routing behavior during generation. -With both ERNIE-4.5 models installed and ready, we can now observe how their tuning objectives affect output—even though both share the same MoE architecture and parameter count (~21B total, ~3B activated at runtime): -- PT (Post-Trained): General-purpose, trained on multilingual corpora. -- Thinking: Tuned for multi-step reasoning, long context, and structured response generation. +Both ERNIE-4.5 models share the same MoE architecture and parameter count (around 21 B total, around 3 B activated at runtime), but they're tuned for different objectives: +- PT: General-purpose model trained on multilingual corpora for versatile tasks +- Thinking: Tuned for multi-step reasoning, long context, and structured response generation -### Example Task: Product Behavior Analysis -Copy the following prompt into a file named `prompt1.txt`: +You can now observe how these different tuning objectives affect output behavior. -``` +### Example task: Product behavior analysis + +Using an editor, copy the following prompt into a file named `prompt1.txt`: + +```txt You are a fitness brand strategist. User profile: Buys protein powder + dumbbells + gym wear, works out at home 4‑5× per week, shares results online, now exploring recovery nutrition and smart gym gear. Task: 1. Identify their top motivation and one hidden pain point. 2. Propose one new product line. -3. Create a short marketing tagline (≤ 15 words). +3. Create a short marketing tagline (≤ 15 words). ``` -Run the prompt using both models: +Run the prompt using both models. + +Here is the PT variant: -***PT Variant:*** ```bash ./bin/llama-cli \ - --jinja \ - -m ~/models/ernie-4.5/ERNIE-4.5-21B-A3B-PT-Q4_0.gguf \ + -m $HOME/models/ernie-4.5/ERNIE-4.5-21B-A3B-PT-Q4_0.gguf \ -f prompt1.txt \ -c 4096 -t 12 \ --jinja ``` -The answer will looks like: +The answer looks like this: -``` +```output Assistant: 1. **Top Motivation**: Achieving visible results and maintaining progress through efficient recovery nutrition. **Hidden Pain Point**: Balancing high-intensity training with optimal recovery nutrition during busy workouts. 2. **New Product Line**: *Smart Recovery Meal Kits* – Customizable, nutrient-dense, and easy-to-prepare post-workout meals designed for quick consumption and recovery optimization. @@ -49,24 +52,23 @@ Assistant: 1. **Top Motivation**: Achieving visible results and maintaining prog The answer shows: - Delivers conclusions directly: motivations and pain points are briefly mentioned with little reasoning. -- Product ideas are sensible but templated: suggestions like “Smart Recovery Meal Kits” are plausible but lack contextual grounding in user behavior. -- Tagline reads like a standard advertisement: e.g., “Fuel Progress, Recover Faster” feels promotional rather than personalized. +- Product ideas are sensible but templated: suggestions like "Smart Recovery Meal Kits" are plausible but lack contextual grounding in user behavior. +- Tagline reads like a standard advertisement: for example, "Fuel Progress, Recover Faster" feels promotional rather than personalized. +Here is the Thinking variant: -***Thinking Variant:*** ```bash ./bin/llama-cli \ - --jinja \ - -m ~/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf \ + -m $HOME/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf \ -f prompt1.txt \ -c 4096 -t 12 \ --jinja ``` -You will see the more comprehensive answer like: +You see a more comprehensive answer like this: -``` - assistant +```output +assistant Let's tackle this user profile. The user buys protein powder, dumbbells, and gym wear. They work out 4-5 times a week at home and share results online. Now they're exploring recovery nutrition and smart gym gear. @@ -93,32 +95,28 @@ So to sum up: 3. **Product Line**: **Smart Recovery Tracker** (wearable monitoring sleep/nutrition, syncing with protein powder/dumbbells to optimize recovery). 4. **Tagline**: *"Track. Recover. Share. Smart Gear for Your Journey."* (8 words) - ``` The answer shows: - Responses show deeper layering: begins with inferring user motivation, iterates through possible product strategies, and only then crafts a refined tagline. -- Transparent reasoning process: the model “thinks out loud” (“let me try again… maybe that’s too long…”), mimicking human deliberation. -- Marketing language reflects user mindset: taglines like “Share. Track. Conquer.” directly appeal to community sharing and progress motivation. +- Transparent reasoning process: the model "thinks out loud" ("let me try again… maybe that's too long…"), mimicking human deliberation. +- Marketing language reflects user mindset: taglines like "Share. Track. Conquer." directly appeal to community sharing and progress motivation. - -### Compare the Outputs +### Compare the outputs After execution, review the responses and compare them along the following dimensions: | **Aspect** | **PT Model** | **Thinking Model** | |----------------------|---------------------------------------------------|--------------------------------------------------------------------------------------| -| `Language Style` | Direct, ad-like tone | Layered and natural, mimics human internal reasoning | -| `Reasoning Depth` | High-level summarization with minimal exploration | Step-by-step inference with intermediate reflections and multiple alternatives | -| `Suggested Actions` | Pre-packaged idea (e.g., meal kits) | Context-aware concepts (e.g., smart gear that integrates with current user behavior) | - +| Language Style | Direct, ad-like tone | Layered and natural, mimics human internal reasoning | +| Reasoning Depth | High-level summarization with minimal exploration | Step-by-step inference with intermediate reflections and multiple alternatives | +| Suggested Actions | Pre-packaged idea (for example, meal kits) | Context-aware concepts (for example, smart gear that integrates with current user behavior) | +## Observe MoE expert routing with debug logs -## Observe MoE Expert Routing with Debug Logs +To look under the hood of the MoE model, you now add debug logging to observe internal MoE behavior and which experts are routed during inference. -If you want to look under the hood of the MoE model, you’ll now add debug log to observe internal MoE behavior which experts are routed during inference. - -Open `src/models/ernie4‑5‑moe.cpp` in the llama.cpp repository and locate the function build_moe_ffn(). -Insert a print statement right after the top‑k expert selection. For example: +Use an editor to open `src/models/ernie4-5-moe.cpp` in the llama.cpp repository and locate the function `build_moe_ffn()`. +Insert a print statement right before calling `build_moe_ffn()`. ```c printf("---[DEBUG]--- entering build_moe_ffn at layer %d with %d experts (use %d)\n", il, n_expert, n_expert_used); @@ -127,12 +125,13 @@ printf("---[DEBUG]--- entering build_moe_ffn at layer %d with %d experts (use %d Rebuild llama.cpp: ```bash -cd ~/llama.cpp +cd $HOME/llama.cpp/build make -j$(nproc) ``` -Run inference with the same prompt and monitor the console for lines such as: -``` +Run inference with the same prompt and monitor the console for lines like this: + +```output ---[DEBUG]--- entering build_moe_ffn at layer 1 with 64 experts (use 64) ---[DEBUG]--- entering build_moe_ffn at layer 2 with 64 experts (use 64) ---[DEBUG]--- entering build_moe_ffn at layer 3 with 64 experts (use 64) @@ -162,18 +161,18 @@ Run inference with the same prompt and monitor the console for lines such as: ---[DEBUG]--- entering build_moe_ffn at layer 27 with 64 experts (use 64) ``` -This reveals how many experts (e.g., 6) and how many tokens (e.g., 16) were routed at that layer. +This reveals how many experts (for example, 6) and how many tokens (for example, 16) were routed at that layer. {{% notice Note %}} -You can also trace the function `llm_graph_context::build_moe_ffn()` inside the `src/llama-graph.cpp` about how to select the expert. +You can also trace the function `llm_graph_context::build_moe_ffn()` in `src/llama-graph.cpp` to see how expert selection works. {{% /notice %}} -What to observe: -- Whether the number of active experts changes between the PT and Thinking models. -- Patterns in routing: e.g., different token batches routing to differing expert sets. -- Correlate routing behaviour with output differences: deeper routing variety may align with more detailed responses. +Remove the print statement from `src/models/ernie4-5-moe.cpp` before moving to the next section. + +As you review the debug output, observe whether the number of active experts changes between the PT and Thinking models. Look for patterns in routing, such as different token batches routing to differing expert sets. You can also correlate routing behavior with output differences, as deeper routing variety might align with more detailed responses. + +## Summary -## Takeaway -This task highlights the advantage of MoE fine-tuning: even under the same architecture, thoughtful tuning can significantly change a model’s reasoning behavior. It also reinforces that Thinking is better suited for applications requiring analytical depth—ideal for edge AI scenarios like customer profiling or real-time recommendations. +This task highlights the advantage of MoE fine-tuning. Even with the same architecture, thoughtful tuning can significantly change a model's reasoning behavior. The Thinking model is better suited for applications that need analytical depth, making it ideal for edge AI scenarios like customer profiling or real-time recommendations. -In the next section, you’ll switch focus from model behavior to system-level performance — compiling with Armv9 instruction sets and measuring the impact on inference speed. +In the next section, you switch focus from model behavior to system-level performance by compiling with Armv9 instruction sets and measuring the impact on inference speed. diff --git a/content/learning-paths/cross-platform/ernie_moe_v9/4_v9_optimization.md b/content/learning-paths/cross-platform/ernie_moe_v9/4_v9_optimization.md index 41dc5e9be1..3f0848d27f 100644 --- a/content/learning-paths/cross-platform/ernie_moe_v9/4_v9_optimization.md +++ b/content/learning-paths/cross-platform/ernie_moe_v9/4_v9_optimization.md @@ -1,27 +1,25 @@ --- -title: Arm v9 Optimization and MoE Efficiency +title: Optimize performance with Armv9 hardware features weight: 5 layout: "learningpathall" --- -## Accelerate ERNIE-4.5 with Armv9 Optimizations +## Accelerate ERNIE-4.5 with Armv9 optimizations -In previous modules, you've learned how MoE enables large model deployment on CPUs, and how to observe inference behavior with ERNIE-4.5. Now, we'll optimize performance using Armv9 architecture features and benchmark the improvements. +In previous sections, you learned how MoE enables large model deployment on CPUs, and how to observe inference behavior with ERNIE-4.5. You'll now optimize performance using Armv9 architecture features and benchmark the improvements. This section shows how to benchmark performance under two scenarios: with and without Armv9 vector instruction optimizations. -We’ll compare: -- Baseline: regular CPU build -- Optimized: Armv9-specific build with SVE/i8mm/dotprod enabled +You compare a baseline using a regular CPU build against an optimized version that's an Armv9-specific build with SVE, i8mm, and dotprod instructions enabled. -To establish a baseline performance, let’s first compile llama.cpp without Armv9 optimizations. +To establish baseline performance, first compile llama.cpp without Armv9 optimizations. -### Disable llama.cpp v9 Optimizations +### Disable llama.cpp Armv9 optimizations This step builds `llama.cpp` without Armv9 vector features to establish a baseline. ```bash -cd ~/llama.cpp +cd $HOME/llama.cpp mkdir build_v9_off && cd build_v9_off cmake \ -DLLAMA_CURL=OFF \ @@ -38,17 +36,16 @@ cmake \ -DGGML_F16C=off \ -DGGML_CPU_KLEIDIAI=OFF \ .. - make -j$(nproc) ``` -Then run benchmark in `build_v9_off` directory. +Run the benchmark in the `build_v9_off` directory: ```bash -./bin/llama-bench -m ~/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf -pg 128,128 -t 8 +./bin/llama-bench -m $HOME/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf -pg 128,128 -t 8 ``` -The result for 24GB Radxa O6 will be: +The output is similar to: | model | size | params | backend | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: | @@ -56,14 +53,14 @@ The result for 24GB Radxa O6 will be: | ernie4_5-moe 21B.A3B Q4_0 | 11.64 GiB | 21.83 B | CPU | 8 | tg128 | 12.03 ± 0.02 | | ernie4_5-moe 21B.A3B Q4_0 | 11.64 GiB | 21.83 B | CPU | 8 | pp128+tg128 | 13.51 ± 0.03 | -With the baseline captured, we now recompile with Armv9 vector extensions enabled. +With the baseline captured, recompile with Armv9 vector extensions enabled. -### Enable llama.cpp v9 Optimizations +### Enable llama.cpp Armv9 optimizations -Now rebuild with vector extensions enabled (i8mm, dotprod, SVE) by following configuration setting. +Rebuild with vector extensions enabled (i8mm, dotprod, SVE) using the following configuration settings: ```bash -cd ~/llama.cpp +cd $HOME/llama.cpp mkdir build_v9_on && cd build_v9_on cmake \ -DLLAMA_CURL=OFF \ @@ -81,21 +78,20 @@ cmake \ -DGGML_CPU_ARM_ARCH=armv9-a+i8mm+dotprod+sve \ -DGGML_CPU_KLEIDIAI=ON \ .. - make -j$(nproc) ``` {{% notice Note %}} -We disable GPU and other backend support to focus exclusively on CPU performance and optimization for this learning path. +This build disables GPU and other backend support to focus on CPU performance and optimization for this Learning Path. {{% /notice %}} -Then re-run benchmark in `build_v9_on` directory. +Re-run the benchmark in the `build_v9_on` directory: ```bash -./bin/llama-bench -m ~/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf -pg 128,128 -t 8 +./bin/llama-bench -m $HOME/models/ernie-4.5/ERNIE-4.5-21B-A3B-Thinking-Q4_0.gguf -pg 128,128 -t 8 ``` -The result for 24GB Radxa O6 will be: +The output is similar to: | model | size | params | backend | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: | @@ -104,9 +100,9 @@ The result for 24GB Radxa O6 will be: | ernie4_5-moe 21B.A3B Q4_0 | 11.64 GiB | 21.83 B | CPU | 8 | pp128+tg128 | 21.58 ± 0.11 | -Let’s now compare the results side-by-side to see how much performance is gained. +Compare the results side by side to see the performance gained. -### Comparing Performance: Armv9 Optimization Results +### Compare performance: Armv9 optimization results After running benchmarks with and without Armv9-specific instructions, the results show significant gains. @@ -116,17 +112,10 @@ After running benchmarks with and without Armv9-specific instructions, the resul | tg128 | 12.03 token/s | 15.96 token/s | 1.32x | | pp128 + tg128 | 13.51 token/s | 21.58 token/s | 1.59x | -- Vectorized kernels (i8mm, dotprod, SVE) drastically improve inference throughput. -- The pp512 test shows the most significant acceleration, delivering a 2.57× improvement. -- Other patterns like tg128 and pp128+tg128 also achieve measurable gains, demonstrating the broad benefit of hardware-aware builds. -- Armv9 optimization enables practical real-time inference for 21B models on edge-class hardware. - +Vectorized kernels (i8mm, dotprod, SVE) drastically improve inference throughput. The pp512 test shows the most significant acceleration with a 2.57× improvement. Other patterns like tg128 and pp128+tg128 also achieve measurable gains. These results demonstrate the broad benefit of hardware-aware builds and show that Armv9 optimization enables practical real-time inference for 21 B models on edge-class hardware. ### Summary -Over this learning path, you've walked through every stage of deploying a 21B parameter Chinese MoE model on edge-class Armv9 hardware. You: -- Understood how MoE reduces memory usage by only activating a small subset of parameters per token. -- Set up llama.cpp and deployed ERNIE-4.5 on a Radxa O6 board. -- Compared ERNIE-4.5 Thinking and PT model behaviors and examined expert routing logic with debug instrumentation. -- Applied Armv9 hardware optimizations to unlock over 2.5× speed improvements in token throughput. -You now have the full-stack capabilities to deploy, profile, and tune Chinese LLMs for efficient inference on modern Arm CPUs. +Throughout this Learning Path, you deployed a 21 B parameter Chinese MoE model on edge-class Armv9 hardware. You started by understanding how MoE reduces memory usage by activating only a small subset of parameters per token. After setting up llama.cpp on a Radxa O6 board, you compared ERNIE-4.5 Thinking and PT model behaviors while examining expert routing logic with debug instrumentation. Finally, you applied Armv9 hardware optimizations and achieved over 2.5× speed improvements in token throughput. + +You can now deploy, profile, and tune Chinese LLMs for efficient inference on modern Arm CPUs. diff --git a/content/learning-paths/cross-platform/ernie_moe_v9/_index.md b/content/learning-paths/cross-platform/ernie_moe_v9/_index.md index 3dac628c68..4b7b3ccd7c 100644 --- a/content/learning-paths/cross-platform/ernie_moe_v9/_index.md +++ b/content/learning-paths/cross-platform/ernie_moe_v9/_index.md @@ -1,5 +1,5 @@ --- -title: Running ERNIE Mixture of Experts (MoE) Models on Armv9 with llama.cpp +title: Run ERNIE-4.5 Mixture of Experts model on Armv9 with llama.cpp draft: true cascade: @@ -7,15 +7,15 @@ cascade: minutes_to_complete: 60 -who_is_this_for: This learning path is designed for developers and engineers looking to deploy Mixture-of-Experts (MoE) models — such as ERNIE-4.5 — on edge-class devices. MoE architectures allow massive LLMs (21B+ parameters) to run with only a fraction of their weights active per inference, making them ideal for resource-constrained environments. +who_is_this_for: This Learning Path is for developers and engineers who want to deploy Mixture of Experts (MoE) models, such as ERNIE 4.5, on edge devices. MoE architectures allow large LLMs with 21 billion or more parameters to run with only a fraction of their weights active per inference, making them ideal for resource constrained environments. learning_objectives: - - Understand how MoE models like ERNIE-4.5 enable large-scale inference on edge devices. - - Set up and execute ERNIE-4.5 (PT and Thinking versions) using llama.cpp and compare the inference behavior. - - Analyze the performance impact of enabling Armv9-specific hardware optimizations. + - Deploy MoE models like ERNIE-4.5 on edge devices using llama.cpp + - Compare inference behavior between ERNIE-4.5 PT and Thinking versions + - Measure performance impact of Armv9-specific hardware optimizations prerequisites: - - One Arm V9 device at least 32GB of available disk space. In this learning path, I use [Radxa O6](https://radxa.com/products/orion/o6/) + - An Armv9 device with at least 32 GB of available disk space, for example, Radxa Orion O6 author: Odin Shen @@ -42,17 +42,17 @@ shared_between: further_reading: - resource: - title: ERNIE-4.5-21B Modelscope link + title: ERNIE-4.5-21B Modelscope link: https://modelscope.cn/models/unsloth/ERNIE-4.5-21B-A3B-PT-GGUF type: website - resource: - title: llama.cpp github repo - link: https://github.com/ggml-org/llama.cpp.git + title: llama.cpp GitHub repository + link: https://github.com/ggml-org/llama.cpp type: documentation - resource: - title: Arm Learning Path - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/llama_cpp_streamline/ - type: Learning Path + title: Build and run llama.cpp with Arm CPU optimizations + link: /learning-paths/servers-and-cloud-computing/llama_cpp_streamline/ + type: website ### FIXED, DO NOT MODIFY diff --git a/content/learning-paths/cross-platform/gitlab-managed-runners/pipeline.md b/content/learning-paths/cross-platform/gitlab-managed-runners/pipeline.md index e8cb26a591..2a51972858 100644 --- a/content/learning-paths/cross-platform/gitlab-managed-runners/pipeline.md +++ b/content/learning-paths/cross-platform/gitlab-managed-runners/pipeline.md @@ -8,7 +8,7 @@ layout: learningpathall ## How to Create a CI/CD Pipeline with Gitlab-hosted Runners? -To create the pipeline we only need to create a new **`.gitlab-ci.yml`** file in our Project and define it's stages. Nothing else is needed since Gtilab-hosted runners are readily avilable to any Project and doesn't need to be created or instantiated by the Gitlab users. +To create the pipeline we only need to create a new **`.gitlab-ci.yml`** file in our Project and define it's stages. Nothing else is needed since Gtilab-hosted runners are readily available to any Project and doesn't need to be created or instantiated by the Gitlab users. Once we run our pipeline with the correct **`tags`** Gitlab will create everything that we need and yep it is as simple as that. @@ -19,7 +19,7 @@ Once we run our pipeline with the correct **`tags`** Gitlab will create everythi 2. We can choose to create **`.gitlab-ci.yml`** file by using one of the 2 options circled in red in the image below. ![CI-CD-New #center](_images/ci-cd-new.webp) -Option1: We can Click on **`Set up CI/CD`** button/link and follow the wizad to create an empty **`.gitlab-ci.yml`** file. +Option1: We can Click on **`Set up CI/CD`** button/link and follow the wizard to create an empty **`.gitlab-ci.yml`** file. Option2: Click on the "+" button. From the popup menu click on **`New File`** option and name the file **`.gitlab-ci.yml`** and then click on **`Commit Changes`** button on the top right hand side like in the image below (Add any message as your commit message). ![New-YML #center](_images/new-yml.png) diff --git a/content/learning-paths/cross-platform/gitlab-managed-runners/results.md b/content/learning-paths/cross-platform/gitlab-managed-runners/results.md index c4680bf19e..afde4d2267 100644 --- a/content/learning-paths/cross-platform/gitlab-managed-runners/results.md +++ b/content/learning-paths/cross-platform/gitlab-managed-runners/results.md @@ -12,7 +12,7 @@ The Pipeline script has multiple sections where each section instructs the pipel ### First Section: Stages -In this section we are describing how many squentional stages will our pipeline have and what are their names (ex. **`Build, Test and Deploy`**). If we would like all the stages or jobs to run simultinously then we simply don't define this section. +In this section we are describing how many sequential stages will our pipeline have and what are their names (ex. **`Build, Test and Deploy`**). If we would like all the stages or jobs to run simultaneously then we simply don't define this section. ### Second Section: Build-Job part of the Build stage @@ -25,17 +25,17 @@ Gitlab offers 3 Arm64 based Instances that use Linux as their OS. - saas-linux-medium-arm64 - saas-linux-large-arm64 -For more information about all Arm and other avilable Gitlab-hosted runners check [Gitlab-Hosted Runners](https://docs.gitlab.com/ci/runners/hosted_runners/linux/) page. +For more information about all Arm and other available Gitlab-hosted runners check [Gitlab-Hosted Runners](https://docs.gitlab.com/ci/runners/hosted_runners/linux/) page. {{%/notice%}} ### Other Sections: -The rest of the other sections follow the same patthern. You will notice that the **`Test`** stage for example has 2 Jobs in it (unit-test-job and lint-test-job). The **`Deploy`** stage here has only 1 Job called **`deploy-job`**. +The rest of the other sections follow the same pattern. You will notice that the **`Test`** stage for example has 2 Jobs in it (unit-test-job and lint-test-job). The **`Deploy`** stage here has only 1 Job called **`deploy-job`**. As you get to learn more YML scripting you will be able to add a lot more complex functionality to your pipelines. {{%notice Note%}} -Gitlab offers a lot of documentions on how to create pipeline that fits different needs and also offer common templates for them as well. You can access then from [Use CI/CD to build your application](https://docs.gitlab.com/topics/build_your_application/) page. +Gitlab offers a lot of documentation on how to create pipeline that fits different needs and also offer common templates for them as well. You can access then from [Use CI/CD to build your application](https://docs.gitlab.com/topics/build_your_application/) page. {{%/notice%}} ## How to run your pipeline for testing and to check the results: @@ -52,5 +52,5 @@ If you navigate to your pipeline editor from before you will notice that there a ### The other Tabs are: -1. Visualize: which can visulaize your pipeline for you as you edit it's componenets which can be very helpful especially for complex pipelines. -2. Validate: which can validate your pipeline script as you are editting them and saving from time to time so that you can catch any issues with you code early on. +1. Visualize: which can visualize your pipeline for you as you edit it's components which can be very helpful especially for complex pipelines. +2. Validate: which can validate your pipeline script as you are editing them and saving from time to time so that you can catch any issues with you code early on. diff --git a/content/learning-paths/cross-platform/kleidiai-explainer/_index.md b/content/learning-paths/cross-platform/kleidiai-explainer/_index.md index f84de2efc8..e85b36da02 100644 --- a/content/learning-paths/cross-platform/kleidiai-explainer/_index.md +++ b/content/learning-paths/cross-platform/kleidiai-explainer/_index.md @@ -11,7 +11,7 @@ learning_objectives: - Run a basic C++ matrix multiplication example to showcase the speedup that KleidiAI micro-kernels can deliver. prerequisites: - - An Arm-based Linux machine that implements the Int8 Matrix Multiplication (*i8mm*) architecture feature. The example in this Learning Path is run on an AWS Graviton 3 instance. Instructions on setting up an Arm-based server are [found here](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/aws/). + - An Arm-based Linux machine that implements the Int8 Matrix Multiplication (*i8mm*) architecture feature. The example in this Learning Path is run on an AWS Graviton 3 instance. Instructions on setting up an Arm-based server are [found here](/learning-paths/servers-and-cloud-computing/csp/aws/). - A basic understanding of linear algebra terminology, such as dot product and matrix multiplication. author: Zach Lasiuk diff --git a/content/learning-paths/cross-platform/kleidiai-explainer/page1.md b/content/learning-paths/cross-platform/kleidiai-explainer/page1.md index 37d0006556..d78dd8097e 100644 --- a/content/learning-paths/cross-platform/kleidiai-explainer/page1.md +++ b/content/learning-paths/cross-platform/kleidiai-explainer/page1.md @@ -40,13 +40,13 @@ KleidiAI uses modern Arm CPU instructions to accelerate matrix multiplication an ## What Arm features does KleidiAI leverage? Each KleidiAI matrix multiplication micro-kernel uses a specific Arm architecture feature to enhance AI inference. In this section you can read a description of each architecture feature that KleidiAI uses to accelerate matrix multiplication. -* **Dot Product**: KleidiAI uses the `vdotq_s32` intrinsic, which is a vector dot product, introduced as part of SIMD. It computes the dot product of two vector 8-bit integers, and accumulates the result into a 32-bit integer. View the `vdot` documentation [here](https://developer.arm.com/documentation/ddi0597/2024-03/SIMD-FP-Instructions/VDOT--by-element---BFloat16-floating-point-indexed-dot-product--vector--by-element--). +* **Dot Product**: KleidiAI uses the `vdotq_s32` intrinsic, which is a vector dot product, introduced as part of SIMD. It computes the dot product of two vector 8-bit integers, and accumulates the result into a 32-bit integer. View the [`vdot` documentation](https://developer.arm.com/documentation/ddi0597/2024-03/SIMD-FP-Instructions/VDOT--by-element---BFloat16-floating-point-indexed-dot-product--vector--by-element--). -* **SMMLA**: KleidiAI also makes use of the Int8 Matrix Multiplication (i8mm) feature including the `SMMLA` instruction (*Signed 8-bit integer matrix multiply-accumulate*). It multiplies a 2x8 matrix of 8-bit integers by a 8x2 matrix of 8-bit integers, which is accumulated into a 2x2 matrix of 32-bit integers. For more information, view the *SMMLA* and *i8mm* documentation [here](https://developer.arm.com/documentation/ddi0602/latest/SIMD-FP-Instructions/SMMLA--vector---Signed-8-bit-integer-matrix-multiply-accumulate--vector--). +* **SMMLA**: KleidiAI also makes use of the Int8 Matrix Multiplication (i8mm) feature including the `SMMLA` instruction (*Signed 8-bit integer matrix multiply-accumulate*). It multiplies a 2x8 matrix of 8-bit integers by a 8x2 matrix of 8-bit integers, which is accumulated into a 2x2 matrix of 32-bit integers. For more information, view the [*SMMLA* and *i8mm* documentation](https://developer.arm.com/documentation/ddi0602/latest/SIMD-FP-Instructions/SMMLA--vector---Signed-8-bit-integer-matrix-multiply-accumulate--vector--). -* **FMLA**: This instruction, *Floating-point Multiply Accumulate*, is for 16-bit operations. It is included as part of the Advanced SIMD extension, multiplying and accumulating two vectors together, each containing eight 16-bit numbers. View the `FMLA` documentation [here](https://developer.arm.com/documentation/ddi0602/2024-03/SIMD-FP-Instructions/FMLA--vector---Floating-point-fused-Multiply-Add-to-accumulator--vector--). +* **FMLA**: This instruction, *Floating-point Multiply Accumulate*, is for 16-bit operations. It is included as part of the Advanced SIMD extension, multiplying and accumulating two vectors together, each containing eight 16-bit numbers. View the [`FMLA` documentation](https://developer.arm.com/documentation/ddi0602/2024-03/SIMD-FP-Instructions/FMLA--vector---Floating-point-fused-Multiply-Add-to-accumulator--vector--). -* **FMOPA**: This instruction stands for *Floating-point outer product and accumulate*. It is included in the Arm Scalable Matrix Extension (SME). The single precision `FMOPA` variant enables optimized matrix multiplication on 32-bit numbers. View the `FMOPA` documentation [here](https://developer.arm.com/documentation/ddi0602/2023-12/SME-Instructions/FMOPA--non-widening---Floating-point-outer-product-and-accumulate-?lang=en). +* **FMOPA**: This instruction stands for *Floating-point outer product and accumulate*. It is included in the Arm Scalable Matrix Extension (SME). The single precision `FMOPA` variant enables optimized matrix multiplication on 32-bit numbers. View the [`FMOPA` documentation](https://developer.arm.com/documentation/ddi0602/2023-12/SME-Instructions/FMOPA--non-widening---Floating-point-outer-product-and-accumulate-?lang=en). Today, Arm-powered hardware containing these instructions exist in cloud servers and smartphones. Here are some examples of the first products from popular vendors that support KleidiAI: diff --git a/content/learning-paths/cross-platform/kleidiai-explainer/page2.md b/content/learning-paths/cross-platform/kleidiai-explainer/page2.md index f09396715f..fe1c0be900 100644 --- a/content/learning-paths/cross-platform/kleidiai-explainer/page2.md +++ b/content/learning-paths/cross-platform/kleidiai-explainer/page2.md @@ -19,7 +19,7 @@ There are essentially two types of KleidiAI micro-kernels today: ![KleidiAI stuff](KleidiAI-src.JPG "KleidiAI src directory") ### What are the quantization levels that KleidiAI supports? -KleidiAI has multiple matrix multiplication micro-kernels, and dynamic quantization routines, to optimally support all model quantization levels. To learn more about model quantization and how selecting the right quantization level affects your AI-based application, refer to [this Learning Path](https://learn.arm.com/learning-paths/servers-and-cloud-computing/llama-cpu/llama-chatbot#quantization-format). +KleidiAI has multiple matrix multiplication micro-kernels, and dynamic quantization routines, to optimally support all model quantization levels. To learn more about model quantization and how selecting the right quantization level affects your AI-based application, refer to [this Learning Path](/learning-paths/servers-and-cloud-computing/llama-cpu/llama-chatbot#quantization-format). KleidiAI currently has three matrix multiplication directories that each handle input/output types differently, and which will evolve to broaden the reach of their support: diff --git a/content/learning-paths/cross-platform/kleidiai-explainer/page3.md b/content/learning-paths/cross-platform/kleidiai-explainer/page3.md index 247011d455..f53234fd18 100644 --- a/content/learning-paths/cross-platform/kleidiai-explainer/page3.md +++ b/content/learning-paths/cross-platform/kleidiai-explainer/page3.md @@ -25,7 +25,7 @@ This reference code is functionally-identical to KleidiAI's micro-kernels, and i Follow these steps to build and run the KleidiAI library and example script: 1. Create an Ubuntu 24.04 Arm Linux machine on an AWS EC2 instance. -For more details view the Learning Path on [setting up AWS EC2 Graviton instances](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/aws/). Use an M7g-medium instance type, which uses the Graviton 3 SoC supporting the *i8mm* Arm architecture feature. The 1 CPU and 4 GB of RAM in the M7g-medium are sufficient for this basic example run. +For more details view the Learning Path on [setting up AWS EC2 Graviton instances](/learning-paths/servers-and-cloud-computing/csp/aws/). Use an M7g-medium instance type, which uses the Graviton 3 SoC supporting the *i8mm* Arm architecture feature. The 1 CPU and 4 GB of RAM in the M7g-medium are sufficient for this basic example run. 2. Initialize your system by installing essential packages: ```bash diff --git a/content/learning-paths/cross-platform/llm-fine-tuning-for-web-applications/how-to-9.md b/content/learning-paths/cross-platform/llm-fine-tuning-for-web-applications/how-to-9.md index b8c0718a02..cc32ba4986 100644 --- a/content/learning-paths/cross-platform/llm-fine-tuning-for-web-applications/how-to-9.md +++ b/content/learning-paths/cross-platform/llm-fine-tuning-for-web-applications/how-to-9.md @@ -27,8 +27,8 @@ layout: learningpathall --output_name="llama3_kv_sdpa_xnn_qe_4_32.pte" ``` -- Build the Llama Runner binary for [Android](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/5-run-benchmark-on-android/). -- Build and Run [Android](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/6-build-android-chat-app/). +- Build the Llama Runner binary for [Android](/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/5-run-benchmark-on-android/). +- Build and Run [Android](/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/6-build-android-chat-app/). - Open Android Studio and choose "Open an existing Android Studio project" to navigate to examples/demo-apps/android/LlamaDemo and Press Run (^R) to build and launch the app on your phone. - Tap the Settings widget to select a model, configure its parameters, and set any prompts. - After choosing the model, tokenizer, and model type, click "Load Model" to load it into the app and return to the main Chat activity. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/mcp-ai-agent/intro-to-mcp-uv.md b/content/learning-paths/cross-platform/mcp-ai-agent/intro-to-mcp-uv.md index 94ff72c1f7..fdd7644ecd 100644 --- a/content/learning-paths/cross-platform/mcp-ai-agent/intro-to-mcp-uv.md +++ b/content/learning-paths/cross-platform/mcp-ai-agent/intro-to-mcp-uv.md @@ -46,7 +46,7 @@ Each component in the diagram plays a distinct role in enabling AI agents to int - The **Remote services** are external APIs the server can call on the host’s behalf. {{% notice Learning Tip %}} -Learn more about AI Agents in the Learning Path [Deploy an AI Agent on Arm with llama.cpp and llama-cpp-agent using KleidiAI](https://learn.arm.com/learning-paths/servers-and-cloud-computing/ai-agent-on-cpu/). +Learn more about AI Agents in the Learning Path [Deploy an AI Agent on Arm with llama.cpp and llama-cpp-agent using KleidiAI](/learning-paths/servers-and-cloud-computing/ai-agent-on-cpu/). {{% /notice %}} ## Section summary diff --git a/content/learning-paths/cross-platform/memory-latency/_index.md b/content/learning-paths/cross-platform/memory-latency/_index.md index ddd715b8ed..1bd10818a6 100644 --- a/content/learning-paths/cross-platform/memory-latency/_index.md +++ b/content/learning-paths/cross-platform/memory-latency/_index.md @@ -38,7 +38,7 @@ shared_between: further_reading: - resource: title: Write a Dynamic Memory Allocator - link: https://learn.arm.com/learning-paths/cross-platform/dynamic-memory-allocator/ + link: /learning-paths/cross-platform/dynamic-memory-allocator/ type: website - resource: title: Memory Latency diff --git a/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/_index.md b/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/_index.md index 771da15452..e637f13a5b 100644 --- a/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/_index.md +++ b/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/_index.md @@ -52,7 +52,7 @@ further_reading: type: documentation - resource: title: Port Code to Arm Scalable Vector Extension (SVE) - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/sve + link: /learning-paths/servers-and-cloud-computing/sve type: website - resource: title: Introducing the Scalable Matrix Extension for the Armv9-A Architecture @@ -72,7 +72,7 @@ further_reading: type: blog - resource: title: Build adaptive libraries with multiversioning - link: https://learn.arm.com/learning-paths/cross-platform/function-multiversioning/ + link: /learning-paths/cross-platform/function-multiversioning/ type: website - resource: title: SME Programmer's Guide diff --git a/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/overview.md b/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/overview.md index ed46aca044..97e21c5094 100644 --- a/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/overview.md +++ b/content/learning-paths/cross-platform/multiplying-matrices-with-sme2/overview.md @@ -47,4 +47,4 @@ On SVE and SME: - [Arm Scalable Matrix Extension (SME) Introduction (Part 1) - Zenon Xiu](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-scalable-matrix-extension-introduction) - [Arm Scalable Matrix Extension (SME) Introduction (Part 2) - Zenon Xiu](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/arm-scalable-matrix-extension-introduction-p2) - [Matrix-matrix multiplication. Neon, SVE, and SME compared (Part 3)](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/.matrix-matrix-multiplication-neon-sve-and-sme-compared) -- [Learn about function multiversioning - Alexandros Lamprineas, Arm](https://learn.arm.com/learning-paths/cross-platform/function-multiversioning/) \ No newline at end of file +- [Learn about function multiversioning - Alexandros Lamprineas, Arm](/learning-paths/cross-platform/function-multiversioning/) \ No newline at end of file diff --git a/content/learning-paths/cross-platform/psa-tfm/run.md b/content/learning-paths/cross-platform/psa-tfm/run.md index efbbe6c6d3..c68fccc510 100644 --- a/content/learning-paths/cross-platform/psa-tfm/run.md +++ b/content/learning-paths/cross-platform/psa-tfm/run.md @@ -8,7 +8,7 @@ weight: 4 # 1 is first, 2 is second, etc. layout: "learningpathall" --- {{% notice MPS3%}} -MPS3 users can jump to [here](#mps3). +MPS3 users can jump to the [MPS3 section](#mps3). {{% /notice %}} diff --git a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md index 164e7b605f..7d49ae2dd9 100644 --- a/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md +++ b/content/learning-paths/cross-platform/pytorch-digit-classification-arch-training/intro.md @@ -47,7 +47,7 @@ If Python3 is not installed, you can download and install it from [python.org](h Alternatively, you can also install Python3 using package managers such as Homebrew or APT. -If you are using Windows on Arm, see the [Python install guide](https://learn.arm.com/install-guides/py-woa/). +If you are using Windows on Arm, see the [Python install guide](/install-guides/py-woa/). Next, if you do not already have it, download and install [Visual Studio Code](https://code.visualstudio.com/download). diff --git a/content/learning-paths/cross-platform/simd-loops/_index.md b/content/learning-paths/cross-platform/simd-loops/_index.md index d9cfa835ba..2826a50e88 100644 --- a/content/learning-paths/cross-platform/simd-loops/_index.md +++ b/content/learning-paths/cross-platform/simd-loops/_index.md @@ -49,7 +49,7 @@ further_reading: type: documentation - resource: title: Port Code to Arm Scalable Vector Extension (SVE) - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/sve + link: /learning-paths/servers-and-cloud-computing/sve type: website - resource: title: Introducing the Scalable Matrix Extension for the Armv9-A Architecture @@ -69,7 +69,7 @@ further_reading: type: blog - resource: title: Build adaptive libraries with multiversioning - link: https://learn.arm.com/learning-paths/cross-platform/function-multiversioning/ + link: /learning-paths/cross-platform/function-multiversioning/ type: website - resource: title: SME Programmer's Guide diff --git a/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part1.md b/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part1.md index 0f66eb4015..328c39d82a 100644 --- a/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part1.md +++ b/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part1.md @@ -282,7 +282,7 @@ Feature detection in particular refers to minor extensions in the ISA that are n #endif ``` -A full list of current extensions for Arm can be found [here](https://doc.rust-lang.org/std/arch/macro.is_aarch64_feature_detected.html) while the full list of supported intrinsics is [here](https://doc.rust-lang.org/core/arch/aarch64/index.html). +A [full list of current extensions for Arm](https://doc.rust-lang.org/std/arch/macro.is_aarch64_feature_detected.html) is available, and the [full list of supported intrinsics](https://doc.rust-lang.org/core/arch/aarch64/index.html) is also documented. ## An alternative way with Rust using std::simd diff --git a/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part4.md b/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part4.md index 969ce2ae27..68bef0be1b 100644 --- a/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part4.md +++ b/content/learning-paths/cross-platform/simd-on-rust/simd-on-rust-part4.md @@ -225,7 +225,7 @@ fff6 0000 0000 0000 A 4x4 matrix `a` is initialized and a `fDCT` function is called on it. The function carries out 2 passes of the same algorithm on the elements of the array, calls the 2 butterfly functions (for one and two coefficients respectively) and transposes the results in between the calculations. The result is rounded and stored in the output buffer `dct`. -The assembly output is linked [here](/learning-paths/cross-platform/simd-on-rust/butterfly1.asm) instead of being displayed due to its size. +The [assembly output](/learning-paths/cross-platform/simd-on-rust/butterfly1.asm) is available separately due to its size. Now create a Rust version of this algorithm and save the contents below in a file called `butterfly2.rs`: @@ -393,7 +393,7 @@ ff71 0000 0000 0000 fff6 0000 0000 0000 ``` -The disassembly output is linked [here](/learning-paths/cross-platform/simd-on-rust/butterfly2.asm) for size reasons. You will see that it is very similar to the C version, apart from the cpu feature check at the start. +The [disassembly output](/learning-paths/cross-platform/simd-on-rust/butterfly2.asm) is available separately for size reasons. You will see that it is very similar to the C version, apart from the cpu feature check at the start. ### Comments diff --git a/content/learning-paths/cross-platform/topdown-compare/_index.md b/content/learning-paths/cross-platform/topdown-compare/_index.md index bf454668bb..406f43b14e 100644 --- a/content/learning-paths/cross-platform/topdown-compare/_index.md +++ b/content/learning-paths/cross-platform/topdown-compare/_index.md @@ -51,7 +51,7 @@ further_reading: type: documentation - resource: title: How to use the Arm Performance Monitoring Unit and System Counter - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/arm_pmu/ + link: /learning-paths/servers-and-cloud-computing/arm_pmu/ type: website diff --git a/content/learning-paths/cross-platform/vectorization-comparison/_index.md b/content/learning-paths/cross-platform/vectorization-comparison/_index.md index b67b5168fb..c1f37d0f0c 100644 --- a/content/learning-paths/cross-platform/vectorization-comparison/_index.md +++ b/content/learning-paths/cross-platform/vectorization-comparison/_index.md @@ -42,7 +42,7 @@ further_reading: type: documentation - resource: title: Port code to Arm Scalable Vector Extension (SVE) - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/sve + link: /learning-paths/servers-and-cloud-computing/sve type: website - resource: title: Introducing the Scalable Matrix Extension for the Armv9-A Architecture @@ -54,7 +54,7 @@ further_reading: type: blog - resource: title: Build adaptive libraries with multiversioning - link: https://learn.arm.com/learning-paths/cross-platform/function-multiversioning/ + link: /learning-paths/cross-platform/function-multiversioning/ type: website - resource: title: SME Programmer's Guide diff --git a/content/learning-paths/embedded-and-microcontrollers/asm/intro.md b/content/learning-paths/embedded-and-microcontrollers/asm/intro.md index 315610ba72..eb10c5f1f5 100644 --- a/content/learning-paths/embedded-and-microcontrollers/asm/intro.md +++ b/content/learning-paths/embedded-and-microcontrollers/asm/intro.md @@ -22,6 +22,6 @@ The assembly level functions will conform to the [Arm Procedure Call Standard](h [Keil MDK](https://www.keil.arm.com/) can support a number of different environments. -Keil Studio is a collection of `Visual Studio Code` Extensions to provide a complete development IDE. For installation instructions, refer to the [Arm Keil Studio for VS Code](/install-guides/keilstudio_vs/) install guide. Windows, Linux, and MacOS are supported hosts. +Keil Studio is a collection of `Visual Studio Code` Extensions to provide a complete development IDE. For installation instructions, refer to the [Arm Keil Studio for VS Code](/install-guides/keilstudio_vs/) install guide. Windows, Linux, and macOS are supported hosts. You can also use the legacy μVision IDE. For installation instructions, refer to the [Keil μVision](/install-guides/mdk/) install guide. Note that only Windows is a supported host. diff --git a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md index 195ad5f4cc..2c3a0c4822 100644 --- a/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md +++ b/content/learning-paths/embedded-and-microcontrollers/customer-support-chatbot-with-llama-and-executorch-on-arm-based-mobile-devices/how-to-2.md @@ -8,7 +8,7 @@ layout: learningpathall ## Android NDK and Android Studio - Environment Setup -#### Plartform Required +#### Platform Required - An AWS Graviton4 r8g.16xlarge instance to test Arm performance optimizations, or any [Arm based instance](/learning-paths/servers-and-cloud-computing/csp/) from a cloud service provider or an on-premise Arm server or Arm based laptop. - An Arm-powered smartphone with the i8mm feature running Android, with 16GB of RAM. - A USB cable to connect your smartphone to your development machine. diff --git a/content/learning-paths/embedded-and-microcontrollers/edge/_index.md b/content/learning-paths/embedded-and-microcontrollers/edge/_index.md index a0fa9edcf2..2e2a10645f 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge/_index.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge/_index.md @@ -12,7 +12,7 @@ learning_objectives: - Control LEDs by turning them on and off based on model predictions. prerequisites: - - Completion of [Embedded programming with Arduino on the Raspberry Pi Pico](https://learn.arm.com/learning-paths/embedded-and-microcontrollers/arduino-pico/) if you're an absolute beginner. + - Completion of [Embedded programming with Arduino on the Raspberry Pi Pico](/learning-paths/embedded-and-microcontrollers/arduino-pico/) if you're an absolute beginner. - An [Edge Impulse Studio](https://studio.edgeimpulse.com/signup) account. - The [Arduino IDE](/install-guides/arduino-pico/) with the RP2040 board support package installed on your computer. - An [Arduino Nano RP2040 Connect board](https://store.arduino.cc/products/arduino-nano-rp2040-connect-with-headers). diff --git a/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/edgeimpulseprojectbuild.md b/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/edgeimpulseprojectbuild.md index 9e5582253b..917653d9d6 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/edgeimpulseprojectbuild.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/edgeimpulseprojectbuild.md @@ -74,7 +74,7 @@ Clicking on "Object Detection" on the left, you will see some detail on the mode ![Edge Impulse](./images/EI_Project_3.png) -In our project, the "Impulse" is fully created, trained, and optimized so we won't have to walk through those steps. Edge Impulse has a ton of examples and docs available [here](https:://docs.edgeimpulse.com) to walk you through your first "Impulse" creation: +In our project, the "Impulse" is fully created, trained, and optimized so we won't have to walk through those steps. Edge Impulse has a ton of [examples and documentation](https:://docs.edgeimpulse.com) to walk you through your first "Impulse" creation: ![Edge Impulse](./images/EI_Project_4.png) diff --git a/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/overview.md b/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/overview.md index a196732bfb..17296d8266 100644 --- a/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/overview.md +++ b/content/learning-paths/embedded-and-microcontrollers/edge_impulse_greengrass/overview.md @@ -23,9 +23,9 @@ The Edge Impulse integration with AWS IoT Core and AWS IoT Greengrass is structu * The Edge Impulse "Runner" service can relay inference results into IoT Core for further processing in the cloud * The Edge Impulse "Runner" service relays model performance metrics, at configurable intervals, into IoTCore for further processing. * The Edge Impulse "Runner" service has accessible commands that can be used to configure the service real-time as well as retrieve information about the model/service/configuration. -* More information regarding the Edge Impulse "Runner" service itself can be found [here](https://docs.edgeimpulse.com/docs/tools/edge-impulse-for-linux/linux-node-js-sdk). +* More information regarding the Edge Impulse "Runner" service itself can be found in the [Edge Impulse for Linux Node.js SDK documentation](https://docs.edgeimpulse.com/docs/tools/edge-impulse-for-linux/linux-node-js-sdk). -Edge Impulse has several custom Greengrass components that can be deployed and run on the Greengrass-enabled edge device to enable this integration. The component recipes and artifacts can be found [here](https://github.com/edgeimpulse/aws-greengrass-components). Lets examine one of those components that we'll used for this workshop! +Edge Impulse has several custom Greengrass components that can be deployed and run on the Greengrass-enabled edge device to enable this integration. The component recipes and artifacts can be found in the [AWS Greengrass components repository](https://github.com/edgeimpulse/aws-greengrass-components). Lets examine one of those components that we'll used for this workshop! ### The "EdgeImpulseLinuxRunnerServiceComponent" Greengrass Component @@ -45,7 +45,7 @@ Command results are published to the following topic: /edgeimpulse/device//command/output -The command reference, including JSON structure details, can be found [here](https://docs.edgeimpulse.com/docs/integrations/aws-greengrass#commands-january-2025-integration-enhancements). +The command reference, including JSON structure details, can be found in the [Edge Impulse AWS Greengrass integration documentation](https://docs.edgeimpulse.com/docs/integrations/aws-greengrass#commands-january-2025-integration-enhancements). Lets dive deeper into this integration starting with setting up our own edge device! diff --git a/content/learning-paths/embedded-and-microcontrollers/llama-python-cpu/llama-python-chatbot.md b/content/learning-paths/embedded-and-microcontrollers/llama-python-cpu/llama-python-chatbot.md index 1ac931ddee..5ed148deb5 100644 --- a/content/learning-paths/embedded-and-microcontrollers/llama-python-cpu/llama-python-chatbot.md +++ b/content/learning-paths/embedded-and-microcontrollers/llama-python-cpu/llama-python-chatbot.md @@ -26,8 +26,7 @@ You use [llama.cpp](https://github.com/ggerganov/llama.cpp), an open source C/C+ Memory size is an important factor to consider when selecting an LLM because many LLMs have memory requirements that are too large for edge devices, such as the Raspberry Pi 5. An idea of the required memory size can be obtained by looking at the number of parameters of the model. A higher number of parameters means more memory used. -You can also use the [Memory Model Calculator](https://huggingface.co/docs/accelerate/main/en/usage_guides/model_size_estimator -) from Hugging Face to estimate memory size. +You can also use the [Memory Model Calculator](https://huggingface.co/docs/accelerate/main/en/usage_guides/model_size_estimator) from Hugging Face to estimate memory size. Copy the string below: diff --git a/content/learning-paths/embedded-and-microcontrollers/project-migration-cmsis-v6/project_format.md b/content/learning-paths/embedded-and-microcontrollers/project-migration-cmsis-v6/project_format.md index bf2eca3c17..1f62c5c4ba 100644 --- a/content/learning-paths/embedded-and-microcontrollers/project-migration-cmsis-v6/project_format.md +++ b/content/learning-paths/embedded-and-microcontrollers/project-migration-cmsis-v6/project_format.md @@ -10,4 +10,4 @@ layout: learningpathall The following section is only relevant to users of Keil MDK v5. It explains how to migrate from a uvprojx-based project file to the new Open-CMSIS-Pack csolution project file format. -The learning path [Convert uvprojx-based projects to csolution](https://learn.arm.com/learning-paths/embedded-and-microcontrollers/uvprojx-conversion/) explains how to import, convert, and build uvprojx-based projects in [Keil Studio for VS Code](https://learn.arm.com/install-guides/keilstudio_vs/). It also shows how to convert and build uvprojx-based projects on the command line. +The learning path [Convert uvprojx-based projects to csolution](/learning-paths/embedded-and-microcontrollers/uvprojx-conversion/) explains how to import, convert, and build uvprojx-based projects in [Keil Studio for VS Code](/install-guides/keilstudio_vs/). It also shows how to convert and build uvprojx-based projects on the command line. diff --git a/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/dev-env.md b/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/dev-env.md index 76d82897cf..348b8eb84f 100644 --- a/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/dev-env.md +++ b/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/dev-env.md @@ -9,7 +9,7 @@ layout: learningpathall This Learning Path has been validated on Ubuntu 22.04 LTS and macOS. {{% notice %}} -If you are running Windows, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [Get started with Windows Subsystem for Linux (WSL) on Arm](https://learn.arm.com/learning-paths/laptops-and-desktops/wsl2/setup/) to learn more. +If you are running Windows, you can use Ubuntu through Windows subsystem for Linux 2 (WSL2). Check out [Get started with Windows Subsystem for Linux (WSL) on Arm](/learning-paths/laptops-and-desktops/wsl2/setup/) to learn more. {{% /notice %}} ## Install software tools diff --git a/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/flash-and-run.md b/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/flash-and-run.md index 90ce184e42..ac03163972 100644 --- a/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/flash-and-run.md +++ b/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/flash-and-run.md @@ -53,7 +53,7 @@ The COM identifier is prefixed with **tty**, so you can use this to help you ide {{< tab header="Linux" language="shell">}} sudo grep -i 'tty' /var/log/dmesg {{< /tab >}} - {{< tab header="MacOS" language="shell">}} + {{< tab header="macOS" language="shell">}} ls /dev/tty.* {{< /tab >}} {{< /tabpane >}} diff --git a/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/web-toolkit.md b/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/web-toolkit.md index db5f254f0f..63ad577719 100644 --- a/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/web-toolkit.md +++ b/content/learning-paths/embedded-and-microcontrollers/yolo-on-himax/web-toolkit.md @@ -54,7 +54,7 @@ Run the script corresponding to the OS of your host machine. {{< tab header="Linux" language="shell">}} ./we2_local_image_gen project_case1_blp_wlcsp.json {{< /tab >}} - {{< tab header="MacOS" language="shell">}} + {{< tab header="macOS" language="shell">}} ./we2_local_image_gen_macOS_arm64 project_case1_blp_wlcsp.json {{< /tab >}} {{< /tabpane >}} diff --git a/content/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/_index.md b/content/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/_index.md index 5d5a37757f..51238f0414 100644 --- a/content/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/_index.md +++ b/content/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/_index.md @@ -46,7 +46,7 @@ further_reading: type: documentation - resource: title: Profile llama.cpp performance with Arm Streamline and KleidiAI LLM kernels Learning Path - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/llama_cpp_streamline/ + link: /learning-paths/servers-and-cloud-computing/llama_cpp_streamline/ type: blog - resource: title: Arm-Powered NVIDIA DGX Spark Workstations to Redefine AI diff --git a/content/learning-paths/laptops-and-desktops/dgx_spark_rag/_index.md b/content/learning-paths/laptops-and-desktops/dgx_spark_rag/_index.md index fc29aba1a5..e9b2796c6b 100644 --- a/content/learning-paths/laptops-and-desktops/dgx_spark_rag/_index.md +++ b/content/learning-paths/laptops-and-desktops/dgx_spark_rag/_index.md @@ -42,7 +42,7 @@ further_reading: type: documentation - resource: title: Unlock quantized LLM performance on Arm-based NVIDIA DGX Spark - link: https://learn.arm.com/learning-paths/laptops-and-desktops/dgx_spark_llamacpp/ + link: /learning-paths/laptops-and-desktops/dgx_spark_llamacpp/ type: Learning Path diff --git a/content/learning-paths/laptops-and-desktops/electron/_index.md b/content/learning-paths/laptops-and-desktops/electron/_index.md index 6ff92a6562..3852fd958a 100644 --- a/content/learning-paths/laptops-and-desktops/electron/_index.md +++ b/content/learning-paths/laptops-and-desktops/electron/_index.md @@ -11,7 +11,7 @@ learning_objectives: prerequisites: - A Windows on Arm computer such as the Lenovo Thinkpad X13s running Windows 11 or a Windows on Arm [virtual machine](/learning-paths/cross-platform/woa_azure/). - - Node.js for Arm64. You can find the installer [here](https://nodejs.org/dist/v20.10.0/node-v20.10.0-arm64.msi) + - Node.js for Arm64. You can find the [Node.js installer](https://nodejs.org/dist/v20.10.0/node-v20.10.0-arm64.msi). - Any code editor; we recommend using [Visual Studio Code for Arm64](https://code.visualstudio.com/docs/?dv=win32arm64user). author: Dawid Borycki diff --git a/content/learning-paths/laptops-and-desktops/electron/how-to-1.md b/content/learning-paths/laptops-and-desktops/electron/how-to-1.md index 9cc294f2d4..8222ca37f7 100644 --- a/content/learning-paths/laptops-and-desktops/electron/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/electron/how-to-1.md @@ -15,10 +15,10 @@ In this learning path you will learn how to create a Desktop application for Win Here, you will use the posts endpoint of the `JSONPlaceholder`. This endpoint enables you to retrieve the list of hypothetical posts. -You can find the the complete code used in the learning path [here](https://github.com/dawidborycki/electron-sample-app.git) +You can find the [complete code on GitHub](https://github.com/dawidborycki/electron-sample-app.git). ## Before you begin -Before you begin, install Node.JS for Arm64. You can find the installer [here](https://nodejs.org/en/download). In this learning path, you will use version 20.10.0. The installation process is automatic. However, make sure to check the "Automatically install the necessary tools" checkbox so that it automatically installs the build tools for the NPM packages: +Before you begin, install Node.JS for Arm64. You can find the [Node.js installer](https://nodejs.org/en/download). In this learning path, you will use version 20.10.0. The installation process is automatic. However, make sure to check the "Automatically install the necessary tools" checkbox so that it automatically installs the build tools for the NPM packages: ![fig1](figures/01.png) diff --git a/content/learning-paths/laptops-and-desktops/kleidicv-on-mac/_index.md b/content/learning-paths/laptops-and-desktops/kleidicv-on-mac/_index.md index f479de08a5..8ae70746e2 100644 --- a/content/learning-paths/laptops-and-desktops/kleidicv-on-mac/_index.md +++ b/content/learning-paths/laptops-and-desktops/kleidicv-on-mac/_index.md @@ -39,7 +39,7 @@ further_reading: type: blog - resource: title: Learn about function multiversioning - link: https://learn.arm.com/learning-paths/cross-platform/function-multiversioning/ + link: /learning-paths/cross-platform/function-multiversioning/ type: website diff --git a/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/_index.md b/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/_index.md index e2763e261d..a1f0c8e12a 100644 --- a/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/_index.md +++ b/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/_index.md @@ -12,8 +12,8 @@ learning_objectives: prerequisites: - An Arm64-powered machine, either virtual or physical. This Learning Path demonstration uses an Arm64-powered VM with Ubuntu 22.04. - - A DockerHub account. You can set up a free account [here](https://hub.docker.com/signup). - - A GitHub account. You can sign up [here](https://github.com/signup). + - A DockerHub account. You can [set up a free DockerHub account](https://hub.docker.com/signup). + - A GitHub account. You can [sign up for GitHub](https://github.com/signup). author: Dawid Borycki diff --git a/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/how-to-1.md b/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/how-to-1.md index 49241d38fc..c5c9f6eb03 100644 --- a/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/self_hosted_cicd_github/how-to-1.md @@ -10,7 +10,7 @@ layout: "learningpathall" In this Learning Path, you discover how to configure and use an Arm64 runner that builds a .NET application for Arm64. Additionally, the CI/CD pipeline you create generates an Arm64 Docker image of the application and then pushes the image to a Docker Hub repository. -Before completing this Learning Path you can complete the Hello World [example](https://learn.arm.com/learning-paths/laptops-and-desktops/windows_cicd_github/), which provides a basic "hello world" scenario. +Before completing this Learning Path you can complete the Hello World [example](/learning-paths/laptops-and-desktops/windows_cicd_github/), which provides a basic "hello world" scenario. You will extend that knowledge here with a comprehensive set of operations critical for real-world application deployment: diff --git a/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-1.md index 9747e7cc58..96ca3c6572 100644 --- a/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-1.md @@ -20,7 +20,7 @@ To follow this learning path: 1. Install [Visual Studio 2022 or higher](/install-guides/vs-woa) with Arm64 build tools. 2. Install [Python](/install-guides/py-woa) on your machine. In this learning path, Python version 3.11.3 was used. -The complete project code used in this learning path is hosted [here](https://github.com/dawidborycki/ARM64EC.Porting). +The [complete project code](https://github.com/dawidborycki/ARM64EC.Porting) is hosted on GitHub. ## Project Setup To set up the project, start by creating the dependencies (the DLLs). In this example you will use CMake in Visual Studio 2022 to create the base project for your dependencies. You can also use MS Build/Visual C++ project templates to compile to Arm64EC by adding the architecture to your build configuration. To access CMake, open Visual Studio 2022 and click Create a new project and look for CMake Project in the window that appears. diff --git a/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-3.md b/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-3.md index da6bb5410c..0dbef7ceeb 100644 --- a/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-3.md +++ b/content/learning-paths/laptops-and-desktops/win_arm64ec_porting/how-to-3.md @@ -121,7 +121,7 @@ Then, right-click on the Arm64EC.Porting.Vectors file and open the Properties wi The compiled DLL will be available under the ARM64EC subfolder. You can then use it in the **Main-app** in the same way as the DLLs built using CMake. -To use MSBuild for the second DLL, proceed in the same way. For your convenience, you can refer to the full source code in this [git repository](https://github.com/dawidborycki/Arm64EC.Porting.MSBuild/). +To use MSBuild for the second DLL, proceed in the same way. For your convenience, you can refer to the [full source code on GitHub](https://github.com/dawidborycki/Arm64EC.Porting.MSBuild/). ## Summary This learning path demonstrated how to port C/C++ DLLs to Arm64 using Arm64EC. It showed you how to use CMake and MSBuild projects. You also learned how to use DLLs in the Python app with ctypes. This information can help you rapidly build UIs and keep computation-intensive work in a separate DLL. You can then build those DLLs for Arm64 and mix them with your existing x64 dependencies. diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_aws_iot/how-to-1.md index 98e37fa169..4baf846600 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot/how-to-1.md @@ -27,8 +27,8 @@ In this learning path, you create an application that emulates the weather stati ## Before you begin Before you start, ensure you have installed and prepared the following tools: -1. Download and install Node.js for ARM64 [here](https://nodejs.org/dist/v21.7.3/node-v21.7.3-arm64.msi). -2. Download and configure AWS CLI [here](https://aws.amazon.com/cli/). +1. Download and install [Node.js for ARM64](https://nodejs.org/dist/v21.7.3/node-v21.7.3-arm64.msi). +2. Download and configure the [AWS CLI](https://aws.amazon.com/cli/). ## Create the project You now create a Node.js project. Open the Command Prompt and type the following commands: diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb/how-to-1.md index f4a956cc78..72db09e435 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb/how-to-1.md @@ -7,7 +7,7 @@ weight: 3 layout: "learningpathall" --- ## Before you begin -Before you begin, prepare the weather station emulator and connect it to AWS IoT Core by completing this [Learning Path](/learning-paths/laptops-and-desktops/win_aws_iot). +Before you begin, prepare the weather station emulator and connect it to AWS IoT Core by completing [Create IoT applications with Windows on Arm and AWS IoT Core](/learning-paths/laptops-and-desktops/win_aws_iot). ## Create the rule Start by creating the rule: diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/create-rule.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/create-rule.md index 082479d20c..ecfe9c6a2a 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/create-rule.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/create-rule.md @@ -7,7 +7,7 @@ weight: 4 layout: "learningpathall" --- ## Before you begin -Before you begin, prepare the weather station emulator and connect it to AWS IoT Core by completing this [Learning Path](/learning-paths/laptops-and-desktops/win_aws_iot). Note that this Learning Path also includes the rule created in the [Learning Path about Amazon DynamoDB for IoT applications running on Arm64](/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb). +Before you begin, prepare the weather station emulator and connect it to AWS IoT Core by completing [Create IoT applications with Windows on Arm and AWS IoT Core](/learning-paths/laptops-and-desktops/win_aws_iot). Note that this Learning Path also includes the rule created in [Use Amazon DynamoDB for your IoT applications running on Arm64](/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb). ## Create the rule Start by creating the rule: diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/implement-lambda.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/implement-lambda.md index c36dcd150c..b3389ad91a 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/implement-lambda.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda/implement-lambda.md @@ -262,7 +262,7 @@ Click the *Confirm subscription* link. Then, invoke the *temperature-high-level* ![fig21](figures/21.webp) -Now, you can start the weather station emulator you created in this [Learning Path](/learning-paths/laptops-and-desktops/win_aws_iot/). Observe the values generated by the emulator. When the temperature exceeds the threshold of 35, you will receive an email notification: +Now, you can start the weather station emulator you created in [Create IoT applications with Windows on Arm and AWS IoT Core](/learning-paths/laptops-and-desktops/win_aws_iot/). Observe the values generated by the emulator. When the temperature exceeds the threshold of 35, you will receive an email notification: ![fig22](figures/22.webp) diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb/create-lambda.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb/create-lambda.md index 0a611f7b91..1924d9ab13 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb/create-lambda.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb/create-lambda.md @@ -7,7 +7,7 @@ weight: 3 layout: "learningpathall" --- ## Objective -You will now implement the AWS Lambda function to scan the records from the DynamoDB. We have assumed that the records have already been written to the table as demonstrated in this [learning path on creating IoT applications with WoA and AWS IoT Core](/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb/). +You will now implement the AWS Lambda function to scan the records from the DynamoDB. We have assumed that the records have already been written to the table as demonstrated in [Use Amazon DynamoDB for your IoT applications running on Arm64](/learning-paths/laptops-and-desktops/win_aws_iot_dynamodb/). ## Create and configure an AWS Lambda function Go the AWS Lambda console. Then, create the function as shown below: diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/add-lambda.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/add-lambda.md index 8b94e6df9b..7c76a773b5 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/add-lambda.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/add-lambda.md @@ -9,7 +9,7 @@ layout: "learningpathall" ### AWS Lambda -You will now use the AWS Lambda console to retrieve the AWS Lambda endpoint for the static website. Before you begin make sure to prepare the `GetAverageTemperature` AWS Lambda function as explained in this [Learning Path](/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb/). Then proceed as follows: +You will now use the AWS Lambda console to retrieve the AWS Lambda endpoint for the static website. Before you begin make sure to prepare the `GetAverageTemperature` AWS Lambda function as explained in [Integrate AWS Lambda with DynamoDB for IoT applications running Windows on Arm](/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb/). Then proceed as follows: 1. Go to the AWS Lambda console, and click the `GetAverageTemperature` Lambda function. 2. In the Lambda function dashboard, click the **Configuration** tab and then the **Function URL** as shown below: diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/background.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/background.md index e74ffa5dd7..8ff067d026 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/background.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/background.md @@ -14,4 +14,4 @@ In addition to its core storage capabilities, Amazon S3 can also be used for sta Amazon S3 provides Software Development Kits (SDKs) that simplify the integration of S3 into applications by providing comprehensive APIs that facilitate file uploads, downloads, and management directly from the codebase. The AWS CLI allows developers and administrators to interact with S3 from the command line, accelerating many programming and administrative tasks. -In this Learning Path, you will learn how to use Amazon S3 to host a static website that interacts with AWS Lambda. Specifically, the Lambda function will consume data from a DynamoDB table, which is populated by a hypothetical IoT device streaming data to the cloud as explained in this [Learning Path](/learning-paths/laptops-and-desktops/win_aws_iot_lambda/). This setup not only demonstrates the seamless connectivity between various AWS services but also serves as a foundation for building an efficient dashboard for IoT solutions, providing real-time insights and data visualization. +In this Learning Path, you will learn how to use Amazon S3 to host a static website that interacts with AWS Lambda. Specifically, the Lambda function will consume data from a DynamoDB table, which is populated by a hypothetical IoT device streaming data to the cloud as explained in [Use AWS Lambda for IoT applications running on Arm64](/learning-paths/laptops-and-desktops/win_aws_iot_lambda/). This setup not only demonstrates the seamless connectivity between various AWS services but also serves as a foundation for building an efficient dashboard for IoT solutions, providing real-time insights and data visualization. diff --git a/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/deploy.md b/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/deploy.md index 4089ffbb17..5b81734e80 100644 --- a/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/deploy.md +++ b/content/learning-paths/laptops-and-desktops/win_aws_iot_s3/deploy.md @@ -10,7 +10,7 @@ layout: "learningpathall" In this section you will deploy the website to Amazon S3 using the Amazon Command Line interface (AWS CLI) version 2. If you don't already have it, start by [installing AWS CLI](/install-guides/aws-cli/). ### AWS CLI -To configure the AWS CLI you first need to create the [AWS CLI user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console). Then, you need to generate the access keys by following this [tutorial](https://docs.aws.amazon.com/cli/v1/userguide/cli-authentication-user.html). +To configure the AWS CLI you first need to create an [AWS CLI user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console). Then, you need to generate the access keys by following the [AWS CLI authentication tutorial](https://docs.aws.amazon.com/cli/v1/userguide/cli-authentication-user.html). Once you have the access key, go to the Command Prompt and type the following: diff --git a/content/learning-paths/laptops-and-desktops/win_forms/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_forms/how-to-1.md index 22cf649a92..bbfb60a809 100644 --- a/content/learning-paths/laptops-and-desktops/win_forms/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_forms/how-to-1.md @@ -179,7 +179,7 @@ namespace Arm64.DesktopApp.WindowsForms.Helpers } ``` -The above code implements two classes: `MatrixHelper` and `PerformanceHelper`. The `MatrixHelper` class implements two private methods: `MatrixMultiplication` and `GenerateRandomMatrix`. The `MatrixMultiplication` method takes two square matrices as input and calculates their product using the mathematical formula explained [here](https://en.wikipedia.org/wiki/Matrix_multiplication). Three 'for' loops are used and the result of the matrix multiplication is stored in the variable result, which is returned by the `MatrixMultiplication` method. The `GenerateRandomMatrix` method is used to generate a square matrix with elements pseudo-randomly generated using the `NextDouble` method of the Random class. Finally, the `MatrixHelper` class also implements the `SquareMatrixMultiplication` method, which generates two matrices of a given size and then calculates their product. +The above code implements two classes: `MatrixHelper` and `PerformanceHelper`. The `MatrixHelper` class implements two private methods: `MatrixMultiplication` and `GenerateRandomMatrix`. The `MatrixMultiplication` method takes two square matrices as input and calculates their product using the [mathematical formula for matrix multiplication](https://en.wikipedia.org/wiki/Matrix_multiplication). Three 'for' loops are used and the result of the matrix multiplication is stored in the variable result, which is returned by the `MatrixMultiplication` method. The `GenerateRandomMatrix` method is used to generate a square matrix with elements pseudo-randomly generated using the `NextDouble` method of the Random class. Finally, the `MatrixHelper` class also implements the `SquareMatrixMultiplication` method, which generates two matrices of a given size and then calculates their product. The `PerformanceHelper` class has one method, MeasurePerformance. This method works by invoking a function using the Action delegate, passed as the first parameter of the MeasurePerformance method. The function is invoked several times (as specified by the second parameter, executionCount). After that, the MeasurePerformance method returns the time taken to execute the specific code. To measure the execution time, you will use the System.Diagnostics.Stopwatch class. Specifically, an instance of this class has two useful methods: Restart and Stop. Restart resets and starts the stopwatch, while Stop halts the stopwatch, allowing us to read the time elapsed since the last restart. diff --git a/content/learning-paths/laptops-and-desktops/win_net_maui/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_net_maui/how-to-1.md index 442e518570..e8c56d57f2 100644 --- a/content/learning-paths/laptops-and-desktops/win_net_maui/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_net_maui/how-to-1.md @@ -16,7 +16,7 @@ Similar to Windows Presentation Foundation (WPF) and Xamarin.Forms, .NET MAUI ut In this learning path, you will explore how to create a .NET MAUI application and discover how .NET MAUI application performance is enhanced on Arm64 devices, particularly through the computationally intensive task of performing the multiply-add operation on two vectors. This operation, commonly used in many artificial neural network architectures, serves as an excellent example of Arm64's capabilities in handling performance-demanding tasks. -You can find the complete project code used in this learning path [here](https://github.com/dawidborycki/Arm64.MobileApp.MAUI.git). +You can find the [complete project code on GitHub](https://github.com/dawidborycki/Arm64.MobileApp.MAUI.git). ## Before you begin Before you begin the implementation, install Visual Studio 2022 with the following workloads: diff --git a/content/learning-paths/laptops-and-desktops/win_python/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_python/how-to-1.md index 46de98818c..ee29b2b2d2 100644 --- a/content/learning-paths/laptops-and-desktops/win_python/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_python/how-to-1.md @@ -22,7 +22,7 @@ Before going further, let's make sure you have installed Python v3.12 using the 1. [x64](https://www.python.org/ftp/python/3.12.0/python-3.12.0-amd64.exe) 2. [Arm64](https://www.python.org/ftp/python/3.12.0/python-3.12.0-arm64.exe) -Install Python v3.12 for Arm64 and x64. The installation process is the same as described [here](/install-guides/py-woa/). +Install Python v3.12 for Arm64 and x64. The installation process is the same as described in [Python on Windows on Arm](/install-guides/py-woa/). To run the Python interpreter for x64, open the command prompt and type the following command: @@ -170,7 +170,7 @@ Successfully installed numpy-1.26.2 ``` ## Arm build tools -To install the NumPy package, we will need to provide the Arm build tools. Install Visual Studio 2022 Community with the **Desktop development with C++** workload and **Arm build tools** as explained in this [installation guide](https://developer.arm.com/documentation/102528/0100/Install-Visual-Studio). +To install the NumPy package, we will need to provide the Arm build tools. Install Visual Studio 2022 Community with the **Desktop development with C++** workload and **Arm build tools** as explained in the [Visual Studio installation guide](https://developer.arm.com/documentation/102528/0100/Install-Visual-Studio). After installing the build tools, restart your machine and install NumPy package with the following command: diff --git a/content/learning-paths/laptops-and-desktops/win_python/how-to-2.md b/content/learning-paths/laptops-and-desktops/win_python/how-to-2.md index ef4a2ff87f..efb5bada96 100644 --- a/content/learning-paths/laptops-and-desktops/win_python/how-to-2.md +++ b/content/learning-paths/laptops-and-desktops/win_python/how-to-2.md @@ -10,7 +10,7 @@ layout: "learningpathall" ## Objective In this section, you will use the NumPy package you installed previously to create a sample application. The application will use NumPy to perform fast Fourier transforms (FFTs) of the synthesized sine waves corrupted by the noise. The application will run the FFTs several times for the variable lengths of the input waves. The application will measure the code execution time and we can then analyze the performance boost of the Python interpreter and NumPy package on Arm64. -You can find the complete code [here](https://github.com/dawidborycki/PythonOnWoA) +You can find the [complete code on GitHub](https://github.com/dawidborycki/PythonOnWoA). ## Creating the application Start by creating a new file, `sample.py`: diff --git a/content/learning-paths/laptops-and-desktops/win_python_onnx/intro.md b/content/learning-paths/laptops-and-desktops/win_python_onnx/intro.md index 5832063ed7..43e7be04a9 100644 --- a/content/learning-paths/laptops-and-desktops/win_python_onnx/intro.md +++ b/content/learning-paths/laptops-and-desktops/win_python_onnx/intro.md @@ -18,7 +18,7 @@ ONNX Runtime, in particular, provides optimized inference capabilities, supporti Several major ML frameworks currently support exporting models directly to the ONNX format, including Azure AI Custom Vision, Core ML, PyTorch, TensorFlow, and scikit-learn, streamlining the workflow from model development to deployment. -The companion code is available [here](https://github.com/dawidborycki/ONNX.WoA/tree/main) +The [companion code](https://github.com/dawidborycki/ONNX.WoA/tree/main) is available on GitHub. ## Objective In this hands-on learning path, you will explore the practical aspects of running inference on an ONNX-formatted model for an image classification task. Specifically, the demonstration uses the widely used Modified National Institute of Standards and Technology (MNIST) dataset, illustrating how ONNX can be applied to accurately recognize handwritten digits, showcasing both the flexibility and simplicity offered by this standardized format. @@ -59,7 +59,7 @@ py -V:3.13-arm64 -m venv venv-arm64 By using different virtual environments, you can compare the performance of the same code across different architectures. However, at the time of writing, ONNX Runtime is unavailable as a Python wheel for Windows ARM64. Therefore, the subsequent instructions apply only to Windows x64. ### A Code Editor -For this demonstration we use Visual Studio Code. Available for download [here](https://code.visualstudio.com/download) +For this demonstration we use Visual Studio Code. [Download Visual Studio Code](https://code.visualstudio.com/download). ## Pre-trained Models Many pre-trained models are available in the ONNX format ([ONNX Model Zoo](https://github.com/onnx/models)). There are models for image classification, image recognition, machine translation, language modeling, speech, audio processing, and more. diff --git a/content/learning-paths/laptops-and-desktops/win_win32_dll_porting/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_win32_dll_porting/how-to-1.md index 92d3a4d881..60de6125e0 100644 --- a/content/learning-paths/laptops-and-desktops/win_win32_dll_porting/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_win32_dll_porting/how-to-1.md @@ -23,7 +23,7 @@ In this learning path, we will demonstrate how to update your Win32 DLL for Arm6 ## Before you begin To follow this learning path you will need to install [Visual Studio 2022 or higher](/install-guides/vs-woa) with Arm64 build tools and desktop development with C++ workload. -The complete project code used in this learning path is hosted [here](https://github.com/dawidborycki/Win32.Porting.git). +The [complete project code](https://github.com/dawidborycki/Win32.Porting.git) is hosted on GitHub. ## Project Setup To set up the project, start by creating the Dynamic-Link Library (DLL). Open Visual Studio and click **Create a new project** button. This will open a new screen, in which you type DLL in the search box. This will filter the list of project templates, from which you select Dynamic-Link Library with exports (DLL): diff --git a/content/learning-paths/laptops-and-desktops/win_winui3/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_winui3/how-to-1.md index 3687f01b8f..78156f5831 100644 --- a/content/learning-paths/laptops-and-desktops/win_winui3/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_winui3/how-to-1.md @@ -17,7 +17,7 @@ WinUI 3 is an open-source project, offering backwards compatibility with existin In this learning path you will implement a Win UI 3 application, which will perform square matrix multiplication. The idea is to reproduce the same functionality used in [Windows Forms learning path](/learning-paths/laptops-and-desktops/win_forms). You will also be able to measure performance improvements on Arm64 architecture. -You can find the complete code used in this learning path [here](https://github.com/dawidborycki/Arm64.WinUIApp.git). +You can find the [complete code on GitHub](https://github.com/dawidborycki/Arm64.WinUIApp.git). ## Before you begin Before you begin the implementation, install Visual Studio 2022 with the following workloads: diff --git a/content/learning-paths/laptops-and-desktops/win_wpf/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_wpf/how-to-1.md index 3b37680a46..92dc9479a6 100644 --- a/content/learning-paths/laptops-and-desktops/win_wpf/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_wpf/how-to-1.md @@ -17,7 +17,7 @@ Through its comprehensive feature set and the support for modern application dev In this learning path, you will create a WPF application from scratch. The application will perform the computationally intensive task of the multiply-add operation on two vectors. By measuring execution times of these operations, you will see the enhancements provided by Arm64 architecture-based devices. Along the way, you will learn how to use Syncfusion's chart library, a third-party tool, to create a column series chart demonstrating the computation times. -You can find the complete project code used in this learning path [here](https://github.com/dawidborycki/Arm64.DesktopApp.WPF). +You can find the [complete project code on GitHub](https://github.com/dawidborycki/Arm64.DesktopApp.WPF). ## Create the project You will now create the WPF project. Open Visual Studio and click **Create a new project**: @@ -47,7 +47,7 @@ Next, open the Solution Explorer (View -> Solution Explorer) to view the created 1. App.xaml and App.xaml.cs. These files represent the entry point of the WPF application. App.xaml is used to define application-wide resources, such as styles, control templates, and other XAML resources. App.xaml.cs is the code-behind file for App.xaml and contains the application's startup logic, event handling for application-level events (like Startup, Exit, and DispatcherUnhandledException), and any other application-wide code. 2. MainWindow.xaml and MainWindow.xaml.cs. These files implement a main window that serves as the primary UI for the application. MainWindow.xaml is used to define the layout and elements of the main window, including control-like buttons, text boxes, and data grids. MainWindow.xaml.cs is the code-behind file that contains the logic to handle events and interactions within the main window. -Now that your project is set up, you are ready to begin development. As mentioned earlier, the application will perform intense computational tasks. You will use the multiply-add vector operations, implemented in the [.NET MAUI learning path](/learning-paths/laptops-and-desktops/win_net_maui). +Now that your project is set up, you are ready to begin development. As mentioned earlier, the application will perform intense computational tasks. You will use the multiply-add vector operations, implemented in the [.NET MAUI learning path](/learning-paths/laptops-and-desktops/win_net_maui/). ## Helpers Start by creating two new helper classes: **PerformanceHelper** and **VectorHelper**. The first class will be used to measure code execution times, while the second class will implement vector operations. diff --git a/content/learning-paths/laptops-and-desktops/win_xamarin_forms/how-to-1.md b/content/learning-paths/laptops-and-desktops/win_xamarin_forms/how-to-1.md index 4b30d9cb51..fe418ce4e9 100644 --- a/content/learning-paths/laptops-and-desktops/win_xamarin_forms/how-to-1.md +++ b/content/learning-paths/laptops-and-desktops/win_xamarin_forms/how-to-1.md @@ -11,7 +11,7 @@ Xamarin Forms is a cross-platform UI toolkit that allows developers to create na In this learning path, you will learn how to use Xamarin Forms to create a cross-platform application. Then, you will launch this application on Windows on Arm using x64 and Arm64 build configurations and compare the performance. By doing so, we will be able to see how Arm64 can accelerate Xamarin Forms applications. -You can find the complete project code used in this learning path [here](https://github.com/dawidborycki/Arm64.MobileApp.XamarinForms.git). +You can find the [complete project code on GitHub](https://github.com/dawidborycki/Arm64.MobileApp.XamarinForms.git). ## Before you begin Before you begin the implementation, install Visual Studio 2022 with the following workloads: @@ -230,7 +230,7 @@ protected override void OnLaunched(LaunchActivatedEventArgs e) ## Helpers You will now supplement the shared project with two helper classes: -1. **MatrixHelper**: this class implements matrix multiplication based on the mathematical formula detailed [here](https://en.wikipedia.org/wiki/Matrix_multiplication). +1. **MatrixHelper**: this class implements matrix multiplication based on the [mathematical formula for matrix multiplication](https://en.wikipedia.org/wiki/Matrix_multiplication). 2. **PerformanceHelper**: this class provides functionality to measure code execution time. To implement these classes, follow these steps: diff --git a/content/learning-paths/laptops-and-desktops/windowsperf/windowsperf.md b/content/learning-paths/laptops-and-desktops/windowsperf/windowsperf.md index 92254bf53a..fc5d94e84d 100644 --- a/content/learning-paths/laptops-and-desktops/windowsperf/windowsperf.md +++ b/content/learning-paths/laptops-and-desktops/windowsperf/windowsperf.md @@ -18,7 +18,7 @@ Learn more in this [blog](https://community.arm.com/arm-community-blogs/b/infras ## WindowsPerf releases -You can find all binary releases of `WindowsPerf` [here](https://github.com/arm-developer-tools/windowsperf/releases). +You can find all [WindowsPerf binary releases](https://github.com/arm-developer-tools/windowsperf/releases). # Installation diff --git a/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/1-prerequisites.md b/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/1-prerequisites.md index 8e9b4eeb63..95dc06c51e 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/1-prerequisites.md +++ b/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/1-prerequisites.md @@ -45,7 +45,7 @@ docker --version Docker version 27.3.1, build ce12230 ``` -If you see "`docker: command not found`," follow the [Docker Install Guide](https://learn.arm.com/install-guides/docker/). +If you see "`docker: command not found`," follow the [Docker Install Guide](/install-guides/docker/). {{% notice Note %}} You might need to log in again or restart your machine for the changes to take effect. diff --git a/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/_index.md b/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/_index.md index 43afa62b2c..bd1c3efb5f 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/ai-camera-pipelines/_index.md @@ -33,15 +33,15 @@ operatingsystems: further_reading: - resource: title: Accelerate Generative AI Workloads Using KleidiAI - link: https://learn.arm.com/learning-paths/cross-platform/kleidiai-explainer + link: /learning-paths/cross-platform/kleidiai-explainer type: website - resource: title: LLM Inference on Android with KleidiAI, MediaPipe, and XNNPACK - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/ + link: /learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/ type: website - resource: title: Vision LLM Inference on Android with KleidiAI and MNN - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/ + link: /learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/ type: website - resource: title: TensorFlow Lite is now LiteRT diff --git a/content/learning-paths/mobile-graphics-and-gaming/ams/ams.md b/content/learning-paths/mobile-graphics-and-gaming/ams/ams.md index 4936d5ca4e..6db0edfda6 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/ams/ams.md +++ b/content/learning-paths/mobile-graphics-and-gaming/ams/ams.md @@ -18,7 +18,7 @@ layout: "learningpathall" ## Download and Install Arm Performance Studio -Arm Performance Studio is supported on Windows, Linux, and macOS hosts. Get the installation package [here](https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Studio#Downloads). +Arm Performance Studio is supported on Windows, Linux, and macOS hosts. Get the [Arm Performance Studio installation package](https://developer.arm.com/Tools%20and%20Software/Arm%20Performance%20Studio#Downloads). Refer to the [Arm Performance Studio install guide](/install-guides/ams/) for installation instructions. diff --git a/content/learning-paths/mobile-graphics-and-gaming/android_sve2/part1.md b/content/learning-paths/mobile-graphics-and-gaming/android_sve2/part1.md index 0ab2a8356a..ee953e8e9d 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/android_sve2/part1.md +++ b/content/learning-paths/mobile-graphics-and-gaming/android_sve2/part1.md @@ -21,7 +21,7 @@ You will need a development computer with [Android Studio](https://developer.and You will also need a Armv8 powered smartphone running Android. We tested the application on a Samsung Galaxy S22. -You can find the complete source code [here](https://github.com/dawidborycki/Arm.SVE2). +You can find the [complete source code on GitHub](https://github.com/dawidborycki/Arm.SVE2). ## Create a project and enable SVE2 support The process of creating and configuring this project is similar to that used for NEON. Follow these steps: diff --git a/content/learning-paths/mobile-graphics-and-gaming/android_webgpu_dawn/2-env-setup.md b/content/learning-paths/mobile-graphics-and-gaming/android_webgpu_dawn/2-env-setup.md index 7f3683f1d6..0d9b3edba5 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/android_webgpu_dawn/2-env-setup.md +++ b/content/learning-paths/mobile-graphics-and-gaming/android_webgpu_dawn/2-env-setup.md @@ -56,7 +56,7 @@ One of the components of Performance Studio is Streamline. Streamline captures d Install Arm Performance Studio using the [Arm Performance Studio Install Guide](/install-guides/ams/). {{% notice Tip %}} -To learn more about Arm Performance Studio and Streamline, see [Get started with Arm Performance Studio for mobile](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ams/ams/). +To learn more about Arm Performance Studio and Streamline, see [Get started with Arm Performance Studio for mobile](/learning-paths/mobile-graphics-and-gaming/ams/ams/). {{% /notice %}} Android Studio and Arm Performance Studio are now installed, and you are ready to create a WebGPU Android application. diff --git a/content/learning-paths/mobile-graphics-and-gaming/best-practices-for-hwrt-lumen-performance/1-ray-tracing.md b/content/learning-paths/mobile-graphics-and-gaming/best-practices-for-hwrt-lumen-performance/1-ray-tracing.md index ecd5573948..35114a8685 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/best-practices-for-hwrt-lumen-performance/1-ray-tracing.md +++ b/content/learning-paths/mobile-graphics-and-gaming/best-practices-for-hwrt-lumen-performance/1-ray-tracing.md @@ -8,7 +8,7 @@ layout: learningpathall ## Overview -In 2023, Arm developed a demo to showcase the new frontier of next-generation graphics technologies of the **Immortalis** GPU via the Unreal Lumen rendering system. If you are not familiar with Lumen and global illumination, please review this [learning path](/learning-paths/mobile-graphics-and-gaming/how-to-enable-hwrt-on-lumen-for-android-devices/) before proceeding. +In 2023, Arm developed a demo to showcase the new frontier of next-generation graphics technologies of the **Immortalis** GPU via the Unreal Lumen rendering system. If you are not familiar with Lumen and global illumination, please review [How to Enable Hardware Ray Tracing on Lumen for Android Devices](/learning-paths/mobile-graphics-and-gaming/how-to-enable-hwrt-on-lumen-for-android-devices/) before proceeding. The demo is named **Steel Arms**. Created with Unreal Engine 5.3, Steel Arms brings desktop-level Bloom, Motion Blur, and Depth of Field (DOF) effects, alongside Physically Based Rendering (PBR), to smartphones. diff --git a/content/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/1-dev-env-setup.md b/content/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/1-dev-env-setup.md index 454856d8be..4dc0d7dada 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/1-dev-env-setup.md +++ b/content/learning-paths/mobile-graphics-and-gaming/build-llama3-chat-android-app-using-executorch-and-xnnpack/1-dev-env-setup.md @@ -36,7 +36,7 @@ Start by downloading and installing the latest version of Android Studio by navi https://developer.android.com/studio/ ``` -### For MacOS: Using UI +### For macOS: Using UI Follow these steps to configure Android Studio: diff --git a/content/learning-paths/mobile-graphics-and-gaming/debugging_with_mte_on_pixel8/how-to-3.md b/content/learning-paths/mobile-graphics-and-gaming/debugging_with_mte_on_pixel8/how-to-3.md index dab202cb4d..9e10c4f8b5 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/debugging_with_mte_on_pixel8/how-to-3.md +++ b/content/learning-paths/mobile-graphics-and-gaming/debugging_with_mte_on_pixel8/how-to-3.md @@ -72,4 +72,4 @@ You can find detailed information on MTE modes in the [MTE Guide for Android OS] The last step is to enable MTE in the phone settings. -For this, follow the indications provided in the Learning Path [Memory Tagging Extension on Google Pixel 8](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/mte_on_pixel8/). +For this, follow the indications provided in the Learning Path [Memory Tagging Extension on Google Pixel 8](/learning-paths/mobile-graphics-and-gaming/mte_on_pixel8/). diff --git a/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/06-create-matrix-mul-model.md b/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/06-create-matrix-mul-model.md index 925d1fd415..9eac17a7b2 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/06-create-matrix-mul-model.md +++ b/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/06-create-matrix-mul-model.md @@ -102,4 +102,4 @@ model/matrix_mul_pf32_gemm.pte ``` These files are the inputs for upcoming executor_runner benchmarks, where you’ll measure and compare KleidiAI micro-kernel performance. -The complete source code is available [here](../export-matrix-mul.py). +The [complete source code](../export-matrix-mul.py) is available. diff --git a/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/_index.md b/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/_index.md index 5cbda27058..acc28266a8 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/measure-kleidiai-kernel-performance-on-executorch/_index.md @@ -13,7 +13,7 @@ learning_objectives: prerequisites: - An x86_64 Linux host machine running Ubuntu, with at least 15 GB of free disk space - - An Arm64 target system with support for SME or SME2 - see the Learning Path [Devices with native SME2 support](https://learn.arm.com/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started/#devices-with-native-sme2-support) + - An Arm64 target system with support for SME or SME2 - see the Learning Path [Devices with native SME2 support](/learning-paths/cross-platform/multiplying-matrices-with-sme2/1-get-started/#devices) author: Qixiang Xu diff --git a/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/4-model-explorer.md b/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/4-model-explorer.md index 998c8e71a6..4be92b9bc3 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/4-model-explorer.md +++ b/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/4-model-explorer.md @@ -44,6 +44,6 @@ model-explorer --extensions=vgf_adapter_model_explorer Use the file browser to open the `.vgf` model exported earlier in your training workflow. -## Wrapping up +Continue to the next section to learn about bringing your own use-cases to the Model Gym. + -Through this Learning Path, you’ve learned what neural graphics is and why it matters for game performance. You’ve stepped through the process of training and evaluating an NSS model using PyTorch and the Model Gym, and seen how to export that model into VGF (.vgf) for real-time deployment. You’ve also explored how to visualize and inspect the model’s structure using Model Explorer. You can now explore the Model Training Gym repository for deeper integration and to keep building your skills. diff --git a/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/5-train-own-model.md b/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/5-train-own-model.md new file mode 100644 index 0000000000..79ef422511 --- /dev/null +++ b/content/learning-paths/mobile-graphics-and-gaming/model-training-gym/5-train-own-model.md @@ -0,0 +1,169 @@ +--- +title: Defining your own use cases +weight: 6 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Beyond NSS: working with your own model + +While NSS is a powerful demonstration of neural graphics, Model Gym is designed to support custom models for various use cases. You can add your own model architecture, register it with the toolkit, and run the same training, evaluation, and export workflows you used for NSS. + +This section walks you through the process of integrating a custom model into Model Gym, using the Python API you used in previous sections. + +### Create a custom model + +To add a custom model to the Model Gym, you need to: + +1. Create a Python class that inherits from `BaseNGModel` +2. Mark it with the `@register_model()` decorator +3. Implement required methods +4. Accept `params` as a constructor argument + +The resulting structure looks like this: + +```python +from ng_model_gym.core.model.base_ng_model import BaseNGModel +from ng_model_gym.core.model.model_registry import register_model +from ng_model_gym.core.utils.config_model import ConfigModel + +@register_model(name="custom_model", version="1") +class CustomModel(BaseNGModel): + def __init__(self, params: ConfigModel): + super().__init__(params) + # Define your model architecture here + + def forward(self, x): + # Implement forward pass + pass +``` + +The `@register_model()` decorator makes your model discoverable by Model Gym. The `name` and `version` parameters will be used later in your configuration file. + +### Register your model + +For your model to be available in Model Gym, the file defining it must be imported. This triggers the registration process. + +Place your model file in the `src/ng_model_gym/usecases/` directory within the Model Gym installation. To find the installed location, make sure the virtual environment you used with the example notebooks is activated. Then, run the following: + +```bash +whereis ng_model_gym +``` + +This should point to your `nb-env` virtual environment. The Model Gym source code in this case will sit in the following directory: + +```output +/nb-env/lib/python3.12/site-packages/ng_model_gym/usecases +``` + +Each use case directory must contain an `__init__.py` file. To understand how the configuration files work, revisit the example notebooks for NSS in previous sections. + +Verify registration succeeded by listing registered models: + +```python +from ng_model_gym.core.model.model_registry import MODEL_REGISTRY + +MODEL_REGISTRY.list_registered() +``` + +### Update your configuration file + +You've already worked with the configuration file in the training and evaluation notebooks. To use your custom model, update the same JSON configuration file to reference the model name and version you used when registering. You can generate a new config template with `ng-model-gym init`, or modify an existing one. + +Update the model section: + +```json +{ + "model": { + "name": "custom_model", + "version": "1" + } +} +``` + +If you're working in Python, set these values programmatically: + +```python +import ng_model_gym as ngmg +from pathlib import Path + +config = ngmg.load_config_file(Path("config.json")) +config.model.name = "custom_model" +config.model.version = "1" +``` + +### Run training with your custom model + +Once your model is registered and your config is updated, you can use all the standard Model Gym workflows that were covered in previous sections. For example training: + +```bash +trained_model_path = ngmg.do_training(config, training_mode=TrainEvalMode.FP32) +``` + +Your custom model will go through the same pipeline as NSS: training, quantization, evaluation, and export to `.vgf` format. + +### Add custom datasets + +Similar to models, you can register custom datasets by marking your dataset class with the `@register_dataset()` decorator: + +```python +from torch.utils.data import Dataset +from ng_model_gym.core.data.dataset_registry import register_dataset + +@register_dataset(name="custom_dataset", version="1") +class CustomDataset(Dataset): + def __init__(self, config): + # Initialize your dataset + pass + + def __len__(self): + # Return dataset size + pass + + def __getitem__(self, idx): + # Return a single sample + pass +``` + +Place the dataset implementation in the same `usecases` directory as your model. Then, update your configuration to use the custom dataset: + +```json +{ + "dataset": { + "name": "custom_dataset", + "version": "1" + } +} +``` + +## Summary of self-defined use cases + +You can group related models, datasets, and configurations into custom use cases. This is useful when working on a specific neural graphics application. To understand what the final structure should look like, use the NSS use case as a reference implementation. Your use case logic executes when you specify its model and dataset in your configuration file. + +## Explore the example notebook + +The `neural-graphics-model-gym-examples` repository includes a walkthrough notebook called `custom_model_example.ipynb`. This notebook demonstrates: + +- How to structure a custom model class +- Registration and verification steps +- Modifying configuration files +- Running training workflows with custom models + +To work through the example, follow the same process as before: + +```bash +cd neural-graphics-model-gym-examples +source nb-env/bin/activate +jupyter lab +``` + +Navigate to `tutorials/nss/custom_model_example.ipynb` and step through the cells. + +## Wrapping up + +You've now learned how to extend Model Gym beyond NSS with your own models and datasets. This opens up possibilities for experimenting with different neural graphics techniques: denoising, frame interpolation, or custom upscaling approaches tailored to your content. + +For more information on model registration, dataset integration, and use case development, see the [Model Gym GitHub repository](https://github.com/arm/neural-graphics-model-gym). + +Through this Learning Path, you’ve learned what neural graphics is and why it matters for game performance. You’ve stepped through the process of training and evaluating a model using PyTorch and the Model Gym, and seen how to export that model into VGF (.vgf) for real-time deployment. You’ve also explored how to visualize and inspect the model’s structure using Model Explorer. You can now explore the Model Training Gym repository for deeper integration and to keep building your skills. \ No newline at end of file diff --git a/content/learning-paths/mobile-graphics-and-gaming/optimizing-vertex-efficiency/_index.md b/content/learning-paths/mobile-graphics-and-gaming/optimizing-vertex-efficiency/_index.md index c89cdc852c..f071f66ee1 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/optimizing-vertex-efficiency/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/optimizing-vertex-efficiency/_index.md @@ -40,7 +40,7 @@ further_reading: type: documentation - resource: title: Analyze a Frame with Frame Advisor - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/analyze_a_frame_with_frame_advisor/ + link: /learning-paths/mobile-graphics-and-gaming/analyze_a_frame_with_frame_advisor/ type: blog - resource: title: Arm Performance Studio diff --git a/content/learning-paths/mobile-graphics-and-gaming/profiling-ml-on-arm/nn-profiling-executorch.md b/content/learning-paths/mobile-graphics-and-gaming/profiling-ml-on-arm/nn-profiling-executorch.md index 2c35a45492..e45e330e49 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/profiling-ml-on-arm/nn-profiling-executorch.md +++ b/content/learning-paths/mobile-graphics-and-gaming/profiling-ml-on-arm/nn-profiling-executorch.md @@ -11,7 +11,7 @@ You can use [ExecuTorch](https://pytorch.org/executorch/stable/index.html) for r To get started on ExecuTorch, you can follow the instructions on the [PyTorch website](https://pytorch.org/executorch/stable/getting-started-setup). To then deploy on Android, you can also find instructions on the [Pytorch website](https://pytorch.org/executorch/stable/demo-apps-android.html). If you do not already have ExecuTorch running on Android, follow these instructions first. -ExecuTorch comes with a set of profiling tools, but currently they are aimed at Linux, and not Android. The instructions to profile on Linux are [here](https://pytorch.org/executorch/main/tutorials/devtools-integration-tutorial.html), and you can adapt them for use on Android. +ExecuTorch comes with a set of profiling tools, but currently they are aimed at Linux, and not Android. The [ExecuTorch profiling instructions for Linux](https://pytorch.org/executorch/main/tutorials/devtools-integration-tutorial.html) can be adapted for use on Android. ## Profiling on Android diff --git a/content/learning-paths/mobile-graphics-and-gaming/ray_tracing/rt09_optimizing.md b/content/learning-paths/mobile-graphics-and-gaming/ray_tracing/rt09_optimizing.md index 37eb7f7020..3ad90d96e3 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/ray_tracing/rt09_optimizing.md +++ b/content/learning-paths/mobile-graphics-and-gaming/ray_tracing/rt09_optimizing.md @@ -18,6 +18,6 @@ Developers can see [our ray tracing best practices](https://developer.arm.com/do - [Mali Offline Compiler](https://developer.arm.com/Tools%20and%20Software/Mali%20Offline%20Compiler) is an extremely useful tool. It is recommended that you use it to check the behavior of your ray tracing shaders. For ray tracing, it is important to ensure that ray traversal is hardware accelerated and not emulated. -See the [Arm Performance Studio learning path](https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ams) for more information about these tools. +See the [Arm Performance Studio learning path](/learning-paths/mobile-graphics-and-gaming/ams) for more information about these tools. On some occasions, like conditionally evaluating Ray Query, Arm GPUs are not able to use the ray tracing hardware and will instead use a slower solution using software emulation. Mali Offline compiler can help to detect this problem. Check that your shaders do not produce this line `Has slow ray traversal: true`. Proper hardware ray tracing should show this line `Has slow ray traversal: false`. \ No newline at end of file diff --git a/content/learning-paths/mobile-graphics-and-gaming/render-graph-optimization/_index.md b/content/learning-paths/mobile-graphics-and-gaming/render-graph-optimization/_index.md index e39d5273a2..a5e37dabb9 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/render-graph-optimization/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/render-graph-optimization/_index.md @@ -27,11 +27,11 @@ further_reading: type: website - resource: title: Get started with Arm Performance Studio for mobile - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/ams/fa + link: /learning-paths/mobile-graphics-and-gaming/ams/fa type: website - resource: title: Analyze a frame with Frame Advisor - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/analyze_a_frame_with_frame_advisor + link: /learning-paths/mobile-graphics-and-gaming/analyze_a_frame_with_frame_advisor type: website - resource: title: Video tutorial – Capture and analyze a problem frame with Frame Advisor diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md index c1ba894988..ae9cc6b834 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md @@ -35,7 +35,7 @@ Download and install [Python version 3.10](https://www.python.org/downloads/rele {{< tab header="Linux">}} sudo apt install -y python3.10 python3.10-venv {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} brew install python@3.10 brew link python@3.10 --force {{< /tab >}} @@ -56,7 +56,7 @@ CMake is an open-source tool that automates the build process for software proje sudo apt update sudo apt install cmake g++ git {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} brew install cmake {{< /tab >}} {{< /tabpane >}} @@ -81,7 +81,7 @@ wget https://github.com/bazelbuild/bazel/releases/download/{$BAZEL_VERSION}/baze sudo bash bazel-7.4.1-installer-linux-x86_64.sh export PATH="/usr/local/bin:$PATH" {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} cd $WORKSPACE export BAZEL_VERSION=7.4.1 curl -fLO "https://github.com/bazelbuild/bazel/releases/download/{$BAZEL_VERSION}/bazel-{$BAZEL_VERSION}-installer-darwin-arm64.sh" @@ -111,7 +111,7 @@ export ANDROID_HOME=$WORKSPACE/Android export ANDROID_SDK_HOME=$ANDROID_HOME/Sdk $WORKSPACE/cmdline-tools/bin/sdkmanager --sdk_root=$ANDROID_SDK_HOME --install "platform-tools" "platforms;android-35" "build-tools;35.0.0" {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} cd $WORKSPACE wget https://dl.google.com/android/repository/commandlinetools-mac-13114758_latest.zip unzip -o commandlinetools-linux-13114758_latest.zip @@ -130,7 +130,7 @@ cd $WORKSPACE wget https://dl.google.com/android/repository/android-ndk-r27b-linux.zip unzip android-ndk-r27b-linux.zip {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} cd $WORKSPACE wget https://dl.google.com/android/repository/android-ndk-r27b-darwin.zip unzip android-ndk-r27b-darwin.zip @@ -145,7 +145,7 @@ export NDK_PATH=$WORKSPACE/android-ndk-r27b/ export ANDROID_NDK_HOME=$NDK_PATH export PATH=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} export NDK_PATH=$WORKSPACE/android-ndk-r27b/ export ANDROID_NDK_HOME=$NDK_PATH export PATH=$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin/:$PATH diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md index f36934bf2e..3c95117e14 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md @@ -26,7 +26,7 @@ Download and copy the configuration file `model_config.json` and the model itsel ls -lha $WORKSPACE/model_config.json $WORKSPACE/model.ckpt ``` -You can learn more about this model [here](https://huggingface.co/stabilityai/stable-audio-open-small). +You can learn more about [stable-audio-open-small on Hugging Face](https://huggingface.co/stabilityai/stable-audio-open-small). ### Good prompting practices diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md index db62d0bb1e..2677ecccfe 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md @@ -110,7 +110,7 @@ python3 ./scripts/export_dit_autoencoder.py --model_config "$WORKSPACE/model_con After successful conversion, you now have `dit_model.tflite` and `autoencoder_model.tflite` models in your current directory. -A more detailed explanation of the above scripts is available [here](https://github.com/ARM-software/ML-examples/blob/main/kleidiai-examples/audiogen/scripts/README.md). +A more detailed explanation of the above scripts is available in the [audiogen scripts README](https://github.com/ARM-software/ML-examples/blob/main/kleidiai-examples/audiogen/scripts/README.md). For easy access, add all the required models to one directory: diff --git a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/4-building-litert.md b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/4-building-litert.md index 1d8531458e..0901659937 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/4-building-litert.md +++ b/content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/4-building-litert.md @@ -36,7 +36,7 @@ export NDK_PATH=$WORKSPACE/android-ndk-r25b/ export ANDROID_NDK_HOME=$NDK_PATH export PATH=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} export NDK_PATH=$WORKSPACE/android-ndk-r25b/ export ANDROID_NDK_HOME=$NDK_PATH export PATH=$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin/:$PATH @@ -76,7 +76,7 @@ bazel build -c opt --config android_arm64 //tensorflow/lite:libtensorflowlite.so --define tflite_with_xnnpack_qs8=true \ --define tflite_with_xnnpack_qu8=true {{< /tab >}} - {{< tab header="MacOS">}} + {{< tab header="macOS">}} bazel build -c opt --config macos //tensorflow/lite:libtensorflowlite.so \ --define tflite_with_xnnpack=true \ --define xnn_enable_arm_i8mm=true \ diff --git a/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/2-overview.md b/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/2-overview.md index c5b4056d13..4a6e8b65e5 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/2-overview.md +++ b/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/2-overview.md @@ -50,7 +50,7 @@ You can build the pipeline for various platforms and independently benchmark the Currently, this module uses [whisper.cpp](https://github.com/ggml-org/whisper.cpp) and wraps the backend library with a thin C++ layer. The module also provides JNI bindings for developers targeting Android based applications. {{% notice %}} -You can get more information on how to build and use this module [here](https://gitlab.arm.com/kleidi/kleidi-examples/speech-to-text/-/blob/main/README.md?ref_type=heads) +You can get more information on how to build and use this module in the [speech-to-text README](https://gitlab.arm.com/kleidi/kleidi-examples/speech-to-text/-/blob/main/README.md?ref_type=heads) {{% /notice %}} ## Large Language Model @@ -85,7 +85,7 @@ Currently, this module provides a thin C++ layer as well as JNI bindings for dev {{% notice %}} -You can get more information on how to build and use this module [here](https://gitlab.arm.com/kleidi/kleidi-examples/large-language-models/-/blob/main/README.md?ref_type=heads) +You can get more information on how to build and use this module in the [large-language-models README](https://gitlab.arm.com/kleidi/kleidi-examples/large-language-models/-/blob/main/README.md?ref_type=heads) {{% /notice %}} ## Text-to-Speech diff --git a/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/_index.md b/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/_index.md index 9443da7334..be327214ca 100644 --- a/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/_index.md +++ b/content/learning-paths/mobile-graphics-and-gaming/voice-assistant/_index.md @@ -36,17 +36,17 @@ further_reading: - resource: title: Accelerate Generative AI workloads using KleidiAI - link: https://learn.arm.com/learning-paths/cross-platform/kleidiai-explainer + link: /learning-paths/cross-platform/kleidiai-explainer type: website - resource: title: LLM inference on Android with KleidiAI, MediaPipe, and XNNPACK - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/ + link: /learning-paths/mobile-graphics-and-gaming/kleidiai-on-android-with-mediapipe-and-xnnpack/ type: website - resource: title: Vision LLM inference on Android with KleidiAI and MNN - link: https://learn.arm.com/learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/ + link: /learning-paths/mobile-graphics-and-gaming/vision-llm-inference-on-android-with-kleidiai-and-mnn/ type: website ### FIXED, DO NOT MODIFY diff --git a/content/learning-paths/servers-and-cloud-computing/_index.md b/content/learning-paths/servers-and-cloud-computing/_index.md index c60510d465..c6e8098b80 100644 --- a/content/learning-paths/servers-and-cloud-computing/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/_index.md @@ -8,8 +8,8 @@ key_ip: maintopic: true operatingsystems_filter: - Android: 3 -- Linux: 204 -- macOS: 13 +- Linux: 207 +- macOS: 15 - Windows: 14 pinned_modules: - module: @@ -20,10 +20,10 @@ pinned_modules: subjects_filter: - CI-CD: 11 - Containers and Virtualization: 36 -- Databases: 24 +- Databases: 23 - Libraries: 9 - ML: 35 -- Performance and Architecture: 74 +- Performance and Architecture: 78 - Storage: 2 - Web: 17 subtitle: Optimize cloud native apps on Arm for performance and cost @@ -66,19 +66,19 @@ tools_software_languages_filter: - Azure: 1 - Azure CLI: 2 - Azure Portal: 1 -- Bash: 3 - bash: 2 +- Bash: 3 - Bastion: 3 - BOLT: 2 - bpftool: 1 - Buildkite: 1 -- C: 10 +- C: 11 - C#: 2 -- C++: 13 +- C++: 15 - Capstone: 1 - Cargo: 1 - cassandra-stress: 1 -- CCA: 8 +- CCA: 9 - CircleCI: 2 - Clair: 1 - Clang: 13 @@ -88,13 +88,14 @@ tools_software_languages_filter: - Cloud Build: 1 - CMake: 1 - conda: 1 +- Confidential containers: 1 - Couchbase: 1 - cqlsh: 1 - Criterion: 1 - Daytona: 1 - Demo: 3 - Django: 2 -- Docker: 26 +- Docker: 28 - Docker Buildx: 1 - Envoy: 3 - Erlang: 1 @@ -105,7 +106,7 @@ tools_software_languages_filter: - Flink: 2 - Fortran: 1 - FunASR: 1 -- FVP: 7 +- FVP: 8 - Gardener: 1 - GCC: 25 - gdb: 1 @@ -115,6 +116,7 @@ tools_software_languages_filter: - GitHub: 6 - GitHub Actions: 1 - GitHub CLI: 1 +- GitHub Copilot: 1 - GitLab: 2 - GKE: 1 - glibc: 1 @@ -138,6 +140,7 @@ tools_software_languages_filter: - JAX: 1 - JMH: 1 - Kafka: 2 +- Kata containers: 1 - KEDA: 1 - Kedify: 1 - Keras: 2 @@ -148,14 +151,15 @@ tools_software_languages_filter: - libbpf: 1 - Linaro Forge: 1 - Litmus7: 1 -- llama.cpp: 2 - Llama.cpp: 2 +- llama.cpp: 2 - LLM: 11 - llvm-mca: 1 - LM Evaluation Harness: 1 - LSE: 1 - MariaDB: 1 - Maven: 1 +- MCP: 1 - Memcached: 2 - MLPerf: 1 - ModelScope: 1 @@ -167,8 +171,8 @@ tools_software_languages_filter: - NEON: 7 - Networking: 1 - Nexmark: 1 -- nginx: 1 - NGINX: 4 +- nginx: 1 - Node.js: 5 - node.js: 1 - npm: 3 @@ -197,7 +201,7 @@ tools_software_languages_filter: - Redis: 4 - redis-benchmark: 1 - Remote.It: 2 -- RME: 8 +- RME: 9 - Ruby: 2 - Runbook: 71 - Rust: 3 @@ -220,11 +224,11 @@ tools_software_languages_filter: - ThirdAI: 1 - topdown-tool: 1 - Trusted Firmware: 1 -- Trustee: 1 +- Trustee: 2 - TSan: 1 - TypeScript: 2 - Vectorscan: 1 -- Veraison: 2 +- Veraison: 3 - Visual Studio Code: 5 - vLLM: 3 - vvenc: 1 diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/1-overview.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/1-overview.md new file mode 100644 index 0000000000..46d2a90f81 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/1-overview.md @@ -0,0 +1,84 @@ +--- +title: Arm MCP Server Overview +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## What is the Arm MCP Server? + +The Arm MCP Server is a tool that enables AI-powered developer tools to become Arm cloud migration and optimization experts. It implements the Model Context Protocol (MCP), an open standard that allows AI assistants to access external tools and data sources. + +By connecting your AI coding assistant to the Arm MCP Server, you gain access to Arm-specific knowledge, container image inspection tools, and code analysis capabilities that streamline the process of migrating applications from x86 to Arm. + +## How to interact with the Arm MCP Server + +There are multiple ways to interact with the Arm MCP Server, depending on your development environment and workflow: + +### 1. Direct AI Chat + +You can ask your AI assistant natural language questions, and it will automatically use the MCP tools when appropriate. For example: + +```text +Check if the nginx:latest Docker image supports Arm architecture +``` + +### 2. Prompt Files + +Many AI coding tools support prompt files that provide structured instructions. These files can reference MCP tools and guide the AI through complex workflows like full codebase migrations. + +### 3. Agentic Workflows + +Tools like GitHub Copilot Agent Mode, Claude Code, Kiro, and OpenAI Codex support autonomous agent workflows where the AI can execute multi-step migration tasks with minimal intervention. These fully agentic workflows can be combined with prompt files and direct chat to create an extremely powerful development system. + +## Available Arm MCP Server Tools + +The Arm MCP Server provides several specialized tools for migration and optimization: + +### knowledge_base_search + +Searches an Arm knowledge base of learning resources, Arm intrinsics, and software version compatibility using semantic similarity. Given a natural language query, it returns matching resources with URLs, titles, and content snippets ranked by relevance. + +**Use case:** Finding documentation, tutorials, or version compatibility information for Arm migration. + +### check_image + +Checks Docker image architectures. Provide an image in `name:tag` format and get a report of supported architectures. + +**Use case:** Quickly verify if a container base image supports `arm64` before starting a migration. + +### skopeo + +A container image architecture inspector that inspects container images remotely without downloading them to check architecture support (especially ARM64 compatibility). + +**Use case:** Detailed inspection of container manifests and multi-architecture support before migrating workloads to Arm-based infrastructure. + +### migrate_ease_scan + +Runs a migrate-ease scan against a workspace or remote Git repository. Supported scanners include: cpp, python, go, js, and java. Returns analysis results identifying x86-specific code that needs attention. + +**Use case:** Automated scanning of codebases to identify architecture-specific dependencies, build flags, intrinsics, and libraries that need to be changed for Arm. + +### mca (Machine Code Analyzer) + +An assembly code performance analyzer that predicts performance on different CPU architectures and identifies bottlenecks. It estimates Instructions Per Cycle (IPC), execution time, and resource usage. + +**Use case:** Analyzing and optimizing performance-critical assembly code when migrating between processor types. + +### sysreport_instructions + +Provides instructions for installing and using sysreport, a tool that obtains system information related to system architecture, CPU, memory, and other hardware details. + +**Use case:** Understanding the target Arm system's capabilities before deployment. + +## Setting up the Arm MCP Server + +To use the Arm MCP Server with an AI coding assistant, you must configure the assistant to connect to the MCP server. This connection allows the assistant to query Arm-specific tools, documentation, and capabilities exposed through the Model Context Protocol (MCP). +The required configuration steps vary by AI coding assistant. Refer to the installation guides below for step-by-step instructions on connecting the following AI coding assistants to the Arm MCP server: + + [GitHub Copilot](/install-guides/github-copilot/) + [Gemini CLI](/install-guides/gemini/) + [Kiro CLI](/install-guides/kiro-cli/) + +Continue to the next section to see the Arm MCP Server in action. diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/2-docker-check.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/2-docker-check.md new file mode 100644 index 0000000000..e4b519dfd8 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/2-docker-check.md @@ -0,0 +1,91 @@ +--- +title: Direct AI Chat +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Checking Base Images for Arm Compatibility + +This section demonstrates just one example of using direct AI chat with the Arm MCP Server. You can use similar natural language prompts to check library compatibility, search for Arm documentation, or analyze code for migration issues. + +A common first step when migrating a containerized application to Arm is verifying that the base container images support the arm64 architecture. The Arm MCP Server simplifies this process by allowing you to ask this question directly using a natural language prompt, without manually inspecting image manifests or registry metadata. + +## Example: Legacy CentOS 6 Application + +Consider an application built on CentOS 6, a legacy Linux distribution that has reached end of life (EOL). The following Dockerfile represents a typical x86-optimized, compute-heavy benchmark application that you might encounter when migrating older workloads. + +Copy this Dockerfile into VS Code using GitHub Copilot or another agentic IDE connected to the Arm MCP Server: + +```dockerfile +FROM centos:6 + +# CentOS 6 reached EOL, need to use vault mirrors +RUN sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/CentOS-Base.repo && \ + sed -i 's|^#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-Base.repo + +# Install EPEL repository (required for some development tools) +RUN yum install -y epel-release && \ + sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/epel.repo && \ + sed -i 's|^#baseurl=http://download.fedoraproject.org/pub/epel|baseurl=http://archives.fedoraproject.org/pub/archive/epel|g' /etc/yum.repos.d/epel.repo + +# Install Developer Toolset 2 for better C++11 support (GCC 4.8) +RUN yum install -y centos-release-scl && \ + sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/CentOS-SCLo-scl.repo && \ + sed -i 's|^mirrorlist=|#mirrorlist=|g' /etc/yum.repos.d/CentOS-SCLo-scl-rh.repo && \ + sed -i 's|^# baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-SCLo-scl.repo && \ + sed -i 's|^# baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-SCLo-scl-rh.repo + +# Install build tools +RUN yum install -y \ + devtoolset-2-gcc \ + devtoolset-2-gcc-c++ \ + devtoolset-2-binutils \ + make \ + && yum clean all + +# Set working directory +WORKDIR /app + +# Copy all header files +COPY *.h ./ + +# Copy all C++ source files +COPY *.cpp ./ + +# Build the application with optimizations using devtoolset-2 (GCC 4.8) +# AVX2 intrinsics are used in the code for x86-64 platforms +RUN scl enable devtoolset-2 "g++ -O2 -mavx2 -o benchmark \ + main.cpp \ + matrix_operations.cpp \ + hash_operations.cpp \ + string_search.cpp \ + memory_operations.cpp \ + polynomial_eval.cpp \ + -std=c++11" + +# Create a startup script +COPY start.sh . +RUN chmod +x start.sh + +# Run the application +CMD ["./start.sh"] +``` + +This Dockerfile has several x86-specific elements: +- The `centos:6` base image +- The `-mavx2` compiler flag for x86 AVX2 SIMD instructions +- C++ source files containing x86 intrinsics (which you will examine in the next section) + +## Using the Arm MCP Server to Check Compatibility + +With the Arm MCP Server connected to your AI assistant, you can quickly verify base image compatibility using a simple natural language prompt: + +```text +Check this base image for Arm compatibility +``` + +The AI assistant will use the `check_image` or `skopeo` tool to inspect the image and return a report. For `centos:6`, you would discover that this legacy image does **not** support `arm64` architecture. + +This simple interaction demonstrates how direct AI chat can quickly surface compatibility issues. In the next section, you'll see how to resolve these issues automatically using a fully agentic migration workflow with prompt files. diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md new file mode 100644 index 0000000000..424a07ba2b --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/3-simd-migration.md @@ -0,0 +1,240 @@ +--- +title: Fully Agentic Migration with Prompt Files +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Migrating SIMD Code with AI Assistance + +When migrating applications from x86 to Arm, you may encounter SIMD (Single Instruction, Multiple Data) code that is written using architecture-specific intrinsics. On x86 platforms, SIMD is commonly implemented with SSE, AVX, or AVX2 intrinsics, while Arm platforms use NEON and SVE intrinsics to provide similar vectorized capabilities. Updating this code manually can be time-consuming and challenging. By combining the Arm MCP Server with a well-defined prompt file, you can automate much of this work and guide an AI assistant through a structured, architecture-aware migration of your codebase. + +## Sample x86 Code with AVX2 Intrinsics + +The following example shows a matrix multiplication implementation using x86 AVX2 intrinsics. This is representative of performance-critical code found in compute benchmarks and scientific workloads. Copy this code into a file named `matrix_operations.cpp`: + +```cpp +#include "matrix_operations.h" +#include +#include +#include +#include +#include // AVX2 intrinsics + +Matrix::Matrix(size_t r, size_t c) : rows(r), cols(c) { + data.resize(rows, std::vector(cols, 0.0)); +} + +void Matrix::randomize() { + std::random_device rd; + std::mt19937 gen(rd()); + std::uniform_real_distribution<> dis(0.0, 10.0); + + for (size_t i = 0; i < rows; i++) { + for (size_t j = 0; j < cols; j++) { + data[i][j] = dis(gen); + } + } +} + +Matrix Matrix::multiply(const Matrix& other) const { + if (cols != other.rows) { + throw std::runtime_error("Invalid matrix dimensions for multiplication"); + } + + Matrix result(rows, other.cols); + + // x86-64 optimized using AVX2 for double-precision + for (size_t i = 0; i < rows; i++) { + for (size_t j = 0; j < other.cols; j++) { + __m256d sum_vec = _mm256_setzero_pd(); + size_t k = 0; + + // Process 4 elements at a time with AVX2 + for (; k + 3 < cols; k += 4) { + __m256d a_vec = _mm256_loadu_pd(&data[i][k]); + __m256d b_vec = _mm256_set_pd( + other.data[k+3][j], + other.data[k+2][j], + other.data[k+1][j], + other.data[k][j] + ); + sum_vec = _mm256_add_pd(sum_vec, _mm256_mul_pd(a_vec, b_vec)); + } + + // Horizontal add using AVX + __m128d sum_high = _mm256_extractf128_pd(sum_vec, 1); + __m128d sum_low = _mm256_castpd256_pd128(sum_vec); + __m128d sum_128 = _mm_add_pd(sum_low, sum_high); + + double sum_arr[2]; + _mm_storeu_pd(sum_arr, sum_128); + double sum = sum_arr[0] + sum_arr[1]; + + // Handle remaining elements + for (; k < cols; k++) { + sum += data[i][k] * other.data[k][j]; + } + + result.data[i][j] = sum; + } + } + + return result; +} + +double Matrix::sum() const { + double total = 0.0; + for (size_t i = 0; i < rows; i++) { + for (size_t j = 0; j < cols; j++) { + total += data[i][j]; + } + } + return total; +} + +void benchmark_matrix_ops() { + std::cout << "\n=== Matrix Multiplication Benchmark ===" << std::endl; + + const size_t size = 200; + Matrix a(size, size); + Matrix b(size, size); + + a.randomize(); + b.randomize(); + + auto start = std::chrono::high_resolution_clock::now(); + Matrix c = a.multiply(b); + auto end = std::chrono::high_resolution_clock::now(); + + auto duration = std::chrono::duration_cast(end - start); + + std::cout << "Matrix size: " << size << "x" << size << std::endl; + std::cout << "Time: " << duration.count() << " ms" << std::endl; + std::cout << "Result sum: " << c.sum() << std::endl; +} +``` + +Create the header file `matrix_operations.h`: + +```cpp +#ifndef MATRIX_OPERATIONS_H +#define MATRIX_OPERATIONS_H + +#include +#include + +// Matrix class with x86 SSE2 optimizations +class Matrix { +private: + std::vector> data; + size_t rows; + size_t cols; + +public: + Matrix(size_t r, size_t c); + void randomize(); + Matrix multiply(const Matrix& other) const; + double sum() const; + + size_t getRows() const { return rows; } + size_t getCols() const { return cols; } +}; + +// Benchmark function +void benchmark_matrix_ops(); + +#endif // MATRIX_OPERATIONS_H +``` + +Create `main.cpp` to run the benchmark: + +```cpp +#include "matrix_operations.h" +#include + +int main() { + std::cout << "x86-64 AVX2 Matrix Operations Benchmark" << std::endl; + std::cout << "========================================" << std::endl; + +#if defined(__x86_64__) || defined(_M_X64) + std::cout << "Running on x86-64 architecture with AVX2 optimizations" << std::endl; +#else + #error "This code requires x86-64 architecture with AVX2 support" +#endif + + benchmark_matrix_ops(); + + return 0; +} +``` + +## The Arm Migration Prompt File + +To automate migration, you can define a prompt file that instructs the AI assistant how to analyze and transform the project using the Arm MCP Server. +Create the following example prompt file to use with GitHub Copilot `.github/prompts/arm-migration.prompt.md`: +```markdown +--- +tools: ['search/codebase', 'edit/editFiles', 'arm-mcp/skopeo', 'arm-mcp/check_image', 'arm-mcp/knowledge_base_search', 'arm-mcp/migrate_ease_scan', 'arm-mcp/mca', 'arm-mcp/sysreport_instructions'] +description: 'Scan a project and migrate to ARM architecture' +--- + +Your goal is to migrate a codebase from x86 to Arm. Use the mcp server tools to help you with this. Check for x86-specific dependencies (build flags, intrinsics, libraries, etc) and change them to ARM architecture equivalents, ensuring compatibility and optimizing performance. Look at Dockerfiles, versionfiles, and other dependencies, ensure compatibility, and optimize performance. + +Steps to follow: +* Look in all Dockerfiles and use the check_image and/or skopeo tools to verify ARM compatibility, changing the base image if necessary. +* Look at the packages installed by the Dockerfile send each package to the knowledge_base_search tool to check each package for ARM compatibility. If a package is not compatible, change it to a compatible version. When invoking the tool, explicitly ask "Is [package] compatible with ARM architecture?" where [package] is the name of the package. +* Look at the contents of any requirements.txt files line-by-line and send each line to the knowledge_base_search tool to check each package for ARM compatibility. If a package is not compatible, change it to a compatible version. When invoking the tool, explicitly ask "Is [package] compatible with ARM architecture?" where [package] is the name of the package. +* Look at the codebase that you have access to, and determine what the language used is. +* Run the migrate_ease_scan tool on the codebase, using the appropriate language scanner based on what language the codebase uses, and apply the suggested changes. Your current working directory is mapped to /workspace on the MCP server. +* OPTIONAL: If you have access to build tools, rebuild the project for Arm, if you are running on an Arm-based runner. Fix any compilation errors. +* OPTIONAL: If you have access to any benchmarks or integration tests for the codebase, run these and report the timing improvements to the user. + +Pitfalls to avoid: + +* Make sure that you don't confuse a software version with a language wrapper package version -- i.e. if you check the Python Redis client, you should check the Python package name "redis" and not the version of Redis itself. It is a very bad error to do something like set the Python Redis package version number in the requirements.txt to the Redis version number, because this will completely fail. +* NEON lane indices must be compile-time constants, not variables. + +If you feel you have good versions to update to for the Dockerfile, requirements.txt, etc. immediately change the files, no need to ask for confirmation. + +Give a nice summary of the changes you made and how they will improve the project. +``` +This prompt file encodes best practices, tool usage, and migration strategy, allowing the AI assistant to operate fully agentically. + +## Running the Migration + +With the prompt file in place and the Arm MCP Server connected, invoke the migration workflow from your AI assistant: + +```text +/arm-migration +``` +The assistant will: + * Detect x86-specific intrinsics + * Rewrite SIMD code using NEON + * Remove architecture-specific build flags + * Update container and dependency configurations as needed + +## Verifying the Migration + +After reviewing and accepting the changes, build and run the application on an Arm system: + +```bash +g++ -O2 -o benchmark matrix_operations.cpp main.cpp -std=c++11 +./benchmark +``` + +If everything works, a successful migration produces output similar to the following: + +```bash +ARM-Optimized Matrix Operations Benchmark +========================================== +Running on ARM64 architecture with NEON optimizations + +=== Matrix Multiplication Benchmark === +Matrix size: 200x200 +Time: 12 ms +Result sum: 2.01203e+08 +``` +If compilation or runtime issues occur, feed the errors back to the AI assistant. This iterative loop allows the agent to refine the migration until the application is correct, performant, and Arm-native. + diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md new file mode 100644 index 0000000000..6d05b11412 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/4-agentic-systems.md @@ -0,0 +1,95 @@ +--- +title: Configure Different Agentic Systems +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Agentic AI Systems for Migration + +Different AI coding tools provide different mechanisms for defining persistent instructions, such as prompt files or configuration documents. This section shows how to set up Arm migration workflows in a few popular agentic AI systems. + +Although the configuration details vary by tool, the goal is the same across all systems: provide the AI with clear, structured instructions so it can use the Arm MCP Server effectively and carry out multi-step migration workflows autonomously. + +## Kiro Steering Documents + +[Kiro](https://kiro.dev/) uses "steering documents" - markdown files stored in `.kiro/steering/` that provide persistent context to the AI. Steering documents can be configured with different inclusion modes to control when and how the instructions are applied. + +### Create Arm Migration Steering Document + +Create the following file at `.kiro/steering/arm-migration.md`: + +```markdown +--- +inclusion: manual +--- + +Your goal is to migrate a codebase from x86 to Arm. Use the MCP server tools to help you with this. Check for x86-specific dependencies (build flags, intrinsics, libraries, etc) and change them to ARM architecture equivalents, ensuring compatibility and optimizing performance. Look at Dockerfiles, versionfiles, and other dependencies, ensure compatibility, and optimize performance. + +Steps to follow: +* Look in all Dockerfiles and use the check_image and/or skopeo tools to verify ARM compatibility, changing the base image if necessary. +* Look at the packages installed by the Dockerfile and send each package to the knowledge_base_search tool to check each package for ARM compatibility. If a package is not compatible, change it to a compatible version. When invoking the tool, explicitly ask "Is [package] compatible with ARM architecture?" where [package] is the name of the package. +* Look at the contents of any requirements.txt files line-by-line and send each line to the knowledge_base_search tool to check each package for ARM compatibility. If a package is not compatible, change it to a compatible version. +* Look at the codebase that you have access to, and determine what the language used is. +* Run the migrate_ease_scan tool on the codebase, using the appropriate language scanner based on what language the codebase uses, and apply the suggested changes. +* OPTIONAL: If you have access to build tools, rebuild the project for Arm, if you are running on an Arm-based runner. Fix any compilation errors. +* OPTIONAL: If you have access to any benchmarks or integration tests for the codebase, run these and report the timing improvements to the user. + +Pitfalls to avoid: + +* Make sure that you don't confuse a software version with a language wrapper package version -- i.e. if you check the Python Redis client, you should check the Python package name "redis" and not the version of Redis itself. +* NEON lane indices must be compile-time constants, not variables. + +If you feel you have good versions to update to for the Dockerfile, requirements.txt, etc. immediately change the files, no need to ask for confirmation. + +Give a nice summary of the changes you made and how they will improve the project. +``` + +Reference this steering document in chat with `#arm-migration`. + +## OpenAI Codex Prompt Files + +[OpenAI Codex](https://openai.com/codex/) uses markdown prompt files stored in `~/.codex/prompts/` or `$CODEX_HOME/prompts/`. The filename determines the command name used to invoke the prompt. + +### Create Arm Migration Prompt + +Create the following file at `~/.codex/prompts/arm-migrate.md`: + +```markdown +--- +description: Migrate codebase from x86 to Arm architecture +--- + +Your goal is to migrate a codebase from x86 to Arm. Use the MCP server tools to help you with this. Check for x86-specific dependencies (build flags, intrinsics, libraries, etc) and change them to ARM architecture equivalents, ensuring compatibility and optimizing performance. Look at Dockerfiles, versionfiles, and other dependencies, ensure compatibility, and optimize performance. + +Steps to follow: +* Look in all Dockerfiles and use the check_image and/or skopeo tools to verify ARM compatibility, changing the base image if necessary. +* Look at the packages installed by the Dockerfile and send each package to the knowledge_base_search tool to check each package for ARM compatibility. If a package is not compatible, change it to a compatible version. When invoking the tool, explicitly ask "Is [package] compatible with ARM architecture?" where [package] is the name of the package. +* Look at the contents of any requirements.txt files line-by-line and send each line to the knowledge_base_search tool to check each package for ARM compatibility. If a package is not compatible, change it to a compatible version. +* Look at the codebase that you have access to, and determine what the language used is. +* Run the migrate_ease_scan tool on the codebase, using the appropriate language scanner based on what language the codebase uses, and apply the suggested changes. +* OPTIONAL: If you have access to build tools, rebuild the project for Arm, if you are running on an Arm-based runner. Fix any compilation errors. +* OPTIONAL: If you have access to any benchmarks or integration tests for the codebase, run these and report the timing improvements to the user. + +Pitfalls to avoid: + +* Make sure that you don't confuse a software version with a language wrapper package version -- i.e. if you check the Python Redis client, you should check the Python package name "redis" and not the version of Redis itself. +* NEON lane indices must be compile-time constants, not variables. + +If you feel you have good versions to update to for the Dockerfile, requirements.txt, etc. immediately change the files, no need to ask for confirmation. + +Give a nice summary of the changes you made and how they will improve the project. +``` + +### Running the Codex Prompt + +Invoke the prompt with: + +```bash +codex /prompts:arm-migrate +``` + +## Other AI assistants + +You should now have a general understanding of how agentic systems support persistent migration instructions. For other AI coding assistants, consult their documentation to learn how to define equivalent prompt files or configuration mechanisms and adapt the same Arm migration workflow. diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/_index.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/_index.md new file mode 100644 index 0000000000..cc56711fff --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/_index.md @@ -0,0 +1,57 @@ +--- +title: Migrate an x86 application to Arm using the Arm MCP Server + +draft: true +cascade: + draft: true + +minutes_to_complete: 20 + +who_is_this_for: This is an advanced topic for software developers who want to use AI-powered tools to migrate x86 applications to Arm-based cloud instances. + +learning_objectives: + - Understand how the Arm MCP Server enables AI developer tools to assist with cloud migration + - Use AI prompts to check Docker images for Arm compatibility + - Perform a fully automated C++ application migration using prompt files with the Arm MCP Server + - Configure different agentic systems to automate Arm migration workflows + +prerequisites: + - An AI-powered IDE such as VS Code with agentic tools like GitHub Copilot, Claude Code, Cursor, or similar + - Basic familiarity with Docker and C/C++ development + - Access to an Arm-based cloud instance or local Arm computer for testing + +author: Joe Stech + +### Tags +skilllevels: Advanced +subjects: Performance and Architecture +armips: + - Neoverse +tools_software_languages: + - MCP + - Docker + - C++ + - GitHub Copilot +operatingsystems: + - Linux + + + +further_reading: + - resource: + title: Arm MCP Server GitHub Repository + link: https://github.com/arm/mcp + type: website + - resource: + title: Model Context Protocol Documentation + link: https://modelcontextprotocol.io/ + type: documentation + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/_next-steps.md new file mode 100644 index 0000000000..727b395ddd --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/arm-mcp-server/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # The weight controls the order of the pages. _index.md always has weight 1. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/aws-copilot/app.md b/content/learning-paths/servers-and-cloud-computing/aws-copilot/app.md index d4bc2b44f2..2fa419e411 100644 --- a/content/learning-paths/servers-and-cloud-computing/aws-copilot/app.md +++ b/content/learning-paths/servers-and-cloud-computing/aws-copilot/app.md @@ -17,7 +17,7 @@ This guide is applicable to both Linux and macOS users. Before starting, ensure you have Docker and Copilot installed on your computer. -For Docker installation, refer to the [Docker install guide](https://learn.arm.com/install-guides/docker/). +For Docker installation, refer to the [Docker install guide](/install-guides/docker/). For Copilot installation, refer to the [AWS Copilot CLI install guide](/install-guides/aws-copilot). diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/_index.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/_index.md index dcc30f94be..d1076daa1c 100644 --- a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/_index.md @@ -1,9 +1,5 @@ --- -title: Deploy Cassandra on a Google Axion C4A virtual machine - -draft: true -cascade: - draft: true +title: Deploy Cassandra on a Google Axion C4A virtual machine minutes_to_complete: 30 @@ -18,7 +14,7 @@ learning_objectives: prerequisites: - A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled - - Familiarity with Cassandra architecture, replication, and [Cassandra partitioning & event-driven I/O](https://cassandra.apache.org/doc/stable/cassandra/architecture/) + - Familiarity with Cassandra architecture, replication, and [Cassandra partitioning and event-driven I/O](https://cassandra.apache.org/doc/stable/cassandra/architecture/) author: Pareena Verma diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/backgraound.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/backgraound.md deleted file mode 100644 index ba3c04462c..0000000000 --- a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/backgraound.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Getting started with Cassandra on Google Axion C4A (Arm Neoverse-V2) - -weight: 2 - -layout: "learningpathall" ---- - -## Google Axion C4A Arm instances in Google Cloud - -Google Axion C4A is a family of Arm-based virtual machines built on Google’s custom Axion CPU, which is based on Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines offer strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. - -The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. - -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. - -## Cassandra - -Cassandra is a highly scalable, distributed NoSQL database designed to handle large amounts of data across many commodity servers without a single point of failure. - -It provides high availability, fault tolerance, and linear scalability, making it ideal for real-time big data applications and high-throughput workloads. - -Cassandra is widely used for time-series data, IoT applications, recommendation engines, and large-scale cloud services. Learn more from the [Cassandra official website](https://cassandra.apache.org/) and its [documentation](https://cassandra.apache.org/doc/latest/). diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/background.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/background.md new file mode 100644 index 0000000000..18e21e993b --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/background.md @@ -0,0 +1,23 @@ +--- +title: Get started with Cassandra on Google Axion C4A + +weight: 2 + +layout: "learningpathall" +--- + +## Explore Google Axion C4A Arm instances + +Google Axion C4A is a family of Arm-based virtual machines built on Google's custom Axion CPU, based on Arm Neoverse-V2 cores. These virtual machines deliver strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. + +The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of Arm architecture in Google Cloud. + +To learn more about Google Axion, see the Google blog [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu). + +## Explore Cassandra + +Cassandra is a highly scalable, distributed NoSQL database designed to handle large amounts of data across many commodity servers without a single point of failure. + +It provides high availability, fault tolerance, and linear scalability, making it ideal for real-time big data applications and high-throughput workloads. + +Cassandra is widely used for time-series data, IoT applications, recommendation engines, and large-scale cloud services. To learn, see the [Cassandra website](https://cassandra.apache.org/) and the [Cassandra documentation](https://cassandra.apache.org/doc/latest/). diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/baseline.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/baseline.md index dfdf7a687a..b39f0a9d87 100644 --- a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/baseline.md @@ -1,17 +1,14 @@ --- -title: Apache Cassandra baseline testing on Google Axion C4A Arm Virtual machine +title: Test Cassandra baseline functionality weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Overview -Since Cassandra has been successfully installed on your GCP C4A Arm virtual machine, follow these steps to verify that it is running and functioning properly. - -## Baseline Testing for Apache Cassandra - -This guide helps verify the installation and perform baseline testing of **Apache Cassandra**. +Now that Cassandra is installed on your GCP C4A Arm virtual machine, verify that it's running and functioning properly. ## Start Cassandra @@ -21,20 +18,23 @@ Run Cassandra in the background: cassandra -R ``` -The `-R` flag allows Cassandra to run in the background as a daemon, so you can continue using the terminal. The first startup may take **30–60 seconds** as it initializes the necessary files and processes. +The `-R` flag allows Cassandra to run in the background as a daemon. The first startup may take 30–60 seconds as it initializes. Check logs to ensure Cassandra started successfully: ```console tail -f ~/cassandra/logs/system.log ``` -Look for the message **"Startup complete"**, which indicates Cassandra is fully initialized. -### Check Cassandra Status +Look for the message "Startup complete", which indicates Cassandra is fully initialized. + +## Check Cassandra status + ```console nodetool status ``` -You should see an output similar to: + +The output is similar to: ```output Datacenter: datacenter1 @@ -44,29 +44,34 @@ Status=Up/Down -- Address Load Tokens Owns (effective) Host ID Rack UN 127.0.0.1 162.51 KiB 16 100.0% 78774686-39f3-47e7-87c3-3abc4f02a835 rack1 ``` -The `nodetool status` command displays the health and status of your Cassandra node(s). For a single-node setup, the output should indicate that the node is **Up (U)** and **Normal (N)**. This confirms that your Cassandra instance is running and ready to accept queries. -### Connect with CQLSH (Cassandra Query Shell) -**cqlsh** is the interactive command-line shell for Cassandra. It allows you to run Cassandra Query Language (CQL) commands to interact with your database, create keyspaces and tables, insert data, and perform queries. +For a single-node setup, the output should indicate that the node is Up (U) and Normal (N), confirming that your Cassandra instance is running and ready to accept queries. + +## Connect with CQLSH + +`cqlsh` is the interactive command-line shell for Cassandra that allows you to run Cassandra Query Language (CQL) commands. ```console cqlsh ``` -You’ll enter the CQL (Cassandra Query Language) shell. -### Create a Keyspace (like a database) -A **keyspace** in Cassandra is similar to a database in SQL systems. Here, we create a simple keyspace `testks` with a **replication factor of 1**, meaning data will only be stored on one node (suitable for a single-node setup). +You'll enter the CQL (Cassandra Query Language) shell. + +## Create a keyspace + +A keyspace in Cassandra is similar to a database in SQL systems. Create a simple keyspace `testks` with a replication factor of 1 (suitable for a single-node setup): ```sql CREATE KEYSPACE testks WITH replication = {'class':'SimpleStrategy','replication_factor' : 1}; ``` -Check if created: + +Verify the keyspace was created: ```sql DESCRIBE KEYSPACES; ``` -You should see an output similar to: +The output is similar to: ```output cqlsh> DESCRIBE KEYSPACES; @@ -75,8 +80,9 @@ system system_distributed system_traces system_virtual_schema system_auth system_schema system_views testks ``` -### Create a Table -Tables in Cassandra are used to store structured data. This step creates a `users` table with three columns: `id` (unique identifier), `name` (text), and `age` (integer). The `id` column is the primary key. +## Create a table + +Create a `users` table with three columns: ```sql USE testks; @@ -88,22 +94,24 @@ CREATE TABLE users ( ); ``` -### Insert Data -We insert two sample rows into the `users` table. The `uuid()` function generates a unique identifier for each row, which ensures that every user entry has a unique primary key. +## Insert data + +Insert two sample rows into the `users` table. The `uuid()` function generates a unique identifier for each row: ```sql INSERT INTO users (id, name, age) VALUES (uuid(), 'Alice', 30); INSERT INTO users (id, name, age) VALUES (uuid(), 'Bob', 25); ``` -### Query Data -This command retrieves all rows from the `users` table. Successful retrieval confirms that data insertion works correctly and that queries return expected results. +## Query data + +Retrieve all rows from the `users` table: ```sql SELECT * FROM users; ``` -You should see an output similar to: +The output is similar to: ```output id | age | name @@ -114,6 +122,6 @@ You should see an output similar to: (2 rows) ``` -This baseline test verifies that Cassandra 5.0.5 is installed and running correctly on the VM. It confirms the node status, allows connection via `cqlsh`, and ensures basic operations like creating a keyspace, table, inserting, and querying data work as expected. +This baseline test verifies that Cassandra 5.0.5 is installed and running correctly on the VM, confirming node status, CQLSH connectivity, and basic database operations. -Please now press "Ctrl-D" to exit the Cassandra Query Shell. +Press `Ctrl-D` to exit the Cassandra Query Shell. diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/benchmnarking.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/benchmarking.md similarity index 74% rename from content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/benchmnarking.md rename to content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/benchmarking.md index b86e744d7e..534bcd74cb 100644 --- a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/benchmnarking.md +++ b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/benchmarking.md @@ -1,32 +1,32 @@ --- -title: Cassandra Benchmarking +title: Benchmark Cassandra performance weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Cassandra Benchmarking by Cassandra-Stress -Cassandra benchmarking can be performed using the built-in `cassandra-stress` tool, which helps measure database performance under different workloads such as write, read, and mixed operations. +## Benchmark Cassandra with cassandra-stress -### Steps for Cassandra Benchmarking with Cassandra-Stress -**Verify cassandra-stress Installation:** +You can perform Cassandra benchmarking using the built-in `cassandra-stress` tool, which measures database performance under different workloads such as write, read, and mixed operations. -Cassandra comes with a built-in tool called **cassandra-stress** that is used for testing performance. It is usually located in the `tools/bin/` folder of your Cassandra installation. +## Verify cassandra-stress installation + +Cassandra comes with a built-in tool called `cassandra-stress` that is used for testing performance. It's located in the `tools/bin/` folder of your Cassandra installation. ```console ls ~/cassandra/tools/bin | grep cassandra-stress ``` -If you see cassandra-stress in the list, it means the tool is installed and ready to use. -**Run the version check:** +If you see `cassandra-stress` in the list, the tool is installed and ready to use. -To make sure the tool works correctly, check its help options. +Check the tool's help options to verify it works correctly: ```console ~/cassandra/tools/bin/cassandra-stress help ``` -You should see output similar to the following: + +The output is similar to: ```output Usage: cassandra-stress [options] @@ -43,10 +43,10 @@ help : Print help for a command or option print : Inspect the output of a distribution definition version : Print the version of cassandra stress ``` -If the tool is working, you will see a list of commands and options that you can use to run benchmarks. -This confirms that your setup is correct and you’re ready to start testing Cassandra’s performance. -### Basic Write Test +The list of commands and options confirms that your setup is correct and you're ready to start testing Cassandra's performance. + +## Run a basic write test Insert 10,000 rows with 50 concurrent threads using `cassandra-stress`: ```console @@ -56,7 +56,7 @@ Insert 10,000 rows with 50 concurrent threads using `cassandra-stress`: - **n=10000** → Specifies the number of rows to insert during the benchmark test. - **-rate threads=50** → Sets the number of concurrent worker threads simulating multiple clients writing to the cluster. -You should see output similar to the following: +The output is similar to: ```output ******************** Stress Settings ******************** @@ -186,13 +186,16 @@ Total operation time : 00:00:00 END ``` -### Read Test -The following command runs a **read benchmark** on your Cassandra database using `cassandra-stress`. It simulates multiple clients reading from the cluster at the same time and records performance metrics such as **throughput** and **latency**. +## Run a read test + +Run a read benchmark on your Cassandra database using `cassandra-stress`. This simulates multiple clients reading from the cluster at the same time and records performance metrics such as throughput and latency. ```console ~/cassandra/tools/bin/cassandra-stress read n=10000 -rate threads=50 ``` -You should see output similar to the following: + +The output is similar to: + ```output ******************** Stress Settings ******************** Command: @@ -322,21 +325,24 @@ Total operation time : 00:00:02 END ``` -## Benchmark Results Table Explained: +## Understand benchmark results + +The metrics below explain what each value in the `cassandra-stress` output represents: + +- Op rate (operations per second): the number of read operations Cassandra successfully executed per second. +- Partition rate: the number of partitions read per second. Since this is a read test, the partition rate equals the op rate. +- Row rate: the number of rows read per second. Again, for this test it equals the op rate. +- Latency mean: the average time taken for each read request to complete. +- Latency median: the 50th percentile latency - half of the operations completed faster than this time. +- Latency max: the slowest single read request during the test. +- Total partitions: the total number of partitions read during the test. +- Total errors: the number of failed read operations. +- GC metrics (Garbage Collection): shows whether JVM garbage collection paused Cassandra during the test. +- Total operation time: the total wall-clock time taken to run the benchmark. -- **Op rate (operations per second):** The number of read operations Cassandra successfully executed per second. -- **Partition rate:** Number of partitions read per second. Since this is a read test, the partition rate equals the op rate. -- **Row rate:** Number of rows read per second. Again, for this test it equals the op rate. -- **Latency mean:** The average time taken for each read request to complete. -- **Latency median:** The 50th percentile latency — half of the operations completed faster than this time. -- **Latency max:** The slowest single read request during the test. -- **Total partitions:** The total number of partitions read during the test. -- **Total errors:** Number of failed read operations. -- **GC metrics (Garbage Collection):** Shows whether JVM garbage collection paused Cassandra during the test. -- **Total operation time:** The total wall-clock time taken to run the benchmark. +## Benchmark summary for Arm64 -### Benchmark summary on Arm64 -Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SuSE shown, Ubuntu results were very similar): +Results from the run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE shown; Ubuntu results were very similar): | Metric | Write Test | Read Test | |----------------------------|----------------------|----------------------| @@ -356,12 +362,12 @@ Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm6 | Total GC Time | 0.0 s | 0.0 s | | Total Operation Time | 0:00:00 | 0:00:02 | -### Cassandra performance benchmarking notes -When examining the benchmark results, you will notice that on the Google Axion C4A Arm-based instances: +## What you've accomplished and what's next + +You've successfully deployed Apache Cassandra 5.0.5 on a Google Axion C4A Arm-based virtual machine, validated its functionality, and measured its performance using cassandra-stress. The benchmark results on Google Axion C4A Arm-based instances demonstrate strong performance characteristics. + +Write operations achieved high throughput of 10,690 op/s, while read operations reached 4,962 op/s on the `c4a-standard-4` Arm64 VM. Write latency was notably low with a mean of 3.7 ms compared to reads at 6.3 ms, indicating fast write processing on this Arm64 VM. The 95th and 99th percentile latencies show consistent performance, with writes significantly faster than reads. Zero errors or GC overhead confirm stable and reliable benchmarking results. -- The write operations achieved a high throughput of **10,690 op/s**, while read operations reached **4,962 op/s** on the `c4a-standard-4` Arm64 VM. -- Latency for writes was very low (mean: **3.7 ms**) compared to reads (mean: **6.3 ms**), indicating fast write processing on this Arm64 VM. -- The 95th and 99th percentile latencies show consistent performance, with writes significantly faster than reads. -- There were no errors or GC overhead, confirming stable and reliable benchmarking results. +The Arm64 VM provides efficient and predictable performance, making it suitable for high-throughput Cassandra workloads. The low write latencies and high operation rates demonstrate that Arm-based infrastructure can effectively handle database operations that require both speed and consistency. These results provide a solid baseline for evaluating Cassandra performance on Arm64 architecture and can guide decisions about instance sizing and configuration for production deployments. -Overall, the Arm64 VM provides efficient and predictable performance, making it suitable for high-throughput Cassandra workloads. +To continue building on this foundation, you can explore advanced Cassandra configurations such as multi-node cluster deployments, replication strategies for high availability, or performance tuning for specific workload patterns. You might also investigate integrating Cassandra with application frameworks or comparing performance across different Arm-based instance types to optimize for your use case. diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/installation.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/installation.md index a90264da85..5a843be673 100644 --- a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/installation.md @@ -6,11 +6,11 @@ weight: 4 layout: learningpathall --- -## Apache Cassandra Installation on Ubuntu or SuSE VM -This guide will help you install **Apache Cassandra** on a Ubuntu or SuSE Linux virtual machine. Cassandra is a highly scalable NoSQL database designed for high availability and fault tolerance. +## Install Apache Cassandra on Ubuntu or SUSE -### Update System Packages -Updating system packages ensures that your system has the latest security patches and dependencies required for Cassandra. +This guide shows you how to install Apache Cassandra on an Ubuntu or SUSE Linux virtual machine. Cassandra is a highly scalable NoSQL database designed for high availability and fault tolerance. + +## Update system packages {{< tabpane code=true >}} {{< tab header="Ubuntu" language="bash">}} @@ -22,8 +22,9 @@ sudo zypper update -y {{< /tab >}} {{< /tabpane >}} -### Install Java -Cassandra requires a Java runtime environment. You can use either Java 11 or Java 17. This example uses Java 17 for optimal performance and compatibility with Cassandra 5.0.5. +## Install Java + +Cassandra requires a Java runtime environment. This example uses Java 17 for optimal performance and compatibility with Cassandra 5.0.5. {{< tabpane code=true >}} {{< tab header="Ubuntu" language="bash">}} @@ -34,44 +35,47 @@ sudo zypper install -y java-17-openjdk java-17-openjdk-devel {{< /tab >}} {{< /tabpane >}} -### Download Cassandra -Download the latest stable release of Apache Cassandra 5.0.5 from the official Apache repository. +## Download Cassandra ```console wget https://downloads.apache.org/cassandra/5.0.5/apache-cassandra-5.0.5-bin.tar.gz ``` + {{% notice Note %}} -Apache Cassandra 5.0 is a major release introducing significant performance, usability, and scalability enhancements. Key features include Storage Attached Indexes (SAI) for flexible querying, Trie-based memtables/SSTables for better efficiency, and the Unified Compaction Strategy (UCS) for automated data management. It also supports JDK 17 for up to 20% performance gains and adds vector search for AI applications. The release marks the end-of-life for the 3.x series, urging users to upgrade for continued support. -You can view [this release note](https://cassandra.apache.org/_/blog/Apache-Cassandra-5.0-Announcement.html) +Apache Cassandra 5.0 is a major release introducing significant performance, usability, and scalability enhancements. Key features include Storage Attached Indexes (SAI) for flexible querying, Trie-based memtables/SSTables for better efficiency, and the Unified Compaction Strategy (UCS) for automated data management. It also supports JDK 17 for up to 20% performance gains and adds vector search for AI applications. The release marks the end-of-life for the 3.x series, urging users to upgrade for continued support. To learn more, see the [Apache Cassandra 5.0 announcement](https://cassandra.apache.org/_/blog/Apache-Cassandra-5.0-Announcement.html). -The [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends PHP version 5.0.0 as the minimum recommended on the Arm platforms. +The [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends Cassandra version 5.0.0 as the minimum recommended on Arm platforms. {{% /notice %}} -### Extract and Setup Cassandra -Extract the downloaded archive and move it to a dedicated directory for Cassandra. +## Extract and set up Cassandra ```console tar -xvzf apache-cassandra-5.0.5-bin.tar.gz mv apache-cassandra-5.0.5 ~/cassandra ``` -### Enable Running Cassandra from Anywhere +## Add Cassandra to PATH + To run Cassandra commands from any location, add the `bin` directory to your PATH environment variable: ```console echo 'export PATH="$HOME/cassandra/bin:$PATH"' >> ~/.bashrc source ~/.bashrc ``` -Now you can run `Cassandra` or `cqlsh` from any terminal without specifying the full path. -### Verify Installation +You can now run `Cassandra` or `cqlsh` from any terminal without specifying the full path. + +## Verify installation Check the installed Cassandra version to confirm the installation: ```console cassandra -v ``` -You should see an output similar to: + +The output is similar to: + ```output 5.0.5 ``` -Cassandra's installation is complete. You can now proceed with the baseline testing. + +Cassandra is now installed and ready for baseline testing. diff --git a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/instance.md index bd8a3d9b32..c02e6f7e58 100644 --- a/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/cassandra-on-gcp/instance.md @@ -1,5 +1,5 @@ --- -title: Create a Google Axion C4A Arm virtual machine on GCP +title: Create a Google Axion C4A Arm virtual machine weight: 3 ### FIXED, DO NOT MODIFY @@ -8,13 +8,13 @@ layout: learningpathall ## Overview -In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. +This section shows you how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} -## Provision a Google Axion C4A Arm VM in Google Cloud Console +## Provision a Google Axion C4A Arm VM To create a virtual machine based on the C4A instance type: - Navigate to the [Google Cloud Console](https://console.cloud.google.com/). @@ -24,35 +24,35 @@ To create a virtual machine based on the C4A instance type: - Set **Series** to `C4A`. - Select `c4a-standard-4` for machine type. - ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") + ![Screenshot of the Google Cloud Console showing the Compute Engine VM instance creation page with Machine configuration section expanded. The Series dropdown shows C4A selected, and the Machine type field displays c4a-standard-4 with specifications of 4 vCPUs and 16 GB memory visible alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") - Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server** or **Ubuntu**. - - If using use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type. - - If using **Ubuntu**, under the **Version** tab, please scroll down and select the aarch64 version of **Ubuntu 22.04 LTS**. -- Once appropriately selected, please Click **Select**. + - If using **SUSE Linux Enterprise Server**, select "Pay As You Go" for the license type. + - If using **Ubuntu**, under the **Version** tab, scroll down and select the aarch64 version of **Ubuntu 22.04 LTS**. +- Once appropriately selected, select **Select**. - Under **Networking**, enable **Allow HTTP traffic**. -- Click **Create** to launch the instance. -- Once created, you should see a "SSH" option to the right in your list of VM instances. Click on this to launch a SSH shell into your VM instance: +- Select **Create** to launch the instance. +- Once created, you see an **SSH** option to the right in your list of VM instances. Select this to launch an SSH shell into your VM instance: -![Invoke a SSH session via your browser alt-text#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance") +![Screenshot of the Google Cloud Console VM instances list showing a running instance with an SSH button highlighted on the right side of the row alt-text#center](images/gcp-ssh.png "SSH button in VM instances list") -- A window from your browser should come up and you should now see a shell into your VM instance: +- A browser window opens and displays a shell into your VM instance: -![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") +![Screenshot of a terminal shell interface showing a command prompt in a Google Cloud VM instance with standard Linux shell environment alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") -## Explore your instance -### Run uname +## Explore your instance: run uname Use the [uname](https://en.wikipedia.org/wiki/Uname) utility to verify that you are using an Arm-based server. For example: ```console uname -m ``` -will identify the host machine as `aarch64`. -### Run hello world +The output identifies the host machine as `aarch64`. + +## Run hello world Install the `gcc` compiler: @@ -83,14 +83,14 @@ gcc hello.c -o hello ./hello ``` -The output is shown below: +The output is: ```output hello world ``` -## Automating Arm Based Infrastructure Deployment +## Automate Arm infrastructure deployment -Cloud infrastructure deployment is typically done via Infrastructure as code (IaC) automation tools. There are Cloud Service Provider specific tools like [Google Cloud Deployment Manager](https://cloud.google.com/deployment-manager/docs/). +Cloud infrastructure deployment is typically done via Infrastructure as Code (IaC) automation tools. There are Cloud Service Provider-specific tools like [Google Cloud Deployment Manager](https://cloud.google.com/deployment-manager/docs/). -There are also Cloud Service Provider agnostic tools like [Terraform](https://www.terraform.io/).There is a [deploying Arm VMs on (GCP) using Terraform learning path](/learning-paths/servers-and-cloud-computing/gcp) that should be reviewed next. \ No newline at end of file +There are also Cloud Service Provider-agnostic tools like [Terraform](https://www.terraform.io/). Review the Learning Path[Deploy Arm virtual machines on Google Cloud Platform (GCP) using Terraform](/learning-paths/servers-and-cloud-computing/gcp) next. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/cca-container/_index.md b/content/learning-paths/servers-and-cloud-computing/cca-container/_index.md index b9ed716d62..a82168bc2f 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-container/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-container/_index.md @@ -10,19 +10,22 @@ learning_objectives: - Create a virtual machine in a Realm running guest Linux using a pre-built docker container. - Run a simple application in a Realm running guest Linux. - Obtain a CCA attestation token from the virtual guest in a Realm. + - Run the CCA software stack using MEC (Memory Encryption Contexts) prerequisites: - - An AArch64 or x86_64 computer running Linux. You can use cloud instances, refer to the list of [Arm cloud service providers](/learning-paths/servers-and-cloud-computing/csp/). + - An AArch64 or x86_64 computer running Linux or macOS. You can use cloud instances, refer to the list of [Arm cloud service providers](/learning-paths/servers-and-cloud-computing/csp/). -author: Pareena Verma +author: + - Pareena Verma + - Arnaud de Grandmaison ### Tags skilllevels: Introductory subjects: Performance and Architecture armips: - - Neoverse + - Neoverse operatingsystems: - - Linux + - Linux tools_software_languages: - GCC - FVP @@ -31,8 +34,12 @@ tools_software_languages: - Docker - Runbook - + further_reading: + - resource: + title: Learn the architecture - Introducing Arm Confidential Compute Architecture + link: https://developer.arm.com/documentation/den0125 + type: documentation - resource: title: Arm Confidential Compute Architecture link: https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture diff --git a/content/learning-paths/servers-and-cloud-computing/cca-container/cca-container.md b/content/learning-paths/servers-and-cloud-computing/cca-container/cca-container.md index fa81e923aa..ad2c806da8 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-container/cca-container.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-container/cca-container.md @@ -9,16 +9,16 @@ layout: "learningpathall" --- ## Download the docker image -Start by downloading the docker container image. +Start by downloading the docker container image. -This docker image contains the pre-built binaries for the Arm CCA reference software stack and the Armv-A Base Architecture Envelope Model (AEM) FVP with support for RME extensions. +This docker image contains the pre-built binaries for the Arm CCA reference software stack and the Armv-A Base Architecture Envelope Model (AEM) FVP with support for RME extensions. Install [docker engine](/install-guides/docker/docker-engine) on your machine. Pull the docker image from DockerHub: ```console -docker pull armswdev/aemfvp-cca-v2-image +docker pull armswdev/cca-learning-path:cca-simulation-v3 ``` Confirm that the docker container image was downloaded successfully: @@ -29,31 +29,34 @@ docker image list The output should be similar to: ```output -REPOSITORY TAG IMAGE ID CREATED SIZE -armswdev/aemfvp-cca-v2-image latest e1c36b91d3d7 5 weeks ago 1.34GB +IMAGE ID DISK USAGE CONTENT SIZE EXTRA + +... +armswdev/cca-learning-path:cca-simulation-v3 21500198bb93 1.18GB 0B +... ``` Run the docker container: ```console -docker run -it armswdev/aemfvp-cca-v2-image /bin/bash +docker run --rm -it armswdev/cca-learning-path:cca-simulation-v3 ``` -You are now inside the `/tmp/cca-stack` directory of the running `armswdev/aemfvp-cca-v2-image` container. +You are now inside the home directory (`/home/cca`) of user `cca` in the running `armswdev/cca-learning-path:cca-simulation-v3` container. ```output -ubuntu@84eb170a69b9:/tmp/cca-stack$ +cca@a9866f863546:~$ ``` ## Run the software stack -The pre-built binaries for the Arm CCA reference software stack are present in the `output/aemfvp-a-rme` directory. +The pre-built binaries for the Arm CCA reference software stack are present in the `cca-3world/` directory. ```console -ls output/aemfvp-a-rme/ +ls cca-3world/ ``` -This includes the Trusted Firmware binaries, the host root filesystem and the host Linux kernel image: +This includes the Realm Management Monitor (`rmm.img`), the host root filesystem (`host-rootfs.ext2`) and the host Linux kernel image (`Image`) and the trusted firmware binaries: ```output -Image KVMTOOL_EFI.fd bl1-uefi.bin bl1.bin fip-std-tests.bin fip-uefi.bin fip.bin host-fs.ext4 +FVP_AARCH64_EFI.fd Image bl1.bin bl2.bin bl31.bin dt_bootargs.dtb fip.bin host-rootfs.ext2 kselftests.tgz modules.tgz rmm.img ``` These binaries can run on an Armv-A Base Architecture Envelope Model (AEM) FVP with support for RME extensions. AEM FVPs are fixed configuration virtual platforms of Armv8-A and Armv9-A architectures with comprehensive system IP. The FVP is also contained within this docker container. @@ -68,71 +71,75 @@ Launch the `run-cca-fvp.sh` script to run the Arm CCA pre-built binaries on the A number of `Info` and `Warning` messages will be emitted by the FVP. These can safely be ignored. {{% /notice %}} -The `run-cca-fvp.sh` script uses the `screen` command to connect to the different UARTs in the FVP. +The `run-cca-fvp.sh` script uses the `screen` command to connect to the different UARTs in the FVP. You should see the host Linux kernel boot on your terminal: ```output -udhcpc: started, v1.31.1 -udhcpc: sending discover -udhcpc: sending select for 172.20.51.1 -udhcpc: lease of 172.20.51.1 obtained, lease time 86400 +udhcpc: started, v1.36.1 +udhcpc: broadcasting discover +udhcpc: broadcasting select for 172.20.51.1, server 172.20.51.254 +udhcpc: lease of 172.20.51.1 obtained from 172.20.51.254, lease time 86400 deleting routers adding dns 172.20.51.254 -FAIL -Starting dropbear sshd: OK +OK +Starting chrony: OK +Starting crond: OK +Setting up macvtap... [ 16.681271] smc91x 1a000000.ethernet eth0: entered promiscuous mode +OK -Welcome to Buildroot -buildroot login: +Welcome to the CCA host +host login: ``` -You will be prompted to log in to buildroot. Enter `root` as both the username and password. +You will be prompted to log in to the CCA host. Enter `root` as the username (no password is required). -You have successfully booted four worlds (Root, Secure, Non-secure and Realm) on the FVP at this point: +You have successfully booted 3 worlds (Root, Non-secure and Realm) on the FVP at this point: * Trusted Firmware-A is running in Root. * Realm Management Monitor (RMM) in Realm. * Host Linux in Non-secure. -* Hafnium in Secure. ## Create a virtual guest in a Realm -Guest VMs can be launched in a Realm using `kvmtool` from your host Linux prompt. The kernel `Image` and filesystem `realm-fs.ext4` for the Realm are packaged into the buildroot host file system. +Guest VMs can be launched in a Realm using `kvmtool` from your host Linux prompt. The realm disk image `guest-disk.img` is included into the host file system. Use `kvmtool` to launch guest Linux in a Realm: ```console -lkvm run --realm --restricted_mem -c 2 -m 256 -k /realm/Image -d /realm/realm-fs.ext4 -p earlycon +cd /cca +./lkvm run --realm --disable-sve --irqchip=gicv3-its --firmware KVMTOOL_EFI.fd -c 1 -m 512 --no-pvtime --force-pci --disk guest-disk.img --measurement-algo=sha256 --restricted_mem ``` + You should see the guest Linux kernel starting to boot in a Realm. This step can take several minutes. -After boot up, you will be prompted to log in at the guest Linux buildroot prompt. Use `root` again as both the username and password. +After boot up, you will be prompted to log in at the guest Linux prompt, use the `root` username (no password required): ```output -Starting network: udhcpc: started, v1.31.1 -udhcpc: sending discover -udhcpc: sending select for 192.168.33.15 -udhcpc: lease of 192.168.33.15 obtained, lease time 14400 +udhcpc: started, v1.36.1 +udhcpc: broadcasting discover +udhcpc: broadcasting select for 192.168.33.15, server 192.168.33.1 +udhcpc: lease of 192.168.33.15 obtained from 192.168.33.1, lease time 14400 deleting routers adding dns 172.20.51.254 -OK -Starting dropbear sshd: OK +FAIL +Starting chrony: OK +Starting crond: OK +Setting up macvtap... OK -Welcome to Buildroot -buildroot login: +Welcome to the CCA realm +realm login: ``` + You have successfully created a virtual guest in a Realm using the Arm CCA reference software stack. ## Obtain a CCA attestation token from the virtual guest in a Realm -Attestation tokens are small reports that are produced by a device upon request. Those tokens are composed of key/value pairs called claims. A CCA attestation token is a collection of claims about the state of a Realm and the CCA platform on which the Realm is running. +Attestation tokens are small reports that are produced by a device upon request. Those tokens are composed of key/value pairs called claims. A CCA attestation token is a collection of claims about the state of a Realm and the CCA platform on which the Realm is running. Refer to [section A7.2 of the Realm Management Monitor Specification](https://developer.arm.com/documentation/den0137/latest/) to learn about the details of the CCA attestation token. -To retrieve a CCA attestation token from the running guest, mount the `configfs` filesystem: -```console -mount -t configfs none /sys/kernel/config -``` +The retrieval of a CCA attestation token from a running guest is done by reading from `/sys/kernel/config/tsm/report/`. This is available when Linux's `configfs` has been mounted, which has been done automatically as part of the guest boot process --- if you are curious, this is the `configfs /sys/kernel/config configfs defaults 0 0` line in `/etc/fstab`. You can now generate an attestation token by running the following commands: @@ -145,62 +152,73 @@ hexdump -C $report/outblob The output should look like: ```output -00000340 00 00 00 00 19 ac cd 58 61 04 76 f9 88 09 1b e5 |.......Xa.v.....| -00000350 85 ed 41 80 1a ec fa b8 58 54 8c 63 05 7e 16 b0 |..A.....XT.c.~..| -00000360 e6 76 12 0b bd 0d 2f 9c 29 e0 56 c5 d4 1a 01 30 |.v..../.).V....0| -00000370 eb 9c 21 51 78 99 dc 23 14 6b 28 e1 b0 62 bd 3e |..!Qx..#.k(..b.>| -00000380 a4 b3 15 fd 21 9f 1c bb 52 8c b6 e7 4c a4 9b e1 |....!...R...L...| -00000390 67 73 73 4f 61 a1 ca 61 03 1b 2b bf 3d 91 8f 2f |gssOa..a..+.=../| -000003a0 94 ff c4 22 8e 50 91 95 44 ae 19 ac cc 67 73 68 |...".P..D....gsh| -000003b0 61 2d 32 35 36 19 ac d0 67 73 68 61 2d 32 35 36 |a-256...gsha-256| -000003c0 19 ac ce 58 20 d7 6c b0 e0 f4 d1 00 4d 51 5f e6 |...X .l.....MQ_.| -000003d0 c9 20 a8 e4 72 9d 26 61 0c cd 53 6b 8f 37 a3 65 |. ..r.&a..Sk.7.e| -000003e0 aa 03 b0 a2 2c 19 ac cf 84 58 20 00 00 00 00 00 |....,....X .....| -000003f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| -00000400 00 00 00 00 00 00 00 00 00 00 00 58 20 00 00 00 |...........X ...| -00000410 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| -00000420 00 00 00 00 00 00 00 00 00 00 00 00 00 58 20 00 |.............X .| -00000430 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| -00000440 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 58 |...............X| -00000450 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ...............| -00000460 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| -00000470 00 58 60 7c 2e 85 d2 b5 ba d8 ee e1 43 0c 5d f9 |.X`|........C.].| -00000480 38 b8 83 64 a0 75 8d d5 02 a2 43 56 53 ba 2f bc |8..d.u....CVS./.| -00000490 f1 a9 c7 82 b4 d5 b4 63 15 45 71 5c 50 ea eb a0 |.......c.Eq\P...| -000004a0 21 68 c4 7f 1a e5 00 b6 9a a5 3a 78 38 80 c6 96 |!h........:x8...| -000004b0 c8 f6 eb 92 62 f8 80 43 fe dd 7b e6 af 16 f0 04 |....b..C..{.....| -000004c0 54 95 6e 87 aa 53 4a bc e2 a4 ab 4d 84 10 b1 c8 |T.n..SJ....M....| -000004d0 84 0e 06 |...| -000004d3 +00000000 d9 01 8f a2 19 ac ca 59 05 ee d2 84 44 a1 01 38 |.......Y....D..8| +00000010 22 a0 59 05 81 a9 19 01 09 78 23 74 61 67 3a 61 |".Y......x#tag:a| +00000020 72 6d 2e 63 6f 6d 2c 32 30 32 33 3a 63 63 61 5f |rm.com,2023:cca_| +00000030 70 6c 61 74 66 6f 72 6d 23 31 2e 30 2e 30 0a 58 |platform#1.0.0.X| +00000040 20 0d 22 e0 8a 98 46 90 58 48 63 18 28 34 89 bd | ."...F.XHc.(4..| +00000050 b3 6f 09 db ef eb 18 64 df 43 3f a6 e5 4e a2 d7 |.o.....d.C?..N..| +00000060 11 19 09 5c 58 20 7f 45 4c 46 02 01 01 00 00 00 |...\X .ELF......| +00000070 00 00 00 00 00 00 03 00 3e 00 01 00 00 00 50 58 |........>.....PX| +00000080 00 00 00 00 00 00 19 01 00 58 21 01 07 06 05 04 |.........X!.....| +00000090 03 02 01 00 0f 0e 0d 0c 0b 0a 09 08 17 16 15 14 |................| +000000a0 13 12 11 10 1f 1e 1d 1c 1b 1a 19 18 19 09 61 44 |..............aD| +000000b0 cf cf cf cf 19 09 5b 19 30 03 19 09 62 67 73 68 |......[.0...bgsh| +000000c0 61 2d 32 35 36 19 09 60 78 3a 68 74 74 70 73 3a |a-256..`x:https:| +000000d0 2f 2f 76 65 72 61 69 73 6f 6e 2e 65 78 61 6d 70 |//veraison.examp| +000000e0 6c 65 2f 2e 77 65 6c 6c 2d 6b 6e 6f 77 6e 2f 76 |le/.well-known/v| +000000f0 65 72 61 69 73 6f 6e 2f 76 65 72 69 66 69 63 61 |eraison/verifica| +00000100 74 69 6f 6e 19 09 5f 8d a4 01 69 52 53 45 5f 42 |tion.._...iRSE_B| +00000110 4c 31 5f 32 05 58 20 53 78 79 63 07 53 5d f3 ec |L1_2.X Sxyc.S]..| +00000120 8d 8b 15 a2 e2 dc 56 41 41 9c 3d 30 60 cf e3 22 |......VAA.=0`.."| +00000130 38 c0 fa 97 3f 7a a3 02 58 20 9a 27 1f 2a 91 6b |8...?z..X .'.*.k| +00000140 0b 6e e6 ce cb 24 26 f0 b3 20 6e f0 74 57 8b e5 |.n...$&.. n.tW..| +00000150 5d 9b c9 4f 6f 3f e3 ab 86 aa 06 67 73 68 61 2d |]..Oo?.....gsha-| +00000160 32 35 36 a4 01 67 52 53 45 5f 42 4c 32 05 58 20 |256..gRSE_BL2.X | +00000170 53 78 79 63 07 53 5d f3 ec 8d 8b 15 a2 e2 dc 56 |Sxyc.S]........V| +... +00000800 88 cf 5b 66 ce b5 30 59 0a d4 81 79 3d e5 02 dc |..[f..0Y...y=...| +00000810 ac 70 bc dc b7 05 b2 cc 40 f1 b6 05 a5 52 57 04 |.p......@....RW.| +00000820 26 7a 24 c5 2e 88 6e a7 b6 18 59 2e 9f e8 58 8d |&z$...n...Y...X.| +00000830 a6 ea 0b 9b 18 90 62 62 07 f0 17 90 b4 27 04 e3 |......bb.....'..| +00000840 ec 89 dd 67 5f 6b 07 47 55 4d a9 7b c1 be d2 03 |...g_k.GUM.{....| +00000850 4f 5d d1 d0 55 d1 |O]..U.| +00000856 ``` + The output is a CCA attestation token from the guest in the Realm. The CCA attestation token is a Concise Binary Object Representation (CBOR) map, in which the map values are the Realm token and the CCA platform token. -You have successfully generated a CCA attestation token from the guest. In later learning paths, you will learn how to use these tokens as part of the Arm CCA attestation flow. +You have successfully generated a CCA attestation token from the guest. In later learning paths, you will learn how to use these tokens as part of the Arm CCA attestation flow. You can now shutdown the guest. Use the `poweroff` command. You should see the following output from the guest: ```output -Stopping dropbear sshd: OK -Stopping network: OK -Saving random seed: OK +(realm) # Destroying macvtap... OK +Stopping crond: stopped /usr/sbin/crond (pid 120) OK +Stopping chrony: OK +Stopping network: ifdown: interface lo not configured OK Stopping klogd: OK -Stopping syslogd: OK +Stopping syslogd: stopped /sbin/syslogd (pid 66) OK umount: devtmpfs busy - remounted read-only -[ 42.595975] EXT4-fs (vda): re-mounted 9e9fa588-c41f-404a-a627-6616bb8491b1 ro. Quota mode: none. +[ 1172.990117] EXT4-fs (vda2): re-mounted b984c902-aed2-4217-bbf0-da44ee66446c ro. The system is going down NOW! Sent SIGTERM to all processes -logout Sent SIGKILL to all processes Requesting system poweroff -[ 44.697156] reboot: Power down +[ 1175.167522] reboot: Power down Info: KVM session ended normally. ``` + The guest has shut down and you are back at the host Linux kernel prompt. -To exit the simulation, use `Ctrl-a + d`. You will be placed back into the running docker container. +To exit the host session and the simulation, use `poweroff`. You will be placed back into the running docker container. To exit the docker container, run `exit`. In the next section, you will learn how to run a simple application inside the Realm. + +{{% notice Note %}} +The docker session has been started with the `--rm` option, which means the container will be destroyed when it is exited, allowing you to experiment with the images without fear: at the next session, you will get working pristine images ! If you intend your changes to persist across docker sessions, omit the `--rm` option to docker. +{{% /notice %}} diff --git a/content/learning-paths/servers-and-cloud-computing/cca-container/cca-mec.md b/content/learning-paths/servers-and-cloud-computing/cca-container/cca-mec.md new file mode 100644 index 0000000000..c2dedec81a --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cca-container/cca-mec.md @@ -0,0 +1,110 @@ +--- +# User change +title: "Use memory encryption" + +weight: 5 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +## Before you begin + +Download the [`armswdev/cca-learning-path:cca-simulation-v3`](https://hub.docker.com/r/armswdev/cca-learning-path:cca-simulation-v3) docker container. + +## About Memory Encryption Context (MEC) + +Memory Encryption Contexts (MEC) is an extension of the Arm Realm Management Extension (RME). MEC extends the existing support for memory encryption, allowing multiple encryption contexts in the Realm Physical Address Space. In an RME system with MEC, each access to a physical address is tagged with a Memory Encryption Context Identifier (MECID), which associates the access with a specific memory encryption context. The Arm Confidential Compute Architecture (CCA) requires that the Realm, Secure, and Root PASes are encrypted. Without MEC, the encryption context used within each PAS is global to that PAS. For example, all Realm memory would use the same encryption context. + +With MEC, this model is extended. Non-secure, Secure, and Root PAS accesses use a default MECID value (0), while the Realm PAS supports multiple MECIDs. This allows each Realm to use a distinct memory encryption context, providing additional defense in depth beyond the isolation already provided by RME. The Realm Management Monitor (RMM) itself can also use a separate encryption context. + + +## Run the FVP without FEAT_MEC + +When the FVP starts, it can advertise support for MEC by exposing the `FEAT_MEC` CPU feature. The CCA software stack detects whether `FEAT_MEC` is available and enables MEC support accordingly. + +First, start the docker container: + +```console +docker run --rm -it armswdev/cca-learning-path:cca-simulation-v3 +``` + +Then start a CCA host in the FVP without enabling MEC: + +```console +./run-cca-fvp.sh +``` + +This boots the 3 worlds. Switch to the second `screen` console by pressing `ctrl+a 2`, which corresponds to the RMM output console. The output on the console should look similar to: + +```output +Trying ::1... +Connection failed: Connection refused +Trying 127.0.0.1... +Connected to localhost. +Escape character is '^]'. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +Booting RMM v.0.8.0(release) tf-rmm-v0.8.0 Built: Dec 4 2025 13:47:33 with GCC 14.3.1 +RMM-EL3 Interface v.0.6 +Boot Manifest Interface v.0.5 +RMI ABI revision v1.1 +RSI ABI revision v1.1 +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD=2 is set but FEAT_MEC is not present. +SMC_RMI_VERSION 10000 > RMI_SUCCESS 10000 10001 +``` + +The messages indicate that the RMM has detected that FEAT_MEC is not available. + +You can now bring down the FVP simulation. Switch back to the main `screen` console with `ctrl+a 1`, log in as `root` (no password) and `poweroff` to exit. + +## Run the FVP with FEAT_MEC enabled + +Next, start the CCA host in the FVP with `FEAT_MEC` enabled: + +```console +./run-cca-fvp.sh --enable-mec +``` + +This again boots the three worlds. Switch to the RMM output console (ctrl+a 2). It should look similar to: + +```output +Trying ::1... +Connection failed: Connection refused +Trying 127.0.0.1... +Connected to localhost. +Escape character is '^]'. +RMM_MEM_SCRUB_METHOD 2 is selected. +Booting RMM v.0.8.0(release) tf-rmm-v0.8.0 Built: Dec 4 2025 13:47:33 with GCC 14.3.1 +RMM-EL3 Interface v.0.6 +Boot Manifest Interface v.0.5 +RMI ABI revision v1.1 +RSI ABI revision v1.1 +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD 2 is selected. +RMM_MEM_SCRUB_METHOD 2 is selected. +SMC_RMI_VERSION 10000 > RMI_SUCCESS 10000 10001 +``` + +With FEAT_MEC enabled, the RMM detects MEC support and no longer reports it as missing. The RMM will now make use of multiple memory encryption contexts, assigning distinct MECIDs to Realms that you create using the lkvm command in the previous sections of this learning path. + +You can now shut down the FVP simulation. Switch back to the main screen console with ctrl+a 1, log in as root (no password), and run poweroff to exit. You can then exit the docker container with exit. + +You have learned how to enable Memory Encryption Contexts (MEC) on the FVP and verified that the RMM detects and uses this capability. diff --git a/content/learning-paths/servers-and-cloud-computing/cca-container/hello-realm.md b/content/learning-paths/servers-and-cloud-computing/cca-container/hello-realm.md index 37dfef8ce7..620ba11d71 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-container/hello-realm.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-container/hello-realm.md @@ -10,143 +10,144 @@ layout: "learningpathall" ## Before you begin -Download the [`armswdev/aemfvp-cca-v2-image`](https://hub.docker.com/r/armswdev/aemfvp-cca-v2-image) docker container. - -Install [GCC](/install-guides/gcc/) on your machine. Depending on the architecture of your machine, `aarch64` or `x86_64`, you will need to install either the native compiler or the cross-compiler. +Download the [`armswdev/cca-learning-path:cca-simulation-v3`](https://hub.docker.com/r/armswdev/cca-learning-path:cca-simulation-v3) docker container. ## Running an application in a Realm -In the previous section, you were able to boot a guest virtual machine as the Realm. In this section, you will learn how to run an application within that Realm. The application inherits the confidential protection of the guest virtual machine. - -A convenient way to run an application inside a Realm, within the context of this example, is to package the application as the "init" process for the guest Linux kernel. -Linux kernels contain a gzipped cpio format archive. When the kernel boots up, this archive is extracted into the root filesystem. The kernel then checks whether the root filesystem contains a file named "init" and tries to run it. At this point the kernel has booted, and the "init" application executes the system the rest of the way. +In the previous section, you were able to boot a guest virtual machine as the Realm. In this section, you will learn how to run your own application within that Realm. The application inherits the confidential protection of the guest virtual machine it is running in. -The use of the "init" process to run the application also means that you will not see a shell prompt when you boot the guest virtual machine. The guest kernel will boot, run the application and then exit. +A convenient way to run an application inside a Realm, within the context of this example, is to *inject* the application into the guest filesystem. In this section, you will inject a simple **hello world** program into the guest filesystem. -In this section, you will package a simple **hello world** application into the `initramfs` for the guest Linux kernel. +## Inject a simple application in the guest filesystem -## Create a simple initramfs for the guest Linux kernel +### Compile the application -Create a directory in which you will build the simple `initramfs`: +Create a directory in which you will build the simple **hello world** application: ```console -mkdir ~/FM-share -cd ~/FM-share +mkdir ~/CCA-docker-share +cd ~/CCA-docker-share ``` + Using a file editor of your choice, create a file named `hello.c` in this directory. Copy the contents below into the file: ```console -#include #include -#include - -#include -#include +#include int main(int argc, char *argv[]) { - printf("\n\n"); - printf("******* Hello from the Realm ! *******\n"); - printf("\n\n"); + printf("\n******* Hello from the Realm ! *******\n\n"); - sleep(30); - - printf("Shutting down the realm (this may take a while)...\n"); - reboot(LINUX_REBOOT_CMD_POWER_OFF); - - while(1); + return EXIT_SUCCESS; } ``` -Now compile the application as the init executable and package it into the `initramfs`: +Now start the container: ```console -aarch64-linux-gnu-gcc -static hello.c -o init -echo init | cpio -o -H newc | gzip > test.cpio.gz +docker run -v ~/CCA-docker-share:/home/cca/CCA-docker-share --rm --privileged -it armswdev/cca-learning-path:cca-simulation-v3 ``` -`test.cpio.gz` is the compressed archive you will use as the `initramfs` for the guest Linux kernel. -## Run the application inside the Realm +{{% notice Note %}} +The docker session is started with the `--rm` option, which means the container will be destroyed when it is exited, allowing you to experiment with the images without fear: at the next session, you will get working pristine images ! If you intend your changes to persist across docker sessions, omit the `--rm` option to docker. -Start the docker container with the `FM-share` directory mounted: +The `--privileged` option is needed to be able to mount the host and guest filesystems inside the container. -```console -docker run -v ~/FM-share:/home/ubuntu/FM-share -it armswdev/aemfvp-cca-v2-image /bin/bash -``` -Inside the running container, run the Arm CCA reference stack pre-built binaries on the FVP: +The `~/CCA-docker-share` directory on your development machine is available within the container at `/home/cca/CCA-docker-share` thanks to the `-v ~/CCA-docker-share:/home/cca/CCA-docker-share` command line option to `docker run`. +{{% /notice %}} -```console -./run-cca-fvp.sh -``` -After the host Linux kernel has booted up, you will prompted to log in. Use `root` for both the login and password: - -```output -[ 3.879183] VFS: Mounted root (ext4 filesystem) readonly on device 254:0. -[ 3.879643] devtmpfs: mounted -[ 3.897918] Freeing unused kernel memory: 9024K -[ 3.901685] Run /sbin/init as init process -[ 3.965637] EXT4-fs (vda): re-mounted 34fdcd00-4506-42fc-8af8-a35ca9cf965d r/w. Quota mode: none. -Starting syslogd: OK -Starting klogd: OK -Running sysctl: OK -Saving random seed: OK -Starting network: ip: RTNETLINK answers: File exists -udhcpc: started, v1.31.1 -udhcpc: sending discover -udhcpc: sending select for 172.20.51.1 -udhcpc: lease of 172.20.51.1 obtained, lease time 86400 -deleting routers -adding dns 172.20.51.254 -FAIL -Starting dropbear sshd: OK +Inside the running container, compile the **hello world** application, the source code file `hello.c` should be present in the shared directory `/home/cca/CCA-docker-share` : -Welcome to Buildroot -buildroot login: root -Password: -# +```console { output_lines="2" } +ls CCA-docker-share/ +hello.c +aarch64-linux-gnu-gcc -O1 -static -o hello CCA-docker-share/hello.c ``` -With the host Linux kernel running, you can now enable the sharing of files between your host machine and the running simulation. Mount a `9p` drive on the host Linux kernel using `/etc/fstab`: -```console -vi /etc/fstab +You now have a `hello` statically linked binary for the **hello world** application inside `/home/cca/`: + +```console { output_lines="2" } +ls +CCA-docker-share FastRAM.cfg cca-3world hello run-cca-fvp.sh ``` -In the `/etc/fstab` file, add the following line to it: + +### Inject the application into the guest filesystem. + +While still in the docker container, mount the host filesystem, then the guest filesystem (which lives inside the host filesystem), then copy the `hello` binary file to the guest filesystem and then unmount the guest and host filesystems: ```console -FM /mnt 9p trans=virtio,version=9p2000.L,aname=/mnt 0 0 +sudo mkdir -p /mnt/host-fs /mnt/guest-fs +sudo mount -t auto -w -o loop cca-3world/host-rootfs.ext2 /mnt/host-fs +fs_start=$(fdisk -l /mnt/host-fs/cca/guest-disk.img |grep "Linux filesystem"| tr -s ' ' | cut -f2 -d" ") +sudo mount -t auto -w -o loop,offset=$(($fs_start*512)) /mnt/host-fs/cca/guest-disk.img /mnt/guest-fs +sudo cp hello /mnt/guest-fs/cca/ +sudo umount /mnt/guest-fs +sudo umount /mnt/host-fs ``` -Save and close the file. Now run the `mount` command: + +### Execute **hello world** in the realm + +Now, as previously, start the CCA host in the FVP: ```console -mount -a +./run-cca-fvp.sh ``` -This command mounts the `FM-share` directory in your docker container to the `/mnt` directory of the host Linux kernel running in the FVP. Inspect the contents of `/mnt`: ```console -ls /mnt +udhcpc: started, v1.36.1 +udhcpc: broadcasting discover +udhcpc: broadcasting select for 172.20.51.1, server 172.20.51.254 +udhcpc: lease of 172.20.51.1 obtained from 172.20.51.254, lease time 86400 +deleting routers +adding dns 172.20.51.254 +OK +Starting chrony: OK +Starting crond: OK +Setting up macvtap... [ 16.681271] smc91x 1a000000.ethernet eth0: entered promiscuous mode +OK + +Welcome to the CCA host +host login: ``` -The output should be: -```output -hello.c init test.cpio.gz +Log into the CCA host, using `root` as the username (no password required), then start a realm with: + +```console +cd /cca +./lkvm run --realm --disable-sve --irqchip=gicv3-its --firmware KVMTOOL_EFI.fd -c 1 -m 512 --no-pvtime --force-pci --disk guest-disk.img --measurement-algo=sha256 --restricted_mem ``` -You can now use `kvmtool` which is a virtualization utility to launch Linux guest images. It supports the creation of Realm guests. Use the `-i` switch to point to the `initramfs` root filesystem you created in the previous step. ```console -lkvm run --realm --restricted_mem --irqchip=gicv3-its -p earlycon --network mode=none -c 1 -m 128 -k /realm/Image -i /mnt/test.cpio.gz +udhcpc: started, v1.36.1 +udhcpc: broadcasting discover +udhcpc: broadcasting select for 192.168.33.15, server 192.168.33.1 +udhcpc: lease of 192.168.33.15 obtained from 192.168.33.1, lease time 14400 +deleting routers +adding dns 172.20.51.254 +FAIL +Starting chrony: OK +Starting crond: OK +Setting up macvtap... OK + +Welcome to the CCA realm +realm login: ``` -The Realm guest takes some time to boot up. At the end of the boot, you should see your **hello world** application running in the Realm: -```output -[ 11.907066] Run /init as init process +Log into the CCA realm, using `root` as the username (no password required). + +Now change directory to `/cca`: the **hello world** application should be there as the binary file `hello`, just run it: + +```console { output_lines="3" } +cd /cca +ls +arc hello kbs-client +./hello ******* Hello from the Realm ! ******* -Shutting down the realm (this may take a while)... -[ 22.010458] reboot: Power down - Info: KVM session ended normally. ``` -You have successfully run an application inside the Realm! -The Realm will exit after the application runs because it has been packaged as the guest kernel's "init" process. +You have successfully run your own application inside the Realm ! +As before, with `poweroff` you can now exit the realm, then the host. diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/_index.md b/content/learning-paths/servers-and-cloud-computing/cca-kata/_index.md new file mode 100644 index 0000000000..0b51529761 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cca-kata/_index.md @@ -0,0 +1,66 @@ +--- +title: Run Confidential Containers with encrypted images using Arm CCA and Trustee + +draft: true +cascade: + draft: true + +minutes_to_complete: 60 + +who_is_this_for: This Learning Path is for software developers who want to understand how Confidential Containers can be run in Arm CCA realm. + +learning_objectives: + - Overview of Confidential Containers + - Understand how Trustee services are used for CCA realm attestation to unlock the confidential processing of data. + - Use an encrypted image to deploy a Confidential Containers in a CCA realm on an Armv9-A AEM Base Fixed Virtual Platform (FVP) that has support for RME extensions. + +prerequisites: + - An AArch64 or x86_64 computer running Linux or macOS; you can use cloud instances - see the [Arm cloud service providers](/learning-paths/servers-and-cloud-computing/csp/) + - Completion of the ["Run an end-to-end Attestation with Arm CCA and Trustee"](/learning-paths/servers-and-cloud-computing/cca-trustee) Learning Path + +author: + - Anton Antonov + +### Tags +skilllevels: Advanced +subjects: Performance and Architecture +armips: + - Neoverse + - Cortex-A +operatingsystems: + - Linux + - macOS +tools_software_languages: + - FVP + - RME + - CCA + - Docker + - Veraison + - Trustee + - Confidential containers + - Kata containers + +further_reading: + - resource: + title: Arm Confidential Compute Architecture + link: https://www.arm.com/architecture/security-features/arm-confidential-compute-architecture + type: website + - resource: + title: Arm Confidential Compute Architecture open-source enablement + link: https://www.youtube.com/watch?v=JXrNkYysuXw + type: video + - resource: + title: Learn the architecture - Realm Management Extension + link: https://developer.arm.com/documentation/den0126 + type: documentation + - resource: + title: Realm Management Monitor specification + link: https://developer.arm.com/documentation/den0137/latest/ + type: documentation + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/cca-kata/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cca-kata/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/attestation.png b/content/learning-paths/servers-and-cloud-computing/cca-kata/attestation.png new file mode 100644 index 0000000000..3719960e9a Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cca-kata/attestation.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/cca-kata.md b/content/learning-paths/servers-and-cloud-computing/cca-kata/cca-kata.md new file mode 100644 index 0000000000..5c16ef27be --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cca-kata/cca-kata.md @@ -0,0 +1,72 @@ +--- +# User change +title: "Overview of Confidential Containers and Arm CCA Attestation with Trustee" + +weight: 2 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + + +## Confidential Containers + +["Confidential Containers"](https://github.com/confidential-containers/confidential-containers) is an open source community +working to enable cloud native confidential computing by leveraging Trusted Execution Environments to protect containers and data. + +## Design overview + +Confidential computing projects are largely defined by what is inside the enclave and what is not. +For Confidential Containers, the enclave contains the workload pod and helper processes and daemons that facilitate the workload pod. +Everything else, including the hypervisor, other pods, and the control plane, is outside of the enclave and untrusted. + +### Kata Containers + +Confidential Containers and ["Kata Containers"](https://github.com/kata-containers/kata-containers) are closely linked, +but the relationship might not be obvious at first. Kata Containers is an existing open source project that encapsulates pods inside of VMs and +VMs can be run in TEEs. In this Learning Path the guest VM will be run in an ARM CCA realm and you will be able to confirm it by checking kernel messages. + +Given the pod-centric design of Confidential Containers this is a perfect match. But if Kata runs pods inside of VM, +why do we need the Confidential Containers project at all? There are crucial changes needed on top of Kata Containers to preserve confidentiality. + +### Image Pulling + +When using Kata Containers container images are pulled on the worker node with the help of a CRI runtime like **containerd**. +The images are exposed to the guest via filesystem passthrough. This is not suitable for confidential workloads +because the container images are exposed to the untrusted host. With Confidential Containers images are pulled and unpacked inside of the guest. +This requires additional components such as **image-rs** to be part of the guest rootfs. These components are beyond the scope of +traditional Kata deployments and live in the Confidential Containers ["guest components"](https://github.com/confidential-containers/guest-components) repository. + +On the host, a **nydus snapshotter** is used to pre-empt image pull and divert control flow to **image-rs** inside the guest. +This is a simplified diagram showing the interaction of **containerd**, the **nydus snapshotter**, and **image-rs**. +![Image pulling alt-text#center](image_pulling.png "Image pulling") + +### Attestation + +Confidential Containers also provides components inside the guest and elsewhere to facilitate attestation. +Attestation is a crucial part of confidential computing and a direct requirement of many guest operations. +For example, to unpack an encrypted container image, the guest must retrieve a secret key. +Inside the guest the **confidential-data-hub** and **attestation-agent** handle operations involving secrets and attestation. +Again, these components are beyond the scope of traditional Kata deployments and are located in the ["guest components"](https://github.com/confidential-containers/guest-components) repository. + +The CDH and AA use the KBS Protocol to communicate with an external, trusted entity. +Confidential Containers provides [Trustee](https://github.com/confidential-containers/trustee) as an attestation service and key management engine that validates the guest TCB and releases secret resources. + +This is a simplified diagram of the attestation process +![Attestation alt-text#center](attestation.png "Attestation") + +In this Learning Path the attestation process will be used to obtain an encryption key required to decrypt a container image. +Learn more about how Trustee services are used to evaluate the trustworthiness of a CCA realm and how attestation policy gates secrets release in +["Run an end-to-end Attestation with Arm CCA and Trustee"](/learning-paths/servers-and-cloud-computing/cca-trustee) + +### Full diagram + +If we take Kata Containers and add guest image pulling and attestation, we arrive at the following diagram, which represents Confidential Containers. +![Confidential Containers alt-text#center](confidential_containers.png "Confidential Containers") + + +For convenience, Trustee services and the Confidential Containers software are packaged in Docker containers, +which you can run on any suitable AArch64 or x86_64 development host. +Because the client runs in a realm, it uses the Fixed Virtual Platform (FVP) and the reference software stack for Arm CCA. + +Proceed to the next section to run a confidential container using the components and architecture described here. diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/confidential_containers.png b/content/learning-paths/servers-and-cloud-computing/cca-kata/confidential_containers.png new file mode 100644 index 0000000000..fce7da64a4 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cca-kata/confidential_containers.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/flow.md b/content/learning-paths/servers-and-cloud-computing/cca-kata/flow.md new file mode 100644 index 0000000000..1729c61da0 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/cca-kata/flow.md @@ -0,0 +1,304 @@ +--- +# User change +title: Run confidential containers with encrypted images using Arm CCA and Trustee + +weight: 3 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- +## Overview +In this section you will + +- run the **Trustee services** (AS, KBS, RVPS) and a local docker image registry. +- publish an encrypted docker image. +- on **Arm FVP** you will start a confidential container using the encrypted image and confirm that it runs in a **CCA realm**. + +## Install dependencies + +Start by installing Docker. On Ubuntu 24.04 LTS, set up Docker’s APT repository: + +```bash +# Add Docker's official GPG key: +sudo apt-get update +sudo apt-get install -y ca-certificates curl +sudo install -m 0755 -d /etc/apt/keyrings +sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc +sudo chmod a+r /etc/apt/keyrings/docker.asc + +# Add the repository to APT sources: +echo \ + "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ + $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \ + sudo tee /etc/apt/sources.list.d/docker.list > /dev/null +sudo apt-get update +``` + +Install Git and Docker packages: +``` +sudo apt-get install -y git docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin +``` + +Add your user name to the Docker group (open a new shell after this so the change takes effect): +``` bash +sudo usermod -aG docker $USER +newgrp docker +``` + +## Start Trustee services containers + +Clone the **kata** branch of the **cca-trustee** repository: +``` bash +git clone -b kata https://github.com/ArmDeveloperEcosystem/cca-trustee.git +``` + +This repository contains configuration to run Trustee services (KBS, AS, RVPS, keyprovider) with CCA attestation support as a simple cluster. The configuration is based on the recommended settings from [KBS Cluster](https://github.com/confidential-containers/trustee/blob/main/kbs/docs/cluster.md). + +Additional Learning Path–specific changes include: + +- External Linaro CCA verifier in the AS configuration + +- Attestation policy with CCA rules + +- An **affirming** resource policy + +- Docker registry service + +- A shared Docker network for all containers in this demo + +Go into the **cca-trustee** directory and generate a self-signed certificate for the docker registry service: +``` bash +cd cca-trustee + +openssl req -x509 -days 365 -config config/registry.cnf -keyout config/registry.key -out config/registry.crt +``` + +Start the Trustee services Docker containers (as detached services): +``` bash { output_lines = "2-9" } +docker compose up -d + ✔ Network cca-trustee Created + ✔ Container cca-trustee-rvps-1 Created + ✔ Container cca-trustee-setup-1 Exited + ✔ Container cca-trustee-registry-1 Created + ✔ Container cca-trustee-as-1 Created + ✔ Container cca-trustee-kbs-1 Created + ✔ Container cca-trustee-keyprovider-1 Created + ✔ Container cca-trustee-kbs-client-1 Created +``` + +While running the demo you can also check logs of the Trustee services in this terminal: +``` bash +docker compose logs +``` +Where **service** is either **as**, **kbs** or **rvps**. + +## Publish an encrypted docker image + +Generate an image encryption key: +``` bash +head -c 32 /dev/urandom | openssl enc >image.key +``` + +Publish the encryption key as a KBS secret resource. This resource can be obtained only with an attestation token with **affirming** status. + +Learn more about how the attestation result is used to evaluate the trustworthiness of a CCA realm and how attestation policy gates secrets release in +["Run an end-to-end Attestation with Arm CCA and Trustee"](/learning-paths/servers-and-cloud-computing/cca-trustee) +``` bash +./publish-key.sh +``` + +Encrypt a **busybox** docker image with the published key and push it into the local docker registry with **busybox_encrypted** name: +``` bash { output_lines = "2-9" } +./encrypt_image.sh +Encrypting docker://busybox image with busybox_encrypted name +Getting image source signatures +Copying blob 5bc51b87d4ec done | +Copying config eade5be814 done | +Writing manifest to image destination + +Inspecting MIMEType of docker://registry:5000/busybox_encrypted image + "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip+encrypted", +``` +By inspecting MIMEType of the published image you can see that it is encrypted. + +{{% notice Encryption key note %}} +Annotations data for an encrypted image contains a path to the key which Confidential Containers workload needs to obtain to decrypt the image. + +You can check it with this command +``` bash { output_lines = "4" } +docker compose exec keyprovider skopeo inspect docker://registry:5000/busybox_encrypted \ + | jq -r '.LayersData[0].Annotations."org.opencontainers.image.enc.keys.provider.attestation-agent"' \ + |base64 -d | jq .kid +"kbs:///cca-trustee/demo-key/encrypt.key" +``` +{{% /notice %}} + +{{% notice Docker image note %}} +**encrypt_image.sh** uses [skopeo](https://github.com/containers/skopeo) in the Trustee **keyprovider** container to pull, encrypt and push docker images. +Without parameters it encrypts **busybox** docker image which will be used in this Learning Path. +You can use this script to encrypt other docker images as well. For example with this command it will encrypt +**alpine** docker image and push it into the local docker registry with **alpine_encrypted** name. + +``` bash +./encrypt_image.sh alpine +``` +{{% /notice %}} + +You have prepared infrastructure and ready to start Confidential Containers in CCA realm. + +## Launch an FVP + +With the Trustee Services running in one terminal, open up a new terminal in which you will run Confidential Containers. + +Pull the Docker image with the pre-built FVP, and then run the container connected to the same Docker network: + +```bash +docker pull armswdev/cca-learning-path:cca-simulation-with-kata-v3 +``` +```bash +docker run --rm -it --network cca-trustee armswdev/cca-learning-path:cca-simulation-with-kata-v3 +``` + +Within your running container, launch the **run-cca-fvp.sh** script to run the Arm CCA pre-built binaries on the FVP: + +```bash +./run-cca-fvp.sh +``` + +The **run-cca-fvp.sh** script uses the **screen** command to connect to the different UARTs in the FVP. + +When the host Linux boots, уnter root as the username: +```output + +Welcome to the CCA host +host login: root +(host) # +``` + +## Inject the local docker registry self-signed certificate + +The local docker registry service was deployed with a self-signed certificate. +To trust this certificate it needs to be added into the CA certificates list in the guest VM image. + +Inject the local docker repository certificate into the image: +``` bash { output_lines = "2-6" } +inject_registry_cert.sh + +### Injecting the certificate into guest file system image +[ 2250.576395] loop0: detected capacity change from 0 to 518144 +[ 2250.588006] EXT4-fs (loop0): mounted filesystem ae947b26-4cdd-4e7c-8018-52b3e1594c9e r/w with ordered data mode. Quota mode: none. +[ 2250.862743] EXT4-fs (loop0): unmounting filesystem ae947b26-4cdd-4e7c-8018-52b3e1594c9e. +``` + +## Calculate Realm Initial Measurement (RIM) + +For a successful attestation of your CCA realm you need to provide the Trustee Reference Values Provider Service (RVPS) with a known good reference value. + +In a production environment, the known good reference value is generated using a deployment-specific process. +This Learning Path includes **rim_calc.sh** script which: +- Runs qemu-system-aarch64-cca-experimental (used by Kata containers to run a guest VM in Arm CCA realm) to dump Device Tree into a file. +- Checks Kata shim configuration file for guest VM delpoyment parameters. +- Uses ["cca-realm-measurements"](https://github.com/veraison/cca-realm-measurements) to calculate the realm initial measurement (RIM). + +Run the script: +``` bash { output_lines = "2-8" } +(host) # rim_calc.sh +Running /opt/kata/bin/qemu-system-aarch64-cca-experimental with dumpdtb=/tmp/kata-qemu.dtb parameter +Running /usr/bin/realm-measurements +RIM: rBD3xqcqqYHrjZ1Tu9aMWqFmwBgwT+NvLxg9jsozZm9rMDhjcaEM5LOQ+qJs+SLdG1jyco33EDuJ8+1/oMdFIQ== +REM0: KSzpB3Vo+rW1bSAJtAbxhN+dvTv6e6SqSwQSKqXFR8vX1mYHlKDA7SJBiHUBx/4UgZ/PLMmrxHrIuGzHaUi6aQ== +REM1: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== +REM2: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== +REM3: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA== +``` +Realm Extensible Measurements (REMs) are not used for attestation in this Learning Path and can be ignored. + + +## Endorse Realm Initial Measurement (RIM) + +In the terminal where you started Trustee services, run **endorse-rim.sh** script with the RIM as a parameter: + +```bash { output_lines = "2-3" } +./endorse-rim.sh "rBD3xqcqqYHrjZ1Tu9aMWqFmwBgwT+NvLxg9jsozZm9rMDhjcaEM5LOQ+qJs+SLdG1jyco33EDuJ8+1/oMdFIQ==" + +Reference Values Updated +``` + +## Launch a Confidential container in CCA realm + +Reduce amount of kernel messages printed to console: +``` bash +dmesg -n 4 +``` + +Run a confidential container from the **busybox_encrypted** image and check its kernel version and RME kernel messages. +Because the container is run in CCA realm on emulated environment (Fixed Virtual Platform (FVP)), it will take some time to launch, be patient. +Depending on the machine you run this Learning Path on it might take between 8 and 15 minutes to run a container. + +``` bash { output_lines = "5-9" } +nerdctl --snapshotter nydus run --runtime io.containerd.kata.v2 \ + --annotation io.kubernetes.cri.image-name=registry:5000/busybox_encrypted \ + --rm -it registry:5000/busybox_encrypted \ + sh -c 'echo; echo "Kernel version:"; uname -a; echo "RME kernel message:"; dmesg | grep RME; echo' + +Kernel version: +Linux 4ce958e5e51f 6.15.0-rc1+ #1 SMP Thu Dec 4 12:55:41 UTC 2025 aarch64 GNU/Linux +RME kernel message: +[ 0.000000] RME: Using RSI version 1.0 +``` + +Verify that the FVP kernel version is different to the Confidential Container version: +``` bash { output_lines = "2" } +uname -a +Linux host 6.15.0-rc1-g916aeec68dd4 #1 SMP PREEMPT @1764597323 aarch64 GNU/Linux +``` + +{{% notice Nydus snapshotter messages note %}} +When you run a confidential container you will see messages from Nydus snapshotter when it loads container snapshot. +``` +registry:5000/busybox_encrypted:latest: resolving |--------------------------------------| +elapsed: 0.1 s total: 0.0 B (0.0 B/s) +registry:5000/busybox_encrypted:latest: resolving |--------------------------------------| +elapsed: 0.2 s total: 0.0 B (0.0 B/s) +registry:5000/busybox_encrypted:latest: resolving |--------------------------------------| +manifest-sha256:8b72055942ded7b1f7f31959ab8ccbee4fbc6355423580275593e5ffeb578f33: waiting |--------------------------------------| +elapsed: 1.1 s total: 0.0 B (0.0 B/s) +registry:5000/busybox_encrypted:latest: resolved |++++++++++++++++++++++++++++++++++++++| +manifest-sha256:8b72055942ded7b1f7f31959ab8ccbee4fbc6355423580275593e5ffeb578f33: downloading |--------------------------------------| 0.0 B/1.3 KiB +elapsed: 1.2 s total: 0.0 B (0.0 B/s) +``` +These messages can be ignored. The docker image will be downloaded by **image-rs** in the guest VM as described previously. +{{% /notice %}} + +{{% notice Failed attestation note %}} +If you forget to endorse RIM for your environment or make a mistake doing that, then CCA realm attestation would fail. +In this case you will see an image decryption error when starting a confidential container with an encrypted image. +``` +FATA[0189] failed to create shim task: rpc status: Status { code: INTERNAL, message: "[CDH] [ERROR]: Image Pull error: Failed to pull image registry:5000/busybox_encrypted from all mirror/mapping locations or original location: image: registry:5000/busybox_encrypted:latest, error: Errors happened when pulling image: Failed to decrypt layer: Failed to decrypt the image layer, please ensure that the decryption key is placed and correct", details: [], special_fields: SpecialFields { unknown_fields: UnknownFields { fields: None }, cached_size: CachedSize { size: 0 } } } +``` + +By checking KBS logs (in the terminal where you run Trustee services) you could see that the encryption key couldn't be obtained because of an AS policy: +``` bash { output_lines = "2-5" } +docker compose logs kbs + +kbs-1 | [2025-12-08T12:25:39Z INFO actix_web::middleware::logger] 172.18.0.7 "POST /kbs/v0/auth HTTP/1.1" 200 74 "-" "attestation-agent-kbs-client/0.1.0" 0.001568 +kbs-1 | [2025-12-08T12:25:40Z INFO actix_web::middleware::logger] 172.18.0.7 "POST /kbs/v0/attest HTTP/1.1" 200 2952 "-" "attestation-agent-kbs-client/0.1.0" 0.420476 +kbs-1 | [2025-12-08T12:25:40Z ERROR kbs::error] PolicyDeny +``` +{{% /notice %}} + +You have successfully run a confidential container with Arm CCA using an encrypted image. + +{{% notice Unencrypted docker images note %}} +In this Learning Path environment you can also run confidential containers in Arm CCA realm from any unencrypted docker images. +Please notice that you need to define the **annotation** parameter so **image-rs** knows where to pull the image from. +For example: +``` +nerdctl run --runtime io.containerd.kata.v2 --rm -it --annotation io.kubernetes.cri.image-name=alpine:latest alpine sh +``` +{{% /notice %}} + +You can stop all Trustee containers with: +```bash +docker compose down +``` diff --git a/content/learning-paths/servers-and-cloud-computing/cca-kata/image_pulling.png b/content/learning-paths/servers-and-cloud-computing/cca-kata/image_pulling.png new file mode 100644 index 0000000000..c2fae549eb Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/cca-kata/image_pulling.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/cca-trustee/cca-trustee.md b/content/learning-paths/servers-and-cloud-computing/cca-trustee/cca-trustee.md index 182b83c8eb..35ba79ad9f 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-trustee/cca-trustee.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-trustee/cca-trustee.md @@ -60,7 +60,7 @@ A Verifier driver parses the attestation evidence provided by the hardware TEE a In this Learning Path, the AS is configured to use an external CCA Verifier. [Linaro](https://www.linaro.org) provides an attestation Verifier for pre-silicon Arm CCA platforms. It is built from the open-source [Veraison](https://github.com/veraison) project. Learn more in -[Get Started with CCA Attestation and Veraison](https://learn.arm.com/learning-paths/servers-and-cloud-computing/cca-veraison/). +[Get Started with CCA Attestation and Veraison](/learning-paths/servers-and-cloud-computing/cca-veraison/). ## Policy engine @@ -84,7 +84,7 @@ You will run this tool in a realm to make requests for an attestation result tok The client tool can also be used to provision the KBS/AS with resources and policies. To prove the environment’s trustworthiness, the KBS Client sends CCA attestation evidence (a CCA attestation token) to the KBS. Learn more about CCA attestation tokens in -[Get Started with CCA Attestation and Veraison](https://learn.arm.com/learning-paths/servers-and-cloud-computing/cca-veraison/). +[Get Started with CCA Attestation and Veraison](/learning-paths/servers-and-cloud-computing/cca-veraison/). For convenience, Trustee services and the client software are packaged in Docker containers, which you can run on any suitable AArch64 or x86_64 development host. Because the client runs in a realm, it uses the Fixed Virtual Platform (FVP) and the reference software stack for Arm CCA. If you are new to running applications in realms with FVP, see [Run an application in a Realm using the Arm Confidential Computing Architecture (CCA)](/learning-paths/servers-and-cloud-computing/cca-container). diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/aws-account-prep.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/aws-account-prep.md index 5ae2079cbf..308e4904d0 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/aws-account-prep.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/aws-account-prep.md @@ -18,7 +18,7 @@ You’ll need the AWS Command-Line Interface (CLI) installed for this section. F ## Set up authentication You'll need to set up your local environment to authenticate with AWS before deploying Veraison. -The recommended method is Single Sign-On (SSO). Follow the steps in Veraison's documentation [here](https://github.com/veraison/services/tree/main/deployments/aws#aws-account). +The recommended method is Single Sign-On (SSO). Follow the steps in [Veraison's AWS account documentation](https://github.com/veraison/services/tree/main/deployments/aws#aws-account). To confirm authentication is configured correctly, run a simple command, such as: diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/overview.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/overview.md index 4a60120e68..dc701875c3 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/overview.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/overview.md @@ -14,7 +14,7 @@ Veraison acts as the Verifier role in the [RATS architecture (RFC9334)](https:// Attestation is essential for confidential computing, and Veraison acts as a verifier for Arm's Confidential Compute Architecture (CCA). {{% notice Learning Tip %}} -If you're new to CCA attestation and Veraison, you will benefit from first completing the Learning Paths [Get Started with CCA Attestation and Veraison](https://learn.arm.com/learning-paths/servers-and-cloud-computing/cca-veraison/) and [Run an end-to-end Attestation Flow with Arm CCA](https://learn.arm.com/learning-paths/servers-and-cloud-computing/cca-essentials/). These two Learning Paths above use a Veraison verification service hosted by [Linaro](https://www.linaro.org). +If you're new to CCA attestation and Veraison, you will benefit from first completing the Learning Paths [Get Started with CCA Attestation and Veraison](/learning-paths/servers-and-cloud-computing/cca-veraison/) and [Run an end-to-end Attestation Flow with Arm CCA](/learning-paths/servers-and-cloud-computing/cca-essentials/). These two Learning Paths above use a Veraison verification service hosted by [Linaro](https://www.linaro.org). {{% /notice %}} In this Learning Path, you'll create and deploy your own Veraison verification service on AWS. After completing this Learning Path, you'll be able to revisit the two Learning Paths mentioned above, using your own AWS-hosted Veraison service instead of the one hosted by Linaro. diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/use-verifier.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/use-verifier.md index 212f44f0a5..1e9b33a62c 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/use-verifier.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison-aws/use-verifier.md @@ -9,7 +9,7 @@ layout: learningpathall ## Use Your AWS Deployment to Verify a CCA Attestation Token Now that your Veraison services are deployed into AWS and initialized with endorsements for the CCA reference platform, you are ready to make use of the verification service to verify a CCA attestation token. -To do this, you should follow the steps set out in the Learning Path [Get Started with CCA Attestation and Veraison](https://learn.arm.com/learning-paths/servers-and-cloud-computing/cca-veraison/). Follow the steps in this Learning Path exactly, except you'll use your AWS-hosted Veraison deployment instead of Linaro's service. +To do this, you should follow the steps set out in the Learning Path [Get Started with CCA Attestation and Veraison](/learning-paths/servers-and-cloud-computing/cca-veraison/). Follow the steps in this Learning Path exactly, except you'll use your AWS-hosted Veraison deployment instead of Linaro's service. The URL for the Veraison server provided by Linaro is `https://veraison.test.linaro.org:8443`. diff --git a/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md b/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md index 342c49d5a0..43599112a3 100644 --- a/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md +++ b/content/learning-paths/servers-and-cloud-computing/cca-veraison/attestation-token.md @@ -61,7 +61,7 @@ Place this file in the `$HOME` folder, while retaining the file name. The rest of this Learning Path uses the notation `$HOME/cca_example_token.cbor` as the file path. {{% notice Note %}} -You will notice that the filename extension on the example token is `.cbor`, which also denotes the format of the data. CBOR is an acronym for Concise Binary Object Representation. You might already be familiar with JSON (the JavaScript Object Notation). JSON provides a standard way of conveying the nested structures of key-value pairs. CBOR is conceptually the same as JSON. The difference is that CBOR is a binary format, rather than a text-based format like JSON. CBOR is designed for compactness and machine-readability, but at the expense of human-readability. You can learn more about CBOR [here](https://cbor.io/). +You will notice that the filename extension on the example token is `.cbor`, which also denotes the format of the data. CBOR is an acronym for Concise Binary Object Representation. You might already be familiar with JSON (the JavaScript Object Notation). JSON provides a standard way of conveying the nested structures of key-value pairs. CBOR is conceptually the same as JSON. The difference is that CBOR is a binary format, rather than a text-based format like JSON. CBOR is designed for compactness and machine-readability, but at the expense of human-readability. You can learn more about [CBOR](https://cbor.io/). {{% /notice %}} ## Build the evcli tool diff --git a/content/learning-paths/servers-and-cloud-computing/circleci-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/circleci-gcp/instance.md index 6daa488285..df43a44caf 100644 --- a/content/learning-paths/servers-and-cloud-computing/circleci-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/circleci-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/benchmarking.md index a57ed3ff12..8c1bee3e3d 100644 --- a/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/benchmarking.md @@ -241,11 +241,11 @@ localhost:9000, queries: 5, QPS: 20.935, RPS: 10467305.309, MiB/s: 79.859, resul ### Benchmark Metrics Explanation - **QPS (Queries Per Second):** Indicates how many complete queries ClickHouse can execute per second. Higher QPS reflects stronger overall query execution capacity. -- **RPS (Rows Per Second):** Shows the number of rows processed every second. Very high RPS values demonstrate ClickHouse’s efficiency in scanning large datasets. +- **RPS (Rows Per Second):** Shows the number of rows processed every second. Very high RPS values demonstrate ClickHouse's efficiency in scanning large datasets. - **MiB/s (Throughput):** Represents data processed per second in mebibytes. High throughput highlights effective CPU, memory, and disk utilization during analytics workloads. - **Latency Percentiles (p50, p95, p99):** Measure query response times. p50 is the median latency, while p95 and p99 show tail latency under heavier load—critical for understanding performance consistency. - **Iterations:** Number of times the same query is executed. More iterations improve measurement accuracy and stability. -- **Concurrency:** Number of parallel query clients. Higher concurrency tests ClickHouse’s ability to scale under concurrent workloads. +- **Concurrency:** Number of parallel query clients. Higher concurrency tests ClickHouse's ability to scale under concurrent workloads. - **Result RPS / Result MiB/s:** Reflects the size and rate of returned query results. Low values are expected for aggregate queries like `COUNT(*)`. - **Insert Benchmark Metrics:** Write tests measure ingestion speed and stability, where consistent latency indicates reliable bulk insert performance. diff --git a/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/instance.md index ddc0fd1bb4..676d08e5d3 100644 --- a/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/clickhouse-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/copilot-extension/2-vector.md b/content/learning-paths/servers-and-cloud-computing/copilot-extension/2-vector.md index 8340bb6795..ad24217ef3 100644 --- a/content/learning-paths/servers-and-cloud-computing/copilot-extension/2-vector.md +++ b/content/learning-paths/servers-and-cloud-computing/copilot-extension/2-vector.md @@ -74,7 +74,7 @@ cd python-rag-extension/vectorstore 2. It is recommended that you use a virtual environment to manage dependencies. -Ensure you have `conda` set up in your development environment. If you need guidance, follow the [Installation Guide](https://docs.anaconda.com/miniconda/install/. +Ensure you have `conda` set up in your development environment. If you need guidance, follow the [Installation Guide](https://docs.anaconda.com/miniconda/install/). 3. To create a new conda environment, use the following command: @@ -104,7 +104,7 @@ python chunk_a_learning_path.py --url Replace `` with the URL of the Learning Path that you want to process. -If no URL is provided, the script defaults to a [known Learning Path URL](https://learn.arm.com/learning-paths/cross-platform/kleidiai-explainer). +If no URL is provided, the script defaults to a [known Learning Path URL](/learning-paths/cross-platform/kleidiai-explainer). The script processes the specified Learning Path and saves the chunks as YAML files in a `./chunks/` directory. diff --git a/content/learning-paths/servers-and-cloud-computing/couchbase-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/couchbase-on-gcp/instance.md index afe7a52935..8f54e5bf1e 100644 --- a/content/learning-paths/servers-and-cloud-computing/couchbase-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/couchbase-on-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/cplusplus_compilers_flags/2.md b/content/learning-paths/servers-and-cloud-computing/cplusplus_compilers_flags/2.md index 33f3ee3e75..fa91db1e73 100644 --- a/content/learning-paths/servers-and-cloud-computing/cplusplus_compilers_flags/2.md +++ b/content/learning-paths/servers-and-cloud-computing/cplusplus_compilers_flags/2.md @@ -11,7 +11,7 @@ layout: learningpathall In this Learning Path, you will build and run a C++ application on an AWS Graviton 4 (`r8g.xlarge`) instance running Ubuntu 24.04 LTS. {{% notice Note %}} -If you are new to cloud computing, see [Getting started with Servers and Cloud Computing](https://learn.arm.com/learning-paths/servers-and-cloud-computing/intro/). It provides an introduction to the Arm servers available from various cloud service providers. +If you are new to cloud computing, see [Getting started with Servers and Cloud Computing](/learning-paths/servers-and-cloud-computing/intro/). It provides an introduction to the Arm servers available from various cloud service providers. {{% /notice %}} ### Get set up diff --git a/content/learning-paths/servers-and-cloud-computing/cpp-profile-guided-optimisation/how-to-3.md b/content/learning-paths/servers-and-cloud-computing/cpp-profile-guided-optimisation/how-to-3.md index 776da293ab..88e050f399 100644 --- a/content/learning-paths/servers-and-cloud-computing/cpp-profile-guided-optimisation/how-to-3.md +++ b/content/learning-paths/servers-and-cloud-computing/cpp-profile-guided-optimisation/how-to-3.md @@ -85,7 +85,7 @@ baseDiv/1500 7.90 us 7.90 us 88512 To inspect what assembly instructions are being executed most frequently, you can use the `perf` command. This is useful for identifying bottlenecks and understanding the performance characteristics of your code. -Install Perf using the [install guide](https://learn.arm.com/install-guides/perf/) before proceeding. +Install Perf using the [install guide](/install-guides/perf/) before proceeding. {{% notice Please Note %}} You may need to set the `perf_event_paranoid` value to -1 with the `sudo sysctl kernel.perf_event_paranoid=-1` command to run the commands below. diff --git a/content/learning-paths/servers-and-cloud-computing/csp/aws.md b/content/learning-paths/servers-and-cloud-computing/csp/aws.md index e11693353e..83d3d3fcdb 100644 --- a/content/learning-paths/servers-and-cloud-computing/csp/aws.md +++ b/content/learning-paths/servers-and-cloud-computing/csp/aws.md @@ -94,8 +94,7 @@ For advanced settings, it is recommended that you search `security groups` and c When all options are set, click `Launch instance` to get started. -![alt-text #center](https://user-images.githubusercontent.com/97123064/243456243-06c6f57d-457a-4b92-9705-8d6abf1870bf.png - "Launch the instance") +![alt-text #center](https://user-images.githubusercontent.com/97123064/243456243-06c6f57d-457a-4b92-9705-8d6abf1870bf.png "Launch the instance") Your compute instance will be created and be available after initialization. Click the `Instance ID` to observe the `Instance state` and other details about your instance. diff --git a/content/learning-paths/servers-and-cloud-computing/csp/google.md b/content/learning-paths/servers-and-cloud-computing/csp/google.md index 7f6fc058a8..854400fc69 100644 --- a/content/learning-paths/servers-and-cloud-computing/csp/google.md +++ b/content/learning-paths/servers-and-cloud-computing/csp/google.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/_index.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/_index.md index 4c1ce1bcd8..ff1cf3601e 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/_index.md @@ -1,19 +1,15 @@ --- title: Deploy Django on Google Cloud C4A (Arm-based Axion VMs) - -draft: true -cascade: - draft: true minutes_to_complete: 30 -who_is_this_for: This learning path is intended for software developers deploying and optimizing Django-based web applications on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. +who_is_this_for: This Learning Path is for software developers deploying and optimizing Django-based web applications on Arm64 Linux environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. learning_objectives: - - Provision an Arm-based SUSE SLES virtual machine on Google Cloud (C4A with Axion processors) + - Provision an Arm-based SUSE Linux Enterprise Server (SLES) virtual machine on Google Cloud (C4A with Axion processors) - Install Django on a SUSE Arm64 (C4A) instance - Verify Django functionality by running the development server and accessing the default welcome page on the Arm64 VM - - Measure Django application performance by benchmarking request handling throughput and latency using the official ApacheBench (ab) tool with Gunicorn on Arm64 (Aarch64) + - Measure Django application performance by benchmarking request handling throughput and latency using the official ApacheBench (ab) tool with Gunicorn on Arm64 (aarch64) prerequisites: - A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/background.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/background.md index 8b2a674186..53ff45dda6 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/background.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/background.md @@ -1,27 +1,36 @@ --- -title: Getting started with Django on Google Axion C4A (Arm Neoverse-V2) +title: Get started with Django on Google Axion C4A (Arm Neoverse-V2) weight: 2 layout: "learningpathall" --- -## Google Axion C4A Arm instances in Google Cloud +## Explore Axion C4A Arm instances in Google Cloud Google Axion C4A is a family of Arm-based virtual machines built on Google’s custom Axion CPU, which is based on Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines offer strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. +To learn more about Google Axion, see the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. -## Django -[Django](https://www.djangoproject.com/) is a high-level, **open-source Python web framework** that encourages **rapid development** and **clean, pragmatic design**. Developed and maintained by the [Django Software Foundation](https://www.djangoproject.com/foundation/), it simplifies web application development by handling much of the boilerplate and providing powerful built-in features. +## Understand Django on Google Axion C4A (Arm Neoverse-V2) -Django follows the **Model–View–Template (MVT)** architectural pattern and includes robust tools for **authentication**, **URL routing**, **form handling**, **ORM (Object Relational Mapping)**, **session management**, and **administration interface** — all out of the box. +Django is a high-level, open-source Python web framework that encourages rapid development and clean, pragmatic design. Developed and maintained by the [Django Software Foundation](https://www.djangoproject.com/foundation/), it simplifies web application development by handling much of the boilerplate and providing powerful built-in features. -Django is known for its focus on **security**, **scalability**, and **maintainability**, making it suitable for everything from small projects to large-scale enterprise applications. It helps developers build secure, high-performance web applications quickly without reinventing common components. +Django follows the Model–View–Template (MVT) architectural pattern and includes robust tools for authentication, URL routing, form handling, ORM (Object Relational Mapping), session management, and administration interface - all out of the box. + +Django is known for its focus on security, scalability, and maintainability, making it suitable for everything from small projects to large-scale enterprise applications. It helps developers build secure, high-performance web applications quickly without reinventing common components. + +Common use cases include web applications, content management systems, APIs, e-commerce platforms, and data-driven dashboards. It integrates seamlessly with popular databases like PostgreSQL, MySQL, SQLite, and Oracle. + +To learn more, visit the [Django website](https://www.djangoproject.com/) and explore the [Django documentation](https://docs.djangoproject.com/en/stable/). + +## Summary and what's next + +You now understand the capabilities of Google Axion C4A Arm-based VMs and why Django is an excellent choice for building web applications on Arm infrastructure. The combination provides a cost-effective, high-performance platform for deploying Python web applications at scale. + +In the next sections, you'll provision your own Arm-based VM on Google Cloud, install Django, and benchmark your application's performance. You're ready to start building! -Common use cases include **web applications**, **content management systems**, **APIs**, **e-commerce platforms**, and **data-driven dashboards**. It integrates seamlessly with popular databases like **PostgreSQL**, **MySQL**, **SQLite**, and **Oracle**. -To learn more, visit the [official Django website](https://www.djangoproject.com/) and explore the [Django documentation](https://docs.djangoproject.com/en/stable/). diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/baseline.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/baseline.md index 955ce42de5..15eca68372 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/baseline.md @@ -1,19 +1,26 @@ --- -title: Django Baseline Testing on Google Axion C4A Arm Virtual Machine +title: Verify Django installation and run the development server weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Django Baseline Testing on GCP SUSE VMs -This baseline testing guide verifies that your **Django installation**, **web server**, and **basic application routing** are functioning correctly on a **Google Cloud SUSE Linux Arm64 (Axion C4A)** virtual machine. -You will first run the Django development server and access it from your browser, then create a simple Django app to ensure routing works. +## Verify Django is working on your Arm-based VM -### Baseline 1 — View Django Welcome Page -This test confirms that Django is installed correctly and the server runs successfully. +In this section, you'll confirm that Django is installed correctly and can serve web requests on your Google Cloud C4A VM. You'll create a Django project, run the development server, and access it from your browser. This hands-on verification ensures your environment is ready for development and testing. + +By the end of this section, you'll have: +- Created a Django project with proper directory structure +- Configured Django to accept requests from your VM's external IP +- Run the development server and accessed it from your browser +- Built a simple Django app with custom routing and views +- Verified that Django can handle HTTP requests and render responses + +Let's get started! + +## Create and test a basic Django project -#### Create a new Django project Run the following command to create a new Django project named `myproject`: ```console @@ -32,33 +39,30 @@ myproject/ ├── asgi.py └── wsgi.py ``` -- `manage.py` is Django’s command-line utility for project management (running server, migrations, etc.). -- The inner `myproject/` folder contains the core configuration files that define your project’s settings and URLs.- -#### Run initial migrations -Migrations prepare your project’s database by creating the required tables for Django’s internal apps (admin, authentication, etc.): +The `manage.py` file is Django's command-line utility for project management. The inner `myproject/` folder contains your project's core settings and URL configuration. + +## Run initial database migrations + +Set up your project's database by running migrations, which create the required tables for Django's built-in apps: ```console python3 manage.py migrate ``` -You should get output showing the Running Migrations (all of which should be "OK"). +The output shows all migrations applied successfully (marked "OK"). -#### Start the Django development server -Before starting the Django development server, you must configure your ALLOWED_HOSTS setting to allow access from your VM’s external IP. -This ensures that Django accepts HTTP requests from outside the localhost (e.g., when testing in a browser or from another machine). +## Configure ALLOWED_HOSTS for external access -**ALLOWED_HOSTS:** is a security setting in Django that defines which host/domain names your Django site can serve. +Before starting the Django development server, you must configure your `ALLOWED_HOSTS` setting to allow access from your VM's external IP. This ensures that Django accepts HTTP requests from outside localhost (for example, when testing in a browser from another machine). -- Navigate to Your Project Settings - Move into your Django project directory where the settings.py file is located. +Navigate to your project settings directory: - ```console - cd ~/myproject/myproject/ - ``` +```console +cd ~/myproject/myproject/ +``` -- Open settings.py File - Use any text editor (like vi or nano) to open the file ("edit" is used as an example below). +Open `settings.py` using a text editor: ```console edit myproject/settings.py @@ -67,10 +71,11 @@ This ensures that Django accepts HTTP requests from outside the localhost (e.g., - Locate the `ALLOWED_HOSTS` Line Inside the file, find the following line: - ```python - ALLOWED_HOSTS = [] - ``` - This setting defines which host/domain names Django will serve. +```python +ALLOWED_HOSTS = [] +``` + +Update it to allow your VM's external IP address: - Allow All Hosts (for Testing Only) To make your Django app accessible from your VM’s external IP address, update it to: @@ -78,49 +83,54 @@ This ensures that Django accepts HTTP requests from outside the localhost (e.g., ALLOWED_HOSTS = ['*'] ``` {{% notice Note %}} -Allowing all hosts `('*')` is suitable **only for development or testing**. -For production, replace `'*'` with specific domain names or IPs, such as your public IP address for your VM that you recorded earlier: +For development and testing only, you can use `ALLOWED_HOSTS = ['*']` to allow all hosts. However, for production deployments, always specify explicit domain names or IP addresses such as `ALLOWED_HOSTS = ['your-external-ip', 'your-domain.com']`. {{% /notice %}} ```python ALLOWED_HOSTS = ['your-external-ip', 'your-domain.com'] ``` -**Now start the Django development server:** +Now that you've configured `ALLOWED_HOSTS`, start the development server: -We can now start the Django development server since we have exposed TCP/8000 in our VM via firewall rules: ```console python3 manage.py runserver 0.0.0.0:8000 ``` -#### View in browser -Open a web browser on your local machine (Chrome, Firefox, Edge, etc.) and enter the following URL in the address bar. Please replace "YOUR_VM_EXTERNAL_IP" with the external IP address of your VM that you saved off earlier: +This starts the Django development server on port 8000, listening on all network interfaces. -```console +## Access Django in your browser + +Open a web browser on your local machine and navigate to: + +``` http://:8000 ``` -- Replace `` with the public IP of your GCP VM. -If everything is set up correctly, you should see the default Django welcome page (“The install worked successfully!”). It looks like this: +Replace `` with the public IP address of your GCP VM. + +You should see the Django welcome page with the message "The install worked successfully!": + +![Screenshot of the Django welcome page displayed in a web browser. The page features a large heading stating The install worked successfully followed by a subheading congratulating the user on successfully installing Django. Below are instructions for the next steps, including editing the settings file and reading the Django documentation. The page has a clean white background with blue highlights. alt-text#center](images/django-welcome-page.png "Django welcome page") -![Django welcome page alt-text#center](images/django-welcome-page.png "Figure 1: Django web page") +## Build a simple Django app with custom routing -### Baseline 2 — Create a Simple Django App -This test ensures Django’s application routing and view rendering work as expected. +This section demonstrates that Django's application routing and view rendering work correctly by creating a simple app with a custom view. -#### Stop the server -Press `Ctrl + C` to stop the Django server if running. +## Stop the server + +Press **Ctrl + C** in your terminal to stop the Django development server. + +## Create a new Django app -#### Create a new app Within your Django project directory, create a new app named `hello`: ```console python3 manage.py startapp hello ``` -**This creates the following directory:** +This generates the following directory structure with files for views, models, configuration, and more: -```markdown +```output hello/ ├── admin.py ├── apps.py @@ -130,19 +140,22 @@ hello/ └── urls.py ``` -#### Create a simple view -Edit `hello/views.py`. Replace your existing file with this: +## Define a view function + +Edit `hello/views.py` and replace the entire file with: ```python from django.http import HttpResponse def home(request): - return HttpResponse("

Hello, Django on GCP SUSE ARM64!

") + return HttpResponse("

Hello, Django on Arm!

") ``` -This defines a simple view function that sends a basic HTML message as the HTTP response. -#### Create app URL configuration -Create a new file `hello/urls.py` and add: +This simple view function returns a basic HTML message as an HTTP response. + +## Create URL configuration for your app + +Create a new file `hello/urls.py`: ```python from django.urls import path @@ -152,10 +165,12 @@ urlpatterns = [ path('', views.home, name='home'), ] ``` -This maps the root URL `(/)`of your app to the `home()` view function. -#### Link the app to the main project -Replace your default `myproject/urls.py` file with this version. +This maps the root URL path to your `home()` view function. + +## Include the app in your project's main URLs + +Edit `myproject/urls.py` to include the `hello` app's URLs: ```python """myproject URL Configuration @@ -182,11 +197,12 @@ urlpatterns = [ path('', include('hello.urls')), ] ``` -This tells Django to delegate routing for the root path (`''`) to the `hello` app’s URLs. -#### Add the app to settings -This makes Django aware of your new app so it can load its configuration and routes. -Edit `myproject/settings.py` → add `'hello'` to INSTALLED_APPS: +This tells Django to route the root path to your `hello` app. + +## Register the app in Django settings + +Django needs to know about your new app. Edit `myproject/settings.py` and add `'hello'` to the `INSTALLED_APPS` list: ```python INSTALLED_APPS = [ @@ -199,18 +215,27 @@ INSTALLED_APPS = [ 'hello', ] ``` -#### Run the server again + +## Start the server again + +Restart the Django development server: ```console python3 manage.py runserver 0.0.0.0:8000 ``` -#### Test your app -Open in browser: +## Test your custom app + +Open your browser and navigate to: -```console -http://:8000 ``` -You should see the Django app. It looks like this: +http://:8000 +``` + +You should now see your custom message displayed: + +![Screenshot of a web browser displaying a Django app with a large heading that reads Hello, Django on Arm centered on a clean white background. The page contains no additional content or navigation, creating a straightforward and welcoming tone. alt-text#center](images/django-app.png "Django custom app") + +## Summary and what's next -![Django App alt-text#center](images/django-app.png "Figure 2: Django App") +You've successfully verified that Django is installed and working on your Arm-based VM. Your application can serve web requests, handle routing, and render custom views. Great job, you're ready to benchmark your Django application! diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/benchmarking.md index 230ecad863..fe5dcd7063 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/benchmarking.md @@ -1,71 +1,74 @@ --- -title: Django Benchmarking +title: Benchmark Django application performance on Arm weight: 7 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Benchmark your Django application with ApacheBench and Gunicorn -## Django Benchmarking using ApacheBench -This section describes how to benchmark a Django web application deployed with **Gunicorn** using **ApacheBench (ab)** — a lightweight HTTP benchmarking tool. -You will measure **throughput (requests per second)** and **latency (response time)** to evaluate the performance of your Django app on an Arm-based GCP SUSE VM. +This section guides you through benchmarking your Django web application using Gunicorn as a production-like WSGI server and ApacheBench (`ab`) for load testing. You'll measure throughput (requests per second) and latency (response time) to evaluate your Django app's performance on an Arm-based Google Cloud C4A VM. -### Stop the server -Press `Ctrl + C` to stop the Django server if running. +## Install ApacheBench -### Ensure ApacheBench is installed -**ApacheBench (ab)** is a command-line tool used to benchmark web servers by simulating multiple HTTP requests. +To begin, if your Django development server is still running, stop it using `Ctrl + C`. -Install it using following command: +ApacheBench (`ab`) is a command-line tool that simulates multiple HTTP requests to measure web server performance. Install it using the following: ```console sudo zypper install -y apache2-utils ``` -**Verify installation:** +## Verify installation -This confirms ApacheBench is correctly installed and available system-wide. +This command confirms ApacheBench is correctly installed and available system-wide: ```console ab -V ``` -**Ensure Django and Gunicorn are installed:** +The output displays the ApacheBench version, confirming successful installation. + +## Install and configure Gunicorn + +Before benchmarking your Django application, you need to install Gunicorn, a production-grade WSGI HTTP server. Gunicorn provides better performance characteristics than Django's development server and more accurately represents real-world deployment scenarios. + +Install both Django and Gunicorn using pip: ```console python3 -m pip install django gunicorn ``` -- **Django** is the Python web framework you’re benchmarking. -- **Gunicorn** is a high-performance WSGI HTTP server for deploying Django apps in production-like environments. -### Run Django with Gunicorn -Use Gunicorn to serve your Django application for benchmarking (run in the background): +This command installs two essential packages. Django is the Python web framework you're benchmarking, while Gunicorn serves as a high-performance WSGI HTTP server that handles multiple concurrent requests efficiently. Unlike Django's built-in development server, Gunicorn is designed for production workloads and provides the multi-worker architecture needed for accurate performance testing. + + +## Run Django with Gunicorn + +Start your Django application using Gunicorn: ```console gunicorn myproject.wsgi:application --bind 0.0.0.0:8000 --workers 4 & ``` -- `--workers 4`: number of worker processes -- `--bind 0.0.0.0:8000`: binds to all interfaces on port 8000 -- `myproject.wsgi:application` your Django project name ("myproject" used in this example). +This command starts Gunicorn with four worker processes to handle concurrent requests. Replace `myproject` with your Django project's actual name. The `--bind 0.0.0.0:8000` flag makes the server accessible on port 8000 from any network interface, while the `&` runs the process in the background so you can continue using your terminal. {{% notice Note %}} -Keep this terminal running during the benchmark. If you’re testing remotely, ensure port 8000 is open in your VM firewall settings. +Ensure your VM's firewall allows inbound traffic on port 8000. See the firewall setup section if you haven't already configured this. {{% /notice %}} -### Benchmark with ApacheBench (ab) -Run ApacheBench to simulate multiple clients hitting your Django server. +## Run the benchmark +Use ApacheBench to test your Django server with simulated traffic: ```console ab -n 1000 -c 10 http://127.0.0.1:8000/ ``` -- `-n 1000`: total number of requests -- `-c 10`: concurrency (simultaneous requests) -You should see an output similar to: +This sends 1000 requests using 10 concurrent connections to your local server's root URL. The `-n` flag sets the total number of requests, while `-c` controls how many run simultaneously. + +The output is similar to: ```output -This is ApacheBench, Version 2.3 <$Revision: 1903618 $> +This is ApacheBench, Version 2.3 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ @@ -82,7 +85,6 @@ Completed 900 requests Completed 1000 requests Finished 1000 requests - Server Software: gunicorn Server Hostname: 127.0.0.1 Server Port: 8000 @@ -119,25 +121,43 @@ Percentage of the requests served within a certain time (ms) 99% 3 100% 5 (longest request) ``` +## Stop the Gunicorn server -### Cleanup +After reviewing the benchmark results, stop the Gunicorn server running in the background: -With the following output (above) seen, you can type "fg" followed by "ctrl-c" to exit the gunicorn server that is running. +```bash +fg +``` -### Benchmark Metrics Explanation +This brings the background Gunicorn process to the foreground. Then press `Ctrl+C` to stop it. -- **Concurrency Level:** Number of requests executed simultaneously during the test. -- **Time Taken for Tests:** Total time required to complete all HTTP requests. -- **Complete Requests:** Total number of successful requests processed. -- **Failed Requests:** Number of requests that failed or returned errors. -- **Total Transferred:** Total amount of data (including headers) sent and received. -- **HTML Transferred:** Amount of actual response content transferred. -- **Requests per Second:** Average number of requests handled by the server per second. -- **Time per Request (mean):** Average time taken to process a single request. -- **Time per Request (across concurrent):** Mean time per request across all concurrent clients. -- **Transfer Rate:** Average network data throughput during the benchmark. +## Interpret your benchmark results + +The ApacheBench output provides key performance metrics that help you evaluate your Django application's capabilities on Arm. Here's what each metric tells you: +### Request handling metrics +- Concurrency Level: number of simultaneous requests the benchmark sent (10 in this example) +- Complete Requests: total successful requests processed (1000 in this test) +- Failed Requests: number of errors or timeouts (0 indicates stable performance) + +### Performance metrics +- Requests per Second: how many requests your server handles per second - higher values indicate better throughput +- Time per Request (mean): average time to complete a single request - lower values mean faster responses +- Time per Request (across concurrent): average latency when factoring in concurrent processing, this shows how well your app handles parallel requests + +### Data transfer metrics +- Total Transferred: all data sent and received, including HTTP headers +- HTML Transferred: actual response content size +- Transfer Rate: network throughput in KB/sec—indicates data handling efficiency +### Timing breakdown +- Time Taken for Tests: total benchmark duration +- Connection Times: shows minimum, mean, median, and maximum times for connecting, processing, and waiting + +These metrics provide a performance baseline for your Django application on Arm. You can use them to compare different configurations, identify bottlenecks, or validate optimizations. + +## Benchmark summary + +The table below summarizes the key performance indicators from the benchmark test. These results demonstrate the capabilities of your Django application running on a Google Cloud C4A Arm-based VM with Gunicorn. -### Benchmark summary Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE): | **Parameter** | **Description** | **Value** | @@ -158,8 +178,12 @@ Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm6 | **Time per Request (across all concurrent requests)** | Average latency considering concurrency | **0.104 ms** | | **Transfer Rate** | Network throughput rate | **2639.00 KB/sec** | -- **Exceptional Throughput:** The Arm64 VM efficiently handled nearly 10K requests per second, showcasing excellent concurrency handling. -- **Low Latency:** Average response time stayed around 1 ms, indicating rapid request processing even under load. -- **High Efficiency:** Zero failed requests demonstrate stable and reliable performance under benchmark conditions. -- **Optimized Networking:** Strong data transfer rate highlights Arm64’s efficient network I/O capabilities. -- **Ideal for Scalable Apps:** The consistent and predictable response times make Arm64 VMs well-suited for high-performance web workloads. +## Key performance insights + +The benchmark results reveal several important characteristics of running Django on Arm-based infrastructure: + +- Exceptional Throughput: the Arm64 VM efficiently handled nearly 10K requests per second, showcasing excellent concurrency handling. +- Low Latency: average response time stayed around 1 ms, indicating rapid request processing even under load. +- High Efficiency: zero failed requests demonstrate stable and reliable performance under benchmark conditions. +- Optimized Networking: strong data transfer rate highlights Arm64's efficient network I/O capabilities. +- Ideal for Scalable Apps: the consistent and predictable response times make Arm64 VMs well-suited for high-performance web workloads. diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/firewall_setup.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/firewall_setup.md index 983fcdb15c..fbe41f5249 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/firewall_setup.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/firewall_setup.md @@ -1,42 +1,55 @@ --- -title: Create a Firewall Rule on GCP +title: Configure firewall rules for Django on Google Cloud weight: 3 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Overview +## Open port 8000 for your Django application -In this section, you will learn how to create a Firewall Rule within Google Cloud Console. For this learning path, we need to expose TCP port 8000. +Before you can access your Django development server from your browser, you need to configure Google Cloud's firewall to allow inbound traffic on port 8000. This section walks you through creating a firewall rule that permits HTTP requests to reach your VM. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} -## Create a Firewall Rule in GCP -For this learning path, we need to expose TCP port 8000. To accomplish this, we first need to create a firewall rule. -- Navigate to the [Google Cloud Console](https://console.cloud.google.com/). -- Go to **VPC Network > Firewall** and press **Create firewall rule**. +## Create a firewall rule in Google Cloud Console -![Create a firewall rule](images/firewall-rule.png "Create a firewall rule") +Navigate to the Google Cloud Console and create a new firewall rule: -- Next, we create the firewall rule that will expose TCP port 8000 for our learning path. -- Set the "Name" of the new rule to "allow-tcp-8000" -- Select your network that you intend to bind to your VM (default is "autoscaling-net" but your organization might have others that you need to use) -- Direction of traffic should be set to "Ingress" -- Allow on match should be set to "Allow" and the "Targets" should be set to "Specified target tags". -- Enter "allow-tcp-8000" to the "Target tags" text field -- Set the "Source IPv4 ranges" text value to "0.0.0.0/0" +- Go to the **Google Cloud Console** +- Select **VPC network** > **Firewall** +- Select **Create firewall rule** -![Create a firewall rule](images/network-rule.png "Creating the TCP/8000 firewall rule") +![Screenshot of the Google Cloud Console showing the Firewall section. The Create firewall rule button is prominently displayed at the top of the page. alt-text#center](images/firewall-rule.png "Google Cloud Console Firewall page") -- Lastly, we select "Specified protocols and ports" under the "Protocols and ports" section -- Select the "TCP" checkbox -- Enter "8000" in the "Ports" text field -- Press "Create" +## Configure the firewall rule -![Specifying the TCP port to expose](images/network-port.png "Specifying the TCP port to expose") +Fill in the rule details to allow traffic on port 8000: -Our network firewall rule is now created so we can continue with the VM creation! \ No newline at end of file +- Set **Name** to `allow-django-8000`. +- Select your network (the default is `default`; your organization may use a different network). +- Set **Direction of traffic** to **Ingress**. +- Set **Action on match** to **Allow**. +- Set **Targets** to **Specified target tags**. +- Enter `django-server` in the **Target tags** field. +- Set **Source IPv4 ranges** to `0.0.0.0/0` (allows traffic from any IP address; restrict this in production). + +![Screenshot showing the firewall rule configuration form. The Name field shows allow-django-8000, Direction is set to Ingress, and Targets is set to Specified target tags. alt-text#center](images/network-rule.png "Firewall rule configuration") + +## Specify the port and protocol + +Configure the protocol and port settings: + +- Under **Protocols and ports**, select **Specified protocols and ports**. +- Check the **TCP** checkbox. +- Enter `8000` in the **Ports** field. +- Click **Create**. + +![Screenshot of the Protocols and ports section. The TCP checkbox is checked and port 8000 is entered in the Ports field. alt-text#center](images/network-port.png "Specifying TCP port 8000") + +## Summary and what's next + +Your firewall rule is now created and active. Your Django application running on port 8000 is now accessible from external IP addresses. You're ready to proceed with VM creation and Django installation! \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/images/firewall_setup.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/images/firewall_setup.md deleted file mode 100644 index 1322281f04..0000000000 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/images/firewall_setup.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -title: Create a Firewall Rule on GCP -weight: 3 - -### FIXED, DO NOT MODIFY -layout: learningpathall ---- - -## Overview - -In this section, you will learn how to create a Firewall Rule within Google Cloud Console. For this learning path, we need to expose TCP port 8000. - -{{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). -{{% /notice %}} - -## Create a Firewall Rule in GCP - -For this learning path, we need to expose TCP port 8000. To accomplish this, we first need to create a firewall rule. -- Navigate to the [Google Cloud Console](https://console.cloud.google.com/). -- Go to **VPC Network > Firewall** and press **Create firewall rule**. - -![Create a firewall rule](images/firewall-rule.png "Create a firewall rule") - -- Next, we create the firewall rule that will expose TCP port 8000 for our learning path. -- Set the "Name" of the new rule to "allow-tcp-8000" -- Select your network that you intend to bind to your VM (default is "autoscaling-net" but your organization might have others that you need to use) -- Direction of traffic should be set to "Ingress" -- Allow on match should be set to "Allow" and the "Targets" should be set to "Specified target tags". -- Enter "allow-tcp-8000" to the "Target tags" text field -- Set the "Source IPv4 ranges" text value to "0.0.0.0/0" - -![Create a firewall rule](images/network-rule.png "Creating the TCP/8000 firewall rule") - -- Lastly, we select "Specified protocols and ports" under the "Protocols and ports" section -- Select the "TCP" checkbox -- Enter "8091" in the "Ports" text field -- Press "Create" - -![Specifying the TCP port to expose](images/network-port.png "Specifying the TCP port to expose") - -Our network firewall rule is now created so we can continue with the VM creation! \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/installation.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/installation.md index 1271b33f9b..ac3bb5cb0c 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/installation.md @@ -1,71 +1,90 @@ --- -title: Install Django +title: Install Django on your Arm-based VM weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Install Django on GCP VM -This guide walks you through installing Django on a **Google Cloud Platform (GCP) SUSE Linux Arm64 VM**, including all dependencies, Python setup, and environment preparation. +## Install Django and dependencies -### Update Your System -Before installing Django, it’s good practice to update your package list and upgrade installed software to ensure you have the latest versions and security patches. +After connecting to your SUSE Linux Enterprise Server (SLES) VM using SSH, you'll update your system, install Python 3.11, and set up a virtual environment for your Django project. + +## Update your system + +Begin by refreshing your package list and upgrading installed software to ensure you have the latest versions and security patches: ```console sudo zypper refresh sudo zypper update -y ``` -### Install Python and Tools -**Django** requires **Python 3.10+**. We will use Python 3.11, which is compatible with Django 5. -You will also install `pip` for package management, and basic developer tools (`git`, `gcc`, and `make`) to build Python packages and work with Django projects. +## Install Python 3.11 and development tools + +Django requires Python 3.10 or later. You'll install Python 3.11 along with pip (Python's package manager) and essential build tools needed for compiling Python packages: + ```console -sudo zypper install -y python311 python311-pip python311-devel -sudo zypper install -y git gcc make +sudo zypper install -y python311 python311-pip python311-devel git gcc make ``` -**Ensure that both Python and pip are installed correctly:** +Verify that Python and pip are installed correctly: -```console +```bash python3.11 --version -pip3 --version +pip3.11 --version ``` -You should see an output similar to: +The output is similar to: + ```output Python 3.11.10 pip 22.3.1 from /usr/lib/python3.11/site-packages/pip (python 3.11) ``` -### Create a Project Folder and Virtual Environment -It’s recommended to create a dedicated project directory and use a **virtual environment** to isolate project dependencies. +## Create a project directory and virtual environment + +Create a dedicated directory for your Django project and set up a Python virtual environment to isolate your project's dependencies: ```console mkdir ~/myproject && cd ~/myproject python3.11 -m venv venv source venv/bin/activate ``` -- `python3.11 -m venv venv` — creates a virtual environment named venv inside your project folder. -- `source venv/bin/activate` — activates the virtual environment. -Once activated, your command prompt will show (venv) at the beginning, indicating that you’re working inside an isolated Python environment. -### Upgrade Pip and Install Django -With your virtual environment active, upgrade pip and install Django using the following commands: +The `python3.11 -m venv venv` command creates an isolated Python environment named `venv` inside your project folder. Running `source venv/bin/activate` activates this environment. + +Once activated, your command prompt displays `(venv)` at the beginning, indicating you're working inside an isolated Python environment where all packages are isolated from your system Python installation. + +## Upgrade pip and install Django + +With your virtual environment active, upgrade pip to the latest version: ```console python3 -m pip install --upgrade pip -python3 -m pip install django ``` -**Confirm that Django is installed correctly by checking its version:** +Now install Django and additional useful packages for web development: -```console +```bash +python3 -m pip install django gunicorn +``` + +This installs: +- **Django** — the web framework for building your application +- **Gunicorn** — a production-ready WSGI (Web Server Gateway Interface) server for running Django applications + +Verify that Django is installed correctly: + +```bash django-admin --version ``` -You should see an output similar to: +The output is similar to: + ```output 5.2.8 ``` -Django is installed successfully and ready for project setup. + +## Summary and what's next + +You have successfully installed Django and all required dependencies on your Arm-based VM. Your environment is now ready for creating Django projects and applications! diff --git a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/instance.md index e708afde53..4a06fbe1fc 100644 --- a/content/learning-paths/servers-and-cloud-computing/django-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/django-on-gcp/instance.md @@ -6,12 +6,12 @@ weight: 4 layout: learningpathall --- -## Overview +## Provision a Google Axion C4A Arm VM -In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. +You'll create a Google Axion C4A Arm-based virtual machine (VM) on Google Cloud Platform (GCP) using the `c4a-standard-4` machine type (4 vCPUs, 16 GB memory). This VM will host your Django application. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For help with GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console @@ -24,7 +24,7 @@ To create a virtual machine based on the C4A instance type: - Set **Series** to `C4A`. - Select `c4a-standard-4` for machine type. - ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") +![Screenshot of the Google Cloud Console showing the Machine configuration section. The Series dropdown is set to C4A and the machine type c4a-standard-4 is selected. alt-text#center](images/gcp-vm.png "Configuring machine type to C4A in Google Cloud Console") - Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. @@ -33,17 +33,23 @@ To create a virtual machine based on the C4A instance type: - Under **Networking**, enable **Allow HTTP traffic**. - Also under **Networking**, in the "Network tags" text field add "allow-tcp-8000" as an additional tag -![Adding the TCP/8000 firewall rule to our VM](images/network-config.png "Adding the TCP/8000 firewall rule to our VM") +![Screenshot showing the Networking configuration section. The Allow HTTP traffic checkbox is enabled and the Network tags field contains django-server. alt-text#center](images/network-config.png "Configuring networking and tags") -- Click **Create** to launch the instance. -- Once created, you should see a "SSH" option to the right in your list of VM instances. You should also see the public IP address for your VM. -- Save off the public IP address for your VM as you will need this in the next step. -- Click on this to launch a SSH shell into your VM instance: +## Create the instance -![Invoke a SSH session via your browser alt-text#center](images/gcp-pubip-ssh.png "Invoke a SSH session into your running VM instance") +Click **Create** to launch your VM instance. Google Cloud provisions the instance, which typically takes one to two minutes. +Once the instance is running, you'll see it listed in the VM instances table with a green checkmark. Note the External IP address displayed in the list—you'll need this to access your Django application later. -- A window from your browser should come up and you should now see a shell into your VM instance: +## Connect using SSH -![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") +Click the **SSH** button next to your running instance to open a browser-based terminal session. -Next, let's install django! \ No newline at end of file +![Screenshot of the VM instances list showing the SSH button next to a running instance. The external IP address is visible in the same row. alt-text#center](images/gcp-pubip-ssh.png "Launching an SSH session from the VM instances list") + +A browser window opens with a terminal shell connected to your VM. You're now ready to install Django. + +![Screenshot of a terminal shell in the browser, connected to the running VM instance. The shell displays a command prompt ready for input. alt-text#center](images/gcp-shell.png "Terminal shell connected to your VM") + +## Summary and what's next + +You have successfully provisioned an Arm-based VM on Google Cloud. Next, you'll install Django and configure your web application! \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/envoy-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/envoy-gcp/instance.md index 04bba3a19e..3aabcab23a 100644 --- a/content/learning-paths/servers-and-cloud-computing/envoy-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/envoy-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/buffer-overflow.md b/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/buffer-overflow.md index bf27b5ccc0..5b06fb9573 100644 --- a/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/buffer-overflow.md +++ b/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/buffer-overflow.md @@ -194,8 +194,7 @@ What have the 3 instructions in the prologue done? Draw the frame layout of function `f` at this point, including where the stack pointer and the frame pointer point to. -The answer to this exercise can be found in the [Answers section]({{< relref -"exercise-answers#answer-to-exercise-3" >}}). +The answer to this exercise can be found in the [Answers section]({{< relref "exercise-answers#answer-to-exercise-3" >}}). {{% /notice %}} diff --git a/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/frame-layout.md b/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/frame-layout.md index 6799592935..a84095e8a6 100644 --- a/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/frame-layout.md +++ b/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/frame-layout.md @@ -69,8 +69,7 @@ From the offset `[sp, #8]` and `[sp, #12]` you can compute the frame layout. Draw the frame layout of function `f`: How are `buffer1` and `buffer2` laid out in the 16 bytes that are reserved? -The answer to this exercise can be found in the [Answers section]({{< relref -"exercise-answers#answer-to-exercise-1" >}}). +The answer to this exercise can be found in the [Answers section]({{< relref "exercise-answers#answer-to-exercise-1" >}}). {{% /notice %}} @@ -133,7 +132,6 @@ Draw the frame layout of function `main`. * Where do the frame pointer (`x29`) and stack pointer (`sp`) point to just before the call to function `f`, which is done by instruction `bl f`? -The answer to this exercise can be found in the [Answers section]({{< relref -"exercise-answers#answer-to-exercise-2" >}}). +The answer to this exercise can be found in the [Answers section]({{< relref "exercise-answers#answer-to-exercise-2" >}}). {{% /notice %}} diff --git a/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/redirect-control-flow-1.md b/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/redirect-control-flow-1.md index a2108800ab..402fda9558 100644 --- a/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/redirect-control-flow-1.md +++ b/content/learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/redirect-control-flow-1.md @@ -177,8 +177,7 @@ You saw previously that the return address will be overwritten with bytes 16 til Create a string that results in overwriting the return address in function `f` with the value `0x0000aaaaaaaa0820`. -The answer to this exercise can be found in the [Answers section]({{< relref -"exercise-answers#answer-to-exercise-4" >}}). +The answer to this exercise can be found in the [Answers section]({{< relref "exercise-answers#answer-to-exercise-4" >}}). {{% /notice %}} diff --git a/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-2.md b/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-2.md index d4dbd95c6f..9470feae9a 100644 --- a/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-2.md +++ b/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-2.md @@ -66,7 +66,7 @@ If the output is blank then SPE is not available. You can install and run a Python script named Sysreport to summarize your system's performance profiling capabilities. -See the Learning Path [Get ready for performance analysis with Sysreport](https://learn.arm.com/learning-paths/servers-and-cloud-computing/sysreport/) to learn how to install and run it. +See the Learning Path [Get ready for performance analysis with Sysreport](/learning-paths/servers-and-cloud-computing/sysreport/) to learn how to install and run it. Look at the Sysreport output and confirm SPE is available by checking the `perf sampling` field. diff --git a/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-4.md b/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-4.md index 61fdaebbfc..39fd13886b 100644 --- a/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-4.md +++ b/content/learning-paths/servers-and-cloud-computing/false-sharing-arm-spe/how-to-4.md @@ -76,7 +76,7 @@ The instructions per cycle (IPC) are also notably different, (0.74 vs. 1.70) and There are many root causes of variations in IPC. -To identify where the bottleneck occurs, we’ll start by using the [Arm Topdown methodology](https://developer.arm.com/documentation/109542/0100/Arm-Topdown-methodology). Install the python script using the [Telemetry Solution Install Guide](https://learn.arm.com/install-guides/topdown-tool/). +To identify where the bottleneck occurs, we’ll start by using the [Arm Topdown methodology](https://developer.arm.com/documentation/109542/0100/Arm-Topdown-methodology). Install the python script using the [Telemetry Solution Install Guide](/install-guides/topdown-tool/). Run the following command to observe the ratio of frontend to backend stall cycles. These indicate which section of the CPU pipeline is waiting on resources and causing slower performance. diff --git a/content/learning-paths/servers-and-cloud-computing/fexpa/_index.md b/content/learning-paths/servers-and-cloud-computing/fexpa/_index.md new file mode 100644 index 0000000000..4f64d1d84f --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fexpa/_index.md @@ -0,0 +1,52 @@ +--- +title: Accelerate the exponential function + +draft: true +cascade: + draft: true + +minutes_to_complete: 15 + +who_is_this_for: This is an introductory topic for developers interested in implementing the exponential function and optimizing it. The Scalable Vector Extension (SVE), introduced with the Armv8-A architecture, includes a dedicated instruction, FEXPA. Although initially not supported in SME, the FEXPA instruction has been made available in Scalable Matrix Extension (SME) version 2.2. + +learning_objectives: + - Implementing with SVE intrinsics the exponential function + - Optimizing it with FEXPA + +prerequisites: + - An AArch64 computer running Linux or macOS. You can use cloud instances, refer to [Get started with Arm-based cloud instances](/learning-paths/servers-and-cloud-computing/csp/) for a list of cloud service providers. + - Some familiarity with SIMD programming and SVE intrinsics. + +author: +- Arnaud Grasset +- Claudio Martino +- Alexandre Romana + +further_reading: + - resource: + title: Scalable Vector Extensions documentation + link: https://developer.arm.com/Architectures/Scalable%20Vector%20Extensions + type: documentation + - resource: + title: FEXPA documentation + link: https://developer.arm.com/documentation/ddi0602/2025-12/SVE-Instructions/FEXPA--Floating-point-exponential-accelerator-?lang=en + type: documentation + +### Tags +skilllevels: Introductory +subjects: Performance and Architecture +armips: + - Neoverse +operatingsystems: + - Linux + - macOS +tools_software_languages: + - C + - C++ + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/fexpa/_next-steps.md b/content/learning-paths/servers-and-cloud-computing/fexpa/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fexpa/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/servers-and-cloud-computing/fexpa/conclusion.md b/content/learning-paths/servers-and-cloud-computing/fexpa/conclusion.md new file mode 100644 index 0000000000..626f2ea3d1 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fexpa/conclusion.md @@ -0,0 +1,18 @@ +--- +title: Conclusion +weight: 5 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Conclusion +The SVE2 FEXPA instruction can speed-up the computation of the exponential function by implementing Look-Up and bit manipulation. + +In conclusion, SME-support for FEXPA lets you embed the expensive exponential approximation directly into the matrix computation path. That translates into: +- Fewer instructions (no back-and-forth to scalar/SVE code) +- Potentially higher aggregate throughput (more exponentials per cycle) +- Lower power & bandwidth (data being kept in SME engine) +- Cleaner fusion with GEMM/GEMV workloads + +All of which makes all exponential heavy workloads significantly faster on ARM CPUs. diff --git a/content/learning-paths/servers-and-cloud-computing/fexpa/fexpa.md b/content/learning-paths/servers-and-cloud-computing/fexpa/fexpa.md new file mode 100644 index 0000000000..80bb34b75b --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fexpa/fexpa.md @@ -0,0 +1,70 @@ +--- +title: FEXPA +weight: 4 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## The FEXPA instruction + +Arm introduced in SVE an instruction called FEXPA: the Floating Point Exponential Accelerator. + +Let’s segment the IEEE754 floating-point representation fraction part into several sub-fields (Index, Exp and Remaining bits) with respective length of Idxb, Expb and Remb bits. + +| IEEE754 precision | Idxb | Expb | Remb | +|-------------------------|------|------|------| +| Half precision (FP16) | 5 | 5 | 0 | +| Single precision (FP32) | 6 | 8 | 9 | +| Double precision (FP64) | 6 | 11 | 35 | + +The FEXPA instruction can be described for any real number x ∈ [2^(Remb + Expb) + 1; 2^(Remb + Expb) + 2^Expb - 1) as: + +$$FEXPA(x)=2^{x-constant}$$ + +where + +$$constant=2^{remBits + expBits} + bias$$ + +The instruction takes the floating-point value x as an input and, by copying some fraction bits y into the result’s exponent, which is then interpreted as 2^(y-bias), and by getting the correct fraction value from a hardware lookup table using the lower fraction bits, the result becomes 2^(x-constant). + +## Usage of lookup tables +Lookup tables can be combined with polynomial approximations. In this approach, the exponential function is reformulated as: + +$$e^x = e^{(m \times 2^L + j) \times ln2⁄2^L +r} = 2^m \times (2^{j⁄2^L} + 2^{j⁄2^L} \times p(r)) $$ + +where + +$$r∈[-ln2/2^{L+1}, +ln2/2^{L+1}], j \in [0, 2^L - 1]$$ + +and p(r) approximates e^r -1. + +If the 2^L possible values of 2^(j⁄2^L) are precomputed in table T, the exponential can be approximated as: +$$ e^x = 2^m \times T[j] \times (1 + p(r)) $$ + +With a table of size 2^L, the evaluation interval for the approximation polynomial is narrowed by a factor of 2^L. This reduction leads to improved accuracy for a given polynomial degree due to the narrower approximation range. Alternatively, for a given accuracy target, the degree of the polynomial—and hence its computational complexity—can be reduced. + +## Exponential implementation with FEXPA +FEXPA can be used to rapidly perform the table lookup. With this instruction a degree-2 polynomial is sufficient to obtain the same accuracy of the implementation we have seen before: + +```C +svfloat32_t lane_consts = svld1rq(pg, ln2_lo); // Load only ln2_lo + +/* Compute k as round(x/ln2) using shift = 1.5*2^(23-6) + 127 */ +svfloat32_t z = svmad_x(pg, svdup_f32(inv_ln2), x, shift); +svfloat32_t k = svsub_x(pg, z, shift); + +/* Compute r as x - k*ln2 with Cody and Waite */ +svfloat32_t r = svmsb_x(pg, svdup_f32(ln2_hi), k, x); + r = svmls_lane(r, k, lane_consts, 0); + +/* Compute the scaling factor 2^k */ +svfloat32_t scale = svexpa(svreinterpret_u32(z)); + +/* Compute poly(r) = exp(r) - 1 (2nd degree polynomial) */ +svfloat32_t p01 = svmla_x (pg, svdup_f32(c0), r, svdup_f32(c1)); // c0 + c1 * r +svfloat32_t poly = svmul_x (pg, r, p01); // r c0 + c1 * r^2 + +/* exp(x) = scale * exp(r) = scale * (1 + poly(r)) */ +svfloat32_t result = svmla_f32_x(pg, scale, poly, scale); +``` diff --git a/content/learning-paths/servers-and-cloud-computing/fexpa/implementation.md b/content/learning-paths/servers-and-cloud-computing/fexpa/implementation.md new file mode 100644 index 0000000000..016a4705b8 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fexpa/implementation.md @@ -0,0 +1,36 @@ +--- +title: First implementation +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## First implementation +Given what we said in the previous chapters, the exponential function can be implemented with SVE intrinsics in the following way: + +```C +svfloat32_t lane_consts = svld1rq(pg, constants); // Load ln2_lo, c0, c2, c4 in register + +/* Compute k as round(x/ln2) using shift = 1.5*2^23 + 127 */ +svfloat32_t z = svmad_x(pg, svdup_f32(inv_ln2), x, shift); +svfloat32_t k = svsub_x(pg, z, shift); + +/* Compute r as x - k*ln2 with Cody and Waite */ +svfloat32_t r = svmsb_x(pg, svdup_f32(ln2_hi), k, x); + r = svmls_lane(r, k, lane_consts, 0); + +/* Compute the scaling factor 2^k */ +svfloat32_t scale = svreinterpret_f32_u32(svlsl_n_u32_m(pg, svreinterpret_u32_f32(z), 23)); + +/* Compute poly(r) = exp(r) - 1 */ +svfloat32_t p12 = svmla_lane(svdup_f32(c1), r, lane_consts, 2); // c1 + c2 * r +svfloat32_t p34 = svmla_lane(svdup_f32(c3), r, lane_consts, 3); // c3 + c4 * r +svfloat32_t r2 = svmul_x(pg, r, r); +svfloat32_t p14 = svmla_x(pg, p12, p34, r2); // c1 + c2 * r + c3 * r^2 + c4 * r^3 +svfloat32_t p0 = svmul_lane(r, lane_consts, 1); // c0 * r +svfloat32_t poly = svmla_x(pg, p0, r2, p14); // c0 * r + c1 * r^2 + c2 * r^3 + c3 * r^4 + c4 * r^5 + +/* exp(x) = scale * exp(r) = scale * (1 + poly(r)) */ +svfloat32_t result = svmla_f32_x(pg, scale, poly, scale); +``` diff --git a/content/learning-paths/servers-and-cloud-computing/fexpa/theory.md b/content/learning-paths/servers-and-cloud-computing/fexpa/theory.md new file mode 100644 index 0000000000..935edc55b7 --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/fexpa/theory.md @@ -0,0 +1,62 @@ +--- +title: Theory +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## The exponential function +The exponential function is a fundamental mathematical function used across a wide range of algorithms for signal processing, High-Performance Computing and Machine Learning. Optimizing its computation has been the subject of extensive research for decades. The precision of the computation depends both on the selected approximation method and on the inherent rounding errors associated with finite-precision arithmetic, and it is directly traded off against performance when implementing the exponential function. + +## Range reduction +Polynomial approximations are among the most widely used methods for software implementations of the exponential function. The accuracy of a Taylor series approximation for exponential function can be improved with the polynomial’s degree but will always deteriorate as the evaluation point moves further from the expansion point. By applying range reduction techniques, the approximation of the exponential function can however be restricted to a very narrow interval where the function is well-conditioned. This approach consists in reformulating the exponential function in the following way: + +$$e^x=e^{k×ln2+r}=2^k \times e^r$$ + +where + +$$x=k×ln2+r, k \in Z, r \in [-ln2/2, +ln2/2]$$ + +Since k is an integer, the evaluation of 2^k can be efficiently performed using bit manipulation techniques, while e^r can be approximated with a polynomial p(r). Hence: + +$$e^x \approx 2^k \times p(r)$$ + +It is important to note that the polynomial p(r) is evaluated exclusively over the interval [-ln2/2, +ln2/2]. So, the computational complexity can be optimized by selecting the polynomial degree based on the required precision of p(r) within this narrow range. Rather than relying on a Taylor polynomial, a minimax polynomial approximation can be used to minimize the maximum approximation error over the considered interval. + +## Decomposition of the input +The decomposition of an input value as x = k × ln2 + r can be done in 2 steps: +- Compute k as: k = round(x⁄ln2), where round(.) is the round-to-nearest function +- Compute r as: r = x - k × ln2 + +Rounding of k is performed by adding an adequately chosen large number to a floating-point value and subtracting it just afterward (the original value is rounded due to the finite precision of floating-point representation). Although explicit rounding instructions are available in both SVE and SME, this method remains advantageous as the addition of the constant can be fused with the multiplication by the reciprocal of ln2. This approach assumes however that the floating-point rounding mode is set to round-to-nearest, which is the default mode in Armv9-A. By integrating the bias into the constant, 2^k can also be directly computed by shifting the intermediate value. + +Rounding error during the second step will introduce a global error as we will have: + +$$ x \approx k \times ln2 + r $$ + +To reduce the rounding errors during the computation of the reduced argument r, the Cody and Waite argument reduction technique is used. + +## Computation of the scaling factor +By leveraging the structure of floating-point number formats, it becomes relatively straightforward to compute 2^k for k∈Z. In the IEEE-754 standard, normalized floating-point numbers in binary interchange format are represented as: + +$$ (-1)^s \times 2^{(exponent - bias)} \times (1.fraction)_2 $$ + +where s is the sign bit and 1.fraction represents the significand. + +The value 2^k can be encoded by setting both the sign and fraction bits to zero and assigning the exponent field the value k + bias. If k is an 8-bits integer, 2^k can be efficiently computed by adding the bias value and positioning the result into the exponent bits of a 32-bit floating-point number using a logical shift. + +Taking this approach a step further, a fast approximation of exponential function can be achieved using bits manipulation techniques alone. Specifically, adding a bias to an integer k and shifting the result into the exponent field can be accomplished by computing an integer i as follows: + +$$i=2^{23} \times (k+bias) = 2^{23} \times k+2^{23} \times bias$$ + +This formulation assumes a 23-bit significand, but the method can be generalized to other floating-point precisions. + +Now, consider the case where k is a real number. The fractional part of k will propagate into the significand bits of the resulting 2^k approximation. However, this side effect is not detrimental, it effectively acts as a form of linear interpolation, thereby improving the overall accuracy of the approximation. To approximate the exponential function, the following identity can be used: + +$$e^x = 2^{x⁄ln2}$$ + +As previously discussed, this value can be approximated by computing a 32-bit integer: + +$$i = 2^{23} \times x⁄ln2 + 2^{23} \times bias = a \times x + b $$ + diff --git a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/_index.md b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/_index.md index 34091b4fec..4c59b0226e 100644 --- a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/_index.md @@ -1,19 +1,15 @@ --- title: Deploy Apache Flink on Google Cloud C4A (Arm-based Axion VMs) - -draft: true -cascade: - draft: true minutes_to_complete: 30 -who_is_this_for: This learning path is intended for software developers deploying and optimizing Apache Flink workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. +who_is_this_for: This is an introductory topic for developers deploying and optimizing Apache Flink workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. learning_objectives: - Provision an Arm-based SUSE SLES virtual machine on Google Cloud (C4A with Axion processors) - - Install Apache Flink on a SUSE Arm64 (C4A) instance - - Validate Flink functionality by starting the Flink cluster and running a simple baseline job (e.g., WordCount) on the Arm64 VM - - Benchmark Flink performance using internal JMH-based micro-benchmarks on Arm64 (Aarch64) architecture + - Install and configure Apache Flink on an Arm64 instance + - Validate Flink functionality by starting the cluster and running a baseline job + - Benchmark Flink performance using JMH-based microbenchmarks prerequisites: - A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled @@ -23,7 +19,7 @@ author: Pareena Verma ##### Tags skilllevels: Introductory -subjects: Databases +subjects: Performance and Architecture cloud_service_providers: Google Cloud armips: @@ -48,7 +44,7 @@ further_reading: - resource: title: Flink documentation - link: https://nightlies.apache.org/flink/flink-docs-lts/ + link: https://nightlies.apache.org/flink/flink-docs-lts/ type: documentation - resource: diff --git a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/background.md b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/background.md index 4b0047cb04..3170d0108b 100644 --- a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/background.md +++ b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/background.md @@ -1,22 +1,22 @@ --- -title: Getting started with Apache Flink on Google Axion C4A (Arm Neoverse-V2) +title: Get started with Apache Flink on Google Axion C4A weight: 2 layout: "learningpathall" --- -## Google Axion C4A Arm instances in Google Cloud +## Explore Google Axion C4A Arm instances Google Axion C4A is a family of Arm-based virtual machines built on Google’s custom Axion CPU, which is based on Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines offer strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. -The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. +The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of Arm architecture in Google Cloud. -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. +To learn more about Google Axion, see the Google blog [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu). -## Apache Flink +## Explore Apache Flink -[Apache Flink](https://flink.apache.org/) is an open-source, distributed stream and batch data processing framework** developed under the [Apache Software Foundation](https://www.apache.org/). +[Apache Flink](https://flink.apache.org/) is an open-source, distributed stream and batch data processing framework developed under the [Apache Software Foundation](https://www.apache.org/). Flink is designed for high-performance, low-latency, and stateful computations on both unbounded (streaming) and bounded (batch) data. It provides a robust runtime and APIs in Java, Scala, and Python for building scalable, fault-tolerant data processing pipelines. diff --git a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/baseline.md b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/baseline.md index 920f3df74a..0455f883c3 100644 --- a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/baseline.md @@ -1,5 +1,5 @@ --- -title: Apache Flink Baseline Testing on Google Axion C4A Arm Virtual Machine +title: Test Flink baseline functionality weight: 5 ### FIXED, DO NOT MODIFY @@ -9,10 +9,10 @@ layout: learningpathall ## Apache Flink Baseline Testing on GCP SUSE VM In this section you will perform baseline testing for Apache Flink after installation on a GCP SUSE VM. Baseline testing validates that your installation is correct, the JVM is functioning properly, and Flink’s JobManager/TaskManager can execute jobs successfully. -### Install Maven (Required to Build and Run Flink Jobs) +## Install Maven (Required to Build and Run Flink Jobs) Before running Flink jobs, ensure that Maven is installed on your VM. Many Flink examples and real-world jobs require Apache Maven to compile Java applications. -Download Maven and extract it: +## Install Maven ```console cd /opt @@ -21,7 +21,7 @@ sudo tar -xvzf apache-maven-3.8.6-bin.tar.gz sudo mv apache-maven-3.8.6 /opt/maven ``` -### Configure Environment Variables +## Configure environment variables Configure the environment so Maven commands can be run system-wide: ```console @@ -29,13 +29,14 @@ echo "export M2_HOME=/opt/maven" >> ~/.bashrc echo "export PATH=\$M2_HOME/bin:\$PATH" >> ~/.bashrc source ~/.bashrc ``` + Verify the Maven installation: ```console mvn -version ``` -The output should look like: +The output is similar to: ```output pache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63) @@ -45,19 +46,17 @@ Default locale: en, platform encoding: UTF-8 OS name: "linux", version: "5.14.21-150500.55.124-default", arch: "aarch64", family: "unix" ``` -At this point, both Java and Maven are installed and ready to use. +## Start the Flink cluster -### Start the Flink Cluster Before launching Flink, open port 8081 in the Google Cloud Firewall Rules so that the Web UI is reachable externally. -Start the standalone Flink cluster using the provided startup script: - ```console cd $FLINK_HOME ./bin/start-cluster.sh ``` -You should see output similar to: +The output is similar to: + ```output Starting cluster. [INFO] 1 instance(s) of standalonesession are already running on lpprojectsusearm64. @@ -65,42 +64,42 @@ Starting standalonesession daemon on host lpprojectsusearm64. Starting taskexecutor daemon on host lpprojectsusearm64. ``` -Verify that the Flink Processes (JobManager and TaskManager) are running: +Verify that the Flink processes (JobManager and TaskManager) are running: ```console jps ``` -You should see output similar to: +The output is similar to: + ```output 21723 StandaloneSessionClusterEntrypoint 2621 Jps 2559 TaskManagerRunner ``` -StandaloneSessionClusterEntrypoint is the JobManager process -TaskManagerRunner is the worker responsible for executing tasks and maintaining state. -### Access the Flink Web UI +`StandaloneSessionClusterEntrypoint` is the JobManager process, and `TaskManagerRunner` is the worker responsible for executing tasks and maintaining state. -In a browser, navigate to: +## Access the Flink Web UI + +In a browser, navigate to `http://:8081`. -```console -http://:8081 -``` You should see the Flink Dashboard: -![Flink Dashboard alt-text#center](images/flink-dashboard.png "Figure 1: Flink Dashboard") + +![Screenshot of the Apache Flink Dashboard web interface showing the Overview page with cluster status, available task slots, running jobs count, and system metrics displayed in a clean web UI alt-text#center](images/flink-dashboard.png "Flink Dashboard") A successfully loaded dashboard confirms the cluster network and UI functionality. This serves as the baseline for network and UI validation. -### Run a Simple Example Job -A basic sanity test is to run the built-in WordCount example: +## Run a simple example job +A basic check is to run the built-in WordCount example: ```console cd $FLINK_HOME ./bin/flink run examples/streaming/WordCount.jar ``` -You can monitor the job in the Web UI or check console logs. A successful WordCount run confirms that your Flink cluster lifecycle works end-to-end. -![Flink Dashboard alt-text#center](images/wordcount.png "Figure 2: Word Count Job") +You can monitor the job in the Web UI or check console logs. + +![Screenshot of the Flink Dashboard showing a completed WordCount job with execution details, task metrics, and job timeline visible in the web interface alt-text#center](images/wordcount.png "WordCount job in Flink Dashboard") -Flink baseline testing has been completed. You can now proceed to Flink benchmarking. +Flink baseline testing is complete. You can now proceed to Flink benchmarking. diff --git a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/benchmarking.md index f31440563c..9c848b4dd0 100644 --- a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/benchmarking.md @@ -1,16 +1,16 @@ --- -title: Apache Flink Benchmarking +title: Benchmark Flink performance weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Benchmark Apache Flink performance -## Apache Flink Benchmarking -This section walks you through running Apache Flink microbenchmarks on a Google Cloud Axion C4A (Arm64) SUSE VM. You will clone the official Flink-benchmarks repository, build the benchmark suite, explore available tests, and run the Remote Channel Throughput Benchmark, one of the key indicators of Flink’s communication and data-transfer performance. +This section walks you through running Apache Flink microbenchmarks on a Google Cloud Axion C4A (Arm64) SUSE VM. You will clone the official Flink-benchmarks repository, build the benchmark suite, and run the Remote Channel Throughput Benchmark. -### Clone the Repository +## Clone the repository Clone the official Flink microbenchmarks repository: ```console @@ -20,39 +20,41 @@ cd flink-benchmarks ``` This repository contains microbenchmarks built using JMH (Java Microbenchmark Harness), widely used for JVM-level performance testing. -### Build the Benchmarks with Maven +## Build the benchmarks with Maven Compile the benchmarks and create the executable JAR: ```console mvn clean package -DskipTests ``` What this does: - * clean removes previous build artifacts - * package compiles the code and produces benchmark JARs + * `clean` removes previous build artifacts + * `package` compiles the code and produces benchmark JARs * -DskipTests speeds up the build since unit tests aren’t needed for benchmarking -After building, the compiled **benchmarks.jar** files appear under: +After building, the compiled benchmarks.jar files appear under: ```output flink-benchmarks/target/ ``` -### Explore the JAR Contents +## Explore the JAR contents Verify that benchmarks.jar was generated: ```console cd target ls ``` -You should see: + +The output is similar to: ```output benchmark-0.1.jar classes generated-test-sources maven-status protoc-plugins benchmarks.jar generated-sources maven-archiver protoc-dependencies test-classes ``` -benchmarks.jar — Contains all Flink microbenchmarks packaged with JMH. -### List Available Benchmarks +`benchmarks.jar` contains all Flink microbenchmarks packaged with JMH. + +## List available benchmarks View all benchmarks included in the JAR: ```console @@ -61,16 +63,18 @@ java -jar benchmarks.jar -l - `-l` → Lists all benchmarks packaged in the JAR. - This helps you identify which benchmarks you want to execute on your VM. -### Run Selected Benchmarks +## Run the Remote Channel Throughput benchmark While the Flink benchmarking project includes multiple suites for state backends, windowing, checkpointing, and scheduler performance, in this Learning path you will run the Remote Channel Throughput benchmark to evaluate network and I/O performance. -**Remote Channel Throughput**: This benchmark measures the data transfer rate between remote channels in Flink, helping to evaluate network and I/O performance. +Remote Channel Throughput: This benchmark measures the data transfer rate between remote channels in Flink, helping to evaluate network and I/O performance. Run the benchmark: ```console java -jar benchmarks.jar org.apache.flink.benchmark.RemoteChannelThroughputBenchmark.remoteRebalance ``` -You should see output similar to: + +The output is similar to: + ```output Result "org.apache.flink.benchmark.RemoteChannelThroughputBenchmark.remoteRebalance": @@ -84,16 +88,18 @@ RemoteChannelThroughputBenchmark.remoteRebalance ALIGNED thrpt 30 17445.341 RemoteChannelThroughputBenchmark.remoteRebalance DEBLOAT thrpt 30 10536.511 ± 60.121 ops/ms ``` -### Flink Benchmark Metrics Explained +## Understand benchmark metrics + +The output from the Remote Channel Throughput benchmark includes several statistical measures that help you interpret Flink's performance on Arm64. Understanding these metrics enables you to assess both the average performance and the consistency of results across multiple iterations. -- **Run Count**: Total benchmark iterations executed, higher count improves reliability. -- **Average Throughput**: Mean operations per second across all iterations. -- **Standard Deviation**: Variation from average throughput, smaller means more consistent. -- **Confidence Interval (99.9%)**: Range where the true average throughput lies with 99.9% certainty. -- **Min Throughput**: The lowest throughput was observed, and it shows worst-case performance. -- **Max Throughput**: Highest throughput observed, shows best-case performance. +- Run Count: total benchmark iterations executed, higher count improves reliability. +- Average Throughput: mean operations per second across all iterations. +- Standard Deviation: variation from average throughput, smaller means more consistent. +- Confidence Interval (99.9%): range where the true average throughput lies with 99.9% certainty. +- Min Throughput: the lowest throughput was observed, and it shows worst-case performance. +- Max Throughput: highest throughput observed, shows best-case performance. -### Benchmark summary on Arm64 +## Benchmark summary for Arm64 Results from the run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE) are summarized below: | Benchmark | Mode | Count | Score (ops/ms) | Error (±) | Min | Max | Stdev | CI (99.9%) | Units | @@ -101,9 +107,9 @@ Results from the run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in | RemoteChannelThroughputBenchmark.remoteRebalance | ALIGNED | 30 | 17445.341 | 153.256 | 10289.593 | 10687.736 | 89.987 | [10476.390, 10596.633] | ops/ms | | RemoteChannelThroughputBenchmark.remoteRebalance | DEBLOAT | 30 | 10536.511 | 60.121 | 10289.593 | 10687.736 | 89.987 | [10476.390, 10596.633] | ops/ms | -### Apache Flink performance benchmarking observations on Arm64 +## What you've accomplished and what's next -- Both the ALIGNED mode and DEBLOAT modes demonstrate a strong throughput on the Arm64 VM. -- The benchmark confirms that the Arm64 architecture efficiently handles Flink's remote channel throughput workloads. +You've successfully deployed Apache Flink on a Google Axion C4A Arm-based virtual machine, validated its functionality, and measured its performance using JMH-based microbenchmarks. +The benchmark results confirm that Google Axion C4A Arm-based instances deliver strong throughput in both ALIGNED and DEBLOAT modes, demonstrating that Arm64 architecture efficiently handles Flink's remote channel throughput workloads. -Overall, Arm64 VMs have shown that they are highly suitable for real-time Flink workloads, especially streaming analytics, ETL pipelines, and JVM-based microbenchmarks. +Arm64 VMs are highly suitable for real-time Flink workloads, including streaming analytics, ETL pipelines, and JVM-based microbenchmarks. The consistent performance across different modes demonstrates that Arm-based infrastructure can effectively handle Flink's demanding real-time processing requirements. diff --git a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/installation.md b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/installation.md index 56aadff047..81ee6fb45c 100644 --- a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/installation.md @@ -6,12 +6,12 @@ weight: 4 layout: learningpathall --- -## Install Apache Flink on GCP VM -In this section you will install Apache Flink and its required dependencies on a Google Cloud Platform (GCP) SUSE Arm64 Virtual Machine (VM). By the end, you will have a fully configured Flink environment ready for local execution, standalone clusters, or benchmarking Flink workloads on Arm. +## Install Apache Flink + +This section shows you how to install Apache Flink and its required dependencies on a Google Cloud Platform (GCP) SUSE Arm64 virtual machine. + +## Update the system and install Java -### Update the System and Install Java -Apache Flink requires a Java runtime (JRE) and development kit (JDK). -Update the system and install Java: ```console sudo zypper refresh sudo zypper update -y @@ -19,7 +19,7 @@ sudo zypper install -y java-17-openjdk java-17-openjdk-devel ``` This step ensures you have the latest system updates and the Java runtime needed to execute Flink applications. -### Download Apache Flink Binary +## Download Apache Flink Binary Navigate to /opt (a standard location for system-wide tools) and download the official Flink distribution: ```console @@ -29,13 +29,12 @@ sudo wget https://dlcdn.apache.org/flink/flink-2.1.1/flink-2.1.1-bin-scala_2.12. This command retrieves the official Flink binary distribution for installation on your VM. {{% notice Note %}} -Flink 2.0.0 introduced Disaggregated State Management architecture, which enables more efficient resource utilization in cloud-native environments, ensuring high-performance real-time processing while minimizing resource overhead. -You can view [this release note](https://flink.apache.org/2025/03/24/apache-flink-2.0.0-a-new-era-of-real-time-data-processing/) +Flink 2.0.0 introduced Disaggregated State Management architecture, which enables more efficient resource utilization in cloud-native environments, ensuring high-performance real-time processing while minimizing resource overhead. See [the Flink 2.0.0 Release Note](https://flink.apache.org/2025/03/24/apache-flink-2.0.0-a-new-era-of-real-time-data-processing/). For best performance on Arm, the [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends using Flink ≥ 2.0.0. {{% /notice %}} -### Extract the Downloaded Archive +## Extract the archive Untar the archive: ```console @@ -49,14 +48,16 @@ sudo mv flink-2.1.1 /opt/flink ``` This makes configuration, upgrades, and scripting easier for your Flink installation. -### Configure Environment Variables +## Configure environment variables Add Flink to your shell environment: ```console echo "export FLINK_HOME=/opt/flink" >> ~/.bashrc echo "export PATH=\$FLINK_HOME/bin:\$PATH" >> ~/.bashrc ``` + Create a logging directory and assign proper permissions: + ```console sudo mkdir -p /opt/flink/log sudo chown -R $(whoami):$(id -gn) /opt/flink/log @@ -69,7 +70,7 @@ source ~/.bashrc ``` Adding Flink to the global PATH lets you use commands like `flink`, `start-cluster.sh`, and `taskmanager.sh` from any terminal. -### Verify the Installation +## Verify the installation To confirm that Flink has been installed correctly, check its version: ```console @@ -81,4 +82,5 @@ You should see an output similar to: ```output Version: 2.1.1, Commit ID: 074f8c5 ``` -This confirms that Apache Flink has been installed and is ready for use. + +Apache Flink is now installed and ready for use. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/instance.md index 4ec8bd98a2..b4d6651083 100644 --- a/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/flink-on-gcp/instance.md @@ -1,5 +1,5 @@ --- -title: Create a Google Axion C4A Arm virtual machine on GCP +title: Create a Google Axion C4A Arm virtual machine weight: 3 ### FIXED, DO NOT MODIFY @@ -8,13 +8,13 @@ layout: learningpathall ## Overview -In this section, you will create a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. +This section shows you how to create a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} -## Provision a Google Axion C4A Arm VM in Google Cloud Console +## Provision a Google Axion C4A Arm VM To create a virtual machine based on the C4A instance type: - Navigate to the [Google Cloud Console](https://console.cloud.google.com/). @@ -24,8 +24,8 @@ To create a virtual machine based on the C4A instance type: - Set **Series** to `C4A`. - Select `c4a-standard-4` for machine type. - ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") + ![Screenshot of the Google Cloud Console showing the Compute Engine VM instance creation page with Machine configuration section expanded. The Series dropdown shows C4A selected, and the Machine type field displays c4a-standard-4 with specifications of 4 vCPUs and 16 GB memory visible alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") -- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. Pick the preferred version for your Operating System. Ensure you select the **Arm image** variant. Click **Select**. +- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. Pick the preferred version for your Operating System. Ensure you select the **Arm image** variant. Select **Select**. - Under **Networking**, enable **Allow HTTP traffic**. -- Click **Create** to launch the instance. +- Select **Create** to launch the instance. diff --git a/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part1/_index.md b/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part1/_index.md index 582e240ce3..8c0541c90a 100644 --- a/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part1/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part1/_index.md @@ -18,7 +18,7 @@ prerequisites: - 'Visual Studio Code: https://code.visualstudio.com/download' - 'Docker Extension for Visual Studio Code: https://code.visualstudio.com/docs/containers/overview' - 'C# Extension for Visual Studio Code: https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.csharp' - - 'Install Docker on Arm64: https://learn.arm.com/install-guides/docker/docker-woa/' + - '[Install Docker on Arm64](/install-guides/docker/docker-desktop/)' author: Dawid Borycki @@ -41,7 +41,7 @@ operatingsystems: further_reading: - resource: title: Terraform on Azure - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/azure-terraform/ + link: /learning-paths/servers-and-cloud-computing/azure-terraform/ type: Tutorial - resource: title: Azure Virtual Machines with Ampere Altra Arm–based processors—generally available diff --git a/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part3/how-to-2.md b/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part3/how-to-2.md index 84c98dd14f..fe86eea69f 100644 --- a/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part3/how-to-2.md +++ b/content/learning-paths/servers-and-cloud-computing/from-iot-to-the-cloud-part3/how-to-2.md @@ -7,7 +7,7 @@ layout: learningpathall --- ## Objective -You will create the managed Kubernetes cluster in Azure, which is integrated with the Azure Container Registry. The cluster will use arm64 VMs as the nodes. This tutorial shows how to manually create the cluster. Optionally you can use Terraform-based deployment as described here: https://learn.arm.com/learning-paths/servers-and-cloud-computing/aks/cluster_deployment/. +You will create the managed Kubernetes cluster in Azure, which is integrated with the Azure Container Registry. The cluster will use arm64 VMs as the nodes. This tutorial shows how to manually create the cluster. Optionally you can use [Terraform-based deployment](/learning-paths/servers-and-cloud-computing/aks/cluster_deployment/). ## Create the Kubernetes cluster Login to the Azure Portal and type **Kubernetes** in the search box. Then, look for the Kubernetes services: diff --git a/content/learning-paths/servers-and-cloud-computing/funasr/3_funasr.md b/content/learning-paths/servers-and-cloud-computing/funasr/3_funasr.md index d30d9c5aa7..6f02daeb70 100644 --- a/content/learning-paths/servers-and-cloud-computing/funasr/3_funasr.md +++ b/content/learning-paths/servers-and-cloud-computing/funasr/3_funasr.md @@ -539,7 +539,7 @@ pip uninstall torchao && cd ao/ && rm -rf build && python setup.py install ``` {{% notice Note %}} -See this Learning Path [Run a Large Language Model Chatbot on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama/) for further information. +See this Learning Path [Run a Large Language Model Chatbot on Arm servers](/learning-paths/servers-and-cloud-computing/pytorch-llama/pytorch-llama/) for further information. {{% /notice %}} Once you have installed the optimized PyTorch, enable bfloat16 fast math kernels by setting DNNL_DEFAULT_FPMATH_MODE. diff --git a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/_index.md b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/_index.md index fc1f9a0288..3bf3aa1254 100644 --- a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/_index.md @@ -1,18 +1,14 @@ --- title: Deploy Gardener on Google Cloud C4A (Arm-based Axion VMs) -draft: true -cascade: - draft: true - minutes_to_complete: 50 -who_is_this_for: This learning path is intended for software developers deploying and optimizing Gardener workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. +who_is_this_for: This is an introductory topic for software developers deploying and optimizing Gardener workloads on Linux Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. learning_objectives: - - Provision an Arm-based SUSE SLES virtual machine on Google Cloud (C4A with Axion processors) + - Provision an Arm-based SUSE Linux Enterprise Server (SLES) virtual machine on Google Cloud (C4A with Axion processors) - Install and configure Gardener on a SUSE Arm64 (C4A) instance - - Deploy Garden, Seed, and Shoot clusters locally using KinD + - Deploy Garden, Seed, and Shoot clusters locally using Kubernetes in Docker (KinD) - Validate Gardener functionality by deploying workloads into a Shoot cluster - Perform baseline security benchmarking of Gardener-managed Kubernetes clusters using kube-bench on Arm64 @@ -62,7 +58,7 @@ further_reading: type: documentation - resource: - title: kube-bench documentation + title: kube-bench security benchmarking tool link: https://github.com/aquasecurity/kube-bench type: documentation diff --git a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/background.md b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/background.md index 83255edbf4..6bd1761517 100644 --- a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/background.md +++ b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/background.md @@ -1,25 +1,35 @@ --- -title: Getting started with Gardener on Google Axion C4A (Arm Neoverse-V2) +title: Get started with Gardener on Google Axion C4A (Arm Neoverse-V2) weight: 2 layout: "learningpathall" --- -## Google Axion C4A Arm instances in Google Cloud +## Explore Google Axion C4A Arm instances -Google Axion C4A is a family of Arm-based virtual machines built on Google’s custom Axion CPU, which is based on Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines offer strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. +Google Axion C4A is a family of Arm-based virtual machines built on Google's custom Axion CPU, which uses Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines deliver strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. +To learn more about Google Axion, see the Google blog [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu). -## Gardener +## Explore Gardener -Gardener is an open-source, Kubernetes-native system for managing and operating Kubernetes clusters at scale. It enables automated creation, update, healing, and deletion of clusters across multiple cloud and on-prem providers. +Gardener is an open-source, Kubernetes-native system for managing and operating Kubernetes clusters at scale. It enables automated creation, updates, healing, and deletion of clusters across multiple cloud providers and on-premises environments. -Gardener uses Kubernetes APIs and CRDs to declaratively manage clusters in a cloud-agnostic way. It follows a **Garden–Seed–Shoot** architecture to separate control planes from workload clusters. +Gardener uses Kubernetes APIs and Custom Resource Definitions (CRDs) to declaratively manage clusters in a cloud-agnostic way. It follows a Garden–Seed–Shoot architecture to separate control planes from workload clusters: -Gardener is widely used to build reliable internal developer platforms and operate thousands of Kubernetes clusters. +- Garden is the central management cluster that runs Gardener components. +- Seed clusters are intermediary clusters that host Shoot control planes. +- Shoot clusters are the workload clusters managed by Gardener for end users. -To learn more, visit the Gardener [official website](https://gardener.cloud/) and explore the [documentation](https://gardener.cloud/docs/). +Organizations use Gardener to build and operate Kubernetes clusters at scale, creating reliable internal developer platforms that serve thousands of teams. + +To learn more, visit the [Gardener website](https://gardener.cloud/) and explore the [Gardener documentation](https://gardener.cloud/docs/). + +## Summary and what's next + +You now understand Google Axion C4A's capabilities as a cost-effective Arm-based platform and how Gardener automates Kubernetes cluster management at scale. Together, these technologies provide a powerful foundation for running enterprise Kubernetes workloads on Arm infrastructure. + +In the next sections, you'll provision your Arm-based VM on Google Cloud, install Gardener, and deploy your first clusters. You're ready to get started! diff --git a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/baseline.md b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/baseline.md index f02c532a9f..02be2cf145 100644 --- a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/baseline.md @@ -1,135 +1,54 @@ --- -title: Gardener Baseline Testing on Google Axion C4A Arm Virtual Machine +title: Verify Gardener cluster health and functionality weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Gardener Baseline Testing on GCP SUSE VMs -This section checks whether your Gardener Local setup is working correctly on an Arm-based GCP Axion (C4A) VM before running real workloads. +## Verify your Gardener cluster is working -### Set Kubeconfig -This tells Kubernetes commands (**kubectl) which cluster to talk to**. Without this, kubectl won’t know where your Gardener cluster is. -``` console +This section confirms that your Gardener Local setup is functioning correctly on an Arm-based Google Cloud C4A VM before running production workloads. You'll check cluster health, deploy test workloads, and validate networking. + +## Set the kubeconfig environment variable + +Configure kubectl to communicate with your Gardener cluster by setting the `KUBECONFIG` variable: + +```console export KUBECONFIG=$PWD/example/gardener-local/kind/local/kubeconfig ``` -### Check Cluster Health -Before testing any workload, verify that the Gardener-local Kubernetes cluster is healthy. This ensures the control plane and node are functional. +This tells kubectl where to find your cluster's authentication credentials. -``` console +## Check cluster health + +Verify that your Gardener Local Kubernetes cluster is healthy by checking node and pod status: + +```console kubectl get nodes -o wide kubectl get pods -A ``` -You should see an output similar to: -```output -NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME -gardener-local-control-plane Ready control-plane 148m v1.32.5 172.18.0.2 Debian GNU/Linux 12 (bookworm) 5.14.21-150500.55.124-default containerd://2.1.1 -extension-networking-calico-8z7jw gardener-extension-networking-calico-94bcb44bf-kmmpj 1/1 Running 0 102m -extension-networking-calico-8z7jw gardener-extension-networking-calico-94bcb44bf-whgtn 1/1 Running 0 135m -extension-provider-local-m7d79 gardener-extension-provider-local-fc75c4494-47szg 1/1 Running 0 137m -extension-provider-local-m7d79 gardener-extension-provider-local-fc75c4494-hkksz 1/1 Running 0 137m -garden dependency-watchdog-prober-d47b5899f-ml6x9 1/1 Running 0 61m -garden dependency-watchdog-prober-d47b5899f-xmzh2 1/1 Running 0 60m -garden dependency-watchdog-weeder-66f8bffd8b-lgx7f 1/1 Running 0 60m -garden dependency-watchdog-weeder-66f8bffd8b-vd9md 1/1 Running 0 61m -garden etcd-0 1/1 Running 0 141m -garden etcd-druid-65d56db866-bstcm 1/1 Running 0 139m -garden etcd-druid-65d56db866-zkfjb 1/1 Running 0 139m -garden fluent-bit-8259c-s5wnv 1/1 Running 0 139m -garden fluent-operator-5b9ff5bfb7-6ffvc 1/1 Running 0 137m -garden fluent-operator-5b9ff5bfb7-cw67l 1/1 Running 0 137m -garden gardener-admission-controller-899c585bf-2mp9g 1/1 Running 2 (141m ago) 141m -garden gardener-admission-controller-899c585bf-xp2f4 1/1 Running 2 (141m ago) 141m -garden gardener-apiserver-54fcdfcd97-5zkgr 1/1 Running 0 141m -garden gardener-controller-manager-77bf4b686f-zxgsh 1/1 Running 3 (140m ago) 141m -garden gardener-extension-admission-local-57d674d98f-6qbcv 1/1 Running 0 136m -garden gardener-extension-admission-local-57d674d98f-zlgpd 1/1 Running 0 135m -garden gardener-resource-manager-cfd685fc5-n9mp7 1/1 Running 0 133m -garden gardener-resource-manager-cfd685fc5-spbn7 1/1 Running 0 134m -garden gardener-scheduler-6599d654c9-vw2q5 1/1 Running 0 141m -garden gardenlet-59cb4b6956-hsmdp 1/1 Running 0 96m -garden kube-state-metrics-seed-f89d48b49-94l46 1/1 Running 0 121m -garden kube-state-metrics-seed-f89d48b49-q95kr 1/1 Running 0 130m -garden nginx-ingress-controller-5bb9b58c44-ck2q7 1/1 Running 0 139m -garden nginx-ingress-controller-5bb9b58c44-r8wwd 1/1 Running 0 139m -garden nginx-ingress-k8s-backend-5547dddffd-fqsfm 1/1 Running 0 139m -garden perses-operator-9f9694dcd-wvl5z 1/1 Running 0 139m -garden plutono-776964667b-225r7 2/2 Running 0 139m -garden prometheus-aggregate-0 2/2 Running 0 87m -garden prometheus-cache-0 2/2 Running 0 22m -garden prometheus-operator-8447dc86f9-6mb25 1/1 Running 0 139m -garden prometheus-seed-0 2/2 Running 0 87m -garden vali-0 2/2 Running 0 139m -garden vpa-admission-controller-76b4c99684-lkf27 1/1 Running 0 30m -garden vpa-admission-controller-76b4c99684-tkg7n 1/1 Running 0 81m -garden vpa-recommender-5b668455db-fctrs 1/1 Running 0 139m -garden vpa-recommender-5b668455db-sdpv6 1/1 Running 0 139m -garden vpa-updater-7dd7dccc6d-dgg7r 1/1 Running 0 131m -garden vpa-updater-7dd7dccc6d-whlqx 1/1 Running 0 133m -gardener-extension-provider-local-coredns coredns-69d964db7f-mrmq9 1/1 Running 0 139m -istio-ingress istio-ingressgateway-5b48596bf9-4pzsw 1/1 Running 0 139m -istio-ingress istio-ingressgateway-5b48596bf9-ff4zp 1/1 Running 0 139m -istio-system istiod-769565bbdb-2hnzz 1/1 Running 0 76m -istio-system istiod-769565bbdb-wlbts 1/1 Running 0 77m -kube-system calico-kube-controllers-bfc8cf74c-pj9hh 1/1 Running 0 148m -kube-system calico-node-88sdt 1/1 Running 0 148m -kube-system coredns-54bf7d48d5-j6zbg 1/1 Running 0 148m -kube-system coredns-54bf7d48d5-zrqqc 1/1 Running 0 148m -kube-system etcd-gardener-local-control-plane 1/1 Running 0 148m -kube-system kube-apiserver-gardener-local-control-plane 1/1 Running 0 148m -kube-system kube-controller-manager-gardener-local-control-plane 1/1 Running 0 148m -kube-system kube-proxy-fxxzc 1/1 Running 0 148m -kube-system kube-scheduler-gardener-local-control-plane 1/1 Running 0 148m -kube-system metrics-server-78b7d676c8-cjwrs 1/1 Running 0 148m -local-path-storage local-path-provisioner-7dc846544d-m825q 1/1 Running 0 148m -registry registry-c85bbb98c-lqtcj 1/1 Running 0 148m -registry registry-europe-docker-pkg-dev-7956694cfb-hbg69 1/1 Running 0 148m -registry registry-gcr-6d4b454594-b9plv 1/1 Running 0 148m -registry registry-k8s-5bf5795799-t44xd 1/1 Running 0 148m -registry registry-quay-84dbcd78b4-dw2pn 1/1 Running 0 148m -shoot--local--local blackbox-exporter-58c4f64c97-l96ct 1/1 Running 0 104m -shoot--local--local blackbox-exporter-58c4f64c97-nlhjj 1/1 Running 0 105m -shoot--local--local cluster-autoscaler-b894888d6-qwrpp 1/1 Running 0 116m -shoot--local--local etcd-events-0 2/2 Running 0 136m -shoot--local--local etcd-main-0 2/2 Running 0 136m -shoot--local--local event-logger-777b7b7c7c-77h9n 1/1 Running 0 133m -shoot--local--local gardener-resource-manager-764b5d4f97-bdd8n 1/1 Running 0 118m -shoot--local--local gardener-resource-manager-764b5d4f97-z48b5 1/1 Running 0 129m -shoot--local--local kube-apiserver-6545887cc9-26h5w 1/1 Running 0 124m -shoot--local--local kube-apiserver-6545887cc9-gf92k 1/1 Running 0 98m -shoot--local--local kube-controller-manager-555b598dbf-45n8v 1/1 Running 0 122m -shoot--local--local kube-scheduler-695d49b6c5-xr7hp 1/1 Running 0 125m -shoot--local--local kube-state-metrics-76cc7bb4f9-xq4g2 1/1 Running 0 130m -shoot--local--local machine-controller-manager-775dc6d574-mntqt 2/2 Running 0 111m -shoot--local--local machine-shoot--local--local-local-68499-nhvjl 1/1 Running 0 131m -shoot--local--local plutono-869d676bb9-jjwcx 2/2 Running 0 133m -shoot--local--local prometheus-shoot-0 2/2 Running 0 95m -shoot--local--local vali-0 4/4 Running 0 133m -shoot--local--local vpa-admission-controller-bcc4c968c-8ndg8 1/1 Running 0 133m -shoot--local--local vpa-admission-controller-bcc4c968c-r6lnt 1/1 Running 0 72m -shoot--local--local vpa-recommender-b49f4dd7c-mk9sx 1/1 Running 0 107m -shoot--local--local vpa-updater-6cc999b5bc-jcrbg 1/1 Running 0 123m -shoot--local--local vpn-seed-server-7497c89db-b5p5c 2/2 Running 0 15m -``` - -### Deploy a Test Nginx Pod -This step deploys a simple web server (nginx) to confirm that workloads can run. -- Creates one nginx pod -- Confirms Kubernetes can pull images and start containers - -When the pod status becomes Running, workload deployment works. +The output is similar to: -``` console +```outputNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE +gardener-local-control-plane Ready control-plane 148m v1.32.5 172.18.0.2 Debian GNU/Linux 12 (bookworm) +``` + +A `Ready` status indicates the control plane is healthy. You should also see numerous pods running across the `garden`, `kube-system`, `istio-system`, and `shoot--local--local` namespaces, confirming that all Gardener components are operational. + +## Deploy a test nginx pod + +Deploy a simple nginx pod to verify that workload deployment works correctly: + +```console kubectl run test-nginx --image=nginx --restart=Never kubectl get pod test-nginx -w ``` - `kubectl run test-nginx` → Creates a single nginx pod. - `kubectl get pod test-nginx -w` → Watches pod status in real time. -You should see an output similar to: +The output is similar to: ```output >pod/test-nginx created @@ -140,24 +59,20 @@ test-nginx 0/1 ContainerCreating 0 1s test-nginx 1/1 Running 0 4s ``` -Now, press "ctrl-c" in the ssh shell to kill the currently running monitor. +When the pod reaches `Running` status, workload deployment is functioning. Press **Ctrl + C** to stop watching the pod. -### Expose the Pod (ClusterIP Service) -Pods cannot be accessed directly by other pods reliably. -So we create a Kubernetes Service. -- The service gives nginx a stable internal IP -- It allows other pods to reach nginx using a name +## Create a service for the pod -This confirms Kubernetes service networking is working. +Kubernetes services provide stable network endpoints for pods. Create a ClusterIP service to expose your nginx pod: -``` console +```console kubectl expose pod test-nginx --port=80 --name=test-nginx-svc kubectl get svc test-nginx-svc ``` - `kubectl expose pod` → Creates a ClusterIP service on port 80. - `kubectl get svc` → Shows the service details. -You should see an output similar to: +The output is similar to: ```output >service/test-nginx-svc exposed @@ -165,23 +80,20 @@ You should see an output similar to: NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE test-nginx-svc ClusterIP 10.2.194.17 80/TCP 9s ``` -A ClusterIP is assigned (example: 10.2.194.17). This confirms that Kubernetes services are functioning. -### Test Service-to-Pod Connectivity -Now we verify that one pod can talk to another pod through a service. +The assigned ClusterIP (such as `10.2.194.17`) confirms that Kubernetes service networking is functioning correctly. -- Start a temporary curl pod -- Send an HTTP request to the nginx service +## Test pod-to-service connectivity -**Start a curl pod:** Create a temporary curl pod. +Verify that one pod can communicate with another pod through a service by running curl inside a temporary pod: -``` console +```console kubectl run curl --image=curlimages/curl -i --tty -- sh ``` -Inside pod shell: +Inside the pod shell, send an HTTP request to the nginx service: -``` console +```console curl http://test-nginx-svc ``` Exit shell: @@ -190,7 +102,7 @@ Exit shell: exit ``` -You should see an output similar to: +The output is similar to: ```output All commands and output from this session will be recorded in container logs, including credentials and sensitive information passed through the command prompt. @@ -224,14 +136,11 @@ Session ended, resume using 'kubectl attach curl -c curl -i -t' command when the ``` This confirms pod-to-service networking. -- Creates a curl container with an interactive shell. -- Uses curl to send an HTTP request to the nginx service. +Successful curl output confirms that pod-to-service networking is working correctly. -### Test DNS Resolution -Ensures CoreDNS is functioning and services resolve properly. Run `nslookup` inside the curl pod to check DNS service discovery. +## Verify DNS resolution -- `nslookup test-nginx-svc` checks if DNS can resolve the service name -- `CoreDNS` is responsible for this +Test DNS service discovery by running `nslookup` inside the curl pod: If DNS resolves correctly, service discovery is healthy. @@ -239,7 +148,7 @@ If DNS resolves correctly, service discovery is healthy. kubectl exec curl -- nslookup test-nginx-svc.default.svc.cluster.local ``` -You should see an output similar to: +The output is similar to: ```output Server: 10.2.0.10 @@ -248,23 +157,21 @@ Address: 10.2.0.10:53 Name: test-nginx-svc.default.svc.cluster.local Address: 10.2.194.17 ``` -If DNS fails, networking or CoreDNS is broken. -### Test Logs and Exec -This confirms two important Kubernetes features: -- Logs – you can debug applications -- Exec – you can run commands inside containers +Successful DNS resolution confirms that CoreDNS is functioning and service discovery is healthy. -If logs show nginx startup and exec returns version info, pod access works. +## Check pod logs and command execution -``` console +Verify that you can access pod logs and execute commands inside containers: + +```console kubectl logs test-nginx | head kubectl exec test-nginx -- nginx -v ``` - `kubectl logs` → Shows nginx pod logs. - `kubectl exec` → Runs nginx -v inside the pod. -You should see an output similar to: +The output is similar to: ```output /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration @@ -281,21 +188,19 @@ You should see an output similar to: > kubectl exec test-nginx -- nginx -v nginx version: nginx/1.29.3 ``` -- Logs show nginx starting. -- Exec shows nginx version (e.g., `nginx version: nginx/1.25.3`). - -### Delete Test Resources -Once testing is complete, temporary resources should be removed. -- Deletes nginx and curl pods -- Deletes the service - -``` console + +Pod logs and command execution working confirms that you can debug and troubleshoot workloads effectively. + +## Clean up test resources + +Remove the test pods and service to keep your cluster clean: + +```console kubectl delete pod test-nginx curl kubectl delete svc test-nginx-svc ``` -Confirms cleanup works and keeps the cluster clean. -You should see an output similar to: +The output is similar to: ```output pod "test-nginx" deleted from default namespace @@ -303,4 +208,6 @@ pod "curl" deleted from default namespace > kubectl delete svc test-nginx-svc service "test-nginx-svc" deleted from default namespace ``` +## Summary and what's next + After completing these steps, you have confirmed that the Kubernetes cluster and Gardener setup are healthy, core components are functioning correctly, pods start successfully, networking and services operate as expected, DNS resolution works, and the cluster is ready to run real workloads. diff --git a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/benchmarking.md index 8923067e7b..8ba9801fc0 100644 --- a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/benchmarking.md @@ -1,26 +1,20 @@ --- -title: Gardener Benchmarking +title: Benchmark Gardener cluster security with kube-bench weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Benchmark your Gardener cluster security -## Gardener Benchmark on GCP SUSE Arm64 VM -This guide shows how to check the security health of a Gardener Kubernetes cluster running on a GCP SUSE Arm64 VM. We use a tool called **kube-bench**, which checks the cluster against CIS security standards +This section shows you how to evaluate the security health of your Gardener Kubernetes cluster running on a Google Cloud Arm64 (C4A) VM. You'll use kube-bench, a security scanning tool that checks your cluster against Center for Internet Security (CIS) Kubernetes benchmarks. These benchmarks provide industry-standard security recommendations for Kubernetes deployments. -### Prerequisites -Before starting, make sure: -- Gardener Local successfully installed -- Garden cluster and Shoot cluster in Ready state -- Docker is running -- Admin access on the VM +## Verify prerequisites -**Why this matters:** -If the cluster is not running or you don’t have admin access, security checks won’t work. +Before benchmarking, confirm that your environment is ready. You need a running Gardener Local installation with Garden and Shoot clusters in Ready state, Docker running, and admin access to your VM. -**Verify cluster:** +Check your cluster status: ```console cd ~/gardener @@ -29,18 +23,21 @@ kubectl apply -f example/provider-local/shoot.yaml kubectl -n garden-local get shoots kubectl get nodes ``` -If the cluster is not ready, benchmarking does not make sense yet. -### Download kube-bench -Download the Arm64-compatible kube-bench binary from the official GitHub release. This tool will be used to check your Kubernetes cluster against CIS security benchmarks. +The output shows your Shoot cluster and node status. Both should display "Ready" state before proceeding. If your cluster isn't ready, wait for all components to start before running security benchmarks. + +## Download kube-bench for Arm64 + +Download the Arm64-compatible kube-bench binary from the official GitHub release. This version is compiled specifically for aarch64 architecture and ensures accurate security scanning on your C4A VM. ```console cd ~ curl -L https://github.com/aquasecurity/kube-bench/releases/download/v0.10.3/kube-bench_0.10.3_linux_arm64.tar.gz --output ./kube-bench_0.10.3_linux_arm64.tar.gz ``` -### Extract and Install kube-bench Configuration -Extract the downloaded file and place the kube-bench binary and configuration files in standard system locations so the tool can run correctly. +## Install kube-bench + +Extract the downloaded archive and place the kube-bench binary and configuration files in standard system locations: ```console tar -xvf kube-bench_0.10.3_linux_arm64.tar.gz @@ -49,15 +46,13 @@ sudo cp -r cfg /etc/kube-bench/ sudo mv kube-bench /usr/local/bin/ ``` -**Verify binary:** - -Confirm that kube-bench is installed in the correct path and make it executable. This ensures the system can successfully run the benchmarking tool. +Verify the installation and make the binary executable: ```console ls -l /usr/local/bin/kube-bench ``` -**Make it executable:** +Make it executable: Execute kube-bench to scan the Kubernetes cluster and evaluate its security based on industry standard CIS checks. @@ -65,13 +60,19 @@ Execute kube-bench to scan the Kubernetes cluster and evaluate its security base sudo chmod +x /usr/local/bin/kube-bench ``` -### Run kube-bench Benchmark -Execute kube-bench to scan the Kubernetes cluster and evaluate its security based on industry standard CIS checks. +The binary is now ready to scan your cluster. + +### Run the security benchmark + +Execute kube-bench to scan your Kubernetes cluster against CIS security standards: ```console sudo /usr/local/bin/kube-bench --config-dir /etc/kube-bench/cfg ``` -You should see an output similar to: + +The tool evaluates your cluster configuration across multiple security domains, including control plane settings, node configurations, Role-Based Access Control (RBAC) policies, and network policies. The scan takes one to two minutes to complete. + +The output is similar to: ```output 5.2.11 Add policies to each namespace in the cluster which has user workloads to restrict the @@ -107,13 +108,12 @@ An example is as below: type: RuntimeDefault 5.7.3 Follow the Kubernetes documentation and apply SecurityContexts to your Pods. For a -suggested list of SecurityContexts, you may refer to the CIS Security Benchmark for Docker +suggested list of SecurityContexts, you may see the CIS Security Benchmark for Docker Containers. 5.7.4 Ensure that namespaces are created to allow for appropriate segregation of Kubernetes resources and that all new resources are created in a specific namespace. - == Summary policies == 4 checks PASS 4 checks FAIL @@ -127,8 +127,20 @@ resources and that all new resources are created in a specific namespace. 0 checks INFO ``` -### Benchmark summary -Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE) will be similar to: +## Interpret the results + +The kube-bench output categorizes security checks into four result types: + +- PASS: your cluster meets the CIS security recommendation +- FAIL: your cluster doesn't meet the recommendation and needs remediation +- WARN: the check requires manual review or represents optional security hardening +- INFO: informational findings that don't require action + +Your benchmark results show 43 passing checks, 38 failures, and 49 warnings. These numbers provide a baseline security score for your Gardener cluster on Arm64 infrastructure. + +## Review the benchmark summary + +The following table shows typical results from running kube-bench on a `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in Google Cloud with SUSE Linux Enterprise Server: | Category / Subsection | PASS | FAIL | WARN | |-------------------------------------------------|:----:|:----:|:----:| @@ -155,11 +167,24 @@ Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm6 | └─ 5.7 General Policies | 0 | 0 | 4 | | **Total** | 43 | 38 | 49 | -### Conclusions -- **Strong Baseline Security:** The cluster passed **43 CIS checks**, indicating a solid foundational security posture out of the box on **Arm64 (C4A)** infrastructure. -- **Control Plane Hardening Gaps:** A significant number of **FAIL results (38)** are concentrated in **control plane file permissions and API server settings**, which are commonly unmet in development and local/KinD-based setups. -- **Healthy Etcd Configuration:** All **Etcd-related checks passed (7/7)**, demonstrating correct encryption, access controls, and secure peer/client configuration on **Arm64**. -- **Worker Node Improvements Needed:** Worker node checks show mixed results, with failures mainly around **kubelet configuration and file permissions**, highlighting clear opportunities for security hardening. -- **Policy-Level Defaults:** Most **Kubernetes policy checks surfaced as WARN**, reflecting features such as **Pod Security Standards, NetworkPolicies, and admission controls** being optional or not strictly enforced by default. -- **Arm64 Parity with x86_64:** The overall benchmark profile aligns with typical results seen on **x86_64 clusters**, confirming that **Arm64 introduces no architecture-specific security limitations**. -- **Production Readiness Signal:** With targeted remediation—especially for **control plane and kubelet configurations**—the **Arm64-based cluster** can achieve **full CIS compliance** while benefiting from **Arm’s cost and energy efficiency**. +## Interpret your security posture + +Your benchmark results reveal several key insights about your Gardener cluster's security on Arm64 infrastructure: + +- Strong foundational security: the cluster passed 43 CIS checks, demonstrating solid baseline security out of the box on C4A Arm64 infrastructure. This indicates that Gardener's default configurations align well with security best practices. + +- Etcd security is excellent: all seven Etcd-related checks passed, confirming correct encryption, access controls, and secure peer and client configuration. Your distributed key-value store, which holds critical cluster state, is properly secured. + +- Control plane needs hardening: the 38 failed checks concentrate in control plane file permissions and API server settings. These failures are typical in development and Kubernetes in Docker (KinD) based setups. For production deployments, you should remediate control plane configuration issues. + +- Worker node improvements available: worker node checks show mixed results, with failures mainly in kubelet configuration and file permissions. These represent clear opportunities for security hardening, particularly around authentication and authorization settings. + +- Policy enforcement is optional: most Kubernetes policy checks appear as warnings, reflecting that features such as Pod Security Standards, network policies, and admission controls aren't strictly enforced by default. You can enable these features based on your security requirements. + +- Arm64 architecture shows no security limitations: the benchmark profile matches typical results on x86_64 clusters, confirming that Arm64 introduces no architecture-specific security constraints. You can achieve the same security standards on Arm as on traditional x86 infrastructure while benefiting from Arm's cost and energy efficiency. + +With targeted remediation of control plane and kubelet configurations, your Arm64-based Gardener cluster can achieve full CIS compliance while maintaining the performance and cost advantages of Google Cloud's C4A instances. + +## Summary and what's next + +You have successfully benchmarked your Gardener cluster's security posture using industry-standard CIS checks. You now understand your cluster's security strengths and areas for improvement. Your next step is to implement security hardening based on the failed and warned checks to prepare your cluster for production workloads. diff --git a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/installation.md b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/installation.md index 6172e82a70..b56c0c6330 100644 --- a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/installation.md @@ -1,34 +1,40 @@ --- -title: Install Gardener +title: Install Gardener on your Arm-based SUSE VM weight: 4 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Install Gardener on GCP SUSE VM -This guide walks you through a clean, corrected, and fully working setup of Gardener Local on a GCP SUSE Arm64 VM, including installation of Go 1.24, the Arm64 yq binary, all required Gardener CLI tools, KinD cluster setup, shoot creation, and kubeconfig retrieval. +## Install Gardener and dependencies -### Update System -This step updates the operating system packages to the latest versions to avoid bugs and compatibility issues. +This section guides you through setting up Gardener Local on a Google Cloud Arm64 (C4A) VM running SUSE Linux Enterprise Server. You'll install system dependencies, CLI tools, and Gardener components to create a fully functional local Kubernetes cluster management system. -``` console +## Update your system + +Update your operating system packages to the latest versions: + +```console sudo zypper refresh sudo zypper update -y ``` -### Enable SUSE Containers Module -This enables SUSE’s official container support, so Docker and container tools can work properly. -``` console +## Enable the SUSE Containers Module + +Enable SUSE's official container support module for Docker and container tools: + +```console sudo SUSEConnect -p sle-module-containers/15.5/arm64 sudo SUSEConnect --list-extensions | grep Containers ``` -You should see "Activated" as part of the output from the above commands. +The output shows "Activated" for the Containers module, confirming successful enablement. -### Install Docker -Docker is required to run KinD and Kubernetes components as containers. This step installs Docker, starts it, and allows your user to run Docker without sudo. -``` console +## Install Docker + +Install Docker to run Kubernetes in Docker (KinD) and Kubernetes components as containers: + +```console sudo zypper refresh sudo zypper install -y docker sudo systemctl enable --now docker @@ -36,22 +42,25 @@ sudo usermod -aG docker $USER exit ``` -Next, re-open a new shell into your VM and type the following: +Open a new SSH session to apply the Docker group membership, then verify Docker is running: ```console docker ps ``` -You should see the following output: +The output is similar to: ```output CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ``` -### Install Go 1.24 (Manual) -Gardener requires a newer Go version than what SUSE provides by default. Here, Go 1.24 is downloaded and installed manually. +An empty container list confirms Docker is installed and running correctly. -``` console +## Install Go 1.24 + +Gardener requires Go 1.24, which isn't available in the default SUSE repositories. Download and install the Arm64-compatible Go binary: + +```console cd /tmp curl -LO https://go.dev/dl/go1.24.0.linux-arm64.tar.gz sudo rm -rf /usr/local/go @@ -60,125 +69,133 @@ echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.consolerc source ~/.consolerc go version ``` -You should see an output similar to: + +The output is similar to: ```output go version go1.24.0 linux/arm64 ``` -### Install Git, Build Tools -These tools are needed to download the Gardener source code and compile components during setup. +## Install build tools -``` console +Install Git and build tools needed to download and compile Gardener components: + +```console sudo zypper install -y git curl tar gzip make gcc ``` -### Install kubectl -`kubectl` is the command-line tool for interacting with Kubernetes clusters. It lets you check nodes, pods, and cluster status. +## Install kubectl -``` console +Install kubectl, the command-line tool for interacting with Kubernetes clusters: + +```console curl -LO https://dl.k8s.io/release/v1.34.0/bin/linux/arm64/kubectl chmod +x kubectl sudo mv kubectl /usr/local/bin/ kubectl version --client ``` -You should see an output similar to: +The output is similar to: ```output Client Version: v1.34.0 Kustomize Version: v5.7.1 ``` -### Install Helm -Helm is used to install and manage Kubernetes applications. Gardener uses Helm internally to deploy its components. +## Install Helm -``` console +Install Helm to manage Kubernetes applications. Gardener uses Helm internally to deploy components: + +```console curl -sSfL https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 > get_helm.sh chmod 755 ./get_helm.sh ./get_helm.sh helm version ``` -You should see an output similar to: +The output is similar to: ```output version.BuildInfo{Version:"v3.19.2", GitCommit:"8766e718a0119851f10ddbe4577593a45fadf544", GitTreeState:"clean", GoVersion:"go1.24.9"} ``` -### Install yq -`yq` is a YAML processing tool used by Gardener scripts to read and modify configuration files. +## Install yq -``` console +Install yq, a YAML processing tool used by Gardener scripts to modify configuration files: + +```console sudo curl -L -o /usr/local/bin/yq https://github.com/mikefarah/yq/releases/download/v4.43.1/yq_linux_arm64 sudo chmod +x /usr/local/bin/yq yq --version ``` -You should see an output similar to: +The output is similar to: ```output yq (https://github.com/mikefarah/yq/) version v4.43.1 ``` -### Install Kustomize -Kustomize helps customize Kubernetes YAML files without changing the original manifests. +## Install Kustomize -``` console +Install Kustomize to customize Kubernetes YAML files without modifying the originals: + +```console curl -LO https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize/v5.3.0/kustomize_v5.3.0_linux_arm64.tar.gz tar -xvf kustomize_v5.3.0_linux_arm64.tar.gz sudo mv kustomize /usr/local/bin/ kustomize version ``` -You should see an output (Kustomize version) that is similar to: +The output is similar to: ```output v5.3.0 ``` -### Install Kind -Kind (Kubernetes in Docker) creates a local Kubernetes cluster inside Docker. Gardener Local runs entirely on this KinD cluster. +## Install Kind -``` console +Install Kind (Kubernetes in Docker) to create a local Kubernetes cluster inside Docker containers: + +```console curl -Lo kind https://kind.sigs.k8s.io/dl/v0.30.0/kind-linux-arm64 chmod +x kind sudo mv kind /usr/local/bin/ kind version ``` -You should see an output similar to: +The output is similar to: ```output kind v0.30.0 go1.24.6 linux/arm64 ``` -### Add Required Loopback IPs -These special loopback IPs are needed so Gardener services and the local API endpoints work correctly. +## Configure network settings -``` console +Add required loopback IP addresses for Gardener services and local API endpoints: + +```console sudo ip addr add 172.18.255.1/32 dev lo sudo ip addr add 172.18.255.22/32 dev lo ip addr show lo ``` -### Add Hosts Entry -This step maps a Gardener domain name to the local machine so services can be accessed by name. +Add a hosts entry to map the Gardener domain name to localhost: -``` console +```console echo "127.0.0.1 garden.local.gardener.cloud" | sudo tee -a /etc/hosts ``` -You should see an output similar to: +The output is similar to: ```output 127.0.0.1 garden.local.gardener.cloud ``` -### Clone Gardener Repo -Here you download the Gardener’s source code and switch to a known, stable release version. +## Clone the Gardener repository -``` console +Download the Gardener source code and check out a stable release version: + +```console cd ~ git clone https://github.com/gardener/gardener.git cd gardener @@ -186,27 +203,28 @@ git fetch --all --tags git checkout v1.122.0 ``` -### Clean Old KinD Network -This removes any leftover KinD network from previous runs to avoid IP or port conflicts. +## Remove the Kind network -``` console +Remove any existing Kind network from previous installations to avoid conflicts: + +```console docker network rm kind ``` -You should get the following output which is correct: +The expected output indicates the network doesn't exist yet: ```output Error response from daemon: network kind not found exit status 1 ``` -You can confirm this by typing: +Verify the Kind network is absent: ```console docker network ls ``` -Your output should look something like this (note the absence of the "kind" name...its not present in the network config): +The output is similar to: ```output NETWORK ID NAME DRIVER SCOPE @@ -215,14 +233,17 @@ aec64365a860 host host local d60c34b45e0a none null local ``` -### Create Gardener KinD Cluster -This step creates the Kubernetes cluster using KinD and prepares it to run Gardener. +The Kind network isn't listed, confirming a clean state for installation. -``` console +## Create the KinD cluster + +Create the Kubernetes cluster using Kind: + +```console make kind-up ``` -You should see an output similar to: +The output is similar to: ```output clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server serverside-applied @@ -242,11 +263,11 @@ certificatesigningrequest.certificates.k8s.io/csr-rnvdk approved Kubelet Serving Certificate Signing Requests approved. ``` -### If the above "make" command fails... - -If the above make command fails, please reset the loopback interfaces as follows and re-try the make command: +The cluster creation takes two to three minutes. -``` console +{{% notice Note %}} +If the `make kind-up` command fails, reset the loopback interfaces and retry: +```console sudo ip addr del 172.18.255.1/32 dev lo sudo ip addr del 172.18.255.22/32 dev lo sudo ip addr add 172.18.255.1/32 dev lo @@ -254,31 +275,36 @@ sudo ip addr add 172.18.255.22/32 dev lo ip addr show lo make kind-up ``` +{{% /notice %}} +## Configure kubectl access -### Export kubeconfig -This config file allows `kubectl` to connect to the newly created Gardener local cluster. +Set the kubeconfig environment variable to connect kubectl to your new cluster: -``` console +```console export KUBECONFIG=$PWD/example/gardener-local/kind/local/kubeconfig kubectl get nodes ``` -You should see an output similar to: + +The output is similar to: ```output NAME STATUS ROLES AGE VERSION gardener-local-control-plane Ready control-plane 41s v1.32.5 ``` -### Deploy Gardener Components -This installs all Gardener control-plane services, including the API server, controller, scheduler, and monitoring tools. +A node in "Ready" status confirms the cluster is operational. -``` console +## Deploy Gardener components + +Install all Gardener control plane services, including the API server, controller, scheduler, and monitoring tools: + +```console make gardener-up kubectl get pods -n garden ``` -You should see an output similar to: +The output is similar to: ```output NAME READY STATUS RESTARTS AGE @@ -322,33 +348,35 @@ vpa-updater-7dd7dccc6d-bcgv8 1/1 Running 0 vpa-updater-7dd7dccc6d-jdxrg 1/1 Running 0 116s ``` -### Verify Seed -This checks whether the “seed” cluster (the infrastructure cluster managed by Gardener) is healthy and ready. +## Verify seed cluster +This checks whether the seed cluster (the infrastructure cluster managed by Gardener) is healthy and ready. ``` console ./hack/usage/wait-for.sh seed local GardenletReady SeedSystemComponentsHealthy ExtensionsReady kubectl get seeds ``` -You should see an output similar to: +The output is similar to: ```output ⏳ Checking last operation state and conditions for seed/local with a timeout of 600 seconds... ✅ Last operation state is 'Succeeded' and all conditions passed for seed/local. -gcpuser@lpprojectsusearm64:~/gardener> kubectl get seeds NAME STATUS LAST OPERATION PROVIDER REGION AGE VERSION K8S VERSION local Ready Reconcile Succeeded (100%) local local 2m48s v1.122.0 v1.32.5 ``` -### Create Shoot Cluster -A Shoot cluster is a user Kubernetes cluster managed by Gardener. This step creates a sample Shoot running locally. +A "Ready" status with "Succeeded" operation confirms the Seed cluster is fully operational. -``` console +## Create a Shoot cluster + +A Shoot cluster is a user-managed Kubernetes cluster created and managed by Gardener. Create a sample Shoot cluster: + +```console kubectl apply -f example/provider-local/shoot.yaml kubectl -n garden-local get shoots ``` -You should see an output similar to: +The output is similar to: ```output shoot.core.gardener.cloud/local created @@ -357,10 +385,13 @@ NAME CLOUDPROFILE PROVIDER REGION K8S VERSION HIBERNATION LAST OPER local local local local 1.33.0 Awake Create Succeeded (100%) healthy 3h45m ``` -### Add shoot DNS -These DNS entries allow your system to resolve the Shoot cluster’s API endpoint correctly. +The Shoot cluster creation takes four to five minutes. Wait for the "STATUS" column to show "healthy" before proceeding. -``` console +## Add Shoot cluster DNS entries + +Add DNS entries to resolve your Shoot cluster's API endpoint: + +```console cat < admin-kubeconf.yaml KUBECONFIG=admin-kubeconf.yaml kubectl get nodes ``` -{{% notice Note %}} -If you get the following result from the "kubectl get nodes" command above: -```output -No resources found -``` -Please wait a bit and retry again. Your nodes are still being generated! -{{% /notice %}} + You should see an output similar to: @@ -398,4 +422,10 @@ NAME STATUS ROLES AGE VERSION machine-shoot--local--local-local-68499-nhvjl Ready worker 12m v1.33.0 ``` -You now have **Gardener Local running on SUSE Arm64** with Go 1.24, Helm, kubectl, yq, Kustomize, Kind, and a working Shoot cluster. +{{% notice Note %}} +If you see "No resources found", your worker nodes are still being created. Wait one to two minutes and retry the command. +{{% /notice %}} + +## Summary and what's next + +You have successfully installed Gardener Local on your Arm64 SUSE VM with all required dependencies and tools. Your Garden, Seed, and Shoot clusters are running and ready for workload deployment. You're now ready to verify cluster health and test deployments! diff --git a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/instance.md index 0aa449deae..4c67dd7506 100644 --- a/content/learning-paths/servers-and-cloud-computing/gardener-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/gardener-gcp/instance.md @@ -1,44 +1,78 @@ --- -title: Create a Google Axion C4A Arm virtual machine on GCP +title: Create a Google Axion C4A Arm virtual machine for Gardener weight: 3 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Overview +## Create your Google Axion C4A VM -In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. +In this section, you'll provision a Google Axion C4A Arm-based virtual machine on Google Cloud Platform (GCP) to host your Gardener installation. The C4A series uses Arm Neoverse-V2 cores, providing cost-effective performance for cloud-native workloads like Kubernetes cluster management. + +You'll use the `c4a-standard-4` machine type, which provides 4 vCPUs and 16 GB of memory, which is sufficient resources for running Gardener Local with Garden, Seed, and Shoot clusters. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For detailed GCP setup instructions, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} -## Provision a Google Axion C4A Arm VM in Google Cloud Console +## Configure machine settings + +Navigate to the [Google Cloud Console](https://console.cloud.google.com/) and go to **Compute Engine > VM Instances**. Click **Create Instance**. + +Under **Machine configuration**, configure the following settings: + +- Set **Instance name** to a descriptive name like `gardener-vm` +- Select your preferred **Region** and **Zone** (C4A instances are available in select regions) +- Set **Series** to **C4A** +- Select **c4a-standard-4** for the machine type + +![GCP VM creation interface displaying Machine configuration section with C4A series selected, c4a-standard-4 machine type highlighted showing 4 vCPUs and 16 GB memory, and Instance name field visible at top alt-text#center](images/gcp-vm.png "Virtual machine creation interface") + +## Select the operating system + +Under **Boot disk**, click **Change** to configure the operating system and storage. + +Select **SUSE Linux Enterprise Server** as your operating system. SUSE provides excellent support for Arm64 architecture and includes the container tools needed for Gardener. + +For the license type, select **Pay As You Go** to use SUSE's subscription-based licensing. + +Increase the disk size from the default 10 GB to 50 GB by setting **Size (GB)** to `50`. Gardener requires additional storage for Docker images, Kubernetes components, and cluster data. + +Click **Select** to confirm your boot disk configuration. + +## Configure networking + +Under **Firewall**, enable both **Allow HTTP traffic** and **Allow HTTPS traffic**. These settings allow your Gardener cluster to serve web traffic and API endpoints. + +## Create and connect to your VM + +Click **Create** to provision your virtual machine. The VM creation takes one to two minutes. + +After creation completes, locate your running instance in the VM instances list. Click **SSH** to open a browser-based terminal connection to your VM. + +![Google Cloud Console VM instances list showing a running instance named gardener-vm with an SSH button highlighted in the Connect column, located next to other action buttons like RDP and Serial console alt-text#center](images/gcp-ssh.png "Google Cloud Console VM instances list") + +A new browser window opens with a terminal shell connected to your VM: + +![Browser-based SSH terminal window showing command prompt with username@gardener-vm format, displaying SUSE Linux Enterprise Server welcome message and system information including kernel version and architecture details alt-text#center](images/gcp-shell.png "Browser-based SSH terminal window") -To create a virtual machine based on the C4A instance type: -- Navigate to the [Google Cloud Console](https://console.cloud.google.com/). -- Go to **Compute Engine > VM Instances** and select **Create Instance**. -- Under **Machine configuration**: - - Populate fields such as **Instance name**, **Region**, and **Zone**. - - Set **Series** to `C4A`. - - Select `c4a-standard-4` for machine type. +## Verify your Arm64 VM - ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") +Confirm that your VM is running on Arm64 architecture: +```console +uname -m +``` -- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. -- If using use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type. -- Edit the Disk size ("Size(GB)" Textfield...) below and change it from "10" to "50" to increase the disk size of the VM to 50 GB... -- Once appropriately selected and configured, please Click **Select**. -- Under **Networking**, enable **Allow HTTP traffic** as well as **Allow HTTPS traffic**. -- Click **Create** to launch the instance. -- Once created, you should see a "SSH" option to the right in your list of VM instances. Click on this to launch a SSH shell into your VM instance: +The output is: -![Invoke a SSH session via your browser alt-text#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance") +```output +aarch64 +``` -- A window from your browser should come up and you should now see a shell into your VM instance: +This confirms your VM is running on Arm64 (aarch64) architecture, ready for Gardener installation. -![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") +## Summary and what's next -Next, let's install Gardner! \ No newline at end of file +You have successfully created a Google Axion C4A Arm-based virtual machine running SUSE Linux Enterprise Server. Your VM is configured with the resources needed to run Gardener Local and is ready for software installation. You're now ready to install Gardener and its dependencies! \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/github-actions-runner/_index.md b/content/learning-paths/servers-and-cloud-computing/github-actions-runner/_index.md index 42121a6e12..5cff9c2d36 100644 --- a/content/learning-paths/servers-and-cloud-computing/github-actions-runner/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/github-actions-runner/_index.md @@ -10,7 +10,7 @@ learning_objectives: - Execute GitHub Actions workflows on Arm runners. prerequisites: - - An [Amazon Web Services account](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/aws/). + - An [Amazon Web Services account](/learning-paths/servers-and-cloud-computing/csp/aws/). - A GitHub account (personal or organizational). author: Cyril Rohr diff --git a/content/learning-paths/servers-and-cloud-computing/github-on-arm/_index.md b/content/learning-paths/servers-and-cloud-computing/github-on-arm/_index.md index 5a5c0314a7..b2208286d1 100644 --- a/content/learning-paths/servers-and-cloud-computing/github-on-arm/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/github-on-arm/_index.md @@ -12,7 +12,7 @@ learning_objectives: prerequisites: - A [Google Cloud Platform (GCP)](https://cloud.google.com/free?utm_source=google&hl=en) account with billing enabled - - A GitHub account; you can sign up [here](https://github.com/signup) + - A GitHub account; you can [sign up for GitHub](https://github.com/signup) author: Annie Tallund diff --git a/content/learning-paths/servers-and-cloud-computing/github-on-arm/instance.md b/content/learning-paths/servers-and-cloud-computing/github-on-arm/instance.md index 753d92cbca..73abefee2f 100644 --- a/content/learning-paths/servers-and-cloud-computing/github-on-arm/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/github-on-arm/instance.md @@ -10,7 +10,7 @@ layout: learningpathall This section walks you through creating a Google Axion C4A Arm virtual machine on Google Cloud with the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type using the Google Cloud Console. You will use this VM later as the host for a GitHub Actions self-hosted runner. -If you don't have a Google Cloud account, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +If you don't have a Google Cloud account, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). ## Create an Arm-based virtual machine (C4A) diff --git a/content/learning-paths/servers-and-cloud-computing/glibc-with-lse/build_glibc_with_lse.md b/content/learning-paths/servers-and-cloud-computing/glibc-with-lse/build_glibc_with_lse.md index 8255fe686d..fdfd623ef7 100644 --- a/content/learning-paths/servers-and-cloud-computing/glibc-with-lse/build_glibc_with_lse.md +++ b/content/learning-paths/servers-and-cloud-computing/glibc-with-lse/build_glibc_with_lse.md @@ -10,7 +10,7 @@ layout: "learningpathall" ## Overview -"Glibc with LSE" refers to the version of [the GNU C Library (glibc)](https://www.gnu.org/software/libc/) that includes support for [LSE (Large Systems Extensions)](https://learn.arm.com/learning-paths/servers-and-cloud-computing/lse/). LSE is an extension to the ARMv8-A architecture that provides enhanced atomic operations and memory model features. +"Glibc with LSE" refers to the version of [the GNU C Library (glibc)](https://www.gnu.org/software/libc/) that includes support for [LSE (Large Systems Extensions)](/learning-paths/servers-and-cloud-computing/lse/). LSE is an extension to the ARMv8-A architecture that provides enhanced atomic operations and memory model features. LSE introduces additional atomic instructions and operations, such as Load-Acquire, Store-Release, and Atomic Compare-and-Swap (CAS). These operations allow for more efficient synchronization and concurrent access to shared memory in multi-threaded applications running on ARMv8-A processors. diff --git a/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/benchmarking.md index c1daaabe2e..a8bd169d8e 100644 --- a/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/benchmarking.md @@ -54,7 +54,7 @@ The first install is usually slower because of following reasons: This warm-up ensures the real benchmark measures Helm performance, not setup overhead. -You should see output (near the top of the output) that is simlar to: +You should see output (near the top of the output) that is similar to: ```output NAME: warmup LAST DEPLOYED: Tue Dec 9 21:10:44 2025 diff --git a/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/instance.md index 5a93ed9c0e..e986d3a1b1 100644 --- a/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/helm-on-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/irq-tuning-guide/_index.md b/content/learning-paths/servers-and-cloud-computing/irq-tuning-guide/_index.md index 67d34163a6..6d6763f218 100644 --- a/content/learning-paths/servers-and-cloud-computing/irq-tuning-guide/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/irq-tuning-guide/_index.md @@ -33,7 +33,7 @@ operatingsystems: further_reading: - resource: title: Perf for Linux on Arm (LinuxPerf) - link: https://learn.arm.com/install-guides/perf/ + link: /install-guides/perf/ type: website - resource: title: Tune network workloads on Arm-based bare-metal instances diff --git a/content/learning-paths/servers-and-cloud-computing/java-on-axion/1-create-instance.md b/content/learning-paths/servers-and-cloud-computing/java-on-axion/1-create-instance.md index 772f20be71..11b156491e 100644 --- a/content/learning-paths/servers-and-cloud-computing/java-on-axion/1-create-instance.md +++ b/content/learning-paths/servers-and-cloud-computing/java-on-axion/1-create-instance.md @@ -12,9 +12,9 @@ Axion is Google's first Arm-based server processor, built using the Armv9 Neover There are several ways to create an Arm-based Google Axion VM: the Google Cloud console, the gcloud CLI tool, or using your choice of IaC (Infrastructure as Code). -This guide will use the gcloud CLI. If you would like to read more about deploying to Google Cloud via IaC, please check out the [Learning Path to Deploy Arm virtual machines on Google Cloud Platform (GCP) using Terraform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/gcp/). +This guide will use the gcloud CLI. If you would like to read more about deploying to Google Cloud via IaC, please check out the [Learning Path to Deploy Arm virtual machines on Google Cloud Platform (GCP) using Terraform](/learning-paths/servers-and-cloud-computing/gcp/). -If you have never used the Google Cloud Platform before, please review [Getting Started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +If you have never used the Google Cloud Platform before, please review [Getting Started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). #### Open and configure the Google Cloud Shell Editor diff --git a/content/learning-paths/servers-and-cloud-computing/lse/_index.md b/content/learning-paths/servers-and-cloud-computing/lse/_index.md index e2cab13013..be8bd0339d 100644 --- a/content/learning-paths/servers-and-cloud-computing/lse/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/lse/_index.md @@ -37,7 +37,7 @@ further_reading: type: blog - resource: title: Learn about glibc with Large System Extensions (LSE) for performance improvement - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/glibc-with-lse/ + link: /learning-paths/servers-and-cloud-computing/glibc-with-lse/ type: website diff --git a/content/learning-paths/servers-and-cloud-computing/microbenchmark-network-iperf3/_index.md b/content/learning-paths/servers-and-cloud-computing/microbenchmark-network-iperf3/_index.md index 933c5aea16..ef145877a8 100644 --- a/content/learning-paths/servers-and-cloud-computing/microbenchmark-network-iperf3/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/microbenchmark-network-iperf3/_index.md @@ -12,7 +12,7 @@ learning_objectives: prerequisites: - Basic understanding of networking principles such as Transmission Control Protocol/Internet Protocol (TCP/IP) and User Datagram Protocol (UDP). - - Access to two [Arm-based cloud instances](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/). + - Access to two [Arm-based cloud instances](/learning-paths/servers-and-cloud-computing/csp/). author: Kieran Hejmadi diff --git a/content/learning-paths/servers-and-cloud-computing/migrate-ease/_index.md b/content/learning-paths/servers-and-cloud-computing/migrate-ease/_index.md index 59b84e8bf7..7390563c20 100644 --- a/content/learning-paths/servers-and-cloud-computing/migrate-ease/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/migrate-ease/_index.md @@ -43,7 +43,7 @@ further_reading: type: website - resource: title: Migrating Applications to Arm Servers - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/migration/ + link: /learning-paths/servers-and-cloud-computing/migration/ type: website diff --git a/content/learning-paths/servers-and-cloud-computing/migration/java.md b/content/learning-paths/servers-and-cloud-computing/migration/java.md index 294359b90f..39f15a1ec7 100644 --- a/content/learning-paths/servers-and-cloud-computing/migration/java.md +++ b/content/learning-paths/servers-and-cloud-computing/migration/java.md @@ -89,7 +89,7 @@ bool UseSimpleArrayEquals = false Depending on your application, you may want to investigate the vector processing flags for SIMD, NEON, SVE, and CRC. -You can try [Process Watch](https://learn.arm.com/learning-paths/servers-and-cloud-computing/processwatch/) to monitor the usage of SIMD and CRC instructions. +You can try [Process Watch](/learning-paths/servers-and-cloud-computing/processwatch/) to monitor the usage of SIMD and CRC instructions. Refer to the [Java documentation](https://docs.oracle.com/en/java/javase/17/docs/specs/man/java.html) for more information about the flags. diff --git a/content/learning-paths/servers-and-cloud-computing/mongodb-on-gcp/create-instance.md b/content/learning-paths/servers-and-cloud-computing/mongodb-on-gcp/create-instance.md index 33af64d5a6..0620cfeec3 100644 --- a/content/learning-paths/servers-and-cloud-computing/mongodb-on-gcp/create-instance.md +++ b/content/learning-paths/servers-and-cloud-computing/mongodb-on-gcp/create-instance.md @@ -10,7 +10,7 @@ layout: learningpathall This section walks you through creating a Google Axion C4A Arm virtual machine on GCP with the `c4a-standard-4` (4 vCPUs, 16 GB Memory) machine type, using the **Google Cloud Console**. -If you haven't set up a Google Cloud account, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +If you haven't set up a Google Cloud account, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). ## Create an Arm-based virtual machine (C4A) diff --git a/content/learning-paths/servers-and-cloud-computing/mongodb/automate_setup_pulumi.md b/content/learning-paths/servers-and-cloud-computing/mongodb/automate_setup_pulumi.md index 95e3778229..80715c68d1 100644 --- a/content/learning-paths/servers-and-cloud-computing/mongodb/automate_setup_pulumi.md +++ b/content/learning-paths/servers-and-cloud-computing/mongodb/automate_setup_pulumi.md @@ -63,7 +63,7 @@ cp build-native-gcc-rel/gatord ~/pulumi-ec2/ ``` ## Install awscli and set environment variables -Use the [awscli](https://learn.arm.com/install-guides/aws-cli/) learning path to install the awscli. +Use the [awscli](/install-guides/aws-cli/) learning path to install the awscli. Set the following environment variables on your local computer to connect to your AWS account: ```bash diff --git a/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/_index.md b/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/_index.md index 0474eaaf90..9d67272dfa 100644 --- a/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/_index.md @@ -12,7 +12,7 @@ learning_objectives: - Use Libamath with different vector accuracy modes in practice prerequisites: - - An Arm computer running Linux with [Arm Performance Libraries](https://learn.arm.com/install-guides/armpl/) version 25.04 or newer installed. + - An Arm computer running Linux with [Arm Performance Libraries](/install-guides/armpl/) version 25.04 or newer installed. ### Tags skilllevels: Introductory @@ -33,7 +33,7 @@ further_reading: type: documentation - resource: title: ArmPL Installation Guide - link: https://learn.arm.com/install-guides/armpl/ + link: /install-guides/armpl/ type: website diff --git a/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/examples.md b/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/examples.md index 38e82af0da..3fe2b2aceb 100644 --- a/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/examples.md +++ b/content/learning-paths/servers-and-cloud-computing/multi-accuracy-libamath/examples.md @@ -10,7 +10,7 @@ layout: learningpathall Here is an example invoking all accuracy modes of the Neon single precision exp function. The file `ulp_error.h` is from the previous section. -Make sure you have [Arm Performance Libraries](https://learn.arm.com/install-guides/armpl/) installed. +Make sure you have [Arm Performance Libraries](/install-guides/armpl/) installed. Use a text editor to save the code below in a file named `example.c`. diff --git a/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/_index.md b/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/_index.md index e3577910c2..1c3c475dd8 100644 --- a/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/_index.md @@ -53,11 +53,11 @@ further_reading: type: documentation - resource: title: Learn how to deploy nginx [Arm Learning Path] - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/nginx/ + link: /learning-paths/servers-and-cloud-computing/nginx/ type: documentation - resource: title: Learn how to tune nginx [Arm Learning Path] - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/nginx_tune/ + link: /learning-paths/servers-and-cloud-computing/nginx_tune/ type: documentation ### FIXED, DO NOT MODIFY diff --git a/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/apply-configuration.md b/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/apply-configuration.md index 0dff6f7bd2..9915833b9f 100644 --- a/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/apply-configuration.md +++ b/content/learning-paths/servers-and-cloud-computing/multiarch_nginx_on_aks/apply-configuration.md @@ -65,7 +65,7 @@ You can now log in to any pod and monitor system performance with `btop`. Many f {{% notice Note %}} Network performance can impact load testing accuracy when running from your local machine. If you experience low request rates (under 70,000 requests/s), consider running the test from an Azure VM in the same region as your cluster. -You can create an [Azure VM instance](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/azure/) and install the `nginx_util.sh` script there for more reliable results.{{% /notice %}} +You can create an [Azure VM instance](/learning-paths/servers-and-cloud-computing/csp/azure/) and install the `nginx_util.sh` script there for more reliable results.{{% /notice %}} Running two btop terminals, one for each pod, is a convenient way to view performance in real time. diff --git a/content/learning-paths/servers-and-cloud-computing/mysql_benchmark/_index.md b/content/learning-paths/servers-and-cloud-computing/mysql_benchmark/_index.md index 145a937aab..d8e6e81cdd 100644 --- a/content/learning-paths/servers-and-cloud-computing/mysql_benchmark/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/mysql_benchmark/_index.md @@ -15,7 +15,7 @@ prerequisites: author: Bolt Liu -skilllevels: introductory +skilllevels: Introductory subjects: Databases armips: - Neoverse @@ -41,7 +41,7 @@ further_reading: type: documentation - resource: title: Learn how to deploy MySQL - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/mysql/ + link: /learning-paths/servers-and-cloud-computing/mysql/ type: documentation diff --git a/content/learning-paths/servers-and-cloud-computing/mysql_tune/_index.md b/content/learning-paths/servers-and-cloud-computing/mysql_tune/_index.md index d7f3c8fc5f..9c594140e1 100644 --- a/content/learning-paths/servers-and-cloud-computing/mysql_tune/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/mysql_tune/_index.md @@ -9,7 +9,7 @@ learning_objectives: - Tune MySQL to increase performance prerequisites: - - Bare-metal or cloud [installation of MySQL](/learning-paths//servers-and-cloud-computing/mysql) + - Bare-metal or cloud [installation of MySQL](/learning-paths/servers-and-cloud-computing/mysql/) author: Julio Suarez diff --git a/content/learning-paths/servers-and-cloud-computing/neoverse-rdv3-swstack/3_rdv3_sw_build.md b/content/learning-paths/servers-and-cloud-computing/neoverse-rdv3-swstack/3_rdv3_sw_build.md index ef11a13325..1539c5c0c4 100644 --- a/content/learning-paths/servers-and-cloud-computing/neoverse-rdv3-swstack/3_rdv3_sw_build.md +++ b/content/learning-paths/servers-and-cloud-computing/neoverse-rdv3-swstack/3_rdv3_sw_build.md @@ -92,7 +92,7 @@ There are two supported methods for building the reference firmware stack: **hos In this Learning Path, you will use the **container-based** approach. -The container image uses your host source directory (~/rdv3) and performs the build inside Docker. Ensure Docker is installed on your machine. You can follow this [installation guide](https://learn.arm.com/install-guides/docker/). +The container image uses your host source directory (~/rdv3) and performs the build inside Docker. Ensure Docker is installed on your machine. You can follow this [installation guide](/install-guides/docker/). After Docker is installed, you’re ready to build the container image. diff --git a/content/learning-paths/servers-and-cloud-computing/net-aspire/gcp.md b/content/learning-paths/servers-and-cloud-computing/net-aspire/gcp.md index da720d2a5d..522a2fd878 100644 --- a/content/learning-paths/servers-and-cloud-computing/net-aspire/gcp.md +++ b/content/learning-paths/servers-and-cloud-computing/net-aspire/gcp.md @@ -13,7 +13,7 @@ In this section, you will learn how to deploy the .NET Aspire application you cr To create an Arm64 VM, follow these steps: 1. Create a Google Cloud Account. If you don’t already have an account, sign up for Google Cloud. -2. Open the Google Cloud Console [here](https://console.cloud.google.com). +2. Open the [Google Cloud Console](https://console.cloud.google.com). 3. Navigate to Compute Engine. In the Google Cloud Console, open the Navigation menu, and go to **Compute Engine** > **VM Instances**. Enable any relevant APIs if prompted. diff --git a/content/learning-paths/servers-and-cloud-computing/node-js-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/node-js-gcp/instance.md index b9b859c089..ecd587360b 100644 --- a/content/learning-paths/servers-and-cloud-computing/node-js-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/node-js-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/openbmc-rdv3/2_openbmc_setup.md b/content/learning-paths/servers-and-cloud-computing/openbmc-rdv3/2_openbmc_setup.md index ed475d44cb..6c2a09de69 100644 --- a/content/learning-paths/servers-and-cloud-computing/openbmc-rdv3/2_openbmc_setup.md +++ b/content/learning-paths/servers-and-cloud-computing/openbmc-rdv3/2_openbmc_setup.md @@ -10,7 +10,7 @@ layout: learningpathall In this section, you prepare your workspace to build and simulate OpenBMC and UEFI firmware on the Neoverse RD-V3 r1 platform using Arm Fixed Virtual Platforms (FVPs). You will install the required tools, configure repositories, and set up a Docker-based build environment for both BMC and host firmware. -Before you start, review the related Learning Path [CSS-V3 pre-silicon software development using Neoverse servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/neoverse-rdv3-swstack). It walks through using the CSSv3 reference design on FVP to perform early development and validation. +Before you start, review the related Learning Path [CSS-V3 pre-silicon software development using Neoverse servers](/learning-paths/servers-and-cloud-computing/neoverse-rdv3-swstack). It walks through using the CSSv3 reference design on FVP to perform early development and validation. You will perform the steps outlined below on your Arm Neoverse-based Linux machine running Ubuntu 22.04 LTS. You will need at least 80 GB of free disk space, 48 GB of RAM. diff --git a/content/learning-paths/servers-and-cloud-computing/openshift/migrationsteps.md b/content/learning-paths/servers-and-cloud-computing/openshift/migrationsteps.md index 48655ee0e1..9dd95a8a07 100644 --- a/content/learning-paths/servers-and-cloud-computing/openshift/migrationsteps.md +++ b/content/learning-paths/servers-and-cloud-computing/openshift/migrationsteps.md @@ -16,8 +16,8 @@ Before migrating, confirm that your applications can run on the Arm 64-bit (`arm Use the following tools to inspect container images for multi-architecture support: -- [KubeArchInspect](https://learn.arm.com/learning-paths/servers-and-cloud-computing/kubearchinspect/): scans container images in your Kubernetes cluster. -- [Docker inspect script](https://learn.arm.com/learning-paths/cross-platform/docker/check-images/): checks local image manifests for platform variants. +- [KubeArchInspect](/learning-paths/servers-and-cloud-computing/kubearchinspect/): scans container images in your Kubernetes cluster. +- [Docker inspect script](/learning-paths/cross-platform/docker/check-images/): checks local image manifests for platform variants. {{% notice Note %}} The OpenShift Pipelines Tutorial supports `arm64` and has no architecture restrictions. diff --git a/content/learning-paths/servers-and-cloud-computing/php-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/php-on-gcp/instance.md index e7b1e2fe77..eaf8c6225e 100644 --- a/content/learning-paths/servers-and-cloud-computing/php-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/php-on-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you’ll provision a Google Cloud Axion C4A Arm virtual machine (VM) using the `c4a-standard-4` (four vCPUs, 16 GB memory) machine type in the Google Cloud Console. This process sets up a high-performance Arm server for PHP workloads, leveraging the scalability and efficiency of Axion C4A and SUSE Linux on Google Cloud. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision your Google Axion C4A instance diff --git a/content/learning-paths/servers-and-cloud-computing/postgresql_tune/_index.md b/content/learning-paths/servers-and-cloud-computing/postgresql_tune/_index.md index df154374cd..9adc6b6491 100644 --- a/content/learning-paths/servers-and-cloud-computing/postgresql_tune/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/postgresql_tune/_index.md @@ -9,7 +9,7 @@ learning_objectives: - Tune PostgreSQL to increase performance prerequisites: - - Bare-metal or cloud [installation of PostgreSQL](/learning-paths//servers-and-cloud-computing/postgresql) + - Bare-metal or cloud [installation of PostgreSQL](/learning-paths/servers-and-cloud-computing/postgresql/) author: Julio Suarez diff --git a/content/learning-paths/servers-and-cloud-computing/processwatch/_index.md b/content/learning-paths/servers-and-cloud-computing/processwatch/_index.md index 342e128a4c..9cce0a0de4 100644 --- a/content/learning-paths/servers-and-cloud-computing/processwatch/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/processwatch/_index.md @@ -36,7 +36,7 @@ operatingsystems: further_reading: - resource: title: Perf for Linux on Arm (LinuxPerf) - link: https://learn.arm.com/install-guides/perf/ + link: /install-guides/perf/ type: website - resource: title: Capstone diff --git a/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/optimization_checklist.md b/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/optimization_checklist.md index 6386ec71da..fac52f4d57 100644 --- a/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/optimization_checklist.md +++ b/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/optimization_checklist.md @@ -18,7 +18,7 @@ Here is a suggested optimization checklist: 1. Check the functions that are the most frontend bound: * If you see high instruction cache miss rate, apply profile-guided optimization to reduce the code size of less important functions. This frees up more instruction cache space for the important hot-functions. - * If you see high instruction TLB misses, apply code layout optimization, using tools such as [Bolt](https://learn.arm.com/learning-paths/servers-and-cloud-computing/bolt/overview/). This improves locality of code accesses, reducing the number of TLB misses. + * If you see high instruction TLB misses, apply code layout optimization, using tools such as [Bolt](/learning-paths/servers-and-cloud-computing/bolt/overview/). This improves locality of code accesses, reducing the number of TLB misses. 1. Check the functions that have the highest bad speculation rate. If you see high branch mispredict rates, use a more predictable branching pattern, or change the software to avoid branches by using conditional selects. diff --git a/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/system_compatibility.md b/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/system_compatibility.md index bdfc9b9e37..47429d5d48 100644 --- a/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/system_compatibility.md +++ b/content/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/system_compatibility.md @@ -9,7 +9,7 @@ layout: learningpathall Before you begin, you can use the Arm Sysreport utility to determine whether your system configuration supports hardware-assisted profiling. Follow the instructions in [Get ready for performance analysis with Sysreport][1] to discover how to download and run this utility. -[1]: https://learn.arm.com/learning-paths/servers-and-cloud-computing/sysreport/ +[1]: /learning-paths/servers-and-cloud-computing/sysreport/ The `perf counters` entry in the generated report indicates how many CPU counters are available. The `perf sampling` entry indicates if SPE is available. You can achieve the best profiles in systems with at least 6 available CPU counters and SPE. diff --git a/content/learning-paths/servers-and-cloud-computing/puppet-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/puppet-on-gcp/instance.md index d31b1fccd8..d8df6bfd37 100644 --- a/content/learning-paths/servers-and-cloud-computing/puppet-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/puppet-on-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -If you need help on setting up GCP, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +If you need help on setting up GCP, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/background.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/background.md index 1d839a0496..1d0e48dd02 100644 --- a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/background.md +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/background.md @@ -12,7 +12,7 @@ Google Axion C4A is a family of Arm-based virtual machines built on Google’s c The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. +To learn more about Google Axion, see the [Introducing Google Axion Processors, Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. ## RabbitMQ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/baseline.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/baseline.md index b6f74f264e..9f6b66be5b 100644 --- a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/baseline.md @@ -1,6 +1,6 @@ --- title: RabbitMQ Baseline Testing on Google Axion C4A Arm Virtual Machine -weight: 5 +weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall @@ -15,7 +15,7 @@ The purpose of this baseline is to confirm: - Queue operations (create, publish, consume) - CLI tooling functionality (`rabbitmqctl` and `rabbitmqadmin`) -### Check RabbitMQ Node Status +### Check RabbitMQ node status Verify that the RabbitMQ node is operational and healthy. ```console @@ -26,7 +26,7 @@ sudo rabbitmqctl status - Listeners are active on ports 5672 and 15672 - Memory and disk space are within safe limits -### Verify Enabled Plugins +### Verify enabled plugins Confirm that the RabbitMQ management plugins are enabled. ```console @@ -42,7 +42,7 @@ You should see an output similar to: [ ] rabbitmq_stream_management 4.2.0 ``` -### Validate RabbitMQ Listeners +### Validate RabbitMQ listeners Ensure RabbitMQ is listening on the required ports. ```console @@ -58,7 +58,7 @@ Interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI Interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0 ``` -### Download RabbitMQ Admin CLI Tool +### Download RabbitMQ Admin CLI tool Download the rabbitmqadmin CLI tool from the local management endpoint. ```console @@ -69,7 +69,7 @@ curl -u guest:guest http://localhost:15672/cli/rabbitmqadmin -o rabbitmqadmin ```console chmod +x rabbitmqadmin ``` -### Validate Queue Creation +### Validate queue creation Create a test queue to validate write operations. ```console @@ -81,7 +81,7 @@ You should see an output similar to: queue declared ``` -### Publish a Test Message +### Publish a test message Send a test message to the queue. ```console @@ -93,7 +93,7 @@ You should see an output similar to: Message published ``` -### Consume Message From Queue +### Consume message from queue Retrieve messages from the queue to verify read functionality. ```console @@ -109,7 +109,7 @@ You should see an output similar to: +-------------+----------+---------------+-------------+---------------+------------------+------------+-------------+ ``` -### Verify Queue State +### Verify queue state Confirm that the queue is empty after consumption. ```console @@ -126,7 +126,7 @@ You should see an output similar to: | testqueue | 1 | ``` -Baseline Validation Summary +### Baseline validation summary - RabbitMQ node is running and healthy - The management plugin is enabled and accessible diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/firewall_setup.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/firewall_setup.md new file mode 100644 index 0000000000..100aa126ad --- /dev/null +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/firewall_setup.md @@ -0,0 +1,36 @@ +--- +title: Create a Firewall Rule on GCP +weight: 3 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Overview + +In this section, you create a Firewall Rule within Google Cloud Console to expose TCP port 15672. + +{{% notice Note %}} +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). +{{% /notice %}} + +## Create a Firewall Rule in GCP + +To expose TCP port 15672, create a firewall rule. + +Navigate to the [Google Cloud Console](https://console.cloud.google.com/), go to **VPC Network > Firewall**, and select **Create firewall rule**. + +![Create a firewall rule](images/firewall-rule.png "Create a firewall rule") + +Next, create the firewall rule that exposes TCP port 15672. +Set the **Name** of the new rule to "allow-tcp-15672". Select your network that you intend to bind to your VM (default is "autoscaling-net" but your organization might have others). + +Set **Direction of traffic** to "Ingress". Set **Allow on match** to "Allow" and **Targets** to "Specified target tags". Enter "allow-tcp-15672" in the **Target tags** text field. Set **Source IPv4 ranges** to "0.0.0.0/0". + +![Create a firewall rule](images/network-rule.png "Creating the TCP/15672 firewall rule") + +Finally, select **Specified protocols and ports** under the **Protocols and ports** section. Select the **TCP** checkbox, enter "15672" in the **Ports** text field, and select **Create**. + +![Specifying the TCP port to expose](images/network-port.png "Specifying the TCP port to expose") + +The network firewall rule is now created and you can continue with the VM creation. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/firewall-rule.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/firewall-rule.png new file mode 100644 index 0000000000..cb2d9bf40a Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/firewall-rule.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-pubip-ssh.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-pubip-ssh.png new file mode 100644 index 0000000000..558745de3e Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-pubip-ssh.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-shell.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-shell.png new file mode 100644 index 0000000000..7e2fc3d1b5 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-shell.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-ssh.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-ssh.png new file mode 100644 index 0000000000..597ccd7fea Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/gcp-ssh.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-config.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-config.png new file mode 100644 index 0000000000..311c498cc9 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-config.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-port.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-port.png new file mode 100644 index 0000000000..0a25905d32 Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-port.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-rule.png b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-rule.png new file mode 100644 index 0000000000..a1cae9180a Binary files /dev/null and b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/images/network-rule.png differ diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/installation.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/installation.md index 9bc7a1e67d..e81746c810 100644 --- a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/installation.md @@ -1,6 +1,6 @@ --- title: Install RabbitMQ -weight: 4 +weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall @@ -9,7 +9,7 @@ layout: learningpathall ## Install RabbitMQ on GCP SUSE Arm64 VM This guide describes a **step-by-step installation of RabbitMQ** on a **Google Cloud Platform SUSE Linux Arm64 virtual machine**, using **RPM packages** for both **Erlang** and **RabbitMQ Server**. -RabbitMQ requires Erlang to be installed before setting up the server. +RabbitMQ needs Erlang to be installed before setting up the server. ### Prerequisites @@ -18,14 +18,14 @@ RabbitMQ requires Erlang to be installed before setting up the server. - Root or sudo privileges - Outbound internet access -### Refresh System Repositories +### Refresh system repositories This step updates the system’s package list so the operating system knows about the latest software available from its repositories. ```console sudo zypper refresh ``` -### Install Required System Utilities +### Install required system utilities You can install the basic tools needed to download and manage packages. ```console @@ -40,7 +40,7 @@ wget https://github.com/rabbitmq/erlang-rpm/releases/download/v26.2.5/erlang-26. sudo rpm -Uvh erlang-26.2.5-1.el8.aarch64.rpm ``` -### Verify Erlang Installation +### Verify Erlang installation Confirm that Erlang is installed correctly. ```console @@ -62,20 +62,20 @@ sudo rpm -Uvh rabbitmq-server-4.2.0-1.el8.noarch.rpm ``` {{% notice Note %}} -RabbitMQ version 3.11.0 introduced significant performance enhancements for Arm-based architectures. This version requires Erlang 25.0 or later, which brings Just-In-Time (JIT) compilation and modern flame graph profiling tooling to both x86 and ARM64 CPUs. These features result in improved performance on ARM64 architectures. +RabbitMQ version 3.11.0 introduced significant performance enhancements for Arm-based architectures. This version needs Erlang 25.0 or later, which brings Just-In-Time (JIT) compilation and modern flame graph profiling tooling to both x86 and Arm64 CPUs. These features result in improved performance on Arm64 architectures. You can view [this release note](https://github.com/rabbitmq/rabbitmq-server/blob/main/release-notes/3.11.0.md) -The [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends RabbitMQ version 3.11.0, the minimum recommended on the Arm platforms. +The [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends RabbitMQ version 3.11.0, the minimum recommended on Arm platforms. {{% /notice %}} -### Enable and Start RabbitMQ Service +### Enable and start RabbitMQ service Enable RabbitMQ to start automatically on boot and start the service immediately. ```console sudo systemctl enable rabbitmq-server --now ``` -### Verify RabbitMQ Service Status +### Verify RabbitMQ service status Check the status of the RabbitMQ service. ```console @@ -84,7 +84,7 @@ sudo systemctl status rabbitmq-server The service should be in an active (running) state. -### Enable RabbitMQ Management Plugin +### Enable RabbitMQ management plugin Enable the RabbitMQ management plugin to access the web-based dashboard. ```console @@ -98,7 +98,7 @@ Restart RabbitMQ to apply plugin changes. sudo systemctl restart rabbitmq-server ``` -### Verify RabbitMQ Version +### Verify RabbitMQ version Confirm the installed RabbitMQ version. ```console @@ -111,12 +111,10 @@ You should see an output similar to: 4.2.0 ``` -### Access RabbitMQ Management UI -Create a new RabbitMQ user for remote access +### Access RabbitMQ management UI +Create a new RabbitMQ user for remote access. -- Create a new `admin` user - -**Run these commands on the VM:** +Create a new `admin` user by running these commands on the VM: ```console sudo rabbitmqctl add_user admin StrongPassword123 @@ -126,19 +124,16 @@ sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*" **Log in to Management UI** +Now, test it from outside the VM. Open a web browser on your local machine (Chrome, Firefox, Edge, etc.) and enter the following URL and credentials in the address bar: + - **URL**: http://:15672 - **Username**: admin - **Password**: StrongPassword123 -This is the production-safe approach. +Replace `` with the public IP of your GCP VM. -Now, let’s test it from outside the VM. Open a web browser on your local machine (Chrome, Firefox, Edge, etc.) and enter the following URL in the address bar: - -```console -[http:///info.php](http://:15672) -``` -- Replace `` with the public IP of your GCP VM. - -If everything is set up correctly, you will see a RabbitMQ login page in your browser. It looks like this: +If everything is configured correctly, you see a RabbitMQ login page in your browser that looks like this: ![RabbitMQ page alt-text#center](images/rabbitmq.png "Figure 1: RabbitMQ Login page") + +This confirms that your RabbitMQ management dashboard is operational. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/instance.md index 2b93bc950d..6f983d3b64 100644 --- a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/instance.md @@ -1,6 +1,6 @@ --- title: Create a Google Axion C4A Arm virtual machine on GCP -weight: 3 +weight: 4 ### FIXED, DO NOT MODIFY layout: learningpathall @@ -8,24 +8,32 @@ layout: learningpathall ## Overview -In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. +In this section, you provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console -To create a virtual machine based on the C4A instance type: -- Navigate to the [Google Cloud Console](https://console.cloud.google.com/). -- Go to **Compute Engine > VM Instances** and select **Create Instance**. -- Under **Machine configuration**: - - Populate fields such as **Instance name**, **Region**, and **Zone**. - - Set **Series** to `C4A`. - - Select `c4a-standard-4` for machine type. +To create a virtual machine based on the C4A instance type, navigate to the [Google Cloud Console](https://console.cloud.google.com/) and go to **Compute Engine > VM Instances**. Select **Create Instance**. + +Under **Machine configuration**, populate fields such as **Instance name**, **Region**, and **Zone**. Set **Series** to `C4A` and select `c4a-standard-4` for machine type. ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") -- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. Pick the preferred version for your Operating System. Ensure you select the **Arm image** variant. Click **Select**. -- Under **Networking**, enable **Allow HTTP traffic**. -- Click **Create** to launch the instance. +Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type and press **Select**. + +Under **Networking**, enable **Allow HTTP traffic** and add "allow-tcp-15672" as a network tag in the **Network tags** text field. + +![Adding the TCP/15672 firewall rule to the VM](images/network-config.png "Adding the TCP/15672 firewall rule to the VM") + +Select **Create** to launch the instance. Once created, you see an **SSH** option and the public IP address for your VM in the list of VM instances. Save the public IP address as you need it in the next step. Select the **SSH** option to launch an SSH shell into your VM instance. + +![Invoke an SSH session via your browser alt-text#center](images/gcp-pubip-ssh.png "Invoke an SSH session into your running VM instance") + +A window opens from your browser and you see a shell into your VM instance. + +![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") + +Next, install RabbitMQ. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case1.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case1.md index c5f4e9b90c..5ef1abeec8 100644 --- a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case1.md +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case1.md @@ -1,6 +1,6 @@ --- title: RabbitMQ Use Case 1 – Event Processing with Python Workers -weight: 6 +weight: 7 ### FIXED, DO NOT MODIFY layout: learningpathall @@ -16,7 +16,7 @@ This use case demonstrates how RabbitMQ enables event-driven architectures using The use case models an **event-driven system**, where order-related events are published and processed asynchronously by workers. -### Use Case Overview +### Use case overview **Scenario:** An application publishes order-related events (`order.created`, `order.updated`, etc.) to RabbitMQ. A background worker consumes these events from a queue and processes them independently. @@ -38,7 +38,7 @@ This architecture improves scalability, fault tolerance, and system decoupling. - Python 3 installed - Network access to RabbitMQ broker -### Declare a Topic Exchange +### Declare a topic exchange Create a durable topic exchange to route events based on routing keys. ```console @@ -46,10 +46,10 @@ Create a durable topic exchange to route events based on routing keys. ``` - Creates a durable topic exchange named events. -- Routes messages using wildcard-based routing keys (e.g., order.*). +- Routes messages using wildcard-based routing keys (for example, order.*). - Ensures the exchange survives broker restarts. -### Declare a Durable Queue +### Declare a durable queue Create a durable queue to store order-related events. ```console @@ -65,7 +65,7 @@ You should see an output similar to: queue declared ``` -### Bind Queue to Exchange +### Bind queue to exchange Bind the queue to the exchange using a topic routing pattern. ```console @@ -86,11 +86,11 @@ This binding ensures the queue receives all messages with routing keys such as: - order.updated - order.completed -### Publish an Event Message +### Publish an event message Publish a sample order event to the exchange. ```console -./rabbitmqadmin publish exchange=events routing_key="order.created" payload='{"order_id":123} +./rabbitmqadmin publish exchange=events routing_key="order.created" payload='{"order_id":123}' ``` - Publishes an event to the events exchange. @@ -102,7 +102,7 @@ You should see an output similar to: Message published ``` -### Install Python Dependencies +### Install Python dependencies Install pip and the pika RabbitMQ client library. ```console @@ -110,13 +110,15 @@ sudo zypper install -y python3-pip pip install pika ``` -### Create the Worker Script +### Create the worker script Create a Python worker file to process messages from a queue. A **Python worker** was created to process messages from a RabbitMQ queue (jobs) using the pika library. The queue is durable, ensuring message persistence. The worker implements fair dispatch (prefetch_count=1) and manual acknowledgments to reliably process each job without loss. Messages were successfully published to the queue using rabbitmqadmin, and the worker consumed them as expected. +Using your favorite editor (the example uses "edit") create your "worker.py" file: + ```console -vi worker.py +edit worker.py ``` **worker.py:** @@ -160,7 +162,7 @@ channel.basic_consume( channel.start_consuming() ``` -### Start the Worker +### Start the worker Run the worker process. ```console @@ -172,8 +174,8 @@ You should see an output similar to: The worker started. Waiting for jobs... ``` -### Publish Job Messages -From another terminal, publish a job message. +### Publish job messages +From another SSH terminal, publish a job message. ```console ./rabbitmqadmin publish routing_key=jobs payload='{"job":"test1"}' @@ -198,13 +200,13 @@ Publish another job: Worker started. Waiting for jobs... [Worker] Received job: {'job': 'hello1'} ``` +Press "CTRL-C" to exit the worker application. -## Use Case Validation +## Use case validation - Event routing via topic exchanges functions correctly - Durable queues and acknowledgments ensure reliable message processing - Worker-based consumption supports safe and controlled job execution - This use case demonstrates how RabbitMQ enables reliable, decoupled, and scalable event processing using topic-based routing and Python workers. The setup provides a strong foundation for production-grade, message-driven architectures on GCP SUSE Arm64 virtual machines. diff --git a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case2.md b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case2.md index 1e4c58b790..9fbf674f90 100644 --- a/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case2.md +++ b/content/learning-paths/servers-and-cloud-computing/rabbitmq-gcp/use-case2.md @@ -1,6 +1,6 @@ --- title: RabbitMQ use case 2 - WhatsApp Notification -weight: 7 +weight: 8 ### FIXED, DO NOT MODIFY layout: learningpathall @@ -10,7 +10,7 @@ layout: learningpathall ## WhatsApp Notification Use Case using RabbitMQ This document demonstrates a **real-world asynchronous messaging use case** where RabbitMQ is used to process WhatsApp notifications reliably using a worker-based architecture. -### Use Case Overview +### Use case overview In many production systems, sending WhatsApp notifications must be: - Reliable @@ -19,7 +19,7 @@ In many production systems, sending WhatsApp notifications must be: RabbitMQ is used as a **message broker** to decouple message production from message consumption. -### Architecture Flow +### Architecture flow 1. Application publishes a message to RabbitMQ 2. RabbitMQ routes the message to a queue @@ -34,7 +34,7 @@ RabbitMQ is used as a **message broker** to decouple message production from mes - Python 3.8+ - `pika` Python client library installed -### Install Python Dependencies +### Install Python dependencies Installs Python and the RabbitMQ Python client needed to build a consumer. ```console @@ -42,7 +42,7 @@ sudo zypper install -y python3 python3-pip pip3 install pika ``` -### RabbitMQ Topology +### RabbitMQ topology This use case uses a direct exchange topology for exact-match routing. **Exchanges** @@ -56,7 +56,7 @@ This use case uses a direct exchange topology for exact-match routing. - Routing key: **whatsapp** – Ensures only WhatsApp-related messages are routed. - Queue: **whatsapp.notifications**– Final destination where messages are delivered for processing. -### Declare RabbitMQ Resources +### Declare RabbitMQ resources Creates the required exchange, queue, and binding for WhatsApp notifications. - `Declare exchange`: Creates a durable direct exchange named notifications to route messages using exact routing keys. @@ -142,7 +142,7 @@ Confirms topology correctness before consuming messages. +---------------+------------------------+------------------------+ ``` -### WhatsApp Worker Implementation +### WhatsApp worker implementation The worker attaches as a **blocking consumer** to the `whatsapp.notifications` queue and processes incoming messages. Create a `whatsapp_worker.py` file with the content below: @@ -202,7 +202,7 @@ print("[DEBUG] Starting consumer loop (this should block)...") channel.start_consuming() ``` -### Start the Worker +### Start the worker Run the worker in a dedicated terminal session: ```console @@ -218,17 +218,13 @@ The worker is running correctly and waiting for messages without exiting. [DEBUG] Declaring queue... [DEBUG] Setting QoS... WhatsApp Worker started. Waiting for messages... -[DEBUG] Starting consumer loop (this should BLOCK)... -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: Your order #1234 has been confirmed -[Worker] Message sent successfully -[Worker] Sending WhatsApp message to +911234567890 +[DEBUG] Starting consumer loop (this should block)... ``` The process must block without returning to the shell prompt. -### Publish a Test Message -From another terminal: Publishes a WhatsApp notification message to RabbitMQ. +### Publish a test message +From another SSH terminal: Publishes a WhatsApp notification message to RabbitMQ. ```console ./rabbitmqadmin publish \ @@ -237,7 +233,15 @@ From another terminal: Publishes a WhatsApp notification message to RabbitMQ. payload='{"phone":"+911234567890","message":"Hello from RabbitMQ"}' ``` -### Message Consumption Validation +You should see the following output from whatsapp_worker.py that is running in the first SSH terminal: + +```output +[Worker] Sending WhatsApp message to +911234567890 +[Worker] Message content: Hello from RabbitMQ +[Worker] Message sent successfully +``` + +### Message consumption validation The worker terminal displays logs similar to: ```output @@ -245,31 +249,10 @@ The worker terminal displays logs similar to: [DEBUG] Declaring queue... [DEBUG] Setting QoS... WhatsApp Worker started. Waiting for messages... -[DEBUG] Starting consumer loop (this should BLOCK)... -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: Your order #1234 has been confirmed -[Worker] Message sent successfully -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: Your order #1234 has been confirmed -[Worker] Message sent successfully -[Worker] Sending WhatsApp message to +9111 -[Worker] Message content: Test-1 -[Worker] Message sent successfully -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: Validation test -[Worker] Message sent successfully -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: Hello from RabbitMQ -[Worker] Message sent successfully +[DEBUG] Starting consumer loop (this should block)... [Worker] Sending WhatsApp message to +911234567890 [Worker] Message content: Hello from RabbitMQ [Worker] Message sent successfully -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: FINAL validation test -[Worker] Message sent successfully -[Worker] Sending WhatsApp message to +911234567890 -[Worker] Message content: FINAL validation test -[Worker] Message sent successfully ``` **What this confirms:** @@ -277,28 +260,28 @@ WhatsApp Worker started. Waiting for messages... - Queue consumption is successful - Manual acknowledgments are applied -End-to-end message flow validated. +This validates the end-to-end message flow. -### Verify Queue State +### Verify queue state ```console ./rabbitmqadmin list queues name messages consumers ``` -Expected output: +Expected output is similar to: ```output +------------------------+----------+-----------+ | name | messages | consumers | +------------------------+----------+-----------+ -| jobs | 1 | 0 | -| order.events | 1 | 0 | +| jobs | 0 | 0 | +| order.events | 2 | 0 | | testqueue | 1 | 0 | | whatsapp.notifications | 0 | 1 | +------------------------+----------+-----------+ ``` -**This confirms that:** +This confirms that: - Messages were consumed successfully - One active consumer is connected diff --git a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/_index.md b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/_index.md index a015e09318..8977695464 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/_index.md @@ -1,19 +1,15 @@ --- -title: Deploy Redis for data searching on Google Cloud C4A (Arm-based Axion VMs) - -draft: true -cascade: - draft: true +title: Deploy Redis for data searching on Google Cloud C4A minutes_to_complete: 30 -who_is_this_for: This learning path is intended for software developers deploying and optimizing Redis-based data searching workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. +who_is_this_for: This is an introductory topic for developers deploying and optimizing Redis-based data searching workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. learning_objectives: - Provision an Arm-based SUSE SLES virtual machine on Google Cloud (C4A with Axion processors) - Install Redis on a SUSE Arm64 (C4A) instance - Verify Redis functionality by running the server and performing baseline data insertion and retrieval tests on the Arm64 VM - - Measure Redis SET (write) and GET (read) performance using the official redis-benchmark tool to evaluate throughput and latency on Arm64 (Aarch64) + - Measure Redis SET (write) and GET (read) performance using the official redis-benchmark tool to evaluate throughput and latency on Arm64 (AArch64) prerequisites: - A [Google Cloud Platform (GCP)](https://cloud.google.com/free) account with billing enabled diff --git a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/background.md b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/background.md index e32b5d281a..d1afdddf1f 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/background.md +++ b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/background.md @@ -1,25 +1,25 @@ --- -title: Getting started with Redis on Google Axion C4A (Arm Neoverse-V2) +title: Get started with Redis on Google Axion C4A weight: 2 layout: "learningpathall" --- -## Google Axion C4A Arm instances in Google Cloud +## Explore Google Axion C4A Arm instances -Google Axion C4A is a family of Arm-based virtual machines built on Google’s custom Axion CPU, which is based on Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines offer strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. +Google Axion C4A is a family of Arm-based virtual machines built on Google's custom Axion CPU, based on Arm Neoverse-V2 cores. These virtual machines deliver strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. -The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. +The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of Arm architecture in Google Cloud. -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. +Learn more about Google Axion from the Google blog [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu). -## Redis +## Explore Redis -[Redis](https://redis.io/) is an open-source, **in-memory data structure store** developed under the [Redis project](https://redis.io/). It is used as a **database**, **cache**, and **message broker**, supporting a variety of data structures such as **strings**, **hashes**, **lists**, **sets**, and **sorted sets**. +[Redis](https://redis.io/) is an open-source, in-memory data structure store developed under the [Redis project](https://redis.io/). It is used as a database, cache, and message broker, supporting a variety of data structures such as strings, hashes, lists, sets, and sorted sets. -Redis is designed for **high performance**, **low latency**, and **high throughput**, making it ideal for real-time applications. It supports **persistence**, **replication**, **Lua scripting**, **transactions**, and **high availability** through Redis Sentinel and **automatic partitioning** with Redis Cluster. +Redis is designed for high performance, low latency, and high throughput, making it ideal for real-time applications. It supports persistence, replication, Lua scripting, transactions, and high availability through Redis Sentinel and automatic partitioning with Redis Cluster. -Redis is widely adopted for **caching**, **session management**, **real-time analytics**, **pub/sub messaging**, and **leaderboards** in gaming and web applications. It integrates seamlessly with programming languages like **Python**, **Java**, **Go**, and **Node.js**. +Redis is widely adopted for caching, session management, real-time analytics, pub/sub messaging, and leaderboards in gaming and web applications. It integrates seamlessly with programming languages like Python, Java, Go, and Node.js. -To learn more, visit the [Redis official website](https://redis.io/) and explore the [documentation](https://redis.io/docs/latest/). +To learn more, visit the [Redis website](https://redis.io/) and explore the [Redis documentation](https://redis.io/docs/latest/). diff --git a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/baseline.md b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/baseline.md index 21a2d88575..373f4de266 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/baseline.md @@ -1,39 +1,39 @@ --- -title: Redis Baseline Testing on Google Axion C4A Arm Virtual Machine +title: Test Redis weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Redis Baseline Testing on GCP SUSE VMs -This section performs baseline testing for Redis running on a GCP SUSE Arm64 VM, focusing on data insertion, retrieval, and search performance. +## Perform Redis baseline testing on GCP SUSE VMs +This section shows you how to perform baseline testing for Redis running on a GCP SUSE Arm64 VM, focusing on data insertion, retrieval, and search performance. ### Prerequisites This command launches the Redis server process in the background. It allows you to run subsequent commands in the same terminal session while Redis continues running. -Start the Redis service in the background: + +Start the Redis service: ```console redis-server & ``` -**Check if Redis is active and responding to commands:** - -The redis-cli ping command sends a simple health check request to the Redis server. A PONG response confirms that the server is running correctly and the client can communicate with it. +Check if Redis is active and responding to commands: ```console redis-cli ping ``` - -output: +The output is similar to: ```output PONG ``` -### Insert Sample Data -These steps populate the Redis database with a sample dataset to validate insertion performance and data persistence. You will create 10,000 key-value pairs using a simple shell loop and verify that the data has been successfully stored. +## Insert sample data + +Populate the Redis database with a sample dataset to validate insertion performance and data persistence. You create 10,000 key-value pairs using a simple shell loop and verify that the data has been successfully stored. + +Use `redis-cli` to insert 10,000 sample key-value pairs: -Use `redis-cli` to insert **10,000 sample key-value pairs**: ```console for i in $(seq 1 10000); do redis-cli SET key:$i "value-$i" > /dev/null @@ -41,24 +41,27 @@ done ``` - This command iterates through numbers **1 to 10,000**, setting each as a Redis key in the format `key:` with the corresponding value `"value-"`. - The `> /dev/null` part suppresses command output to make the insertion process cleaner and faster. +- The `> /dev/null` part suppresses command output to make the insertion process cleaner and faster. -**Verify Data Storage Count:** - -The `DBSIZE` command returns the total number of keys currently stored in the Redis database. +Verify data storage count: ```console redis-cli DBSIZE ``` +The output is similar to: + ```output (integer) 10000 ``` -Seeing `(integer) 10000` confirms that all key-value pairs were inserted successfully. -**Verify Sample Data Retrieval** +This confirms that all key-value pairs were inserted successfully. + +### Verify sample data retrieval Fetch one of the inserted keys to confirm data correctness: +Verify sample data retrieval: ```console redis-cli GET key:5000 ``` @@ -71,11 +74,12 @@ You should see an output similar to: "value-5000" ``` -### Perform Basic Data Search Tests +## Perform basic data search tests This step verifies Redis’s ability to retrieve specific data efficiently using unique keys. The `GET` command fetches the value associated with a given key from Redis. -You can test this by retrieving a known key-value pair: +Verify Redis's ability to retrieve specific data efficiently using unique keys. The `GET` command fetches the value associated with a given key from Redis. +Test this by retrieving a known key-value pair: ```console redis-cli GET key:1234 ``` @@ -87,7 +91,7 @@ You should see an output similar to: ``` This confirms that Redis is storing and retrieving data correctly from memory. -### Search for Multiple Keys Using Pattern Matching +## Search for multiple keys using pattern matching This test demonstrates how Redis can locate multiple keys that match a pattern, useful for exploratory queries or debugging. Use the `KEYS` command to search for keys matching a pattern: @@ -95,9 +99,8 @@ Use the `KEYS` command to search for keys matching a pattern: ```console redis-cli KEYS "key:1*" ``` -`KEYS` is fast but **blocks the server** when handling large datasets, so it’s not recommended in production. -You should see an output similar to: +The output is similar to: ```output 1) "key:1392" @@ -112,8 +115,11 @@ You should see an output similar to: .......... ``` -### Production-Safe Searching with SCAN -This step introduces a production-friendly method for iterating through keys without blocking Redis operations. +{{% notice Note %}} +`KEYS` is fast but blocks the server when handling large datasets, so it's not recommended in production. +{{% /notice %}} + +## Production-safe searching with SCAN Use the `SCAN` command for larger datasets — it is non-blocking and iterates safely. @@ -137,8 +143,9 @@ You should see an output similar to: Redis will return a cursor value (for example, `9792`). Continue scanning by reusing the cursor until it returns `0`, meaning the iteration is complete. -### Measure Data Retrieval Performance -This step measures how quickly Redis can retrieve a single key from memory, helping to establish a baseline for data access latency on the Arm-based VM. +## Measure data retrieval performance + +Measure how quickly Redis can retrieve a single key from memory, helping to establish a baseline for data access latency on the Arm-based VM. **Time Single Key Lookup**: Redis operations are extremely fast since data is stored in-memory. To quantify this, the Unix `time` command is used to measure the latency of retrieving a single key using `redis-cli`. diff --git a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/benchmarking.md index ba90e7ee15..7e68120cbb 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/benchmarking.md @@ -1,44 +1,45 @@ --- -title: Redis Benchmarking +title: Benchmark Redis weight: 6 ### FIXED, DO NOT MODIFY layout: learningpathall --- +## Benchmark Redis using redis-benchmark -## Redis Benchmarking by redis-benchmark -The `redis-benchmark` tool is an official performance testing utility for Redis. It helps measure to throughput (requests per second) and latency (response delay) across different workloads. +The `redis-benchmark` tool is an official performance testing utility for Redis. It measures throughput (requests per second) and latency (response delay) across different workloads. ### Prerequisites -Ensure Redis server is running and accessible: + +Before running benchmarks, verify that Redis is running and accessible: ```console redis-cli ping ``` -If you do not see a "PONG" response, please start redis: +If you don't see a `PONG` response, start Redis: ```console redis-server & redis-cli ping ``` -### Benchmark SET (Write Performance) -The `SET` benchmark helps validate Redis’s ability to handle high insertion rates efficiently on Arm-based servers. +### Benchmark SET (write performance) -The following command benchmarks data insertion performance: +Benchmark data insertion performance: ```console redis-benchmark -t set -n 100000 -c 50 ``` -**Explanation:** -- `-t set`: Runs the benchmark only for **SET** operations. -- `-n 100000`: Performs **100,000 total requests**. -- `-c 50`: Simulates **50 concurrent clients**. +This command: + +- Runs the benchmark for SET operations only (`-t set`) +- Performs 100,000 total requests (`-n 100000`) +- Simulates 50 concurrent clients (`-c 50`) -You should see an output similar to: +The output is similar to: ```output ====== SET ====== @@ -85,19 +86,20 @@ Summary: 0.170 0.056 0.167 0.183 0.191 1.095 ``` -### Benchmark GET (Read/Search Performance) -Now test data retrieval performance separately. +### Benchmark GET (read performance) + +Test data retrieval performance: ```console redis-benchmark -t get -n 100000 -c 50 -``` -**Explanation:** -- `-t get`: Runs the benchmark only for **GET** operations. -- `-n 100000`: Executes **100,000 total requests**. -- `-c 50`: Simulates **50 concurrent clients** performing reads. +Parameters: + +- `-t get`: Runs the benchmark only for GET operations. +- `-n 100000`: Executes 100,000 total requests. +- `-c 50`: Simulates 50 concurrent clients performing reads. -You should see an output similar to: +The output is similar to: ```output ====== GET ====== @@ -141,26 +143,24 @@ Summary: 0.169 0.048 0.167 0.183 0.191 0.807 ``` -### Benchmark Metrics Explanation +## Interpret the benchmark metrics -- **Throughput:** Number of operations (requests) Redis can process per second. -- **Latency:** Latency reflects the time it takes for Redis to complete a single request, measured in milliseconds (ms). -- **avg:** Average time taken to process a request across the entire test. -- **min:** Fastest observed response time (best case). -- **p50:** Median latency — 50% of requests completed faster than this value. -- **p95:** 95th percentile — 95% of requests completed faster than this value. -- **p99:** 99th percentile — shows near worst-case latency, key for reliability analysis. -- **max:** Slowest observed response time (worst case). - -### Benchmark summary -Results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE Enterprise Server): +The following table summarizes the benchmark results from the earlier run on the `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP (SUSE Enterprise Server): | Operation | Total Requests | Concurrent Clients | Avg Latency (ms) | Min (ms) | P50 (ms) | P95 (ms) | P99 (ms) | Max (ms) | Throughput (req/sec) | Description | |------------|----------------|--------------------|------------------|-----------|-----------|-----------|-----------|-----------|-----------------------|--------------| | SET | 100,000 | 50 | 0.170 | 0.056 | 0.167 | 0.183 | 0.191 | 1.095 | 149,700.61 | Measures Redis write performance using SET command | | GET | 100,000 | 50 | 0.169 | 0.048 | 0.167 | 0.183 | 0.191 | 0.807 | 150,375.94 | Measures Redis read performance using GET command | - - **High Efficiency:** Redis achieved over **150K ops/sec** on both read and write workloads, showcasing strong throughput on **Arm64 (C4A)** architecture. -- **Low Latency:** Average latency remained around **0.17 ms**, demonstrating consistent response times under concurrency. -- **Balanced Performance:** Both **SET** and **GET** operations showed nearly identical performance, indicating excellent CPU and memory optimization on **Arm64**. -- **Energy-Efficient Compute:** The **Arm-based C4A VM** delivers competitive performance-per-watt efficiency, ideal for scalable, sustainable Redis deployments. +Redis demonstrated excellent performance on the Arm64-based C4A VM, achieving over 150K operations per second for both read and write workloads with an average latency of approximately 0.17 ms. Both SET and GET operations showed nearly identical performance characteristics, indicating efficient CPU and memory optimization on the Arm architecture. The Arm-based C4A VM delivers competitive performance-per-watt efficiency, making it ideal for scalable, sustainable Redis deployments. + +## What you've accomplished and what's next + +In this section, you: +- Benchmarked Redis SET operations, achieving over 149K requests per second with 0.170 ms average latency +- Benchmarked Redis GET operations, achieving over 150K requests per second with 0.169 ms average latency +- Verified that Redis performs efficiently on Google Axion C4A Arm instances + +You've successfully benchmarked Redis on Google Cloud's C4A Arm-based virtual machines, demonstrating strong performance for in-memory data operations. + +For next steps, consider exploring Redis Cluster for distributed deployments, implementing persistence strategies for production workloads, or testing more advanced data structures like sorted sets and streams. You can also compare performance across different C4A machine types to optimize cost and performance for your specific use case. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/installation.md b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/installation.md index 30958df36d..b58c77ae0b 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/installation.md @@ -7,17 +7,18 @@ layout: learningpathall --- ## Install Redis on GCP VM -This section provides a complete guide to installing Redis on a **GCP SUSE virtual machine**. Redis will be built from source to ensure compatibility with the Arm architecture and to enable TLS support for secure connections. +This section provides a complete guide to installing Redis on a GCP SUSE virtual machine. You will build Redis from source to ensure compatibility with the Arm architecture and to enable TLS support for secure connections. -### Prerequisites -Before building Redis, update your package repositories and install essential build tools and libraries. +## Prerequisites +Before building Redis, update your package repositories and install essential build tools and libraries: ```console sudo zypper refresh sudo zypper install -y gcc gcc-c++ make tcl openssl-devel wget ``` -### Download and Extract Redis Source Code -This step downloads the Redis 8.2.2 source archive directly from the official GitHub repository, extracts the contents, and navigates into the extracted directory for compilation. +## Download and extract Redis source code + +Now download the Redis 8.2.2 source archive directly from the official GitHub repository, extract the contents, and navigate into the extracted directory for compilation: ```console wget https://github.com/redis/redis/archive/refs/tags/8.2.2.tar.gz @@ -25,20 +26,19 @@ tar -xvf 8.2.2.tar.gz cd redis-8.2.2 ``` {{% notice Note %}} -In [this](https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/improve-redis-performance-by-deploying-on-alibaba-cloud-yitian-710-instances) blog, Redis version 6.0.9 is recommended for deployment on Arm-based Alibaba Yitian 710 instances, which deliver up to 36% higher throughput and 20% lower cost compared to equivalent x86-based ECS instances, making it a more efficient and cost-effective choice for Redis workloads. +In this blog [Improve Redis performance up to 36% by deploying on Alibaba Cloud Yitian 710 instances](https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/improve-redis-performance-by-deploying-on-alibaba-cloud-yitian-710-instances), Redis version 6.0.9 is recommended for deployment on Arm-based Alibaba Yitian 710 instances. These instances deliver up to 36% higher throughput and 20% lower cost compared to equivalent x86-based ECS instances, making it a more efficient and cost-effective choice for Redis workloads. The [Arm Ecosystem Dashboard](https://developer.arm.com/ecosystem-dashboard/) recommends Redis version 6.0.9, the minimum recommended on the Arm platforms. {{% /notice %}} -### Build Redis with TLS Support -**Clean any previous build artifacts (if any):** +Clean any previous build artifacts (if any): ```console make distclean ``` This removes any residual files from a previous build, ensuring a clean build environment. -**Now build Redis dependencies and compile Redis:** +Build Redis dependencies and compile Redis: Redis relies on several third-party libraries (such as hiredis, jemalloc, and lua) to optimize performance and functionality. After building dependencies, the Redis source is compiled with BUILD_TLS=yes, enabling support for encrypted TLS connections. @@ -48,31 +48,36 @@ sudo make hiredis jemalloc linenoise lua fast_float cd .. sudo make BUILD_TLS=yes ``` -Note: The BUILD_TLS=yes flag enables TLS (SSL) support for secure Redis connections. -### Verify Redis Binary -After a successful build, check that the redis-server binary exists: +{{% notice Note %}} +The `BUILD_TLS=yes` flag enables TLS (SSL) support for secure Redis connections. +{{% /notice %}} + +## Verify Redis binary + +After a successful build, check that the `redis-server` binary exists: ``` cd src ls -l redis-server ``` -You should see a file similar to: +The output is similar to: ```output -rwxr-xr-x 1 root root 17869216 Oct 23 11:48 redis-server ``` This confirms that Redis compiled successfully and that the `redis-server` binary is present in the `/src` directory. The file’s permissions indicate it is executable. -### Install Redis System-Wide -This command installs Redis binaries (`redis-server` and `redis-cli`) into system-wide directories (typically `/usr/local/bin`), allowing you to run Redis commands from any location. -To make redis-server and redis-cli accessible globally: +## Install Redis system-wide +Use the following command to install Redis binaries (`redis-server` and `redis-cli`) globally, making them accessible from any directory: ```console sudo make install ``` +This places `redis-server` and `redis-cli` into `/usr/local/bin`, allowing you to run Redis commands from anywhere on your system. + Verify installation paths: ```console @@ -80,15 +85,17 @@ which redis-server which redis-cli ``` -Expected: +The expected output is: ```output /usr/local/bin/redis-server /usr/local/bin/redis-cli ``` -This confirms that Redis binaries are available in your system path, verifying a successful installation. -### Verify Installation +This confirms that Redis binaries are installed in your system path. + +### Verify installation + Check Redis versions: ```console @@ -96,7 +103,7 @@ redis-server --version redis-cli --version ``` -Output: +The expected output is: ```output redis-server --version diff --git a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/instance.md b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/instance.md index 4fe413e0f5..959c2667e4 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis-data-searching/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/redis-data-searching/instance.md @@ -1,5 +1,5 @@ --- -title: Create a Google Axion C4A Arm virtual machine on GCP +title: Create a Compute Engine instance weight: 3 ### FIXED, DO NOT MODIFY @@ -11,10 +11,10 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} -## Provision a Google Axion C4A Arm VM in Google Cloud Console +## Create your Google Axion C4A Arm virtual machine To create a virtual machine based on the C4A instance type: - Navigate to the [Google Cloud Console](https://console.cloud.google.com/). @@ -24,20 +24,20 @@ To create a virtual machine based on the C4A instance type: - Set **Series** to `C4A`. - Select `c4a-standard-4` for machine type. - ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") + ![Google Cloud Console showing Create Instance form with Machine configuration section displaying C4A series and c4a-standard-4 machine type selected alt-text#center](images/gcp-vm.png "Create a Google Axion C4A Arm virtual machine") - Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. -- If using use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type. -- Once appropriately selected, please Click **Select**. +- If using **SUSE Linux Enterprise Server**, select **Pay As You Go** for the license type. +- Once appropriately selected, select **Select**. - Under **Networking**, enable **Allow HTTP traffic**. -- Click **Create** to launch the instance. -- Once created, you should see a "SSH" option to the right in your list of VM instances. Click on this to launch a SSH shell into your VM instance: +- Select **Create** to launch the instance. +- Once created, you should see an **SSH** option to the right in your list of VM instances. Select this to launch an SSH shell into your VM instance: -![Invoke a SSH session via your browser alt-text#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance") +![Google Cloud Console showing VM instances list with SSH button highlighted for connecting to the running C4A instance alt-text#center](images/gcp-ssh.png "SSH connection to VM instance") -- A window from your browser should come up and you should now see a shell into your VM instance: +- A window from your browser opens and you see a shell into your VM instance: -![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") + ![Browser-based SSH terminal window showing command prompt connected to the SUSE Linux VM instance alt-text#center](images/gcp-shell.png "Terminal shell in VM instance") -Next, let's install redis! \ No newline at end of file +You're now ready to install Redis. \ No newline at end of file diff --git a/content/learning-paths/servers-and-cloud-computing/redis_tune/tune_redis.md b/content/learning-paths/servers-and-cloud-computing/redis_tune/tune_redis.md index d32f7f87a7..23c1925b17 100644 --- a/content/learning-paths/servers-and-cloud-computing/redis_tune/tune_redis.md +++ b/content/learning-paths/servers-and-cloud-computing/redis_tune/tune_redis.md @@ -10,7 +10,7 @@ Optimizing Redis allows you to gain performance improvement without scaling your ## Redis File Configuration -In the [Configure Redis single-node](https://learn.arm.com/learning-paths/servers-and-cloud-computing/redis/single-node_deployment/) section of the [Learn how to deploy Redis on Arm](https://learn.arm.com/learning-paths/servers-and-cloud-computing/redis/) learning path, a bare minimum file server configuration was discussed. In this section, a tuned file server configuration is discussed. +In the [Configure Redis single-node](/learning-paths/servers-and-cloud-computing/redis/single-node_deployment/) section of the [Learn how to deploy Redis on Arm](/learning-paths/servers-and-cloud-computing/redis/) learning path, a bare minimum file server configuration was discussed. In this section, a tuned file server configuration is discussed. ### Top Level redis.conf diff --git a/content/learning-paths/servers-and-cloud-computing/refinfra-debug/debugging-bl33-uefi-5.md b/content/learning-paths/servers-and-cloud-computing/refinfra-debug/debugging-bl33-uefi-5.md index b3b9239303..fd68b168f4 100644 --- a/content/learning-paths/servers-and-cloud-computing/refinfra-debug/debugging-bl33-uefi-5.md +++ b/content/learning-paths/servers-and-cloud-computing/refinfra-debug/debugging-bl33-uefi-5.md @@ -61,7 +61,7 @@ You can see that it has stopped at the breakpoint. Older versions of Arm Development Studio have Beta support for DWARF 5 formats. EDK2 builds the debug files in DWARF 5 format. -In order to load the debug files properly, follow the instructions [here](https://developer.arm.com/documentation/101470/2023-0/Reference/Standards-compliance-in-Arm-Debugger). +In order to load the debug files properly, follow the [Standards compliance in Arm Debugger](https://developer.arm.com/documentation/101470/2023-0/Reference/Standards-compliance-in-Arm-Debugger) instructions. These instructions state that you must enable the LLVM DWARF parser to use DWARF 5 format. To enable the LLVM DWARF parser, do the following. diff --git a/content/learning-paths/servers-and-cloud-computing/rtp-llm/_index.md b/content/learning-paths/servers-and-cloud-computing/rtp-llm/_index.md index 5dae64ab7b..10c8d1393d 100644 --- a/content/learning-paths/servers-and-cloud-computing/rtp-llm/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/rtp-llm/_index.md @@ -48,7 +48,7 @@ further_reading: type: blog - resource: title: Get started with Arm-based cloud instances - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/ + link: /learning-paths/servers-and-cloud-computing/csp/ type: website diff --git a/content/learning-paths/servers-and-cloud-computing/ruby-on-rails/instance.md b/content/learning-paths/servers-and-cloud-computing/ruby-on-rails/instance.md index b6b0e89cb2..f35a59939a 100644 --- a/content/learning-paths/servers-and-cloud-computing/ruby-on-rails/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/ruby-on-rails/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/_index.md b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/_index.md index 65a5bf5a4f..fb41b3b79f 100644 --- a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/_index.md @@ -1,13 +1,11 @@ --- title: Deploy Rust on Google Cloud C4A (Arm-based Axion VMs) -draft: true -cascade: - draft: true +description: Learn to deploy and benchmark Rust applications on Google Cloud C4A virtual machines powered by Arm-based Axion processors. minutes_to_complete: 30 -who_is_this_for: This learning path is intended for software developers deploying and optimizing Rust workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. +who_is_this_for: This is an introductory topic for developers deploying and optimizing Rust workloads on Linux/Arm64 environments, specifically using Google Cloud C4A virtual machines powered by Axion processors. learning_objectives: - Provision an Arm-based SUSE SLES virtual machine on Google Cloud (C4A with Axion processors) diff --git a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/background.md b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/background.md index f777c8c4d5..5f50e3bc7c 100644 --- a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/background.md +++ b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/background.md @@ -1,24 +1,23 @@ --- -title: Getting started with Rust on Google Axion C4A (Arm Neoverse-V2) +title: Get started with Rust on Google Axion C4A (Arm Neoverse-V2) weight: 2 layout: "learningpathall" --- -## Google Axion C4A Arm instances in Google Cloud +## Explore Google Axion C4A Arm instances in Google Cloud Google Axion C4A is a family of Arm-based virtual machines built on Google’s custom Axion CPU, which is based on Arm Neoverse-V2 cores. Designed for high-performance and energy-efficient computing, these virtual machines offer strong performance for modern cloud workloads such as CI/CD pipelines, microservices, media processing, and general-purpose applications. The C4A series provides a cost-effective alternative to x86 virtual machines while leveraging the scalability and performance benefits of the Arm architecture in Google Cloud. -To learn more about Google Axion, refer to the [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu) blog. +To learn more about Google Axion, see the Google blog [Introducing Google Axion Processors, our new Arm-based CPUs](https://cloud.google.com/blog/products/compute/introducing-googles-new-arm-based-cpu). -## Rust +## Explore Rust [Rust](https://www.rust-lang.org/) is a modern, high-performance systems programming language designed for safety, speed, and concurrency. It provides memory safety without garbage collection, making it ideal for building reliable and efficient software. -Developed by Mozilla, Rust is widely used in system-level programming, web assembly, embedded systems, and performance-critical applications. -Its strong type system and ownership model help prevent common bugs like data races and memory leaks. +Developed by Mozilla, Rust is widely used in system-level programming, web assembly, embedded systems, and performance-critical applications. Its strong type system and ownership model help prevent common bugs like data races and memory leaks. -To learn more, visit the [official Rust website](https://www.rust-lang.org/). +To learn more, visit the [Rust website](https://www.rust-lang.org/). diff --git a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/baseline.md b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/baseline.md index 9d74aaeaeb..5545b93619 100644 --- a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/baseline.md +++ b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/baseline.md @@ -1,16 +1,16 @@ --- -title: Test Rust baseline performance on Google Axion C4A Arm virtual machines +title: Perform baseline testing weight: 5 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Perform baseline testing +## Overview -You can perform baseline testing of Rust on GCP SUSE aarch64 VMs to verify installation, build functionality, and compilation performance on the Arm-based Axion C4A platform. +Now that Rust is installed, verify the installation, build functionality, and compilation performance on your Arm-based Axion C4A instance. -### Create a sample Rust program +## Create a sample Rust program Create and build a simple "Hello, World" application to verify that Rust is working correctly: @@ -33,9 +33,9 @@ The output is similar to: Hello, world! ``` -This confirms that Rust and Cargo are properly configured on your aarch64 VM. +This confirms that Rust and Cargo are properly configured on your Arm64 VM. -### Measure compilation performance +## Measure compilation performance Use the `time` command to measure compilation performance on the Arm64 processor: @@ -59,3 +59,7 @@ sys 0m0.071s ``` The timing results show that Rust compilation performs well on the Arm64 architecture, with the "real" time indicating the total elapsed time for the build process. + +## What you've accomplished and what's next + +You've successfully verified your Rust installation and measured baseline compilation performance on the C4A instance. In the next section, you'll benchmark Rust code execution using Criterion to measure runtime performance and consistency. diff --git a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/benchmarking.md b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/benchmarking.md index b99aaf2547..363a5f981f 100644 --- a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/benchmarking.md +++ b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/benchmarking.md @@ -6,11 +6,11 @@ weight: 6 layout: learningpathall --- -## Benchmark Rust performance +## Overview -This section demonstrates how to benchmark Rust performance using `cargo bench` and the Criterion library to measure code execution speed and performance consistency on aarch64 hardware. +This section demonstrates how to benchmark Rust performance using `cargo bench` and the Criterion library to measure code execution speed and performance consistency on Arm64 hardware. -### Create a benchmark project +## Create a benchmark project Create a new Rust project specifically for benchmarking: @@ -19,9 +19,9 @@ cargo new rust-benchmark cd rust-benchmark ``` -### Configure Criterion as a dependency +## Configure Criterion as a dependency -Criterion is the recommended benchmarking crate for Rust. Edit the `Cargo.toml` file in your project root directory and replace the existing content with: +Criterion is the recommended benchmarking crate for Rust. Open the `Cargo.toml` file in your project root directory and replace the existing content with: ```toml [dependencies] @@ -34,7 +34,7 @@ harness = false This configuration enables Criterion for high-precision benchmarking and disables the default test harness. -### Create the benchmark directory and file +## Create the benchmark directory and file Create the benchmark structure that Cargo expects: @@ -42,13 +42,7 @@ Create the benchmark structure that Cargo expects: mkdir benches ``` -Create a new benchmark file in the `benches/` directory: - -```console -edit benches/my_benchmark.rs -``` - -Add the following benchmark code to measure Fibonacci number calculation performance: +Create a new file named `my_benchmark.rs` in the `benches/` directory and add the following benchmark code to measure Fibonacci number calculation performance: ```rust use criterion::{black_box, Criterion, criterion_group, criterion_main}; @@ -72,7 +66,7 @@ criterion_main!(benches); This code implements a recursive Fibonacci function and measures how efficiently Rust computes the 20th Fibonacci number. The `black_box` function prevents the compiler from optimizing away the benchmark. -### Run the benchmark +## Run the benchmark Execute the benchmark using Cargo: @@ -92,29 +86,13 @@ Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) low mild ``` -### Benchmark Metrics Explanation - -- **Average Time:** Mean execution time across benchmark runs. -- **Outliers:** Represent runs significantly slower or faster than average. -- **Plotting Backend:** Used `plotters` since Gnuplot was not found. -- The results show **consistent performance** with only slight variation across 100 measurements. - -### Understand the results - -The benchmark output provides several key metrics: - -- **Average time**: Mean execution time across benchmark runs -- **Outliers**: Runs significantly slower or faster than average -- **Plotting backend**: Uses plotters since Gnuplot wasn't found - -The results show consistent performance with only slight variation across 100 measurements. - -### Performance summary +## Performance summary +The benchmark output provides several key metrics: the average time represents the mean execution time across benchmark runs, outliers identify runs that were significantly slower or faster than average, and the plotting backend indicates that plotters is being used since Gnuplot wasn't found on the system. -The following table shows results from running the benchmark on a `c4a-standard-4` (4 vCPU, 16 GB memory) aarch64 VM in GCP using SUSE: +The following table shows results from running the benchmark on a `c4a-standard-4` (4 vCPU, 16 GB memory) Arm64 VM in GCP using SUSE: | Benchmark | Average Time (µs) | Min (µs) | Max (µs) | Outliers (%) | Remarks | |---------------|------------------:|---------:|---------:|-------------:|---------| | fibonacci 20 | 12.028 | 12.026 | 12.030 | 1.00% | Stable performance with minimal variation | -The Fibonacci benchmark demonstrates consistent performance on the aarch64 platform. The average execution time of 12.028 µs indicates efficient CPU computation, while only 1% of measurements were outliers. This low variance confirms Rust's reliable execution speed and performance stability on aarch64 architecture. +The Fibonacci benchmark demonstrates consistent performance on the Arm64 platform. The average execution time of 12.028 µs indicates efficient CPU computation, while only 1% of measurements were outliers. This low variance confirms Rust's reliable execution speed and performance stability on Arm64 architecture. diff --git a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/installation.md b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/installation.md index 959917ca11..97fb23f507 100644 --- a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/installation.md +++ b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/installation.md @@ -1,3 +1,4 @@ + --- title: Install Rust weight: 4 @@ -6,11 +7,11 @@ weight: 4 layout: learningpathall --- -## Install Rust +## Overview -This section explains how to install and configure Rust on a GCP SUSE aarch64 VM, preparing your environment for building and benchmarking Rust applications. +This section explains how to install and configure Rust on your GCP SUSE Arm64 VM, preparing your environment for building and benchmarking Rust applications. -### Update your system +## Update your system Update the system and install essential build tools required for compiling Rust programs: @@ -22,7 +23,7 @@ sudo zypper install -y curl gcc make This ensures your system has the latest packages and the necessary compilation tools. -### Install Rust using rustup +## Install Rust using rustup Rust provides an official installer script via `rustup` that handles the setup automatically: @@ -32,7 +33,7 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh When prompted, select option 1 for the default installation. This installs the latest stable version of Rust along with Cargo, Rust's package manager and build system. -### Configure your environment +## Configure your environment Activate Rust's environment variables for your current shell session: @@ -42,7 +43,7 @@ source $HOME/.cargo/env This command adds the Rust toolchain to your PATH, making the `rustc` compiler and `cargo` commands available. -### Verify the installation +## Verify the installation Confirm that Rust and Cargo installed successfully by checking their versions: @@ -58,4 +59,4 @@ rustc 1.91.0 (f8297e351 2025-10-28) cargo 1.91.0 (ea2d97820 2025-10-10) ``` -Your Rust installation is now complete and ready for development on the aarch64 platform. +Your Rust installation is now complete and ready for development on your Arm64 instance. diff --git a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/instance.md index f531adfdd6..f530e66357 100644 --- a/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/rust-on-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you will learn how to provision a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` (4 vCPUs, 16 GB memory) machine type in the Google Cloud Console. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Provision a Google Axion C4A Arm VM in Google Cloud Console @@ -24,20 +24,20 @@ To create a virtual machine based on the C4A instance type: - Set **Series** to `C4A`. - Select `c4a-standard-4` for machine type. - ![Create a Google Axion C4A Arm virtual machine in the Google Cloud Console with c4a-standard-4 selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") + ![Screenshot of Google Cloud Console showing the machine configuration section with C4A series and c4a-standard-4 machine type selected alt-text#center](images/gcp-vm.png "Creating a Google Axion C4A Arm virtual machine in Google Cloud Console") -- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. -- If using use **SUSE Linux Enterprise Server**. Select "Pay As You Go" for the license type. -- Once appropriately selected, please Click **Select**. +- Under **OS and Storage**, select **Change**, then choose an Arm64-based OS image. For this Learning Path, use **SUSE Linux Enterprise Server**. +- If using **SUSE Linux Enterprise Server**, select "Pay As You Go" for the license type. +- Once appropriately selected, click **Select**. - Under **Networking**, enable **Allow HTTP traffic**. - Click **Create** to launch the instance. -- Once created, you should see a "SSH" option to the right in your list of VM instances. Click on this to launch a SSH shell into your VM instance: +- Once created, you should see a "SSH" option to the right in your list of VM instances. Click on this to launch a SSH shell into your VM instance: -![Invoke a SSH session via your browser alt-text#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance") +![Screenshot of VM instances list showing the SSH button in the Connect columnalt-text#center](images/gcp-ssh.png "Invoke a SSH session into your running VM instance") - A window from your browser should come up and you should now see a shell into your VM instance: -![Terminal Shell in your VM instance alt-text#center](images/gcp-shell.png "Terminal shell in your VM instance") +![Screenshot of terminal window showing a command-line shell interface connected to the VM instance#center](images/gcp-shell.png "Terminal shell in your VM instance") -Next, let's install rust! \ No newline at end of file +You're now ready to install Rust on your C4A instance. diff --git a/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/_index.md b/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/_index.md index 0951c01ba6..96e5953132 100644 --- a/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/_index.md @@ -12,7 +12,7 @@ learning_objectives: prerequisites: - A Windows on Arm computer such as the Lenovo Thinkpad X13s running Windows 11 or a Windows on Arm [virtual machine](/learning-paths/cross-platform/woa_azure/). - Any code editor. [Visual Studio Code for Arm64](https://code.visualstudio.com/docs/?dv=win32arm64user) is suitable. - - Completion of this [Learning Path](/learning-paths/servers-and-cloud-computing/serverless-framework-aws-intro/). + - Completion of [Deploy AWS services using the Serverless Framework](/learning-paths/servers-and-cloud-computing/serverless-framework-aws-intro/). author: Dawid Borycki @@ -37,7 +37,7 @@ operatingsystems: further_reading: - resource: title: Terraform on Azure - link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/azure-terraform/ + link: /learning-paths/servers-and-cloud-computing/azure-terraform/ type: Tutorial - resource: title: Azure Virtual Machines with Ampere Altra Arm–based processors—generally available diff --git a/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/objective.md b/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/objective.md index 743e21cca1..155a2bcc1f 100644 --- a/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/objective.md +++ b/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/objective.md @@ -16,4 +16,4 @@ In the previous [Learning Path](/learning-paths/servers-and-cloud-computing/serv While manual resource provisioning has its benefits, it can become increasingly problematic as you start deploying your applications to the cloud. As your solutions grow and become more complex, the challenges associated with manual provisioning escalate. This is where the Serverless Framework comes into play, offering a streamlined and efficient way to manage your cloud resources. -In this learning path, you will learn how to automatically deploy a multi-resource serverless solution to AWS. By leveraging the Serverless Framework, you can simplify the deployment process, enhance scalability, and reduce the operational overhead associated with managing cloud infrastructure manually. Specifically, you will create a solution composed of the DynamoDB table and the AWS Lambda function. The latter will consume the data from the table. The function will calculate the average of numerical values in the selected column. This is similar to what you learned in this [Learning Path](/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb). +In this learning path, you will learn how to automatically deploy a multi-resource serverless solution to AWS. By leveraging the Serverless Framework, you can simplify the deployment process, enhance scalability, and reduce the operational overhead associated with managing cloud infrastructure manually. Specifically, you will create a solution composed of the DynamoDB table and the AWS Lambda function. The latter will consume the data from the table. The function will calculate the average of numerical values in the selected column. This is similar to what you learned in [Integrate AWS Lambda with DynamoDB for IoT applications running Windows on Arm](/learning-paths/laptops-and-desktops/win_aws_iot_lambda_dynamodb). diff --git a/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-s3/configuration.md b/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-s3/configuration.md index e2924e6332..e2f458b5ef 100644 --- a/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-s3/configuration.md +++ b/content/learning-paths/servers-and-cloud-computing/serverless-framework-aws-s3/configuration.md @@ -275,6 +275,6 @@ The code defines the two AWS Lambda functions that interact with a DynamoDB tabl 1. `writeTemperatures` - writes a batch of random temperature records to the DynamoDB table. 2. `getAverageTemperature` - retrieves the last N temperature records from the table, calculates the average, and returns it. -They are the same as in this [Learning Path](/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/). The only difference is that you now have a helper function, **createResponse**, which is a utility function used to standardize HTTP responses for AWS Lambda functions. This function formats the response to ensure it includes the necessary HTTP status code, headers, and body, making it easier to handle CORS (Cross-Origin Resource Sharing) and JSON responses consistently across different Lambda functions. CORS is configured because the S3 bucket domain might differ from the AWS Lambda function endpoints. +They are the same as in [Deploy and integrate AWS Lambda with DynamoDB using the Serverless Framework](/learning-paths/servers-and-cloud-computing/serverless-framework-aws-lambda-dynamodb/). The only difference is that you now have a helper function, **createResponse**, which is a utility function used to standardize HTTP responses for AWS Lambda functions. This function formats the response to ensure it includes the necessary HTTP status code, headers, and body, making it easier to handle CORS (Cross-Origin Resource Sharing) and JSON responses consistently across different Lambda functions. CORS is configured because the S3 bucket domain might differ from the AWS Lambda function endpoints. The service configuration is now ready, and you can move on to prepare the website and supporting files. diff --git a/content/learning-paths/servers-and-cloud-computing/spark-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/spark-on-gcp/instance.md index aa39cf2bf1..c4009b6c08 100644 --- a/content/learning-paths/servers-and-cloud-computing/spark-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/spark-on-gcp/instance.md @@ -10,7 +10,7 @@ layout: learningpathall In this section, you learn how to provision a **Google Axion C4A Arm virtual machine** on Google Cloud Platform (GCP) using the **c4a-standard-4 (4 vCPUs, 16 GB memory)** machine type in the **Google Cloud Console**. -For background on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For background on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). ### Create a Google Axion C4A Arm VM in Google Cloud Console diff --git a/content/learning-paths/servers-and-cloud-computing/sysreport/_index.md b/content/learning-paths/servers-and-cloud-computing/sysreport/_index.md index 0dcc72db18..f0bb527d16 100644 --- a/content/learning-paths/servers-and-cloud-computing/sysreport/_index.md +++ b/content/learning-paths/servers-and-cloud-computing/sysreport/_index.md @@ -32,15 +32,15 @@ operatingsystems: further_reading: - resource: title: Perf for Linux on Arm (LinuxPerf) - link: https://learn.arm.com/install-guides/perf/ + link: /install-guides/perf/ type: website - resource: title: AWS Perf (APerf) - link: https://learn.arm.com/install-guides/aperf/ + link: /install-guides/aperf/ type: website - resource: title: Arm Performance Studio - link: https://learn.arm.com/install-guides/ams/ + link: /install-guides/ams/ type: website diff --git a/content/learning-paths/servers-and-cloud-computing/typescript-on-gcp/instance.md b/content/learning-paths/servers-and-cloud-computing/typescript-on-gcp/instance.md index 9a2aa5bf4a..2773da197a 100644 --- a/content/learning-paths/servers-and-cloud-computing/typescript-on-gcp/instance.md +++ b/content/learning-paths/servers-and-cloud-computing/typescript-on-gcp/instance.md @@ -11,7 +11,7 @@ layout: learningpathall In this section, you'll set up a Google Axion C4A Arm virtual machine on Google Cloud Platform (GCP) using the `c4a-standard-4` machine type. This instance gives you four virtual CPUs and 16 GB of memory. You'll use the Google Cloud Console to complete each step. {{% notice Note %}} -For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/csp/google/). +For support on GCP setup, see the Learning Path [Getting started with Google Cloud Platform](/learning-paths/servers-and-cloud-computing/csp/google/). {{% /notice %}} ## Create the virtual machine diff --git a/content/learning-paths/servers-and-cloud-computing/wordpress/wordpress.md b/content/learning-paths/servers-and-cloud-computing/wordpress/wordpress.md index 3d8117b837..172a70f66d 100644 --- a/content/learning-paths/servers-and-cloud-computing/wordpress/wordpress.md +++ b/content/learning-paths/servers-and-cloud-computing/wordpress/wordpress.md @@ -19,7 +19,7 @@ You will need an [Oracle OCI account](https://cloud.oracle.com/) to complete thi You can deploy an Arm (Ampere) compute instance in OCI via the console or using Terraform. -If you want to deploy a compute instance using Terraform, you can follow the Learning Path [Deploy Arm Instances on Oracle Cloud Infrastructure (OCI) using Terraform](https://learn.arm.com/learning-paths/servers-and-cloud-computing/oci-terraform/). +If you want to deploy a compute instance using Terraform, you can follow the Learning Path [Deploy Arm Instances on Oracle Cloud Infrastructure (OCI) using Terraform](/learning-paths/servers-and-cloud-computing/oci-terraform/). The compute instance should be created with Oracle Linux 9 as the operating system. diff --git a/content/lists/_index.md b/content/lists/_index.md index c6b8788394..fd4aa132c5 100644 --- a/content/lists/_index.md +++ b/content/lists/_index.md @@ -8,18 +8,18 @@ title: "Learn more about Arm Servers and Cloud Computing" |-----------------------------------|---------------| | [Improving Java performance on Neoverse N1 systems](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/java-performance-on-neoverse-n1) | Blog | | [Spark on AWS Graviton2 best practices K-Means clustering case study](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/optimize-spark-on-aws-graviton2-best-practices-k-means-clustering) | Blog | -| [Learn how to Tune MySQL](https://learn.arm.com/learning-paths/servers-and-cloud-computing/mysql_tune/) | Learning Path | -| [Learn how to Tune PostgreSQL](https://learn.arm.com/learning-paths/servers-and-cloud-computing/postgresql_tune/) | Learning Path | -| [Learn how to tune Nginx](https://learn.arm.com/learning-paths/servers-and-cloud-computing/nginx_tune/) | Learning Path | -| [Learn how to tune Envoy](https://learn.arm.com/learning-paths/servers-and-cloud-computing/envoy_tune/) | Learning Path | -| [Learn how to tune Redis](https://learn.arm.com/learning-paths/servers-and-cloud-computing/redis_tune/) | Learning Path | +| [Learn how to Tune MySQL](/learning-paths/servers-and-cloud-computing/mysql_tune/) | Learning Path | +| [Learn how to Tune PostgreSQL](/learning-paths/servers-and-cloud-computing/postgresql_tune/) | Learning Path | +| [Learn how to tune Nginx](/learning-paths/servers-and-cloud-computing/nginx_tune/) | Learning Path | +| [Learn how to tune Envoy](/learning-paths/servers-and-cloud-computing/envoy_tune/) | Learning Path | +| [Learn how to tune Redis](/learning-paths/servers-and-cloud-computing/redis_tune/) | Learning Path | | [DPDK Tuning Guide](https://developer.arm.com/documentation/109701/1-0/?lang=en) | Documentation | ## How can I migrate my application to Arm? | Migration Guides | Resource Type | |-----------------------------------|---------------| -| [Migrating applications to Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/migration/) | Learning Path | +| [Migrating applications to Arm servers](/learning-paths/servers-and-cloud-computing/migration/) | Learning Path | | [Migrating to AWS Graviton with AWS container services](https://www.youtube.com/watch?v=9JZVomrx6uQ) | Video | ## How does Arm Neoverse performance compare? diff --git a/content/migration/_index.md b/content/migration/_index.md index 85da72ff9b..d7fa740e94 100644 --- a/content/migration/_index.md +++ b/content/migration/_index.md @@ -74,7 +74,7 @@ Alibaba Cloud offers [3 instance types](https://www.alibabacloud.com/help/en/ecs {{< /tab >}} {{< /tabpane-normal >}} -Read [Get started with Servers and Cloud Computing](https://learn.arm.com/learning-paths/servers-and-cloud-computing/intro) to learn more and find additional cloud service providers. +Read [Get started with Servers and Cloud Computing](/learning-paths/servers-and-cloud-computing/intro) to learn more and find additional cloud service providers. ## STEP 1: Plan your transition @@ -85,7 +85,7 @@ Newer software is generally easier to migrate because Arm support continues to i Step one in a typical migration journey is understanding the software stack. Make notes about operating system versions, programming languages, development tools, container tools, performance analysis tools, and any other important scripts included in the project. You can reference **Migrating applications to Arm servers** below for tips on how to get started - setting up a development machine, common challenges, and tips to proceed at pace. -[Migrating applications to Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/migration/) +[Migrating applications to Arm servers](/learning-paths/servers-and-cloud-computing/migration/) ### 1.2 Discover Arm compatibility @@ -101,15 +101,15 @@ You can quickly find out if software dependencies are available for Arm using th ### 1.3 Find resources to help The below resources are curated to address migration challenges on specific cloud providers. Look through them for additional context. -- [Porting architecture specific intrinsics](https://learn.arm.com/learning-paths/cross-platform/intrinsics/) - perfect for porting intrinsics from another architecture. -- [Arm software install guides](https://learn.arm.com/install-guides) - good for quickly installing common tools and software. +- [Porting architecture specific intrinsics](/learning-paths/cross-platform/intrinsics/) - perfect for porting intrinsics from another architecture. +- [Arm software install guides](/install-guides) - good for quickly installing common tools and software. - [simd.info](https://simd.info/) - a searchable reference tool for C intrinsics for SIMD engines. -- [migrate-ease](https://github.com/migrate-ease/migrate-ease) - scan your source code for suggestions to migrate to Arm Neoverse, get started [here](https://learn.arm.com/learning-paths/servers-and-cloud-computing/migrate-ease/). +- [migrate-ease](https://github.com/migrate-ease/migrate-ease) - scan your source code for suggestions to migrate to Arm Neoverse, get started with [Migrate applications to Arm servers using migrate-ease](/learning-paths/servers-and-cloud-computing/migrate-ease/). - [Arm Infrastructure Solutions blog](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/) - an Arm-specific technical blog. -- [Arm Learning Paths for Servers and Cloud](https://learn.arm.com/learning-paths/servers-and-cloud-computing/) - general tutorials for Arm servers. You can search for specific cloud service providers, including [AWS](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=aws/#), [Google Cloud](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=google-cloud/#), [Microsoft Azure](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=microsoft-azure/#), and [Oracle](https://learn.arm.com/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=oracle/#). +- [Arm Learning Paths for Servers and Cloud](/learning-paths/servers-and-cloud-computing/) - general tutorials for Arm servers. You can search for specific cloud service providers, including [AWS](/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=aws/#), [Google Cloud](/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=google-cloud/#), [Microsoft Azure](/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=microsoft-azure/#), and [Oracle](/learning-paths/servers-and-cloud-computing/?cloud-service-providers-filter=oracle/#). -- AWS has additional resources such as the [Porting Advisor for Graviton](https://learn.arm.com/install-guides/porting-advisor/) and [AWS Graviton Technical Guide](https://github.com/aws/aws-graviton-getting-started). +- AWS has additional resources such as the [Porting Advisor for Graviton](/install-guides/porting-advisor/) and [AWS Graviton Technical Guide](https://github.com/aws/aws-graviton-getting-started). Lastly, below are some specific migration helpers for various types of software: @@ -121,12 +121,12 @@ Which tools are available for building and running containers on Arm servers? | Tool | Learning Paths | Other Content (Blogs/Videos) | |-----------|----------------|----------------------------------------| -| Docker | [Learn how to use Docker](https://learn.arm.com/learning-paths/cross-platform/docker/) | [How to build cloud-native applications for multi-architecture infrastructure](https://stackoverflow.blog/2024/02/05/how-to-build-cloud-native-applications-for-multi-architecture-infrastructure/) -| AWS CodeBuild | [Build and share Docker images using AWS CodeBuild](https://learn.arm.com/learning-paths/servers-and-cloud-computing/codebuild/) | | -| Docker Build Cloud | [Build multi-architecture container images with Docker Build Cloud](https://learn.arm.com/learning-paths/cross-platform/docker-build-cloud/) | [Supercharge your Arm builds with Docker Build Cloud: Efficiency meets performance](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/supercharge-arm-builds-with-docker-build-cloud) | -| GitHub Actions (GitHub runners) | [Build multi-architecture container images with GitHub Arm-hosted runners](https://learn.arm.com/learning-paths/cross-platform/github-arm-runners/) | [Arm64 on GitHub Actions: Powering faster, more efficient build systems](https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/) | -| GitHub Actions (AWS Graviton runners) | [Managed, self-hosted Arm runners for GitHub Actions](https://learn.arm.com/learning-paths/servers-and-cloud-computing/github-actions-runner/) | -| GitLab (GitLab runners) | [Build a CI/CD pipeline with GitLab on Google Axion](https://learn.arm.com/learning-paths/cross-platform/gitlab/) | | +| Docker | [Learn how to use Docker](/learning-paths/cross-platform/docker/) | [How to build cloud-native applications for multi-architecture infrastructure](https://stackoverflow.blog/2024/02/05/how-to-build-cloud-native-applications-for-multi-architecture-infrastructure/) +| AWS CodeBuild | [Build and share Docker images using AWS CodeBuild](/learning-paths/servers-and-cloud-computing/codebuild/) | | +| Docker Build Cloud | [Build multi-architecture container images with Docker Build Cloud](/learning-paths/cross-platform/docker-build-cloud/) | [Supercharge your Arm builds with Docker Build Cloud: Efficiency meets performance](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/supercharge-arm-builds-with-docker-build-cloud) | +| GitHub Actions (GitHub runners) | [Build multi-architecture container images with GitHub Arm-hosted runners](/learning-paths/cross-platform/github-arm-runners/) | [Arm64 on GitHub Actions: Powering faster, more efficient build systems](https://github.blog/news-insights/product-news/arm64-on-github-actions-powering-faster-more-efficient-build-systems/) | +| GitHub Actions (AWS Graviton runners) | [Managed, self-hosted Arm runners for GitHub Actions](/learning-paths/servers-and-cloud-computing/github-actions-runner/) | +| GitLab (GitLab runners) | [Build a CI/CD pipeline with GitLab on Google Axion](/learning-paths/cross-platform/gitlab/) | | {{< /tab >}} @@ -136,16 +136,16 @@ Which programming languages work on Arm servers? - Nearly all of them. | Languages | Learning Paths | Other Content (Blogs/Videos) | |-----------|----------------|----------------------------------------| -| C/C++ | [Migrating C/C++ applications](https://learn.arm.com/learning-paths/servers-and-cloud-computing/migration/c/) | [What is new in LLVM 18?](https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/p1-whats-new-in-llvm-18) | -| Rust | [Rust Install Guide](https://learn.arm.com/install-guides/rust/) | [Neon Intrinsics in Rust](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/rust-neon-intrinsics) | -| Java | [Java Install Guide](https://learn.arm.com/install-guides/java/) | [Improving Java performance on Neoverse N1 systems](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/java-performance-on-neoverse-n1) | -| | [Migrating Java applications](https://learn.arm.com/learning-paths/servers-and-cloud-computing/migration/java/) | [Java Vector API on AArch64](https://community.arm.com/arm-community-blogs/b/high-performance-computing-blog/posts/java-vector-api-on-aarch64) | -| | [Run Java applications on Google Axion](https://learn.arm.com/learning-paths/servers-and-cloud-computing/java-on-axion/)| [Java on Graviton](https://github.com/aws/aws-graviton-getting-started/blob/main/java.md) | +| C/C++ | [Migrating C/C++ applications](/learning-paths/servers-and-cloud-computing/migration/c/) | [What is new in LLVM 18?](https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/p1-whats-new-in-llvm-18) | +| Rust | [Rust Install Guide](/install-guides/rust/) | [Neon Intrinsics in Rust](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/rust-neon-intrinsics) | +| Java | [Java Install Guide](/install-guides/java/) | [Improving Java performance on Neoverse N1 systems](https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/java-performance-on-neoverse-n1) | +| | [Migrating Java applications](/learning-paths/servers-and-cloud-computing/migration/java/) | [Java Vector API on AArch64](https://community.arm.com/arm-community-blogs/b/high-performance-computing-blog/posts/java-vector-api-on-aarch64) | +| | [Run Java applications on Google Axion](/learning-paths/servers-and-cloud-computing/java-on-axion/)| [Java on Graviton](https://github.com/aws/aws-graviton-getting-started/blob/main/java.md) | | | | [Optimizing Java Workloads on Azure General Purpose D-series v5 VMs with Microsoft's Build of OpenJDK](https://techcommunity.microsoft.com/t5/azure-compute-blog/optimizing-java-workloads-on-azure-general-purpose-d-series-v5/ba-p/3827610) | | | | [Improving Java performance on OCI Ampere A1 compute instances](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/performance-of-specjbb2015-on-oci-ampere-a1-compute-instances) | -| Go | [Go Install Guide](https://learn.arm.com/install-guides/go/) | [Making your Go workloads up to 20% faster with Go 1.18 and AWS Graviton](https://aws.amazon.com/blogs/compute/making-your-go-workloads-up-to-20-faster-with-go-1-18-and-aws-graviton/)| -| .NET | [.NET Install Guide](https://learn.arm.com/install-guides/dotnet/) | [Arm64 Performance Improvements in .NET 7](https://devblogs.microsoft.com/dotnet/arm64-performance-improvements-in-dotnet-7/) | -| | [Deploy .NET application on Azure Cobalt 100 VMs](https://learn.arm.com/learning-paths/servers-and-cloud-computing/azure-cobalt-cicd-aks/) | [Arm64 Performance Improvements in .NET 8](https://devblogs.microsoft.com/dotnet/this-arm64-performance-in-dotnet-8/) | +| Go | [Go Install Guide](/install-guides/go/) | [Making your Go workloads up to 20% faster with Go 1.18 and AWS Graviton](https://aws.amazon.com/blogs/compute/making-your-go-workloads-up-to-20-faster-with-go-1-18-and-aws-graviton/)| +| .NET | [.NET Install Guide](/install-guides/dotnet/) | [Arm64 Performance Improvements in .NET 7](https://devblogs.microsoft.com/dotnet/arm64-performance-improvements-in-dotnet-7/) | +| | [Deploy .NET application on Azure Cobalt 100 VMs](/learning-paths/servers-and-cloud-computing/azure-cobalt-cicd-aks/) | [Arm64 Performance Improvements in .NET 8](https://devblogs.microsoft.com/dotnet/this-arm64-performance-in-dotnet-8/) | | Python | | [Python on Arm](https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/python-on-arm)| | PHP | | [Improving performance of PHP for Arm64 and impact on AWS Graviton2 based EC2 instances](https://aws.amazon.com/blogs/compute/improving-performance-of-php-for-arm64-and-impact-on-amazon-ec2-m6g-instances/) | @@ -156,21 +156,21 @@ Which key libraries are optimized for Arm servers? | Library/Framework | Learn More | Blogs | |-------------------|------------|-------| -| x264/x265 | [Run x265 (H.265 codec) on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/codec/) | [Improve video encoding price/performance by up to 36% with Arm Neoverse based Amazon EC2 C6g instances](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/thirty-six-percent-better-video-encoding-with-aws-graviton2_2d00_based-c6g) | +| x264/x265 | [Run x265 (H.265 codec) on Arm servers](/learning-paths/servers-and-cloud-computing/codec/) | [Improve video encoding price/performance by up to 36% with Arm Neoverse based Amazon EC2 C6g instances](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/thirty-six-percent-better-video-encoding-with-aws-graviton2_2d00_based-c6g) | | | | [Reduce H.265 High-Res Encoding Costs by over 80% with AWS Graviton2](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/reduce-h-265-high-res-encoding-costs-by-over-80-with-aws-graviton2-1207706725) | | | | [Ampere Altra Max Delivers Sustainable High-Resolution H.265 Encoding](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/ampere-altra-max-delivers-sustainable-high-resolution-h-265-video-encoding-without-compromise) | | | | [OCI Ampere A1 Compute instances can significantly reduce video encoding costs versus modern CPUs](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/oracle-cloud-infrastructure-arm-based-a1) | -| ArmPL | [Arm Performance Libraries install guide](https://learn.arm.com/install-guides/armpl/) | [Arm Compiler for Linux and Arm Performance Libraries 24.04](https://community.arm.com/arm-community-blogs/b/high-performance-computing-blog/posts/arm-compiler-for-linux-and-arm-performance-libraries-24-04) | -| ArmRAL | [Get started with the Arm 5G RAN Acceleration Library (ArmRAL)](https://learn.arm.com/learning-paths/servers-and-cloud-computing/ran/) | [The next chapter for Arm RAN Acceleration Library: Open-sourcing the code base & accelerating adoption](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/arm-ral-is-now-open-source) | +| ArmPL | [Arm Performance Libraries install guide](/install-guides/armpl/) | [Arm Compiler for Linux and Arm Performance Libraries 24.04](https://community.arm.com/arm-community-blogs/b/high-performance-computing-blog/posts/arm-compiler-for-linux-and-arm-performance-libraries-24-04) | +| ArmRAL | [Get started with the Arm 5G RAN Acceleration Library (ArmRAL)](/learning-paths/servers-and-cloud-computing/ran/) | [The next chapter for Arm RAN Acceleration Library: Open-sourcing the code base & accelerating adoption](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/arm-ral-is-now-open-source) | | OpenSSL | | | -| VP9 | [Run the AV1 and VP9 codecs on Arm Linux](https://learn.arm.com/learning-paths/servers-and-cloud-computing/codec1/) | [Arm-based cloud instances outperform x86 instances by up to 64% on VP9 encoding](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/arm-outperforms-x86-by-up-to-64-percent-on-vp9) | +| VP9 | [Run the AV1 and VP9 codecs on Arm Linux](/learning-paths/servers-and-cloud-computing/codec1/) | [Arm-based cloud instances outperform x86 instances by up to 64% on VP9 encoding](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/arm-outperforms-x86-by-up-to-64-percent-on-vp9) | | ISA-L | | | | IPSEC-MB | | | -| AV1 | [Run the AV1 and VP9 codecs on Arm Linux](https://learn.arm.com/learning-paths/servers-and-cloud-computing/codec1/) | | +| AV1 | [Run the AV1 and VP9 codecs on Arm Linux](/learning-paths/servers-and-cloud-computing/codec1/) | | | SLEEF | | [A New Pulse for SLEEF](https://sleef.org/2024/10/02/new-pulse.html) | | AES | | [AWS Graviton3 delivers leading AES-GCM encryption performance](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/aes-gcm-optimizations-for-armv8-4-on-neoverse-v1-graviton3) | -| Snappy | [Measure performance of compression libraries on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/snappy/) | [Comparing data compression algorithm performance on AWS Graviton2](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/comparing-data-compression-algorithm-performance-on-aws-graviton2-342166113) | -| Cloudflare zlib | [Learn how to build and use Cloudflare zlib on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/zlib/) | | +| Snappy | [Measure performance of compression libraries on Arm servers](/learning-paths/servers-and-cloud-computing/snappy/) | [Comparing data compression algorithm performance on AWS Graviton2](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/comparing-data-compression-algorithm-performance-on-aws-graviton2-342166113) | +| Cloudflare zlib | [Learn how to build and use Cloudflare zlib on Arm servers](/learning-paths/servers-and-cloud-computing/zlib/) | | {{< /tab >}} @@ -180,17 +180,17 @@ Which databases are available on Arm servers? | Database | Learning Paths | Other Content (Blogs/Videos) | |-----------|----------------|----------------------------------------| -| MySQL | [Deploy WordPress with MySQL on Elastic Kubernetes Service (EKS)](https://learn.arm.com/learning-paths/servers-and-cloud-computing/eks/) | | -| MySQL | [Learn how to deploy MySQL](https://learn.arm.com/learning-paths/servers-and-cloud-computing/mysql/) | | -| MySQL | [Benchmarking MySQL with Sysbench](https://learn.arm.com/learning-paths/servers-and-cloud-computing/mysql_benchmark/) | | -| MySQL | [Learn how to Tune MySQL](https://learn.arm.com/learning-paths/servers-and-cloud-computing/mysql_tune/) | | -| PostgreSQL | [Learn how to deploy PostgreSQL](https://learn.arm.com/learning-paths/servers-and-cloud-computing/postgresql/) | | -| Flink | [Benchmark the performance of Flink on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/flink/) | -| Clickhouse | [Measure performance of ClickHouse on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/clickhouse/) | [Improve ClickHouse Performance up to 26% by using AWS Graviton3](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/improve-clickhouse-performance-up-to-26-by-using-aws-graviton3) | -| MongoDB | [Test the performance of MongoDB on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/mongodb/) | [MongoDB performance on Arm Neoverse based AWS Graviton2 processors](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/mongodb-performance-on-aws-with-the-arm-graviton2) | -| Redis | [Deploy Redis on Arm](https://learn.arm.com/learning-paths/servers-and-cloud-computing/redis/) | [Improve Redis performance up to 36% by deploying on Alibaba Cloud Yitian 710 instances](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/improve-redis-performance-by-deploying-on-alibaba-cloud-yitian-710-instances) | -| Spark | [Learn how to deploy Spark on AWS Graviton2](https://learn.arm.com/learning-paths/servers-and-cloud-computing/spark/) | [Spark on AWS Graviton2 best practices: K-Means clustering case study](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/optimize-spark-on-aws-graviton2-best-practices-k-means-clustering) | -| MariaDB | [Deploy MariaDB on Arm servers](https://learn.arm.com/learning-paths/servers-and-cloud-computing/mariadb/) | +| MySQL | [Deploy WordPress with MySQL on Elastic Kubernetes Service (EKS)](/learning-paths/servers-and-cloud-computing/eks/) | | +| MySQL | [Learn how to deploy MySQL](/learning-paths/servers-and-cloud-computing/mysql/) | | +| MySQL | [Benchmarking MySQL with Sysbench](/learning-paths/servers-and-cloud-computing/mysql_benchmark/) | | +| MySQL | [Learn how to Tune MySQL](/learning-paths/servers-and-cloud-computing/mysql_tune/) | | +| PostgreSQL | [Learn how to deploy PostgreSQL](/learning-paths/servers-and-cloud-computing/postgresql/) | | +| Flink | [Benchmark the performance of Flink on Arm servers](/learning-paths/servers-and-cloud-computing/flink/) | +| Clickhouse | [Measure performance of ClickHouse on Arm servers](/learning-paths/servers-and-cloud-computing/clickhouse/) | [Improve ClickHouse Performance up to 26% by using AWS Graviton3](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/improve-clickhouse-performance-up-to-26-by-using-aws-graviton3) | +| MongoDB | [Test the performance of MongoDB on Arm servers](/learning-paths/servers-and-cloud-computing/mongodb/) | [MongoDB performance on Arm Neoverse based AWS Graviton2 processors](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/mongodb-performance-on-aws-with-the-arm-graviton2) | +| Redis | [Deploy Redis on Arm](/learning-paths/servers-and-cloud-computing/redis/) | [Improve Redis performance up to 36% by deploying on Alibaba Cloud Yitian 710 instances](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/improve-redis-performance-by-deploying-on-alibaba-cloud-yitian-710-instances) | +| Spark | [Learn how to deploy Spark on AWS Graviton2](/learning-paths/servers-and-cloud-computing/spark/) | [Spark on AWS Graviton2 best practices: K-Means clustering case study](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/optimize-spark-on-aws-graviton2-best-practices-k-means-clustering) | +| MariaDB | [Deploy MariaDB on Arm servers](/learning-paths/servers-and-cloud-computing/mariadb/) | | Elasticsearch/Opensearch | | | Spark+Gluten+Velox | | | Pinecone | | @@ -201,9 +201,9 @@ Which databases are available on Arm servers? Which software helps me build web applications on Arm servers? | Software | Learning Paths | Other Content (Blogs/Videos) | |-----------|----------------|----------------------------------------| -| Nginx | [Learn how to deploy Nginx](https://learn.arm.com/learning-paths/servers-and-cloud-computing/nginx/) | [Nginx Performance on AWS Graviton3](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/nginx-performance-on-graviton-3) | -| | [Learn how to tune Nginx](https://learn.arm.com/learning-paths/servers-and-cloud-computing/nginx_tune/) | | -| Django | [Learn how to deploy a Django application](https://learn.arm.com/learning-paths/servers-and-cloud-computing/django/) | | +| Nginx | [Learn how to deploy Nginx](/learning-paths/servers-and-cloud-computing/nginx/) | [Nginx Performance on AWS Graviton3](https://community.arm.com/arm-community-blogs/b/infrastructure-solutions-blog/posts/nginx-performance-on-graviton-3) | +| | [Learn how to tune Nginx](/learning-paths/servers-and-cloud-computing/nginx_tune/) | | +| Django | [Learn how to deploy a Django application](/learning-paths/servers-and-cloud-computing/django/) | | {{< /tab >}} @@ -213,7 +213,7 @@ Which networking software works on Arm servers? | Software | Learning Paths | Other Content (Blogs/Videos) | |-----------|----------------|----------------------------------------| -| Vectorscan | [Install Vectorscan (Hyperscan on Arm) and use it with Snort 3](https://learn.arm.com/learning-paths/servers-and-cloud-computing/vectorscan/) | | +| Vectorscan | [Install Vectorscan (Hyperscan on Arm) and use it with Snort 3](/learning-paths/servers-and-cloud-computing/vectorscan/) | | | DPDK | | [DPDK Tuning Guide](https://developer.arm.com/documentation/109701/1-0/?lang=en) | {{< /tab >}} @@ -250,12 +250,12 @@ Once the application is running you can measure performance. This can be as simp You may have some performance analysis methodologies you already follow, continue to use those. Below are some additional performance analysis tips and methodologies specific to Arm-based servers: -- [Learn the Arm Neoverse N1 performance analysis methodology](https://learn.arm.com/learning-paths/servers-and-cloud-computing/top-down-n1/) -- [Profiling for Neoverse with Streamline CLI Tools](https://learn.arm.com/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/) -- [Learn how to optimize an application with BOLT](https://learn.arm.com/learning-paths/servers-and-cloud-computing/bolt/) -- [How to use the Arm Performance Monitoring Unit and System Counter](https://learn.arm.com/learning-paths/servers-and-cloud-computing/arm_pmu/) +- [Learn the Arm Neoverse N1 performance analysis methodology](/learning-paths/servers-and-cloud-computing/top-down-n1/) +- [Profiling for Neoverse with Streamline CLI Tools](/learning-paths/servers-and-cloud-computing/profiling-for-neoverse/) +- [Learn how to optimize an application with BOLT](/learning-paths/servers-and-cloud-computing/bolt/) +- [How to use the Arm Performance Monitoring Unit and System Counter](/learning-paths/servers-and-cloud-computing/arm_pmu/) - [NVIDIA Grace CPU Benchmarking Guide](https://nvidia.github.io/grace-cpu-benchmarking-guide/index.html) -- [Learn about Large System Extensions (LSE)](https://learn.arm.com/learning-paths/servers-and-cloud-computing/lse/) +- [Learn about Large System Extensions (LSE)](/learning-paths/servers-and-cloud-computing/lse/) ### 3.2 Ask for help