From 2592e76e137d7394c65fe6c08d103f493f8d47d4 Mon Sep 17 00:00:00 2001 From: Ajay Dhangar Date: Tue, 27 Jan 2026 21:14:01 +0530 Subject: [PATCH 1/2] done advance toipic of ml --- .../ai-agents/agent-architecture.mdx | 108 ++++++++++++++++ .../ai-agents/ai-agent-use-cases.mdx | 114 +++++++++++++++++ .../ai-agents/autonomous-task-agents.mdx | 89 +++++++++++++ .../ai-agents/llm-powered-agents.mdx | 107 ++++++++++++++++ .../ai-agents/memory-in-agents.mdx | 121 ++++++++++++++++++ .../ai-agents/multi-agent-systems.mdx | 111 ++++++++++++++++ .../ai-agents/planning-and-reasoning.mdx | 97 ++++++++++++++ .../ai-agents/tool-using-agents.mdx | 112 ++++++++++++++++ .../ai-agents/types-of-ai-agents.mdx | 96 ++++++++++++++ .../ai-agents/what-are-ai-agents.mdx | 88 +++++++++++++ .../ai-engineer-roadmap.mdx | 96 ++++++++++++++ 11 files changed, 1139 insertions(+) diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/agent-architecture.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/agent-architecture.mdx index e69de29..dd43320 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/agent-architecture.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/agent-architecture.mdx @@ -0,0 +1,108 @@ +--- +title: "AI Agent Architecture: The Blueprint of Autonomy" +sidebar_label: Agent Architecture +description: "A deep dive into the structural components of AI agents, including the reasoning core, planning modules, and memory systems." +tags: [ai-agents, architecture, llms, cognitive-architecture, rag] +--- + +While a simple Large Language Model (LLM) is a statistical engine for predicting the next token, an **AI Agent Architecture** is a cognitive framework that wraps around the LLM to give it purpose, memory, and the ability to act. + +Building an agent is less about training a model and more about **system design**. + +## 1. The Four-Layer Framework + +Most modern AI agents (like those built with LangChain or AutoGPT) follow a standardized architecture composed of four main modules: + +### A. The Brain (Reasoning Core) +The LLM serves as the central processing unit. It is responsible for parsing instructions, generating plans, and deciding which tools to use. +* **Key Task:** Converting a vague user request into a structured set of logical steps. + +### B. Planning Module +The agent must break down a complex goal (e.g., "Write a research paper") into smaller sub-tasks. +* **Chain of Thought (CoT):** Encouraging the model to "think step-by-step." +* **Reflection/Self-Criticism:** The agent looks at its own plan or output and corrects errors before finalizing. + +### C. Memory Module +An agent needs to remember what it has done to avoid loops and maintain context. +* **Short-term Memory:** The immediate conversation history (context window). +* **Long-term Memory:** External storage (usually a **Vector Database**) where the agent can retrieve relevant documents or past experiences via RAG (Retrieval-Augmented Generation). + +### D. Action/Tool Layer +This is the interface between the agent and the outside world. +* **Tools:** Set of APIs (Search, Calculator, Calendar) or code executors. +* **Output:** The agent generates a structured command (like JSON) that triggers a real-world action. + +## 2. Advanced Architectural Flow + +The following diagram illustrates how information flows through the agent's internal components during a single task. + +```mermaid +graph TD + User[User Goal] --> Brain[Brain: LLM Reasoning] + + subgraph Cognitive_Process [Internal Reasoning] + Brain --> Plan[Planning: Task Decomposition] + Plan --> Memory_Access[Memory: Retrieve Past Context] + Memory_Access --> Reflect[Reflection: Verify Logic] + end + + Reflect --> Action{Action Needed?} + + subgraph Execution [External Interface] + Action -- Yes --> Tools[Tools: Web Search, Python, APIs] + Tools --> Obs[Observation: Result from World] + end + + Obs --> Brain + Action -- No --> Final[Final Response to User] + + style User fill:#e1f5fe,stroke:#01579b,color:#333 + style Cognitive_Process fill:#fff3e0,stroke:#ef6c00,color:#333 + style Execution fill:#f3e5f5,stroke:#7b1fa2,color:#333 + style Final fill:#c8e6c9,stroke:#2e7d32,color:#333 + +``` + +## 3. Cognitive Architectures: ReAct + +One of the most popular architectures for agents is the **ReAct** (Reason + Act) pattern. It forces the agent to document its "thoughts" before taking an action. + +**Example Flow:** + +1. **Thought:** "The user wants to know the weather in Tokyo. I need to find a weather API." +2. **Action:** `get_weather(city="Tokyo")` +3. **Observation:** "Tokyo: 22°C, Partly Cloudy." +4. **Thought:** "I have the information. I can now answer the user." + +## 4. Memory Architectures: Short vs. Long Term + +Managing memory is the biggest challenge in agent architecture. + +| Memory Type | Implementation | Purpose | +| --- | --- | --- | +| **Short-term** | Context Window | Keeps track of the current conversation flow. | +| **Long-term** | Vector DB (Pinecone/Milvus) | Stores "memories" as embeddings for later retrieval. | +| **Procedural** | System Prompt | The "hard-coded" instructions on how the agent should behave. | + +## 5. Multi-Agent Orchestration + +In complex scenarios, a single agent's architecture might be insufficient. Instead, we use a **Manager-Worker** architecture: + +1. **Manager Agent:** Orchestrates the goal and delegates sub-tasks. +2. **Worker Agents:** Specialized agents (e.g., a "Coder Agent," a "Reviewer Agent," and a "Researcher Agent"). + +## 6. Challenges in Agent Design + +* **Infinite Loops:** The agent gets stuck repeating the same unsuccessful action. +* **Context Overflow:** Long-term memory retrieval provides too much irrelevant information, confusing the brain. +* **Reliability:** The LLM may hallucinate that a tool exists or format a tool call incorrectly. + +## References + +* **Original Paper:** [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) +* **AutoGPT:** [An Experimental Open-Source Objective-Driven AI Agent](https://github.com/Significant-Gravitas/Auto-GPT) +* **LangChain:** [Conceptual Documentation on Agents](https://python.langchain.com/docs/modules/agents/) + +--- + +**Now that you understand the internal architecture, how do these agents actually execute code or call APIs?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/ai-agent-use-cases.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/ai-agent-use-cases.mdx index e69de29..f5c877f 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/ai-agent-use-cases.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/ai-agent-use-cases.mdx @@ -0,0 +1,114 @@ +--- +title: "AI Agent Use Cases: From Theory to Reality" +sidebar_label: Use Cases +description: "Real-world applications of autonomous and multi-agent systems across software development, finance, healthcare, and business operations." +tags: [ai-agents, use-cases, industry-ai, automation, devin, agentforce] +--- + +By 2026, the shift from "Chatbots" to **Agents** has reached a critical tipping point. While chatbots are optimized for **conversation**, Agents are designed for **operation**. They don't just provide information; they execute multi-step workflows across diverse software ecosystems. + +## 1. Software Engineering & DevOps +This is the most mature domain for agentic AI. Agents have evolved from "coding assistants" to "digital coworkers" capable of managing the entire Software Development Life Cycle (SDLC). + +* **Autonomous Engineering:** Agents like **Devin** or **GitHub Copilot Workspace** can ingest a Jira ticket, clone a repository, identify the bug, write a fix, and submit a Pull Request—all while running unit tests to ensure no regressions occur. +* **Self-Healing Infrastructure:** SRE (Site Reliability Engineering) agents monitor server logs in real-time. If they detect a memory leak or a DDoS attack, they can autonomously restart services, scale resources, or update firewall rules. +* **Automated QA:** Agents can browse a web application like a human, identifying edge cases and writing complex Selenium or Playwright tests without manual intervention. + +## 2. Customer Service: The "Level 3" Revolution +We are moving beyond rigid FAQ bots toward **Agentic Support**—systems that possess the authority and tools to actually solve user problems. + +* **End-to-End Resolution:** Instead of explaining *how* to change a flight, the agent connects to the Global Distribution System (GDS), checks availability, processes the payment, and **issues the new ticket**. +* **Proactive Retention:** Agents monitor customer behavior. If a high-value user hasn't logged in for weeks, the agent can reach out with a personalized, goal-oriented incentive to prevent churn. +* **Sentiment-Driven Escalation:** Agents analyze tone and frustration levels. If a situation becomes too complex, they autonomously escalate to a human manager with a concise summary of the case. + +## 3. Finance and Trading +In high-stakes environments, utility-based agents excel at optimizing trade-offs between risk, speed, and reward. + +* **Autonomous Fraud Investigation:** Unlike static rule-based systems, agents act as "investigators," correlating data across internal ledgers, social media, and dark web monitors to flag and pause suspicious transactions. +* **Hyper-Personalized Wealth Management:** Agents create investment strategies by analyzing global market trends alongside an individual's specific tax constraints and life goals (e.g., "Adjust my portfolio to pay for a house in 3 years"). +* **Real-time Compliance:** Agents act as constant auditors, scanning thousands of communications and trades to ensure adherence to SEC, GDPR, or MiFID II regulations. + +## 4. Healthcare Administration & Research +Agents are being deployed to solve the "Administrative Burden" that leads to physician burnout and slow drug discovery. + +* **Autonomous Documentation:** During a consultation, an agent "listens" to the dialogue and autonomously drafts the clinical notes, updates the Electronic Health Record (EHR), and flags potential drug-drug interactions. +* **Patient Triage:** Agents interact with patients before they see a doctor, collecting symptoms and prioritizing cases based on urgency using clinical protocols. +* **AI-Driven Lab Discovery:** Research agents (like those used at **Genentech**) manage complex lab workflows, searching through millions of publications to identify promising molecular structures for testing. + +## 5. Enterprise Operations: The "Glue" Agent +Agents act as a bridge between disconnected SaaS tools (Salesforce, Slack, Gmail, Jira) to automate complex business processes. + +| Use Case | Agent Task | Common Tools Used | +| :--- | :--- | :--- | +| **Sales Ops** | Lead enrichment and personalized outreach. | LinkedIn API, CRM, Gmail | +| **HR Tech** | Screening resumes and scheduling interviews. | PDF Parser, Google Calendar | +| **Supply Chain** | Monitoring inventory and reordering parts. | ERP Systems, Email, Web Search | + +## 6. Mapping the Spectrum of Autonomy + +The following diagram illustrates where different use cases sit on the spectrum of "Simple Reactivity" to "Fully Autonomous Missions." + +```mermaid +graph LR + subgraph Low_Autonomy [Reactive] + QA[Simple Q&A Chatbots] + SUM[Document Summary] + end + + subgraph Medium_Autonomy [Task-Oriented] + T1[Meeting Scheduler] + T2[Lead Enrichment] + end + + subgraph High_Autonomy [Goal-Oriented] + A1[Autonomous Devs - Devin] + A2[Market Research Squads] + A3[Cybersecurity Red-Teaming] + end + + Low_Autonomy --> Medium_Autonomy + Medium_Autonomy --> High_Autonomy + + style High_Autonomy fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#333 + style Low_Autonomy fill:#f5f5f5,stroke:#333,color:#333 + +``` + +## 7. The "Agentic Shift" in Industry + +| Industry | Before AI Agents (Chatbots) | After AI Agents (Operators) | +| --- | --- | --- | +| **Finance** | Manual fraud review. | Agents investigate and file reports autonomously. | +| **Healthcare** | Doctors manually summarizing notes. | Agents transcribe, code for billing, and alert for risks. | +| **E-commerce** | Static recommendation engines. | Personal agents that find, negotiate, and buy products. | + +## 8. Implementation: A "Research Agent" Workflow + +Using a framework like **CrewAI** or **LangGraph**, a multi-agent "Research Squad" is structured like this: + +```python +research_crew = Crew( + agents=[web_searcher, data_analyst, technical_writer], + tasks=[ + Task(description="Search for 2026 AI hardware trends", agent=web_searcher), + Task(description="Analyze specs and price-to-performance", agent=data_analyst), + Task(description="Write a whitepaper for stakeholders", agent=technical_writer) + ], + process=Process.sequential # Data flows from one expert to the next +) + +``` + +:::tip The Personal Agent +Your flight is cancelled. Your personal agent detects this via email, rebooks a new flight, reschedules your 2 PM meeting, and notifies your hotel—all before you've even checked your phone. +::: + +## References + +* **Salesforce:** [Agentforce Use Cases](https://www.salesforce.com/agentforce/use-cases/) +* **Cognition AI:** [Devin - The First AI Software Engineer](https://www.cognition.ai/blog/introducing-devin) +* **Stanford:** [Generative Agents: Interactive Simulacra of Human Behavior](https://arxiv.org/abs/2304.03442) + +--- + +**Use cases show us what is possible. However, as we give agents the power to move money and handle patient data, we must discuss the guardrails. How do we ensure these autonomous systems remain safe?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/autonomous-task-agents.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/autonomous-task-agents.mdx index e69de29..e63656e 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/autonomous-task-agents.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/autonomous-task-agents.mdx @@ -0,0 +1,89 @@ +--- +title: "Autonomous Task Agents: The 'Fire and Forget' AI" +sidebar_label: Autonomous Agents +description: "Understanding fully autonomous agents that navigate open-ended goals through recursive loops and self-correction." +tags: [ai-agents, autonomy, autogpt, babyagi, goal-oriented-ai] +--- + +An **Autonomous Task Agent** is a system capable of completing open-ended objectives with minimal human intervention. Unlike a chatbot that responds to a single prompt, an autonomous agent takes a **goal** (e.g., "Research and write a comprehensive market report on EV trends"), creates its own tasks, executes them, and continues until the goal is met. + +## 1. Defining Autonomy + +What separates an autonomous agent from a standard script or chatbot? It is the ability to handle **uncertainty** and **novelty**. + +* **Self-Directed Planning:** The agent decides *how* to solve the problem. +* **Recursive Loops:** The agent can spawn new sub-tasks based on the results of previous ones. +* **Termination Logic:** The agent knows when the objective has been achieved and stops itself. + +## 2. The Core Execution Loop: "The Agentic Cycle" + +The most famous autonomous agents, like **AutoGPT** and **BabyAGI**, operate on a loop that mimics human task management. + +1. **Objective Input:** The human provides a high-level goal. +2. **Task Creation:** The agent generates a list of steps. +3. **Prioritization:** The agent reorders tasks based on importance and dependencies. +4. **Execution:** The agent performs the top task (using tools). +5. **Memory Storage:** Results are saved to long-term memory. +6. **Refinement:** The agent looks at the results and updates the task list. + +## 3. Architecture of Autonomy + +This diagram shows how an autonomous agent manages its own "To-Do List" without human guidance. + +```mermaid +graph TD + Goal[Global Objective] --> TP[Task Planner] + + subgraph Autonomous_Loop [The Self-Driving Loop] + TP --> Queue[Task Queue / To-Do List] + Queue --> Exec[Executor Agent] + Exec --> Tools[API / Code / Search] + Tools --> Result[Result Observation] + Result --> Memory[(Memory)] + Memory --> Critic[Self-Critic / Evaluator] + Critic --> TP + end + + Critic -- "Goal Accomplished" --> Output[Final Deliverable] + + style Autonomous_Loop fill:#fff8e1,stroke:#ffc107,color:#333,stroke-width:2px + style Critic fill:#fce4ec,stroke:#d81b60,color:#333 + style Queue fill:#e1f5fe,stroke:#01579b,color:#333 + +``` + +## 4. Landmark Autonomous Projects + +| Project | Key Innovation | Best Use Case | +| --- | --- | --- | +| **AutoGPT** | Recursive reasoning and file system access. | General purpose automation and research. | +| **BabyAGI** | Simplified task prioritization loop. | Managing complex, multi-step project tasks. | +| **AgentGPT** | Browser-based UI for autonomous agents. | Accessible, low-code agent deployment. | +| **Devin** | Software engineering autonomy. | Writing code, fixing bugs, and deploying apps. | + +## 5. The Risks of "Going Autonomous" + +High autonomy comes with high unpredictability. Developers must manage several specific risks: + +* **Task Drifting:** The agent gets distracted by a sub-task and loses sight of the primary goal. +* **Infinite Loops:** The agent tries the same unsuccessful action repeatedly, burning through API credits. +* **Hallucinated Success:** The agent believes it has finished the task when it has actually failed or produced a superficial result. +* **Security:** An autonomous agent with "write" access to a file system or database can cause unintended damage if its logic fails. + +## 6. Implementation Strategy: Guardrails + +To make autonomous agents safe for production, we implement **Guardrails**: + +* **Token Caps:** Limiting the maximum number of loops an agent can perform. +* **Human-in-the-Loop (HITL):** Requiring human approval for high-risk actions (e.g., spending money or deleting files). +* **Structured Output:** Forcing the agent to output its reasoning in a specific schema (JSON) to ensure logical consistency. + +## References + +* **AutoGPT GitHub:** [Significant Gravitas - AutoGPT](https://github.com/Significant-Gravitas/Auto-GPT) +* **Yohei Nakajima:** [Task-driven Autonomous Agent (BabyAGI)](https://github.com/yoheinakajima/babyagi) +* **OpenAI:** [Building Autonomous Agents with GPT-4](https://openai.com/blog/gpt-4-api-general-availability) + +--- + +**Autonomous agents work best when they focus on a single mission. But what happens when you need multiple specialists to work together as a team?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/llm-powered-agents.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/llm-powered-agents.mdx index e69de29..1fc85de 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/llm-powered-agents.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/llm-powered-agents.mdx @@ -0,0 +1,107 @@ +--- +title: "LLM-Powered Agents: The Reasoning Engine" +sidebar_label: LLM-Powered Agents +description: "How Large Language Models function as the central nervous system for autonomous agents." +tags: [llm, ai-agents, gpt-4, claude, prompt-engineering, reasoning] +--- + +An **LLM-Powered Agent** is a system where a Large Language Model (like GPT-4, Claude 3.5, or Llama 3) serves as the core controller. In this setup, the LLM is not just generating text; it is performing **high-level cognitive tasks**: reasoning about instructions, planning multi-step trajectories, and deciding when to stop. + +## 1. The LLM as a "Reasoning Engine" + +In a standard RAG or Chatbot setup, the LLM is a **Knowledge Retriever**. In an Agentic setup, the LLM is a **Decision Maker**. + +To move from "model" to "agent," the LLM must handle three critical cognitive functions: +1. **Instruction Following:** Understanding complex, often ambiguous, human goals. +2. **Context Management:** Utilizing the "system prompt" to maintain a persona and operational boundaries. +3. **Self-Correction:** Evaluating its own intermediate outputs and adjusting the plan if a tool returns an error. + +## 2. Prompting Frameworks for Agents + +The way we prompt the LLM determines how effectively it acts as an agent. Several key frameworks have emerged: + +### A. Chain of Thought (CoT) +By asking the model to "think step-by-step," we force the LLM to decompose a complex problem into smaller logical units. This significantly reduces hallucinations in multi-step tasks. + +### B. ReAct (Reason + Act) +The LLM generates a "Thought" followed by an "Action" command. It then pauses and waits for an "Observation" (input from a tool) before generating the next "Thought." + +### C. Tree of Thoughts (ToT) +The agent explores multiple reasoning paths simultaneously, evaluating the "prospects" of each path and backtracking if one leads to a dead end. + +## 3. The Agentic Workflow (Mermaid) + +This diagram visualizes how the LLM interacts with external inputs to drive the agent's progress toward a goal. + +```mermaid +graph TD + User[User Input] --> System[System Prompt: 'You are an autonomous research agent...'] + System --> LLM_Brain{LLM Reasoning} + + LLM_Brain --> Thought[Internal Thought: What is my next step?] + Thought --> Action[Action: Generate Tool Call] + + Action --> Environment[External World: APIs, Database, Web] + Environment --> Observation[Observation: Data returned from Tool] + + Observation --> LLM_Brain + + LLM_Brain --> Final{Goal Achieved?} + Final -- Yes --> Output[Final Response] + Final -- No --> Thought + + style LLM_Brain fill:#f1f8e9,stroke:#558b2f,stroke-width:2px,color:#333 + style System fill:#e1f5fe,stroke:#01579b,color:#333 + style Environment fill:#fff3e0,stroke:#ef6c00,color:#333 + +``` + +## 4. Why LLMs are suited for Agency + +Not all models can power an agent. High-performing LLM-powered agents rely on specific model capabilities: + +* **Large Context Windows:** To store long histories of previous actions and observations. +* **Function Calling Support:** Models fine-tuned to output structured data (like JSON) that external software can execute. +* **Zero-Shot Generalization:** The ability to use a tool correctly even if the agent hasn't seen that specific task during training. + +## 5. Limitations of LLM-Powered Agents + +Despite their power, these agents face significant hurdles: + +* **Context Drift:** As the "Reasoning-Action" loop continues, the LLM may lose track of the original goal. +* **Reliability:** If the LLM produces a single malformed JSON string, the entire agentic loop may crash. +* **Cost/Latency:** Multi-step reasoning requires multiple LLM calls, increasing both the time to respond and the cost per task. + +## 6. Implementation Sketch (Pseudo-Code) + +A conceptual loop for an LLM-powered agent: + +```python +def agent_loop(user_goal): + history = [system_prompt, user_goal] + + while True: + # 1. Ask the LLM for the next step + response = llm.generate(history) + + # 2. Check if the LLM wants to provide a final answer + if response.is_final_answer: + return response.text + + # 3. Otherwise, the LLM wants to use a tool + tool_output = execute_tool(response.tool_name, response.args) + + # 4. Feed the observation back into the history + history.append(f"Observation: {tool_output}") + +``` + +## References + +* **OpenAI:** [Function Calling Documentation](https://platform.openai.com/docs/guides/function-calling) +* **Anthropic:** [Computer Use with Claude 3.5 Sonnet](https://www.anthropic.com/news/3-5-models-and-computer-use) +* **arXiv:** [A Survey on Large Language Model based Autonomous Agents](https://arxiv.org/abs/2308.11432) + +--- + +**The LLM provides the logic, but an agent is nothing without its "senses" and "hands." How does an LLM interact with the real world?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/memory-in-agents.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/memory-in-agents.mdx index e69de29..51e819b 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/memory-in-agents.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/memory-in-agents.mdx @@ -0,0 +1,121 @@ +--- +title: "Memory in AI Agents: Context vs. Persistence" +sidebar_label: Agent Memory +description: "How AI agents store and recall information using short-term context and long-term persistent storage." +tags: [ai-agents, memory, vector-databases, rag, long-term-memory] +--- + +Without memory, an AI agent is essentially a "stateless" function—it treats every new prompt as if it's the first time it has ever spoken to you. To become true collaborators, agents need **Memory**: the ability to store, recall, and synthesize information from previous interactions. + +## 1. The Memory Hierarchy + +AI memory is structured similarly to human cognition, divided into temporal and functional layers. + +| Type | Duration | Technical Implementation | Analogous To | +| :--- | :--- | :--- | :--- | +| **Short-term** | Session-only | Context Window / RAM | Working Memory | +| **Long-term** | Permanent | Vector DB / Knowledge Graph | Hard Drive / Experiences | + +### A. Short-Term (Working) Memory +This is the information the agent can "see" right now in its context window. It includes the last few messages and current task variables. +* **Limit:** Restricted by the model's token limit (e.g., 128k tokens). +* **Reset:** Usually cleared when a new session or "thread" starts. + +### B. Long-Term (Persistent) Memory +This allows agents to remember you across days or weeks. It is often subdivided into: +* **Episodic Memory:** Recalling specific past events ("Last Tuesday, we discussed the marketing budget"). +* **Semantic Memory:** Recalling general facts or learned concepts ("The user prefers Python over Java"). + +## 2. Memory Architecture: How it Works + +Modern agents use a **Retrieval-Augmented Generation (RAG)** pattern for memory. Instead of stuffing everything into the prompt, the agent selectively "pulls" only the relevant memories. + +```mermaid +graph TD + In[New User Input] --> Encode[Encoder: Create Embedding] + + subgraph Memory_Bank [Persistent Storage] + VDB[(Vector Database)] + end + + Encode --> Search[Semantic Search] + VDB --> Search + + Search -- "Top K Relevant Memories" --> Prompt[Augmented Prompt] + In --> Prompt + + Prompt --> LLM[LLM Reasoning] + LLM --> Out[Response] + + Out --> Save[Memory Saver: Extract & Store] + Save --> VDB + + style Memory_Bank fill:#f1f8e9,stroke:#558b2f,color:#333 + style LLM fill:#e1f5fe,stroke:#01579b,color:#333 + +``` + +## 3. Context Window vs. Vector Databases + +There is a constant trade-off between keeping history in the prompt (fast/accurate) versus an external database (infinite/scalable). + +| Feature | Context Window (Short-term) | Vector DB (Long-term) | +| --- | --- | --- | +| **Speed** | Instant | 30-100ms (Search overhead) | +| **Capacity** | Small (Tokens) | Massive (Gigabytes) | +| **Cost** | High (Tokens = $$) | Low (Storage is cheap) | +| **Reliability** | 100% Recall | Search may miss relevant data | + +## 4. Advanced Memory Techniques + +To prevent "Memory Bloat" (where the agent gets confused by too much old data), developers use these strategies: + +1. **Summarization:** Periodically, the agent compresses old chat history into a concise paragraph, saving tokens while preserving the core "gist." +2. **Entity Memory:** The agent maintains a "profile" of entities it knows (e.g., people, projects, preferences) rather than just raw text logs. +3. **Reflective Memory:** The agent "thinks" about its experiences during downtime to derive new insights or consolidate conflicting information. + +## 5. Implementation: Simple Vector Memory + +In frameworks like **LangGraph** or **CrewAI**, memory is often managed by a "Checkpointer" or a dedicated Memory tool. + +```python +# Conceptual Memory Retrieval +def get_context(user_id, current_query): + # 1. Search Long-term memory for past preferences + past_memories = vector_db.search(user_id, current_query, limit=3) + + # 2. Combine with Short-term conversation history + full_prompt = f""" + Context from past: {past_memories} + Current History: {session_history} + User Request: {current_query} + """ + return llm.generate(full_prompt) + +``` + +## 6. Challenges in Agentic Memory + +* **The Forgetting Problem:** How do you decide what to delete? (Recency vs. Importance). +* **Privacy:** Ensuring "User A" doesn't accidentally retrieve "User B's" memories. +* **Inconsistency:** What if a user changes their mind? The agent must resolve the conflict between an old memory ("I love coffee") and a new one ("I'm quitting caffeine"). + +## References + +* **LangChain Docs:** [Memory Types and Implementation](https://python.langchain.com/docs/modules/memory/) +* **Mem0:** [The Memory Layer for AI Agents](https://mem0.ai/) +* **Pinecone:** [Vector Databases for Long-term AI Memory](https://www.pinecone.io/learn/vector-database/) + +--- + + + +
+ +**Memory turns a chatbot into a personal assistant. But what happens when one agent isn't enough? How do multiple agents work together to solve massive problems?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/multi-agent-systems.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/multi-agent-systems.mdx index e69de29..8449deb 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/multi-agent-systems.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/multi-agent-systems.mdx @@ -0,0 +1,111 @@ +--- +title: "Multi-Agent Systems: Collaborative Intelligence" +sidebar_label: Multi-Agent Systems +description: "How multiple specialized AI agents work together to solve complex problems through coordination and debate." +tags: [ai-agents, multi-agent-systems, mas, autogen, crewai, orchestration] +--- + +In the world of AI Agents, a single agent is like a "Generalist." While capable, it can become overwhelmed by highly complex tasks. **Multi-Agent Systems (MAS)** solve this by using a "Team of Specialists." + +In an MAS, different agents are assigned specific roles (e.g., a Coder, a Reviewer, and a Project Manager) and work together through communication and coordination to achieve a common goal. + +## 1. Why Use Multiple Agents? + +Moving from a single agent to a multi-agent system provides several key advantages: + +* **Role Specialization:** You can prompt different agents with different personas, making them "experts" in their specific domain. +* **Error Correction through Debate:** One agent can generate an output, while another agent critiques it, leading to higher quality results. +* **Parallelism:** Multiple agents can work on different sub-tasks simultaneously. +* **Scalability:** Complex problems can be broken down into a hierarchy of manageable steps. + +## 2. Common Multi-Agent Architectures + +How agents interact is defined by the **Orchestration Pattern**: + +### A. Sequential (Waterfall) +Agent A finishes its task and passes the result to Agent B. +* *Example:* Researcher → Writer → Proofreader. + +### B. Manager-Worker (Hub and Spoke) +A "Manager Agent" receives the goal, decomposes it into tasks, and assigns them to "Worker Agents." The Manager reviews the results before finalizing. +* *Example:* A Project Manager delegating code tasks to various developer agents. + +### C. Joint Collaboration (Round Robin) +Agents engage in a group chat or a shared whiteboard space, contributing as needed until the task is complete. +* *Example:* A brainstorm session between a "Creative Agent" and a "Logical Agent." + +## 3. Communication Logic + +This diagram illustrates a **Manager-Worker** architecture where the Manager acts as the central orchestrator. + +```mermaid +graph TD + User[User Goal] --> Manager[Manager Agent] + + subgraph Specialist_Squad [Specialist Agents] + Manager <--> Researcher[Researcher: Web Access] + Manager <--> Coder[Coder: Python Expert] + Manager <--> QA[QA: Logic Tester] + end + + Researcher -- "Found Data" --> Manager + Manager -- "Write Script" --> Coder + Coder -- "Draft Code" --> QA + QA -- "Bug Report" --> Coder + QA -- "Verified" --> Manager + + Manager --> Final[Final Deliverable] + + style Manager fill:#e1f5fe,stroke:#01579b,stroke-width:2px,color:#333 + style Specialist_Squad fill:#f9f9f9,stroke:#333,color:#333,stroke-dasharray: 5 5 + +``` + +## 4. Popular Multi-Agent Frameworks + +Building these systems from scratch is difficult, so several frameworks have emerged to handle the "handshaking" between agents: + +1. **Microsoft AutoGen:** Focused on conversational multi-agent systems. Agents can be "human-in-the-loop" or fully autonomous. +2. **CrewAI:** Uses a "Role-Playing" approach where you define specific roles, goals, and backstories for each agent. +3. **LangGraph (LangChain):** Provides fine-grained control over cycles and state management in multi-agent graphs. +4. **OpenAI Swarm:** An experimental, lightweight framework for orchestrating many small, specialized agents. + +## 5. Challenges in Multi-Agent Systems + +Coordinating a team of AIs introduces new technical hurdles: + +* **Agent Chatter:** Agents might get stuck in an "endless loop" of talking to each other without making progress. +* **Context Fragmentation:** Important information might get lost as it is passed from one agent to another. +* **High Latency/Cost:** Every agent-to-agent interaction requires an LLM call, which can become expensive and slow. +* **Consensus Issues:** Agents might disagree on a strategy, requiring a "tie-breaking" logic in the manager. + +## 6. Implementation Sketch: CrewAI Example + +In **CrewAI**, you define the "Crew" by assigning agents and tasks. + +```python +from crewai import Agent, Task, Crew + +# 1. Define Agents +researcher = Agent(role='Researcher', goal='Find 2026 AI trends', backstory='Expert Analyst') +writer = Agent(role='Writer', goal='Write a blog post', backstory='Tech Journalist') + +# 2. Define Tasks +task1 = Task(description='Search for AI trends', agent=researcher) +task2 = Task(description='Summarize trends into a post', agent=writer) + +# 3. Form the Crew +my_crew = Crew(agents=[researcher, writer], tasks=[task1, task2]) +result = my_crew.start() + +``` + +## References + +* **Microsoft Research:** [AutoGen: Enabling Next-Gen LLM Applications](https://microsoft.github.io/autogen/) +* **CrewAI:** [Multi-agent Orchestration Framework](https://www.crewai.com/) +* **arXiv:** [CAMEL: Communicative Agents for 'Mind' Exploration](https://arxiv.org/abs/2303.17760) + +--- + +**Multi-agent systems represent the peak of current AI agency. But with great power comes great responsibility. How do we ensure these autonomous teams act safely?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/planning-and-reasoning.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/planning-and-reasoning.mdx index e69de29..c193b00 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/planning-and-reasoning.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/planning-and-reasoning.mdx @@ -0,0 +1,97 @@ +--- +title: "Planning & Reasoning: The Agent's Inner Monologue" +sidebar_label: Planning & Reasoning +description: "Exploring how AI agents break down complex tasks, self-correct, and navigate logic using advanced reasoning frameworks." +tags: [ai-agents, reasoning, planning, chain-of-thought, react] +--- + +If tools are the hands of an agent, **Planning and Reasoning** are its frontal cortex. Without these, an agent would simply react to inputs. With them, an agent can look at a high-level goal—like "Plan a 3-day trip to Tokyo"—and strategically decompose it into actionable sub-tasks. + +## 1. Task Decomposition + +The first step in any agentic workflow is breaking a large goal into smaller, manageable steps. This is known as **Task Decomposition**. + +* **Linear Planning:** The agent identifies a sequence (Step 1 → Step 2 → Step 3). +* **Hierarchical Planning:** The agent creates a "Master Plan" with high-level objectives, which are then broken down into "Sub-plans" by specialized sub-routines. + +## 2. Key Reasoning Frameworks + +To move through a plan successfully, agents use specific cognitive frameworks to structure their "thoughts." + +### A. Chain of Thought (CoT) +CoT encourages the LLM to generate intermediate reasoning steps. By "thinking aloud," the model is less likely to jump to a premature (and often incorrect) conclusion. +* **Prompting Technique:** Adding "Let's think step by step" to the instructions. + +### B. ReAct (Reason + Act) +ReAct combines reasoning and acting in an interleaved manner. The agent writes a **Thought**, performs an **Action**, observes the **Result**, and then writes a new **Thought** based on that result. + +### C. Tree of Thoughts (ToT) +For complex problems requiring trial and error, ToT allows the agent to explore multiple reasoning branches simultaneously. If one branch reaches a dead end, the agent "backtracks" and tries a different path. + +## 3. Self-Reflection and Error Correction + +An intelligent agent doesn't just execute; it evaluates. **Self-Reflection** is the process where an agent looks at its own output to find flaws. + +1. **Self-Criticism:** "Does this plan actually solve the user's request?" +2. **Environment Feedback:** If a tool returns an error, the agent reasons: "The API failed because the date format was wrong. I will try again with YYYY-MM-DD." +3. **Human-in-the-Loop:** Pausing to ask the user for clarification when the reasoning becomes too uncertain. + +## 4. The Planning Workflow + +The following diagram illustrates how the Planning module sits between the User Input and the Execution Layer. + +```mermaid +graph TD + Goal[User Goal] --> Decomp[Task Decomposition] + + subgraph Reasoning_Loop [The Cognitive Loop] + Decomp --> Step1[Current Sub-task] + Step1 --> Reason[Reason: How to solve this?] + Reason --> Execute[Action: Use Tool] + Execute --> Obs[Observation: Result] + Obs --> Reflect{Task Complete?} + Reflect -- No --> Step1 + end + + Reflect -- Yes --> Final[Consolidate Results] + Final --> Response[Final Answer] + + style Reasoning_Loop fill:#f9f9f9,stroke:#333,color:#333,stroke-dasharray: 5 5 + style Decomp fill:#fff3e0,stroke:#ef6c00,color:#333 + style Reflect fill:#f3e5f5,stroke:#7b1fa2,color:#333 + +``` + +## 5. Challenges in Agent Reasoning + +* **Hallucination in Planning:** The agent might plan to use a tool that doesn't exist or assume a fact that is false. +* **Infinite Loops:** The agent tries the same failing action repeatedly without changing its strategy. +* **Context Window Fatigue:** As the plan grows longer, the agent may forget the initial constraints of the goal. + +## 6. Implementation Sketch (Plan-and-Execute) + +In high-level frameworks like LangGraph, the "Plan-and-Execute" pattern is often implemented by separating the **Planner** from the **Executor**. + +```python +# 1. Planner Agent creates the list of steps +plan = planner.generate("Research the impact of AI on SEO in 2025") + +# 2. Executor Agent iterates through steps +for task in plan.steps: + result = executor.run(task) + # 3. Re-planner evaluates if the plan needs to change + plan = replanner.update(plan, task, result) + if plan.is_finished: + break + +``` + +## References + +* **Google Research:** [Chain-of-Thought Prompting Elicits Reasoning in LLMs](https://arxiv.org/abs/2201.11903) +* **Princeton/Google:** [ReAct: Synergizing Reasoning and Acting](https://react-lm.github.io/) +* **OpenAI:** [Reasoning with o1 series models](https://openai.com/index/learning-to-reason-with-llms/) + +--- + +**Planning gives the agent a roadmap, but memory allows it to remember where it has already been. How do agents store long-term experiences?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/tool-using-agents.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/tool-using-agents.mdx index e69de29..5beeed5 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/tool-using-agents.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/tool-using-agents.mdx @@ -0,0 +1,112 @@ +--- +title: "Tool-Using Agents: The Hands of AI" +sidebar_label: Tool-Using Agents +description: "A deep dive into how AI agents select, interact with, and execute external tools and APIs." +tags: [ai-agents, tool-use, function-calling, api-integration, autonomy] +--- + +A standalone Large Language Model (LLM) is limited by its training data cutoff and its inability to interact with the real world. **Tool-Using Agents** overcome these limitations by using the LLM as a controller that decides when to call external software, APIs, or scripts to gather information or perform actions. + +## 1. Why Agents Need Tools + +Without tools, an agent is a "brain in a vat." Tools provide the agent with three essential capabilities: + +1. **Access to Live Data:** Browsing the web for current news or stock prices. +2. **Computational Accuracy:** Using a calculator or Python interpreter for complex math instead of relying on probabilistic token prediction. +3. **Real-World Impact:** Sending emails, updating database records, or controlling smart home devices. + +## 2. The Mechanism: Function Calling + +The industry standard for tool use is **Function Calling**. This is a structured communication protocol between the LLM and the application code. + +### The Protocol Flow: +1. **Declaration:** You provide the LLM with a list of tools defined in JSON schema (describing what the tool does and what parameters it needs). +2. **Selection:** The LLM analyzes the user prompt and determines which tool is appropriate. +3. **Request:** Instead of conversational text, the LLM outputs a structured JSON object (e.g., `{"tool": "get_weather", "parameters": {"city": "London"}}`). +4. **Execution:** Your code executes the actual function and returns the raw data to the LLM. +5. **Synthesis:** The LLM reads the tool's output and incorporates it into a natural language response. + +## 3. Types of Common Agent Tools + +| Tool Category | Example Tools | Purpose | +| :--- | :--- | :--- | +| **Search** | Google Search, Arxiv, Wikipedia | Overcoming knowledge cutoffs. | +| **Computation** | Python REPL, Wolfram Alpha | Precise mathematical and logical execution. | +| **Storage** | SQL Databases, Vector DBs | Reading and writing persistent information. | +| **Communication** | Gmail, Slack, Twilio | Interacting with human collaborators. | + +## 4. Logical Workflow: Tool-Use Loop + +The following diagram illustrates the "Observation" phase, which is unique to tool-using agents. The agent must "wait" for the world to respond before it can continue its reasoning. + +```mermaid +graph TD + User[User: 'What is 15% of the current Bitcoin price?'] --> Brain[Brain: LLM Reasoning] + + subgraph Tool_Execution [The External Loop] + Brain -- 'I need BTC price' --> Search[Tool: Crypto API] + Search -- 'Price is $50,000' --> Obs1[Observation 1] + Obs1 --> Brain + + Brain -- 'I need to calculate 15% of 50,000' --> Calc[Tool: Python Interpreter] + Calc -- 'Result: 7,500' --> Obs2[Observation 2] + end + + Obs2 --> Brain + Brain --> Final[Final Response: '15% of BTC is $7,500'] + + style Tool_Execution fill:#f3e5f5,stroke:#7b1fa2,color:#333,stroke-dasharray: 5 5 + style Brain fill:#fff3e0,stroke:#ef6c00,color:#333 + style Final fill:#c8e6c9,stroke:#2e7d32,color:#333 + +``` + +## 5. Tool Selection Strategies + +How does an agent decide which tool to use when it has access to dozens? + +* **Zero-Shot Selection:** The LLM relies on the descriptions provided in the system prompt to choose the best fit. +* **Few-Shot Prompting:** Providing examples of past "User Prompt -> Tool Selection" pairs to guide the model. +* **Router Agents:** A small, fast model acts as a "traffic controller," deciding which specialized sub-agent (with its own specific tools) should handle the request. + +## 6. Challenges and Safety (Human-in-the-Loop) + +Giving an AI the "hands" to click buttons and run code introduces risks: + +* **Tool Hallucination:** The agent tries to use a tool that wasn't provided or invents parameters that don't exist. +* **Prompt Injection:** An external website might contain hidden instructions that trick the agent into using a tool maliciously (e.g., "Delete all my emails"). +* **Execution Errors:** If an API is down, the agent must be resilient enough to try an alternative or report the error gracefully. + +**Best Practice:** Use **Human-in-the-Loop (HITL)** for sensitive tools (e.g., the agent can draft an email, but a human must click "Send"). + +## 7. Implementation Sketch (JSON Schema) + +This is how a tool is described to an LLM-powered agent: + +```json +{ + "name": "get_stock_price", + "description": "Retrieves the current stock price for a given ticker symbol.", + "parameters": { + "type": "object", + "properties": { + "ticker": { + "type": "string", + "description": "The stock symbol (e.g., AAPL, TSLA)" + } + }, + "required": ["ticker"] + } +} + +``` + +## References + +* **OpenAI:** [Introduction to Tool Use](https://platform.openai.com/docs/guides/function-calling) +* **LangChain:** [Tools and Toolkits Documentation](https://python.langchain.com/docs/modules/agents/tools/) +* **Anthropic:** [Claude Tool Use (Function Calling)](https://docs.anthropic.com/claude/docs/tool-use) + +--- + +**Tools allow agents to act, but memory allows them to learn and stay consistent. How do agents remember the results of their tool uses over time?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/types-of-ai-agents.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/types-of-ai-agents.mdx index e69de29..fbaa2fc 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/types-of-ai-agents.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/types-of-ai-agents.mdx @@ -0,0 +1,96 @@ +--- +title: Taxonomy of AI Agents +sidebar_label: Types of Agents +description: "Exploring the various types of AI agents from Simple Reflex to Learning and Multi-Agent Systems." +tags: [ai-agents, robotics, reinforcement-learning, multi-agent-systems, autonomy] +--- + +Not all AI agents are created equal. They vary significantly in their "intelligence" levels and the logic they use to make decisions. In classical AI theory (popularized by Russell & Norvig), agents are categorized based on their degree of perceived intelligence and capability. + +## 1. Simple Reflex Agents +These are the most basic type of agents. They take actions based **only** on the current perception, ignoring the rest of the perceptual history. They follow "Condition-Action" rules. + +* **Logic:** If [Condition], then [Action]. +* **Limitation:** They only work if the environment is fully observable. They have no "memory." +* **Example:** A thermostat that turns on the AC only when the temperature exceeds 25°C. + +## 2. Model-Based Reflex Agents +These agents maintain an internal **state** (a "model" of the world) that allows them to track parts of the environment they cannot see right now. This helps them handle partially observable environments. + +* **Logic:** What is the world like now? + How does the world evolve? +* **Key Feature:** Internal memory of previous states. +* **Example:** A self-driving car that "remembers" a pedestrian is behind a parked truck, even if the camera can no longer see them. + +## 3. Goal-Based Agents +Goal-based agents expand on the model-based approach by having a **destination** or a specific goal. They don't just react to the world; they act to achieve a future state. + +* **Logic:** Will this action get me closer to my goal? +* **Key Feature:** Proactive planning and searching. +* **Example:** A GPS navigation system that plans the shortest route to your destination, rerouting if it detects traffic. + +## 4. Utility-Based Agents +While goal-based agents care *if* they reach the goal, utility-based agents care about *how well* they reach it. They use a "Utility Function" to measure "happiness" or "efficiency." + +* **Logic:** Which action provides the highest "utility" (speed, safety, cost-efficiency)? +* **Key Feature:** Optimization and trade-off analysis. +* **Example:** A flight booking agent that doesn't just find a flight (goal), but finds the cheapest flight with the fewest layovers (high utility). + +## 5. Learning Agents +Learning agents are capable of improving their performance over time. They operate in initially unknown environments and become more competent through experience. + +They are divided into four conceptual components: +1. **Learning Element:** Makes improvements based on feedback. +2. **Performance Element:** Chooses external actions. +3. **Critic:** Tells the learning element how well the agent is doing. +4. **Problem Generator:** Suggests new actions (exploration) to gain more knowledge. + +## 6. Multi-Agent Systems (MAS) +In modern AI, we often see multiple agents working together. These systems are classified by the relationship between the agents: + +| Type | Relationship | Example | +| :--- | :--- | :--- | +| **Cooperative** | Agents share information to solve a common goal. | A swarm of drones performing a light show. | +| **Competitive** | Agents work against each other to win. | AlphaGo playing against a human; Trading bots in a market. | +| **Negotiation-Based** | Agents bargain to reach a mutually beneficial state. | Resource allocation in smart power grids. | + +## 7. Logical Evolution: Hierarchy of Agency + +The following diagram illustrates how agents move from simple reactivity to complex autonomous learning. + +```mermaid +graph TD + Simple[Simple Reflex: Condition-Action] --> Model[Model-Based: Internal State] + Model --> Goal[Goal-Based: Planning] + Goal --> Utility[Utility-Based: Optimization] + Utility --> Learning[Learning Agent: Self-Improvement] + + subgraph Autonomous_System [Advanced Agency] + Learning + MAS[Multi-Agent Systems: Collaboration] + end + + style Simple fill:#f5f5f5,stroke:#333,color:#333 + style Learning fill:#c8e6c9,stroke:#2e7d32,color:#333 + style MAS fill:#e1f5fe,stroke:#01579b,color:#333 + +``` + +## 8. Summary Table + +| Agent Type | Uses Memory? | Uses Goals? | Optimizes? | Learns? | +| --- | :---: | :---: | :---: | :---: | +| **Simple Reflex** | ❌ | ❌ | ❌ | ❌ | +| **Model-Based** | ✅ | ❌ | ❌ | ❌ | +| **Goal-Based** | ✅ | ✅ | ❌ | ❌ | +| **Utility-Based** | ✅ | ✅ | ✅ | ❌ | +| **Learning Agent** | ✅ | ✅ | ✅ | ✅ | + +## References + +* **Textbook:** *Artificial Intelligence: A Modern Approach* by Stuart Russell and Peter Norvig. +* **OpenAI:** [Research on Multi-Agent Hide and Seek](https://openai.com/research/emergent-tool-use) +* **DeepMind:** [Reinforcement Learning and Learning Agents](https://www.deepmind.com/learning-resources) + +--- + +**Understanding the types of agents is the theory. In practice, how do these agents actually "reach out" and touch the digital world?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-agents/what-are-ai-agents.mdx b/docs/machine-learning/advanced-ml-topics/ai-agents/what-are-ai-agents.mdx index e69de29..d3bce66 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-agents/what-are-ai-agents.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-agents/what-are-ai-agents.mdx @@ -0,0 +1,88 @@ +--- +title: What are AI Agents? +sidebar_label: Intro to AI Agents +description: "Defining AI Agents, their core components, and how they differ from standard Chatbots." +tags: [ai-agents, autonomy, llms, reasoning, decision-making] +--- + +An **AI Agent** is an autonomous or semi-autonomous system that perceives its environment, reasons about how to achieve a specific goal, and takes actions using available tools. + +While a standard Large Language Model (LLM) is like a "brain in a box" that waits for a prompt to generate text, an **AI Agent** is like a "brain with hands." It doesn't just talk; it **does**. + +## 1. The Core Architecture: The Agentic Loop + +An agent operates in a continuous cycle often referred to as the **Perception-Reasoning-Action** loop. + +1. **Perceive:** The agent receives a goal (e.g., "Research the latest stock prices for NVIDIA and email a summary"). +2. **Plan:** It breaks the goal into sub-tasks (1. Search web, 2. Extract data, 3. Format email). +3. **Act:** It uses a **Tool** (like a Web Search API). +4. **Observe:** It looks at the tool output and decides if it needs to do more. + +## 2. Key Components of an Agent + +To function effectively, an agent requires four primary modules: + +| Component | Responsibility | Analogous to... | +| :--- | :--- | :--- | +| **Brain (LLM)** | Reasoning, planning, and decision-making. | The Prefrontal Cortex | +| **Planning** | Breaking down complex goals into manageable steps. | Strategic Thinking | +| **Memory** | Short-term (Current context) and Long-term (Vector Databases). | Hippocampus | +| **Tools/Action** | Interfaces to interact with the world (APIs, Code Interpreters). | Hands and Senses | + +## 3. Agents vs. Chatbots: What's the Difference? + +| Feature | Standard Chatbot | AI Agent | +| :--- | :--- | :--- | +| **Initiative** | Reactive (Wait for prompt) | Proactive (Takes steps to reach goal) | +| **Connectivity** | Static knowledge | Dynamic (Can browse, use apps, run code) | +| **Multi-step** | Single turn | Multi-step reasoning and self-correction | +| **Goal-oriented** | Provide an answer | Accomplish a mission | + +## 4. Logical Workflow: Agentic Reasoning + +The following diagram illustrates how an agent handles a complex request using a **ReAct** (Reason + Act) pattern. + +```mermaid +graph TD + User[User: 'Book a flight to NYC'] --> Plan[Brain: I need to search for flights first] + Plan --> Action[Action: Call Flight API] + Action --> Obs[Observation: Found 3 flights] + + Obs --> Reason{Is goal met?} + Reason -- No --> Subtask[Reason: I need to check the user's calendar for conflicts] + Subtask --> Action2[Action: Google Calendar API] + + Reason -- Yes --> Final[Result: Flight booked and confirmed] + + style User fill:#e1f5fe,stroke:#01579b,color:#333 + style Plan fill:#fff3e0,stroke:#ef6c00,color:#333 + style Action fill:#f3e5f5,stroke:#7b1fa2,color:#333 + style Final fill:#c8e6c9,stroke:#2e7d32,color:#333 + +``` + +## 5. Types of AI Agents + +1. **Simple Reflex Agents:** Act only on the basis of the current perception (e.g., a simple chatbot). +2. **Goal-Based Agents:** Take actions specifically to reach a defined destination (e.g., AutoGPT). +3. **Utility-Based Agents:** Not only reach a goal but try to find the "best" or "most efficient" way to do it. +4. **Multi-Agent Systems (MAS):** Multiple agents working together, often with specialized roles (e.g., one agent writes code, another tests it). + +## 6. Popular Agent Frameworks + +If you want to build agents today, these are the most common starting points: + +* **LangChain:** The "Swiss Army Knife" for connecting LLMs to tools. +* **AutoGPT / BabyAGI:** Early experiments in fully autonomous goal-seeking. +* **Microsoft AutoGen:** A framework for multi-agent conversations. +* **CrewAI:** Orchestrating role-playing autonomous agents to work as a team. + +## References + +* **Original Paper:** [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) +* **Lilian Weng Blog:** [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) +* **DeepLearning.ai:** [AI Agents Specialization](https://www.deeplearning.ai/short-courses/ai-agents-in-langgraph/) + +--- + +**Agents are powerful because they can use tools. But how exactly does an LLM know which tool to pick and how to use it?** \ No newline at end of file diff --git a/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx b/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx index e69de29..381f965 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx @@ -0,0 +1,96 @@ +--- +title: "AI Engineer Roadmap (2026)" +sidebar_label: AI Engineer Roadmap +description: "A comprehensive guide to becoming an AI Engineer, focusing on LLMs, RAG, Agents, and Production MLOps." +tags: [ai-engineer, roadmap, career, llmops, agents, rag] +--- + +In 2026, the role of an **AI Engineer** has diverged from a traditional **Data Scientist**. While Data Scientists focus on research and training models ($loss$ functions and gradients), AI Engineers focus on **implementing** those models into scalable, reliable products. + +This roadmap outlines the essential skills required to build "Agentic" and "Cognitive" applications. + +## Phase 1: Foundations (The AI "Brain") +Before building agents, you must understand the engine that powers them. + +* **LLM Fundamentals:** Understand Tokenization, Context Windows, and Temperature. +* **Prompt Engineering:** Master techniques like **Chain-of-Thought (CoT)** and **Few-Shot Prompting**. +* **Model Selection:** Know when to use "Frontier Models" (GPT-4o, Claude 3.5) vs. "Local Models" (Llama 3.x, Mistral) via **Ollama**. + +## Phase 2: Retrieval Augmented Generation (RAG) +LLMs have a knowledge cutoff. RAG allows them to "talk" to your private data. + +* **Embeddings:** Understanding how to turn text into vectors. +* **Vector Databases:** Learning tools like **Pinecone**, **Weaviate**, or **ChromaDB**. +* **Hybrid Search:** Combining semantic search with traditional keyword search (BM25). +* **Advanced RAG:** Implementing **Reranking**, **Query Expansion**, and **Small-to-Big Retrieval**. + +## Phase 3: AI Agents & Autonomy +This is the core of modern AI Engineering—moving from static chat to active doing. + +* **Tool Use:** Implementing **Function Calling** so models can use APIs and Databases. +* **Reasoning Loops:** Mastering the **ReAct** (Reason + Act) pattern. +* **Agent Frameworks:** Gaining proficiency in **LangChain**, **LangGraph**, or **CrewAI**. +* **Memory Systems:** Building short-term and long-term memory for persistent agents. + +## Phase 4: The AI Stack + +The following diagram illustrates the modern architecture an AI Engineer must manage. + +```mermaid +graph TD + User[User Interface] --> Orchestrator[Orchestration: LangGraph / CrewAI] + + subgraph Reasoning [The Brain] + Orchestrator --> LLM[LLM: OpenAI / Anthropic / Llama] + end + + subgraph Memory [The Context] + Orchestrator --> VDB[(Vector DB: RAG)] + VDB --> Docs[Company Data / PDFs] + end + + subgraph Action [The Hands] + Orchestrator --> Tools[Tools: APIs / Python Interpreter] + end + + LLM --> Result[Output] + Result --> User + + style Reasoning fill:#e1f5fe,stroke:#01579b,color:#333 + style Memory fill:#f1f8e9,stroke:#558b2f,color:#333 + style Action fill:#fff3e0,stroke:#ef6c00,color:#333 + +``` + +## Phase 5: Evaluation & LLMOps + +Building an AI app is easy; making it reliable is hard. + +* **Evaluation (Evals):** Using frameworks like **Ragas** or **Arize Phoenix** to measure hallucinations. +* **Observability:** Tracking traces and costs using **LangSmith** or **Weights & Biases**. +* **Guardrails:** Implementing **NeMo Guardrails** or **Pydantic Program** to ensure structured, safe outputs. + +## The 2026 Skill Matrix + +| Skill Area | Beginner | Professional | +| --- | --- | --- | +| **Coding** | Python Basics | Async Python & FastAPI | +| **Data** | CSV/JSON handling | Vector Databases & SQL | +| **Deployment** | Streamlit apps | Docker, Kubernetes, & Serverless AI | +| **Logic** | Simple Chatbots | Multi-Agent Orchestration | + +## How to Start Today + +1. **Build a CLI Agent:** Create a Python script that uses an LLM to manage your local file system (e.g., "Find all PDFs and summarize them"). +2. **Master RAG:** Build a "Chat with your Documentation" tool using a Vector DB. +3. **Deploy:** Put an agentic API behind a **FastAPI** endpoint and containerize it with **Docker**. + +## References + +* **Andrej Karpathy:** [Intro to Large Language Models](https://www.youtube.com/watch?v=DzjkBMFhNj_g) +* **Chip Huyen:** [Operationalizing Machine Learning](https://huyenchip.com/mlops/) +* **DeepLearning.ai:** [AI Engineering Specialization](https://www.deeplearning.ai/) + +--- + +**This roadmap is your guide to the future of software development. Are you ready to build the next generation of autonomous systems?** \ No newline at end of file From 3ef9dc594efda38e4a92c9603f60d295a6b846a4 Mon Sep 17 00:00:00 2001 From: Ajay Dhangar Date: Tue, 27 Jan 2026 21:17:49 +0530 Subject: [PATCH 2/2] done advance toipic of ml --- docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx b/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx index 381f965..1e60f98 100644 --- a/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx +++ b/docs/machine-learning/advanced-ml-topics/ai-engineer-roadmap.mdx @@ -87,7 +87,6 @@ Building an AI app is easy; making it reliable is hard. ## References -* **Andrej Karpathy:** [Intro to Large Language Models](https://www.youtube.com/watch?v=DzjkBMFhNj_g) * **Chip Huyen:** [Operationalizing Machine Learning](https://huyenchip.com/mlops/) * **DeepLearning.ai:** [AI Engineering Specialization](https://www.deeplearning.ai/)