How Powernode agents persist context across executions, retrieve documents via RAG, manage skill capabilities, and link content through the knowledge graph.
- What this concept covers
- Memory tiers
- Memory Pools — Configuration & Examples
- RAG system
- Skill graph
- Content linking and the knowledge graph
- Automated maintenance
- Related concepts
- Materials previously at
Powernode agents need three kinds of persistent context: working state they can manipulate within a single task, long-lived facts and experiences they can recall semantically across sessions, and reusable capabilities they can discover, evolve, and share. Three subsystems handle these:
- Memory — a four-tier hierarchy (working / short-term / long-term / shared) routed by
Ai::Memory::RouterService - RAG — hybrid retrieval over knowledge bases, documents, and a knowledge graph
- Skill graph — versioned, conflict-aware capability registry with proposal workflows
Content linking sits between them: pages and articles get knowledge-graph nodes with embeddings, so author-written wikilinks become typed edges that the same retrieval pipeline can traverse. The end result is that pages, KB articles, missions, agents, and skills coexist as nodes in one queryable graph.
This document covers the data models, services, and operational lifecycle. For how agents use this context, see concepts/agents-and-autonomy.md. For the MCP tool surface that exposes these capabilities to AI sessions, see concepts/mcp-and-tools.md.
The memory system provides agents with persistent, searchable memory across multiple tiers. Working memory lives in Redis for fast access, short-term memory has TTL-based expiration, long-term memory uses pgvector for semantic search, and shared memory enables cross-agent knowledge exchange.
flowchart TB
Router[Ai::Memory::RouterService]
WM[Working Memory<br/>Redis<br/>TTL: mins<br/>key-value]
STM[Short-Term Memory<br/>DB + TTL<br/>TTL: hours<br/>structured]
LTM[Long-Term Memory<br/>pgvector<br/>permanent<br/>embeddings]
Shared[Shared Memory<br/>Pools<br/>cross-agent<br/>collaborative]
Router --> WM
Router --> STM
Router --> LTM
Router --> Shared
| Model | Tier | Purpose |
|---|---|---|
| Redis hash | Working | Session-scoped key-value, manipulated directly |
Ai::AgentShortTermMemory |
Short-term | TTL-based per-agent session entries (default 1 hour TTL) |
Ai::PersistentContext + Ai::ContextEntry |
Long-term | Long-lived containers with pgvector embeddings and access logs |
Ai::MemoryPool |
Shared | Cross-agent collaborative pools |
Ai::SharedContextPool |
Shared (workflow) | Workflow-scoped pools for multi-node data exchange |
MEMORY_TYPES = %w[task_context conversation tool_result reflection observation]
DEFAULT_TTL = 3600 # 1 hour
# Per-agent session entries with TTL
ai_agent_short_term_memory:
session_id, memory_key, memory_value (JSON), memory_type,
ttl_seconds, expires_at, access_count, last_accessed_atKey methods: expired?, touch_access!, refresh_ttl!, self.cleanup_expired!.
Long-lived context containers for agents and knowledge bases.
CONTEXT_TYPES = %w[agent_memory knowledge_base shared_context tool_cache project_context]
SCOPES = %w[agent account team global]
# Container with snapshot/restore + access control
context.create_snapshot
context.restore_from_snapshot(snapshot_id)
context.archive!
context.grant_access(agent_id, level)Individual entries within a PersistentContext, with pgvector embeddings:
ENTRY_TYPES = %w[fact observation reflection decision plan tool_output conversation_summary]
SOURCE_TYPES = %w[agent user system tool workflow]
MEMORY_TYPES = %w[factual experiential working]
has_neighbors :embedding # pgvector integration
# Versioned, importance-weighted, semantically searchable
entry.semantic_search(query_embedding, limit: 10)
entry.update_content(new_content, editor) # creates new version
entry.boost_importance!
entry.effective_relevance_score # combines importance, confidence, recencyCross-agent collaboration with access control:
POOL_TYPES = %w[shared agent_private team_shared task_scoped]
SCOPES = %w[global account team agent task]
# Access control structure (stored as JSON)
{
"read" => ["agent-uuid-1", "agent-uuid-2"],
"write" => ["agent-uuid-1"],
"public_read" => false
}
# Operations
pool.read_data
pool.write_data(data)
pool.merge_data(partial_data)
pool.accessible_by?(agent)
pool.grant_access(agent_id, level)
pool.revoke_access(agent_id)
pool.statistics # entry count, size, access metricsAi::Memory::RouterService routes operations to the appropriate tier:
router = Ai::Memory::RouterService.new(account: account, agent: agent)
value = router.read("session_context", session_id: "sess-123")
router.write("task_result", { output: "..." }, tier: :short_term, session_id: "sess-123")
results = router.semantic_search(query_embedding, limit: 10)
# Promote frequently-accessed short-term entries to long-term
router.consolidate!(session_id: "sess-123")
stats = router.stats
# => { working_memory: {...}, short_term: {...}, long_term: {...}, shared: {...} }Redis-backed fast key-value store for active sessions:
wm = Ai::Memory::WorkingMemoryService.new(agent: agent, account: account, task: task)
# Basic ops
wm.store("key", value, ttl: 300)
wm.retrieve("key")
wm.exists?("key")
wm.remove("key")
wm.keys
wm.clear
# Specialized storage
wm.store_task_state(state_hash)
wm.store_intermediate_result(step_name, result)
wm.store_conversation_context(messages)
wm.append_to_conversation(message)
wm.store_tool_state(tool_name, state)
wm.store_scratch_pad(content)
wm.append_to_scratch_pad(content)
# Cross-agent sharing
wm.share_with_agent(target_agent_id, key, value)
shared_value = wm.retrieve_shared(source_agent_id, key)
# Persist Redis state to database
wm.persist_to_database
wm.load_from_databaseflowchart LR
subgraph STM["Short-Term"]
EntryA["Entry A<br/>access: 5<br/>ttl: expired"]
EntryB["Entry B<br/>access: 1<br/>ttl: expired"]
end
subgraph LTM["Long-Term"]
Promoted[Promoted with embedding]
end
EntryA -- "access >= 3" --> Promoted
EntryB -- "low access" --> Deleted[Deleted]
EntryC["Entry C<br/>sim: 0.95"]
EntryD["Entry D<br/>sim: 0.95"]
Merged[Merged<br/>deduplicated]
EntryC --> Merged
EntryD --> Merged
Memory access is controlled at multiple levels:
- Account scope — all memory is scoped to an account
- Agent scope — private memory is only accessible to the owning agent
- Pool access control — JSONB
access_controlfield with read/write permissions - Context access control — per-agent grants on
PersistentContext
This section zooms in on the shared memory tier — the Ai::MemoryPool model that backs cross-agent collaboration, agent bulletins, and stigmergic coordination. The memory consolidation diagram above shows how STM entries promote into LTM; pools sit alongside that pipeline as a deliberately-named, deliberately-shared substrate.
Ai::MemoryPool validates pool_type against the model's enum and scope against the runtime scope set. Pool data lives in JSONB, and access control plus retention are declarative:
| Field | Values | Purpose |
|---|---|---|
pool_type |
shared, agent_private, team_shared, task_scoped, global |
Coarse-grained ownership semantic |
scope |
execution, persistent, session |
Lifetime relative to surrounding work |
expires_at |
timestamp / nil |
Optional TTL; active scope excludes expired pools |
retention_policy |
JSON | Per-pool retention config (TTL, max size, eviction) |
access_control |
JSON | `{ "agents": [...uuids], "public": true |
data_size_bytes |
integer | Auto-calculated on save; underpins quota enforcement |
persist_across_executions (boolean) flips a pool into the persistent scope filter so workflow ticks can rebind to long-lived state. Versioning is automatic — before_save increments version whenever data changes.
# frozen_string_literal: true
platform.create_memory_pool(
pool_id: "trading_ops",
name: "Trading Operations Shared State",
pool_type: "shared",
scope: "persistent",
retention_policy: { ttl_seconds: 604_800, max_size_bytes: 5_000_000 }
)The tool is registered as create_memory_pool in Ai::Tools::PlatformApiToolRegistry (see reference/auto/mcp-tools.md for the live parameter schema). pool_id is a unique slug; defaults are pool_type: "shared" and scope: "account".
# frozen_string_literal: true
platform.write_shared_memory(
pool_id: "trading_ops",
key: "agent_bulletin.market_regime",
value: { regime: "risk_off", confidence: 0.82 }
)
platform.read_shared_memory(
pool_id: "trading_ops",
key: "agent_bulletin.market_regime"
)
# => { success: true, key: "agent_bulletin.market_regime", value: { ... } }Dot-separated keys index nested JSON, so agent_bulletin.market_regime writes into data["agent_bulletin"]["market_regime"]. Writes to agent_bulletin.* keys broadcast a memory_pool_key_write event with is_bulletin: true over McpChannel — that is the stigmergic-coordination signal subscribers listen for.
Pool access is enforced at the model layer, not the controller:
accessible_by?(agent_id)— owner always, plus any agent listed inaccess_control["agents"], plus any agent whenaccess_control["public"] == truewritable_by?(agent_id)— owner always; shared pools withpublic: trueadditionally permit any accessible agentagent_privatepools default to owner-only;team_sharedpools require explicit grants viagrant_access(agent_id)(the model has no team-membership lookup, so the writer must populateaccess_control["agents"]after ateam_sharedpool is created)- The
defaultpool is special-cased:find_or_create_by!(pool_id: "default")creates it on demand withaccess_control: { "public" => true }, anddelete_memory_poolrefuses to remove it
For the STM→LTM consolidation diagram and pool/tier overview, see Memory tiers above. For the full set of memory MCP actions (search, consolidate, stats, list pools), see reference/auto/mcp-tools.md.
The RAG (Retrieval-Augmented Generation) system provides document ingestion, chunking, embedding, and multi-modal retrieval. It supports vector search, keyword search, graph-based retrieval, and agentic iterative search with automatic query reformulation.
| Model | Purpose |
|---|---|
Ai::KnowledgeBase |
Container for documents with embedding configuration |
Ai::Document |
Source documents with processing lifecycle |
Ai::DocumentChunk |
Chunked document segments with pgvector embeddings |
Ai::RagQuery |
Query records with embedding and retrieval metadata |
Ai::HybridSearchResult |
Search result records across multiple modes |
flowchart TD
Upload[Upload document]
Doc[Document.create<br/>status: pending]
Process[start_processing!<br/>status: processing]
Chunk[Chunking<br/>recursive / semantic]
Embed[Embedding<br/>set_embedding!]
Done[complete_indexing!<br/>status: indexed<br/>update_stats!]
Upload --> Doc --> Process --> Chunk --> Embed --> Done
Chunking strategies:
| Strategy | Behavior |
|---|---|
recursive |
Recursive character splitting with overlap |
semantic |
Semantic boundary detection |
Configurable via chunk_size and chunk_overlap on KnowledgeBase.
ai_knowledge_base:
embedding_model # e.g. "text-embedding-3-small"
embedding_provider # e.g. "openai"
chunking_strategy # recursive | semantic
chunk_size # target character count
chunk_overlap # overlap between adjacent chunks
status # active | indexing | paused | error | archivedLifecycle: start_indexing!, complete_indexing!, pause!, archive!, mark_error!. Each query is logged via record_query!.
Ai::Rag::HybridSearchService combines multiple search strategies with result fusion:
service = Ai::Rag::HybridSearchService.new(account: account)
results = service.search(
query,
mode: :hybrid, # :vector, :keyword, :graph, :hybrid
top_k: 10,
knowledge_base_ids: [kb.id],
rerank: true
)| Mode | Description | Best For |
|---|---|---|
:vector |
Semantic similarity via pgvector embeddings | Meaning-based queries |
:keyword |
Full-text search via PostgreSQL | Exact term matching |
:graph |
Knowledge graph traversal via GraphRagService |
Entity relationship queries |
:hybrid |
Combines vector + keyword with fusion | General-purpose retrieval |
Fusion methods:
| Method | Description |
|---|---|
| Reciprocal Rank Fusion (RRF, default) | Combines rankings using 1 / (k + rank) formula with k=60 |
| Weighted fusion | Weighted combination of normalized scores |
Ai::Rag::GraphRagService uses knowledge graph communities for retrieval:
service = Ai::Rag::GraphRagService.new(account: account)
results = service.retrieve(query, top_k: 10, max_hops: 2, include_summaries: true)
context = service.build_context(query, token_budget: 4000, max_hops: 3)Pipeline:
- Seed node discovery — finds relevant graph nodes via embedding similarity
- Community detection — discovers connected communities within max hops
- Chunk collection — gathers document chunks linked to community nodes
- Scoring — ranks results by relevance
- Summary building — generates community summaries for context
Constants: MAX_SEED_NODES = 5, SEED_DISTANCE_THRESHOLD = 0.8, MAX_COMMUNITIES = 10, COMMUNITY_MIN_SIZE = 3.
Iterative retrieval with LLM-driven query reformulation for complex queries:
service = Ai::Rag::AgenticRagService.new(account: account)
result = service.retrieve(query, max_rounds: 3)
# => { answer: "...", sources: [...], rounds: 2, total_results: 15 }Per-round pipeline:
- Search — runs hybrid search
- Rerank — re-scores results for relevance
- Sufficiency check —
MIN_RELEVANT_RESULTS = 3,MIN_AVG_SCORE = 0.5 - Gap identification — what's missing from the results?
- Query reformulation — LLM rewrites query to fill gaps
- Synthesis — LLM generates answer from accumulated results
Max rounds: 3 (configurable via MAX_ROUNDS).
Ai::RagQuery records every query for analytics:
has_neighbors :query_embedding
ai_rag_query:
query_text, status, retrieval_strategy, top_k,
similarity_threshold, results_count, avg_score,
processing_time_ms
query.quality_score # computed quality metricAi::HybridSearchResult records search results with mode and fusion metadata:
SEARCH_MODES = %w[vector keyword graph hybrid]
FUSION_METHODS = %w[rrf weighted simple]
# Class method for optimization analysis
Ai::HybridSearchResult.avg_latency_for(mode)| Method | Path | Description |
|---|---|---|
GET |
/api/v1/ai/rag/query |
Query a knowledge base |
GET |
/api/v1/ai/rag/search |
Search documents |
POST |
/api/v1/ai/rag/knowledge_bases |
Create knowledge base |
POST |
/api/v1/ai/rag/documents |
Upload document |
POST |
/api/v1/ai/rag/documents/:id/process |
Trigger processing |
MCP tools expose RAG operations via platform.query_knowledge_base, platform.search_documents, platform.add_document, platform.process_document. See reference/auto/mcp-tools.md for the live catalog.
The Skill Graph manages reusable capabilities that agents can possess and execute. Skills are versioned, categorized, and linked to agents via an assignment model. The system includes conflict detection, proposal workflows, gap detection, and automated lifecycle management.
| Model | Purpose |
|---|---|
Ai::Skill |
Core skill definition with category, status, execution context |
Ai::AgentSkill |
Many-to-many link between agents and skills |
Ai::SkillConflict |
Detected conflicts between overlapping skills |
Ai::SkillProposal |
Workflow for proposing new skills (submit → approve → create) |
Ai::SkillUsageRecord |
Tracks skill execution outcomes |
Ai::SkillVersion |
Version history with A/B testing support |
CATEGORIES = %w[
code_generation code_review testing debugging deployment
documentation analysis communication planning research
data_processing security monitoring optimization
integration automation design architecture
project_management devops operations
]
STATUSES = %w[draft active deprecated archived]flowchart LR
Research[Research<br/>topic analysis]
Proposal[SkillProposal<br/>submitted]
Review[under_review]
Approval[approved]
Created[Ai::Skill<br/>active]
Active[in use]
Deprecated[deprecated]
Archived[archived]
Research --> Proposal --> Review --> Approval --> Created --> Active
Active --> Deprecated --> Archived
Ai::SkillGraph::LifecycleService provides end-to-end skill management:
service = Ai::SkillGraph::LifecycleService.new(account: account)
# Research and propose a new skill
proposal = service.research_and_propose(
"Kubernetes deployment automation",
requesting_agent: agent,
requesting_user: user
)
# Submit for review
service.submit_proposal(proposal.id)
# Approve and create
service.approve_proposal(proposal.id, reviewer: admin_user)
skill = service.create_skill_from_proposal(proposal.id)Pipeline: research (via ResearchService) → propose (with inferred category + confidence) → submit → approve (auto-creates sub-proposals for dependencies) → create (builds Skill, initial SkillVersion, and dependency edges in knowledge graph).
Ai::SkillConflict records overlapping or contradictory skills:
CONFLICT_TYPES = %w[overlap contradiction dependency version_mismatch naming]
SEVERITIES = %w[low medium high critical]
STATUSES = %w[detected acknowledged resolved dismissed]
SEVERITY_WEIGHTS = { "low" => 1, "medium" => 2, "high" => 4, "critical" => 8 }
conflict.resolve!
conflict.dismiss!
conflict.calculate_priority!Ai::SkillVersion tracks version history with A/B testing:
CHANGE_TYPES = %w[major minor patch hotfix experimental]
version.record_outcome!(success: true)
version.activate! # Make this version active| Service | Purpose |
|---|---|
ConflictDetectionService |
Scans for overlapping skills daily at 4:15 AM |
HealthScoreService |
Calculates skill health from usage patterns and conflicts |
EvolutionService |
Tracks skill improvement over versions |
OptimizationService |
Suggests skill configuration improvements |
AutoRepairService |
Automatically resolves simple conflicts |
BridgeService |
Bridges skills to knowledge graph nodes |
TraversalService |
Graph traversal for skill dependency chains |
TeamCoverageService |
Analyzes skill coverage across agent teams |
ResearchService |
AI-powered skill research and analysis |
SelfLearningService |
Skill improvement from usage feedback |
ContextEnrichmentService |
Enriches skill execution context |
Skills are exposed to agents via MCP tools:
platform.discover_skills(description: "deploy to kubernetes")
platform.get_skill_context(skill_id: "uuid")
platform.list_skills(category: "deployment", status: "active")Query the live skill registry via platform.list_skills or platform.discover_skills — it is account-scoped and reflects current ai_skills state. The registry is not committed to git (would churn nightly); a local snapshot can be regenerated with cd server && bundle exec rails mcp:sync_docs.
Content Linking extends Page and KnowledgeBase::Article with bidirectional linking on the knowledge graph. Authors reference other content with Obsidian-style wikilinks ([[Title]] or [[Title|Display Text]]), and the backend extracts those references into typed references edges on Ai::KnowledgeGraphEdge. Each referenced page can then render its backlinks panel, unlinked plain-text mentions, and semantically related pages.
The design goal is to give long-form content the same knowledge-graph surface area as structured entities, so pages, KB articles, missions, and agents all coexist as linkable nodes in one graph.
Inside Page#content (Markdown):
[[Feature Development Guide]]— resolves to thePageorKnowledgeBase::Articlewith a matching title[[Feature Development Guide|see the guide]]— same resolution, custom display text- Case-insensitive title matching, or exact match on a slug:
downcase.gsub(/[^a-z0-9\s-]/, "").gsub(/\s+/, "-")
Resolution order:
Pagescoped to the source page'saccountKnowledgeBase::Article(not account-scoped)
Wikilinks that resolve to nothing are silently dropped from the edge set.
service = ContentLinkService.new(account: account)
# Extract wikilinks + persist as KG edges (idempotent per source node)
service.extract_links!(page) # → Integer count of edges created
# Backlinks panel data
service.backlinks_for(page) # → Array of Page | KB::Article records
# Plain-text mentions (not yet wikilinked)
service.unlinked_mentions_for(page)
# → Pages mentioning the title (excludes source, excludes already-wikilinked,
# ordered by updated_at DESC, limit 20)
# Ensure KG node exists for a page
service.find_or_create_page_node(page) # → Ai::KnowledgeGraphNode entity_type "page"
# Generate + store embedding for the page on its KG node
service.generate_page_embedding!(page)
# Find semantically similar pages via cosine similarity
service.related_pages_for(page, limit: 10)
# → Array of [content_record, similarity_float] pairs, DESC by similarityAi::KnowledgeGraphEdge.create!(
account: account,
source_node: source_page_node,
target_node: target_content_node,
relation_type: "references", # always "references" for wikilinks
weight: 1.0,
confidence: 1.0,
bidirectional: false,
metadata: { link_text: "[[Original Link Text]]" }
)On re-extraction, extract_links! deletes all outgoing references edges from the source node first so the set stays canonical for the current content.
Pages and articles get KG nodes with:
node_type: "entity"entity_type: "page"or"article"metadata: { content_type: "page" | "article", content_id: <uuid>, slug: <slug> }
Nodes track mention_count, status: "active", confidence: 1.0 at creation. The node's name is kept in sync with the page title when find_or_create_page_node detects a mismatch.
All endpoints require the admin.access permission.
| Method | Path | Purpose |
|---|---|---|
GET |
/api/v1/admin/pages/:id/backlinks |
Pages/articles that [[link]] to this page |
GET |
/api/v1/admin/pages/:id/unlinked_mentions |
Pages mentioning the title in plain text only |
GET |
/api/v1/admin/pages/:id/related_pages?limit=N |
Semantic neighbors (default 10, max 50) |
POST |
/api/v1/admin/pages/:id/extract_links |
Force re-extraction |
POST |
/api/v1/admin/pages/:id/generate_embedding |
Force regeneration of the page's embedding |
Read endpoints respond:
{
"success": true,
"data": {
"backlinks": [
{ "id": "uuid", "title": "...", "slug": "...", "type": "page", "excerpt": "..." }
]
}
}related_pages entries additionally include similarity (0.0–1.0 cosine similarity).
- Daily Summaries — each summary is a
Page, so wikilinks in a summary surface as backlinks on referenced pages. Daily reports become implicitly traversable through the graph - RAG — page embeddings produced by
generate_page_embedding!feed the same retrieval pipeline used by document search and KB RAG flows - Knowledge graph tooling — all
referencesedges show up underplatform.search_knowledge_graphandplatform.get_graph_neighborswhen filtered onrelation_type: "references"
The knowledge, memory, and skill subsystems run on automated maintenance schedules. These are background jobs in worker/; see concepts/architecture.md for the worker process model.
| Job | Schedule | Action |
|---|---|---|
| Compound learning decay | 3:45 AM daily | importance_score decays exponentially on stale learnings |
| Memory consolidation | 4:00 AM daily | Promotes STM→long-term (access_count >= 3), deduplicates (similarity >= 0.92) |
| Rot detection | 4:00 AM daily | Auto-archives context entries with staleness >= 0.9 |
| Trust score decay | 2:00 AM daily | Decays idle agent trust scores |
| Skill conflict scan | 4:15 AM daily | Detects new skill conflicts |
| Skill stale decay | 5:00 AM weekly | Reduces effectiveness of unused skills |
| Skill re-embedding | 5:00 AM weekly | Updates skill embeddings for discovery |
| Skill gap detection | 3:00 AM monthly | Identifies missing capabilities across teams |
| Shared knowledge maintenance | Daily | Import from learnings, recalculate quality scores, audit stale entries |
| Escalation timeout | Every 15 min | Auto-escalate overdue escalations |
| Goal maintenance | Every 6 hours | Auto-abandon stale goals |
| Observation pipeline | Every 30 min | Collect sensor data for autonomous agents |
| Observation cleanup | Daily | Delete expired and old processed observations |
| Proposal expiry | Every hour | Expire overdue unreviewed proposals |
| Intervention policy tuning | Weekly | Analyze approval patterns and suggest policy adjustments |
concepts/agents-and-autonomy.md— how agents use this knowledgeconcepts/mcp-and-tools.md— MCP tool surface for memory/knowledge/skillsguides/backend.md— backend implementation patternsguides/content-management.md— content authoring workflow
For live, account-scoped registries (KB content, knowledge graph statistics, compound learnings, skills), query MCP directly — these are not committed to git because they churn with normal platform use: platform.search_knowledge, platform.search_knowledge_graph, platform.query_learnings, platform.list_skills. A local Markdown snapshot can be produced with cd server && bundle exec rails mcp:sync_docs.
This concept consolidates content from:
docs/platform/MEMORY_SYSTEM_ARCHITECTURE.mddocs/platform/RAG_SYSTEM_GUIDE.mddocs/platform/SKILL_GRAPH_REFERENCE.mddocs/platform/CONTENT_LINKING.md
Last verified: 2026-05-17